Cloudera News

• 

  Assessment of Apache Impala Performance using Cloudera Manager Metrics – Part 1 of 3
  metrics cdh performance cloudera manager impala apache impala queries tuning • • sorangutan  

  1
  1
  Posts
  99
  Views

  

  For a user-facing system like Apache Impala, bad performance and downtime can have serious negative impacts on your business. Given the complexity of the system and all the moving parts, troubleshooting can be time-consuming and overwhelming.

  In this blog post series, we are going to show how the charts and metrics on Cloudera Manager (CM) can help troubleshoot Impala performance issues. They can also help to monitor the system to predict and prevent future outages.

  Read more

  The post Assessment of Apache Impala Performance using Cloudera Manager Metrics – Part 1 of 3 appeared first on Cloudera Engineering Blog.

  
  
  https://blog.cloudera.com/blog/2018/12/assessment-of-apache-impala-performance-using-cloudera-manager-metrics-part-1-of-3/
• 

  Cloudera Altus Director SDX Integration
  cloud cdh altus cloudera director director data warehouse sdx • • sorangutan  

  1
  1
  Posts
  83
  Views

  

  Cloudera now provides a pathway for migrating an Altus Director managed cluster to a cluster that takes advantage of other Cloudera Altus cloud services. This blog post will outline how to do that.

  SDX for Cloudera Altus persists both Apache Hive metadata and Apache Sentry data access policies independently from clusters in SDX namespaces. In this way, SDX for Cloudera Altus provides the missing link to share Altus Director managed workload with Altus cloud services.

  Read more

  The post Cloudera Altus Director SDX Integration appeared first on Cloudera Engineering Blog.

  
  
  https://blog.cloudera.com/blog/2018/12/cloudera-altus-director-sdx-integration/
• 

  Learn for Free How to Deploy Cloudera Enterprise on Microsoft Azure
  • sorangutan  

  1
  1
  Posts
  62
  Views

  

  At Cloudera, we spend our time helping customers benefit from data. We help them with different types of data—structured, semi-structured, or raw unstructured. We also help them implement solutions for storing, tracing, securing, processing, enriching, analyzing, and visualizing it.

  Over the past several years, we’ve observed that customers are increasingly working with their data in the cloud, and Microsoft’s Azure cloud service is a popular deployment option. Cloudera University is pleased to announce a free course,

  Read more

  The post Learn for Free How to Deploy Cloudera Enterprise on Microsoft Azure appeared first on Cloudera Engineering Blog.

  
  
  https://blog.cloudera.com/blog/2018/11/learn-for-free-how-to-deploy-cloudera-enterprise-on-microsoft-azure/
• 

  Proactive Data Pipeline Alerting with Pulse
  events cdh search logs guest apache solr cloudera search phdata pulse • • sorangutan  

  1
  1
  Posts
  109
  Views

  

  In mid-2017, we were working with one of the world’s largest healthcare companies to put a new data application into production. The customer had grown through acquisition and in order to maintain compliance with the FDA, they needed to aggregate data in real-time from dozens of different divisions of the company. The consumers of this application, of course, did not care how we built the data pipeline. However, they cared greatly that if it broke,

  Read more

  The post Proactive Data Pipeline Alerting with Pulse appeared first on Cloudera Engineering Blog.

  
  
  http://blog.cloudera.com/blog/2018/11/proactive-data-pipeline-alerting-with-pulse/
• 

  Protecting Hadoop Clusters From Malware Attacks
  • sorangutan  

  1
  1
  Posts
  65
  Views

  

  Two new strains of malware–XBash and DemonBot–are targeting Apache Hadoop servers for Bitcoin mining and DDOS purposes. This malware is scanning the internet so vigorously for Hadoop clusters that an infection can occur within minutes of an insecure cluster being placed on the open internet. This blog post describes the mechanism this malware uses and offers specific actions to protect your Hadoop-based clusters.

  A History of Hadoop Malware

  Roughly two years ago there were a spate of attacks against the open source database solution MongoDB,

  Read more

  The post Protecting Hadoop Clusters From Malware Attacks appeared first on Cloudera Engineering Blog.

  
  
  https://blog.cloudera.com/blog/2018/11/protecting-hadoop-clusters-from-malware-attacks/
• 

  New in Cloudera Enterprise 6: Apache Hive 2.1
  cdh hive hive-on-spark • • sorangutan  

  1
  1
  Posts
  120
  Views

  

  We recently released Cloudera Enterprise 6.0 featuring significant improvements across a number of core components. In this blog post, we’re going to focus on Apache Hive 2.1.

  Hive’s Approach to Rebase: Stability and Quality Most Important

  Prior to the release of Cloudera Enterprise 6.0, Cloudera’s supported platform included Apache Hive 1.1 augmented with numerous features, enhancements and fixes from the later Apache Hive releases—all of which were included only after rigorous quality criteria were met.

  Read more

  The post New in Cloudera Enterprise 6: Apache Hive 2.1 appeared first on Cloudera Engineering Blog.

  
  
  http://blog.cloudera.com/blog/2018/10/new-in-cloudera-enterprise-6-apache-hive-2-1/
• 

  Network Security with Cloudera Altus and Apache Spot
  security altus platform security & cyber apache spot spot streamsets • • sorangutan  

  1
  1
  Posts
  135
  Views

  

  Introduction

  In the last few years, IT security threats to enterprise systems have increased, which has necessitated installing log ingestion and analysis solutions in any enterprise network.

  This blog post illustrates how Cloudera built its own scalable solution for log ingestion and analytics using Apache Spot and Cloudera Altus. By leveraging transient workloads in the cloud, Cloudera reduced the solution’s operational costs by 50% when compared to traditional, persistent cluster approaches.

  Use Case

  At Cloudera,

  Read more

  The post Network Security with Cloudera Altus and Apache Spot appeared first on Cloudera Engineering Blog.

  
  
  http://blog.cloudera.com/blog/2018/10/network-security-with-cloudera-altus-and-apache-spot/
• 

  Third-Party Libraries in C6
  security development cdh open source general platform security & cyber dependencies libraries • • sorangutan  

  1
  1
  Posts
  112
  Views

  

  Cloudera has put a significant amount of work into upgrading the third-party libraries used in our just-released C6 version. This major upgrade of our software has given us the opportunity to upgrade many of the libraries we use. These upgrades allow us to avoid security vulnerabilities, use modern versions of libraries, and to standardize versions of libraries across CDH.

  Modern software development relies on reusing other people’s code. Code reused in this fashion is called a “third-party library.”

  Read more

  The post Third-Party Libraries in C6 appeared first on Cloudera Engineering Blog.

  
  
  http://blog.cloudera.com/blog/2018/10/third-party-libraries-in-c6/
• 

  Cloudera Altus Cloud Services—You’re in Control (Part 2)
  aws cloud infrastructure altus • • sorangutan  

  1
  1
  Posts
  157
  Views

  

  In part 1 of this two-part series, we discussed how Cloudera Altus cloud services put you, the Cloudera Altus customer, in control of your cloud resources. In that blog post, we described the problems and solutions in a cloud provider independent manner and then showed the solutions with Azure. In this second blog post, we’ll look at the solutions using AWS.

  We previously introduced two control problems: the infrastructure construction problem (how do Cloudera Altus cloud services allow you to have precise control over how they build and manage a cluster for you);

  Read more

  The post Cloudera Altus Cloud Services—You’re in Control (Part 2) appeared first on Cloudera Engineering Blog.

  
  
  http://blog.cloudera.com/blog/2018/10/cloudera-altus-cloud-services-youre-in-control-part2/
• 

  Cloudera Altus Cloud Services—You’re in Control (Part 1)
  • sorangutan  

  1
  1
  Posts
  99
  Views

  

  Cloudera Altus is a suite of products which enable you to deploy and consume Cloudera Enterprise clusters in the public cloud environment of your choice. With Cloudera Altus cloud services such as Altus Data Engineering or Altus Data Warehouse Cloudera creates and manages clusters on your behalf, using your cloud provider’s infrastructure resources, freeing you up to focus on your workloads and their business value. Cloudera Altus cloud services creates resources and manages them in your account,

  Read more

  The post Cloudera Altus Cloud Services—You’re in Control (Part 1) appeared first on Cloudera Engineering Blog.

  
  
  http://blog.cloudera.com/blog/2018/10/cloudera-altus-cloud-services-youre-in-control-part1/
• 

  What’s New in Cloudera Altus Director 6.0?
  cloud altus cloudera director director administration • • sorangutan  

  1
  1
  Posts
  145
  Views

  

  Cloudera Altus Director 6.0 adds support for deploying CDH 6 clusters on public cloud infrastructure. Altus Director also features improved integration with SDX configured namespaces, allows encryption of its configuration properties, and includes a tool for copying its database. Altus Director’s interfaces have been streamlined, removing deprecated features and old API endpoints.

  Cloudera Altus Director, formerly known as Cloudera Director, helps you deploy, scale, and manage Cloudera clusters in AWS,

  Read more

  The post What’s New in Cloudera Altus Director 6.0? appeared first on Cloudera Engineering Blog.

  
  
  http://blog.cloudera.com/blog/2018/09/whats-new-in-cloudera-altus-director-6-0/
• 

  Support Best Practices Video
  customer success best practices communication video customer experience support • • sorangutan  

  1
  1
  Posts
  99
  Views

  

  Cloudera Support recently received Service Capability & Performance (SCP) certification for the third consecutive year. In order to earn this industry-leading mark of distinction, a support organization must demonstrate an ongoing commitment to exemplary standards and methodologies in over 100 major performance areas. At the heart of this achievement is the fundamental mission of Cloudera Support: Enable total customer success.

  Cloudera strongly believes that timely, accurate, and healthy communication has to exist between the customer and our support organization in order to result in a successful outcome for both parties.

  Read more

  The post Support Best Practices Video appeared first on Cloudera Engineering Blog.

  
  
  http://blog.cloudera.com/blog/2018/09/support-best-practices-video/
• 

  Robust Message Serialization in Apache Kafka Using Apache Avro, Part 3
  cdh avro streaming use case kafka how-to • • sorangutan  

  1
  1
  Posts
  117
  Views

  

  Part 3: Configuring Clients

  Earlier, we introduced Kafka Serializers and Deserializers that are capable of writing and reading Kafka records in Avro format. In this part we will going to see how to configure producers and consumers to use them.

  Setting up a Kafka Topic for use as a Schema Store

  KafkaTopicSchemaProvider works with a Kafka topic as its persistent store. This topic will contain at most thousands of records: the schemas. It does not need multiple partitions,

  Read more

  The post Robust Message Serialization in Apache Kafka Using Apache Avro, Part 3 appeared first on Cloudera Engineering Blog.

  
  
  http://blog.cloudera.com/blog/2018/08/robust-message-serialization-in-apache-kafka-using-apache-avro-part-3/
• 

  Robust Message Serialization in Apache Kafka Using Apache Avro, Part 2
  cdh avro streaming use case kafka how-to • • sorangutan  

  1
  1
  Posts
  169
  Views

  

  Implementing a Schema Store

  In Part 1, we saw the need for an Apache Avro schema provider but did not implement one. In this part we will implement a schema provider that works with Apache Kafka as storage.

  In-Memory SchemaStore

  First we can implement an in-memory store for schemas. This is useful to understand the requirements for such a store and as the cache of the Kafka backed store. A SchemaStore has to be quick in looking up VersionedSchema entries.

  Read more

  The post Robust Message Serialization in Apache Kafka Using Apache Avro, Part 2 appeared first on Cloudera Engineering Blog.

  
  
  http://blog.cloudera.com/blog/2018/07/robust-message-serialization-in-apache-kafka-using-apache-avro-part-2/
• 

  Robust Message Serialization in Apache Kafka Using Apache Avro, Part 1
  cdh avro streaming use case kafka how-to • • sorangutan  

  1
  1
  Posts
  119
  Views

  

  In Apache Kafka, Java applications called producers write structured messages to a Kafka cluster (made up of brokers). Similarly, Java applications called consumers read these messages from the same cluster.  In some organizations, there are different groups in charge of writing and managing the producers and consumers. In such cases, one major pain point can be in the coordination of the agreed upon message format between producers and consumers.

  This example demonstrates how to use Apache Avro to serialize records that are produced to Apache Kafka while allowing evolution of schemas and nonsynchronous update of producer and consumer applications.

  Read more

  The post Robust Message Serialization in Apache Kafka Using Apache Avro, Part 1 appeared first on Cloudera Engineering Blog.

  
  
  http://blog.cloudera.com/blog/2018/07/robust-message-serialization-in-apache-kafka-using-apache-avro-part-1/
• 

  Fine Grained Access Control in Cloudera Manager
  cdh cloudera manager access control cluster isolation fine grained access contr granular permissions role based access control • • sorangutan  

  1
  1
  Posts
  122
  Views

  

  One instance of Cloudera Manager (CM) can manage N clusters. In the current Role Based Access Control (RBAC) model, CM users hold privileges and permissions across everything in CM’s purview (including every cluster that CM manages). For example, Read-Only user John is a user who can perform all the actions of Read-Only users on all clusters managed by CM. The “Cluster Admin” user Chris is a cluster administrator of all the clusters managed by CM.

  Read more

  The post Fine Grained Access Control in Cloudera Manager appeared first on Cloudera Engineering Blog.

  
  
  http://blog.cloudera.com/blog/2018/07/fine-grained-access-control-in-cloudera-manager/
• 

  Deploy Cloudera EDH Clusters Like a Boss Revamped – Part 3: Cloud Considerations
  • sorangutan  

  1
  1
  Posts
  162
  Views

  

  The previous two sections have concentrated on infrastructure considerations and services and role layouts for categories of workloads such as Analytic DB and Operational DB. Many of the concepts described therein apply predominantly to on-premise clusters while others apply to clusters deployed on-premise or in the cloud. This section will concentrate predominantly on those considerations that apply to cloud deployments only.

  At the time of this writing, Cloudera supports 3 Infrastructure as a Service (IaaS) platforms: Amazon Elastic Compute Cloud (AWS),

  Read more

  The post Deploy Cloudera EDH Clusters Like a Boss Revamped – Part 3: Cloud Considerations appeared first on Cloudera Engineering Blog.

  
  
  http://blog.cloudera.com/blog/2018/05/deploy-cloudera-edh-clusters-like-a-boss-revamped-part-3-cloud-considerations/