Cloudera News

• 

  Robust Message Serialization in Apache Kafka Using Apache Avro, Part 3
  cdh avro streaming use case kafka how-to • • sorangutan  

  1
  1
  Posts
  120
  Views

  

  Part 3: Configuring Clients

  Earlier, we introduced Kafka Serializers and Deserializers that are capable of writing and reading Kafka records in Avro format. In this part we will going to see how to configure producers and consumers to use them.

  Setting up a Kafka Topic for use as a Schema Store

  KafkaTopicSchemaProvider works with a Kafka topic as its persistent store. This topic will contain at most thousands of records: the schemas. It does not need multiple partitions,

  Read more

  The post Robust Message Serialization in Apache Kafka Using Apache Avro, Part 3 appeared first on Cloudera Engineering Blog.

  
  
  http://blog.cloudera.com/blog/2018/08/robust-message-serialization-in-apache-kafka-using-apache-avro-part-3/
• 

  Robust Message Serialization in Apache Kafka Using Apache Avro, Part 2
  cdh avro streaming use case kafka how-to • • sorangutan  

  1
  1
  Posts
  173
  Views

  

  Implementing a Schema Store

  In Part 1, we saw the need for an Apache Avro schema provider but did not implement one. In this part we will implement a schema provider that works with Apache Kafka as storage.

  In-Memory SchemaStore

  First we can implement an in-memory store for schemas. This is useful to understand the requirements for such a store and as the cache of the Kafka backed store. A SchemaStore has to be quick in looking up VersionedSchema entries.

  Read more

  The post Robust Message Serialization in Apache Kafka Using Apache Avro, Part 2 appeared first on Cloudera Engineering Blog.

  
  
  http://blog.cloudera.com/blog/2018/07/robust-message-serialization-in-apache-kafka-using-apache-avro-part-2/
• 

  Robust Message Serialization in Apache Kafka Using Apache Avro, Part 1
  cdh avro streaming use case kafka how-to • • sorangutan  

  1
  1
  Posts
  120
  Views

  

  In Apache Kafka, Java applications called producers write structured messages to a Kafka cluster (made up of brokers). Similarly, Java applications called consumers read these messages from the same cluster.  In some organizations, there are different groups in charge of writing and managing the producers and consumers. In such cases, one major pain point can be in the coordination of the agreed upon message format between producers and consumers.

  This example demonstrates how to use Apache Avro to serialize records that are produced to Apache Kafka while allowing evolution of schemas and nonsynchronous update of producer and consumer applications.

  Read more

  The post Robust Message Serialization in Apache Kafka Using Apache Avro, Part 1 appeared first on Cloudera Engineering Blog.

  
  
  http://blog.cloudera.com/blog/2018/07/robust-message-serialization-in-apache-kafka-using-apache-avro-part-1/
• 

  Fine Grained Access Control in Cloudera Manager
  cdh cloudera manager access control cluster isolation fine grained access contr granular permissions role based access control • • sorangutan  

  1
  1
  Posts
  123
  Views

  

  One instance of Cloudera Manager (CM) can manage N clusters. In the current Role Based Access Control (RBAC) model, CM users hold privileges and permissions across everything in CM’s purview (including every cluster that CM manages). For example, Read-Only user John is a user who can perform all the actions of Read-Only users on all clusters managed by CM. The “Cluster Admin” user Chris is a cluster administrator of all the clusters managed by CM.

  Read more

  The post Fine Grained Access Control in Cloudera Manager appeared first on Cloudera Engineering Blog.

  
  
  http://blog.cloudera.com/blog/2018/07/fine-grained-access-control-in-cloudera-manager/
• 

  Deploy Cloudera EDH Clusters Like a Boss Revamped – Part 3: Cloud Considerations
  • sorangutan  

  1
  1
  Posts
  162
  Views

  

  The previous two sections have concentrated on infrastructure considerations and services and role layouts for categories of workloads such as Analytic DB and Operational DB. Many of the concepts described therein apply predominantly to on-premise clusters while others apply to clusters deployed on-premise or in the cloud. This section will concentrate predominantly on those considerations that apply to cloud deployments only.

  At the time of this writing, Cloudera supports 3 Infrastructure as a Service (IaaS) platforms: Amazon Elastic Compute Cloud (AWS),

  Read more

  The post Deploy Cloudera EDH Clusters Like a Boss Revamped – Part 3: Cloud Considerations appeared first on Cloudera Engineering Blog.

  
  
  http://blog.cloudera.com/blog/2018/05/deploy-cloudera-edh-clusters-like-a-boss-revamped-part-3-cloud-considerations/