Managing Python dependencies for Spark workloads in Cloudera Data Engineering

[sorangutan]

Apache Spark is now widely used in many enterprises for building high-performance ETL and Machine Learning pipelines. If the users are already familiar with Python then PySpark provides a python API for using Apache Spark. When users work with PySpark they often use existing python and/or custom Python packages in their program to extend and […]

The post Managing Python dependencies for Spark workloads in Cloudera Data Engineering appeared first on Cloudera Blog.

https://blog.cloudera.com/managing-python-dependencies-for-spark-workloads-in-cloudera-data-engineering/