Trending repositories for topic apache-spark

Open source platform for the machine learning lifecycle

An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All compone...

Code for "Efficient Data Processing in Spark" Course

This is the development repository for sparkMeasure, a tool and library designed for efficient analysis and troubleshooting of Apache Spark jobs. It focuses on easing the collection and examination of...

This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]

lakeFS - Data version control for your data lake | Git for data

Simple and Distributed Machine Learning

An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All compone...

Code for "Efficient Data Processing in Spark" Course

Open source platform for the machine learning lifecycle

This is the development repository for sparkMeasure, a tool and library designed for efficient analysis and troubleshooting of Apache Spark jobs. It focuses on easing the collection and examination of...

This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]

lakeFS - Data version control for your data lake | Git for data

Simple and Distributed Machine Learning

Open source platform for the machine learning lifecycle

lakeFS - Data version control for your data lake | Git for data

This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]

An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All compone...

Fully managed Apache Parquet implementation

Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.

Simple and Distributed Machine Learning

PySpark Tutorial for Beginners - Practical Examples in Jupyter Notebook with Spark version 3.4.1. The tutorial covers various topics like Spark Introduction, Spark Installation, Spark RDD Transformati...

Code for "Efficient Data Processing in Spark" Course

This is the development repository for sparkMeasure, a tool and library designed for efficient analysis and troubleshooting of Apache Spark jobs. It focuses on easing the collection and examination of...

Apache Spark docker image

Fundamentals of Spark with Python (using PySpark), code examples

Spark fires is a anti-pattern playground where we deliberately break Spark applications in various ways so you can observe what happens and potentially recognise the issue when you come across it in y...

SQL data analysis & visualization projects using MySQL, PostgreSQL, SQLite, Tableau, Apache Spark and pySpark.

A curated list of awesome Apache Spark packages and resources.

Feathr – A scalable, unified data and AI engineering platform for enterprise

Interactive and Reactive Data Science using Scala and Spark.

Spark fires is a anti-pattern playground where we deliberately break Spark applications in various ways so you can observe what happens and potentially recognise the issue when you come across it in y...

PySpark Tutorial for Beginners - Practical Examples in Jupyter Notebook with Spark version 3.4.1. The tutorial covers various topics like Spark Introduction, Spark Installation, Spark RDD Transformati...

An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All compone...

Code for "Efficient Data Processing in Spark" Course

Fundamentals of Spark with Python (using PySpark), code examples

Fully managed Apache Parquet implementation

This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]

Open source platform for the machine learning lifecycle

lakeFS - Data version control for your data lake | Git for data

This is the development repository for sparkMeasure, a tool and library designed for efficient analysis and troubleshooting of Apache Spark jobs. It focuses on easing the collection and examination of...

Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.

Apache Spark docker image

SQL data analysis & visualization projects using MySQL, PostgreSQL, SQLite, Tableau, Apache Spark and pySpark.

A curated list of awesome Apache Spark packages and resources.

Feathr – A scalable, unified data and AI engineering platform for enterprise

Simple and Distributed Machine Learning

Interactive and Reactive Data Science using Scala and Spark.

Open source platform for the machine learning lifecycle

lakeFS - Data version control for your data lake | Git for data

Fully managed Apache Parquet implementation

Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.

Performance Observability for Apache Spark

Code for "Efficient Data Processing in Spark" Course

This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]

A curated list of awesome Apache Spark packages and resources.

An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All compone...

SQL data analysis & visualization projects using MySQL, PostgreSQL, SQLite, Tableau, Apache Spark and pySpark.

PySpark Tutorial for Beginners - Practical Examples in Jupyter Notebook with Spark version 3.4.1. The tutorial covers various topics like Spark Introduction, Spark Installation, Spark RDD Transformati...

PySpark Cheat Sheet - example code to help you learn PySpark and develop apps faster

pyspark methods to enhance developer productivity 📣 👯 🎉

The Internals of Apache Spark

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

Simple and Distributed Machine Learning

Spark fires is a anti-pattern playground where we deliberately break Spark applications in various ways so you can observe what happens and potentially recognise the issue when you come across it in y...

Apache Spark docker image

R interface for Apache Spark

Spark fires is a anti-pattern playground where we deliberately break Spark applications in various ways so you can observe what happens and potentially recognise the issue when you come across it in y...

This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgres and Streamlit. The system is built using Docker Compose to e...

PySpark Tutorial for Beginners - Practical Examples in Jupyter Notebook with Spark version 3.4.1. The tutorial covers various topics like Spark Introduction, Spark Installation, Spark RDD Transformati...

Performance Observability for Apache Spark

Code for "Efficient Data Processing in Spark" Course

An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All compone...

⛳️ PASS: Amazon Web Services Certified (AWS Certified) Machine Learning Specialty (MLS-C01) by learning based on our Questions & Answers (Q&A) Practice Tests Exams.

Fully managed Apache Parquet implementation

Built a real-time streaming pipeline to extract stock data, using Apache Nifi, Debezium, Kafka, and Spark Streaming. Loaded the transformed data into Glue database and created real-time dashboards usi...

PySpark Cheat Sheet - example code to help you learn PySpark and develop apps faster

A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer

Apache Tomcat exploit and Pentesting guide for penetration tester

pyspark methods to enhance developer productivity 📣 👯 🎉

Open source platform for the machine learning lifecycle

Real-Time Financial Market Data Processing and Prediction application

This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]

REST API for Apache Spark on K8S or YARN

OSM planet dump high performance data loader. Transform OpenStreetMap World/Region PBF dump into partitioned by H3 regions PostGIS pgsnapshot (lossless) OSM schema representation and/or into ArrowIPC/...

lakeFS - Data version control for your data lake | Git for data

Spark in Action, 2nd edition - chapter 1 - Introduction

Code for "Efficient Data Processing in Spark" Course

Spark fires is a anti-pattern playground where we deliberately break Spark applications in various ways so you can observe what happens and potentially recognise the issue when you come across it in y...

This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgres and Streamlit. The system is built using Docker Compose to e...

This project shows how to capture changes from postgres database and stream them into kafka

Open source platform for the machine learning lifecycle

lakeFS - Data version control for your data lake | Git for data

SQL data analysis & visualization projects using MySQL, PostgreSQL, SQLite, Tableau, Apache Spark and pySpark.

Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.

Code for "Efficient Data Processing in Spark" Course

Simple and Distributed Machine Learning

Fully managed Apache Parquet implementation

Performance Observability for Apache Spark

An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All compone...

This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]

A curated list of awesome Apache Spark packages and resources.

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

PySpark Cheat Sheet - example code to help you learn PySpark and develop apps faster

Feathr – A scalable, unified data and AI engineering platform for enterprise

pyspark methods to enhance developer productivity 📣 👯 🎉

This is the development repository for sparkMeasure, a tool and library designed for efficient analysis and troubleshooting of Apache Spark jobs. It focuses on easing the collection and examination of...

The Internals of Apache Spark

PySpark Tutorial for Beginners - Practical Examples in Jupyter Notebook with Spark version 3.4.1. The tutorial covers various topics like Spark Introduction, Spark Installation, Spark RDD Transformati...

BigDL: Distributed TensorFlow, Keras and PyTorch on Apache Spark/Flink & Ray

Apache Spark docker image

Code for "Efficient Data Processing in Spark" Course

Performance Observability for Apache Spark

PySpark Tutorial for Beginners - Practical Examples in Jupyter Notebook with Spark version 3.4.1. The tutorial covers various topics like Spark Introduction, Spark Installation, Spark RDD Transformati...

An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All compone...

⛳️ PASS: Amazon Web Services Certified (AWS Certified) Machine Learning Specialty (MLS-C01) by learning based on our Questions & Answers (Q&A) Practice Tests Exams.

Built a real-time streaming pipeline to extract stock data, using Apache Nifi, Debezium, Kafka, and Spark Streaming. Loaded the transformed data into Glue database and created real-time dashboards usi...

Apache Tomcat exploit and Pentesting guide for penetration tester

Stream CDC into an Amazon S3 data lake in Apache Iceberg table format with AWS Glue Streaming and DMS

A Spark Connector that reads data from / writes data to Arrow-Flight end-points with Arrow-Flight and Flight-SQL

Master's thesis on Big Data

Fully managed Apache Parquet implementation

A command-line interface for packaging, deploying, and running your EMR Serverless Spark jobs

PySpark Cheat Sheet - example code to help you learn PySpark and develop apps faster

SQL data analysis & visualization projects using MySQL, PostgreSQL, SQLite, Tableau, Apache Spark and pySpark.

Experiment tracking server focused on speed and scalability

REST API for Apache Spark on K8S or YARN

Real-Time Financial Market Data Processing and Prediction application

Apache Spark for data engineers

OSM planet dump high performance data loader. Transform OpenStreetMap World/Region PBF dump into partitioned by H3 regions PostGIS pgsnapshot (lossless) OSM schema representation and/or into ArrowIPC/...

A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer