airscholar / e2e-data-engineering

An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.

Date Created 2023-09-06 (about a year ago)

Commits 3 (last one about a year ago)

Stargazers 195 (0 this week)

Watchers 3 (0 this week)

Forks 90

License unknown

Ranking

RepositoryStats indexes 579,238 repositories, of these airscholar/e2e-data-engineering is ranked #180,930 (69th percentile) for total stargazers, and #420,216 for total watchers. Github reports the primary language for this repository as Python, for repositories using this language it is ranked #32,039/115,001.

airscholar/e2e-data-engineering is also tagged with popular topics, for these it's ranked: docker (#2,459/6172), postgresql (#816/1888), big-data (#197/359), data-engineering (#125/292), cassandra (#59/127), apache-spark (#48/108)

Other Information

airscholar/e2e-data-engineering has Github issues enabled, there are 4 open issues and 2 closed issues.

Homepage URL: https://www.youtube.com/watch?v=GqAcTrqKcrY

All Topics

docker big-data cassandra postgresql apache-kafka apache-spark data-storage etl-pipeline data-pipeline apache-airflow data-processing apache-zookeeper containerization data-engineering real-time-analytics

Star History

Github stargazers over time

Watcher History

Github watchers over time, collection started in '23

Recent Commit History

3 commits on the default branch (main) since jan '22

Yearly Commits

Commits to the default branch (main) per year

Issue History

Languages

The primary language is Python but there's also others...

updated: 2024-11-05 @ 02:18pm, id: 687878380 / R_kgDOKQAw7A