airscholar / e2e-data-engineering

An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.

Date Created 2023-09-06 (about a year ago)
Commits 5 (last one 8 days ago)
Stargazers 229 (0 this week)
Watchers 4 (0 this week)
Forks 109
License unknown
Ranking

RepositoryStats indexes 618,350 repositories, of these airscholar/e2e-data-engineering is ranked #167,653 (73rd percentile) for total stargazers, and #382,403 for total watchers. Github reports the primary language for this repository as Python, for repositories using this language it is ranked #29,983/125,168.

airscholar/e2e-data-engineering is also tagged with popular topics, for these it's ranked: docker (#2,318/6505),  postgresql (#781/2012),  big-data (#188/365),  data-engineering (#125/320),  cassandra (#54/131),  apache-spark (#48/113)

Other Information

airscholar/e2e-data-engineering has Github issues enabled, there are 5 open issues and 3 closed issues.

Homepage URL: https://www.youtube.com/watch?v=GqAcTrqKcrY

Star History

Github stargazers over time

250250200200150150100100505000Oct '23Oct '23Nov '23Nov '23Dec '23Dec '2320242024Feb '24Feb '24Mar '24Mar '24Apr '24Apr '24May '24May '24Jun '24Jun '24Jul '24Jul '24Aug '24Aug '24Sep '24Sep '24Oct '24Oct '24Nov '24Nov '24Dec '24Dec '2420252025Feb '25Feb '25

Watcher History

Github watchers over time, collection started in '23

4444333333222220242024Feb '24Feb '24Mar '24Mar '24Apr '24Apr '24May '24May '24Jun '24Jun '24Jul '24Jul '24Aug '24Aug '24Sep '24Sep '24Oct '24Oct '24Nov '24Nov '24Dec '24Dec '2420252025Feb '25Feb '25

Recent Commit History

5 commits on the default branch (main) since jan '22

554.54.5443.53.5332.52.5221.51.5110.50.500Oct '23Oct '23Nov '23Nov '23Dec '23Dec '2320242024Feb '24Feb '24Mar '24Mar '24Apr '24Apr '24May '24May '24Jun '24Jun '24Jul '24Jul '24Aug '24Aug '24Sep '24Sep '24Oct '24Oct '24Nov '24Nov '24Dec '24Dec '2420252025Feb '25Feb '25

Yearly Commits

Commits to the default branch (main) per year

2222111111000020242024

Issue History

Total Issues
Open Issues
Closed Issues
88776655443322110020242024Feb '24Feb '24Mar '24Mar '24Apr '24Apr '24May '24May '24Jun '24Jun '24Jul '24Jul '24Aug '24Aug '24Sep '24Sep '24Oct '24Oct '24Nov '24Nov '24Dec '24Dec '2420252025Feb '25Feb '25

Languages

The primary language is Python but there's also others...

PythonPythonShellShell

updated: 2025-02-22 @ 06:41pm, id: 687878380 / R_kgDOKQAw7A