Statistics for topic data-integration

RepositoryStats tracks 603,442 Github repositories, of these 59 are tagged with the data-integration topic. The most common primary language for repositories using this topic is Python (21).

Stargazers over time for topic data-integration

Most starred repositories for topic data-integration (view more)

apache
airflow apache
14.5k
38.3k
apache-2.0
766
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
dag elt etl mlops apache python airflow workflow scheduler automation data-science orchestration apache-airflow data-pipelines workflow-engine data-engineering data-integration machine-learning data-orchestrator workflow-orchestration
Created 2015-04-13
27,629 commits to main branch, last one 11 hours ago
Avaiga
taipy Avaiga
1.9k
17.6k
apache-2.0
80
Turns Data and AI algorithms into production-ready web applications in no time.
mlops python data-ops pipeline scenario workflow pipelines taipy-gui automation taipy-core datascience hacktoberfest job-scheduler orchestration developer-tools data-engineering data-integration hacktoberfest2023 scenario-analysis data-visualization
Created 2022-02-18
7,340 commits to develop branch, last one 3 days ago
airbytehq
airbyte airbytehq
4.2k
16.9k
other
186
The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
s3 elt etl data java mssql mysql python bigquery pipeline redshift snowflake postgresql self-hosted data-analysis data-pipeline data-collection data-engineering data-integration change-data-capture
Created 2020-07-27
25,605 commits to master branch, last one 11 hours ago
dagster-io
dagster dagster-io
1.5k
12.2k
apache-2.0
125
An orchestration platform for the development, production, and observation of data assets.
etl mlops python dagster metadata workflow analytics scheduler data-science orchestration data-pipelines data-engineering data-integration data-orchestrator workflow-automation
Created 2018-04-30
21,783 commits to master branch, last one 4 hours ago
apache
seatunnel apache
1.9k
8.2k
apache-2.0
176
SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
cdc elt batch apache offline real-time streaming data-ingestion data-integration high-performance change-data-capture
Created 2017-08-05
4,597 commits to dev branch, last one 19 hours ago
mage-ai
mage-ai mage-ai
784
8.1k
apache-2.0
62
🧙 Build, run, and manage data pipelines for integrating and transforming data.
dbt elt etl sql data spark python pipeline pipelines reverse-etl data-science orchestration data-pipelines transformation data-engineering data-integration machine-learning artificial-intelligence
Created 2022-05-16
5,516 commits to master branch, last one 2 days ago

Trending repositories for topic data-integration (view more)

Last 3 days (new repositories)
no newly created repositories trending in the last 3 days
Last 3 days (absolute gain)
apache
apache/airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

38,255 (+28)
apache-2.0
Avaiga
Avaiga/taipy

Turns Data and AI algorithms into production-ready web applications in no time.

17,594 (+13)
apache-2.0
dagster-io
dagster-io/dagster

An orchestration platform for the development, production, and observation of data assets.

12,250 (+11)
apache-2.0
pracdata
pracdata/awesome-open-source-data-engineering

A curated list of open source tools used in analytics platforms and data engineering ecosystem

174 (+4)
apache
apache/flink-cdc

Flink CDC is a streaming data integration tool

5,902 (+3)
apache-2.0
Last 3 days (relative gain)
pracdata
pracdata/awesome-open-source-data-engineering

A curated list of open source tools used in analytics platforms and data engineering ecosystem

174 (+2%)
artie-labs
artie-labs/transfer

Database replication platform that leverages change data capture. Stream production data from databases to your data warehouse (Snowflake, BigQuery, Redshift, Databricks) in real-time.

621 (+0.5%)
theislab
theislab/scarches

Reference mapping for single-cell genomics

348 (+0.3%)
bsd-3-clause
infinyon
infinyon/fluvio

Lean and mean distributed stream processing system written in rust and web assembly. Alternative to Kafka + Flink in one.

3,960 (+0.2%)
apache-2.0
apache
apache/flink-cdc

Flink CDC is a streaming data integration tool

5,902 (+0.1%)
apache-2.0
Last week (new repositories)
no newly created repositories trending in the last week
Last week (absolute gain)
airbytehq
airbytehq/airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

16,865 (+178)
apache
apache/airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

38,255 (+82)
apache-2.0
dagster-io
dagster-io/dagster

An orchestration platform for the development, production, and observation of data assets.

12,250 (+48)
apache-2.0
Avaiga
Avaiga/taipy

Turns Data and AI algorithms into production-ready web applications in no time.

17,594 (+40)
apache-2.0
apache
apache/flink-cdc

Flink CDC is a streaming data integration tool

5,902 (+20)
apache-2.0
Last week (relative gain)
pracdata
pracdata/awesome-open-source-data-engineering

A curated list of open source tools used in analytics platforms and data engineering ecosystem

174 (+3%)
artie-labs
artie-labs/transfer

Database replication platform that leverages change data capture. Stream production data from databases to your data warehouse (Snowflake, BigQuery, Redshift, Databricks) in real-time.

621 (+1%)
airbytehq
airbytehq/airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

16,865 (+1%)
theislab
theislab/scarches

Reference mapping for single-cell genomics

348 (+0.6%)
bsd-3-clause
dataplane-app
dataplane-app/dataplane

Dataplane is an Airflow inspired unified data platform with additional data mesh and RPA capability to automate, schedule and design data pipelines and workflows. Dataplane is written in Golang with a...

222 (+0.5%)
Last month (new repositories)
no newly created repositories trending in the last month
Last month (absolute gain)
apache
apache/airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

38,255 (+665)
apache-2.0
airbytehq
airbytehq/airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

16,865 (+420)
Avaiga
Avaiga/taipy

Turns Data and AI algorithms into production-ready web applications in no time.

17,594 (+356)
apache-2.0
bruin-data
bruin-data/ingestr

ingestr is a CLI tool to copy data between any databases with a single command seamlessly.

2,835 (+254)
mit
dagster-io
dagster-io/dagster

An orchestration platform for the development, production, and observation of data assets.

12,250 (+192)
apache-2.0
Last month (relative gain)
artie-labs
artie-labs/reader

Perform historical snapshots without database locks and read change data capture logs from databases. Artie Reader is compatible with Debezium and is written in Go.

47 (+24%)
linkml
linkml/linkml-model

Link Modeling Language (LinkML) model

42 (+11%)
bruin-data
bruin-data/ingestr

ingestr is a CLI tool to copy data between any databases with a single command seamlessly.

2,835 (+10%)
mit
starlake-ai
starlake-ai/starlake

Declarative text based tool for data analysts and engineers to extract, load, transform and orchestrate their data pipelines.

67 (+10%)
apache-2.0
pracdata
pracdata/awesome-open-source-data-engineering

A curated list of open source tools used in analytics platforms and data engineering ecosystem

174 (+7%)
Last 12-months (new repositories)
bruin-data
bruin-data/ingestr

ingestr is a CLI tool to copy data between any databases with a single command seamlessly.

2,835
mit
pracdata
pracdata/awesome-open-source-data-engineering

A curated list of open source tools used in analytics platforms and data engineering ecosystem

174
DerwenAI
DerwenAI/ERKG

Demonstrate integration of Senzing and Neo4j to construct an Entity Resolved Knowledge Graph

27
mit
Last 12-months (absolute gain)
Avaiga
Avaiga/taipy

Turns Data and AI algorithms into production-ready web applications in no time.

17,594 (+14,643)
apache-2.0
apache
apache/airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

38,255 (+5,182)
apache-2.0
airbytehq
airbytehq/airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

16,865 (+4,056)
dagster-io
dagster-io/dagster

An orchestration platform for the development, production, and observation of data assets.

12,250 (+2,872)
apache-2.0
bruin-data
bruin-data/ingestr

ingestr is a CLI tool to copy data between any databases with a single command seamlessly.

2,835 (+2,834)
mit
Last 12-months (relative gain)
swimos
swimos/swim-rust

Self-contained distributed software platform for building stateful, massively real-time streaming applications in Rust.

325 (+2,221%)
apache-2.0
pracdata
pracdata/awesome-open-source-data-engineering

A curated list of open source tools used in analytics platforms and data engineering ecosystem

174 (+1,482%)
Avaiga
Avaiga/taipy

Turns Data and AI algorithms into production-ready web applications in no time.

17,594 (+496%)
apache-2.0
starlake-ai
starlake-ai/starlake

Declarative text based tool for data analysts and engineers to extract, load, transform and orchestrate their data pipelines.

67 (+148%)
apache-2.0
Teichlab
Teichlab/cellhint

A tool for semi-automatic cell type harmonization and integration

99 (+125%)
mit