27 results found Sort:

apache
datafusion apache
1.4k
7.0k
apache-2.0
115
Apache DataFusion SQL Query Engine
sql olap rust arrow python big-data dataframe datafusion query-engine
Created 2021-04-17
10,305 commits to main branch, last one 9 hours ago
ibis-project
ibis ibis-project
627
5.7k
apache-2.0
82
the portable Python dataframe library
sql mssql mysql trino duckdb impala pandas polars python sqlite pyarrow pyspark bigquery database snowflake clickhouse datafusion postgresql
Created 2015-04-17
9,539 commits to main branch, last one 5 hours ago
roapi
roapi roapi
188
3.3k
apache-2.0
42
Create full-fledged APIs for slowly moving datasets without writing a single line of code.
s3 sql rust arrow query graphql parquet columnar datasets rest-api analytics datafusion delta-lake blob-storage cloud-native query-frontends static-datasets in-memory-database
Created 2020-12-11
290 commits to main branch, last one 7 days ago
lakesoul-io
LakeSoul lakesoul-io
405
2.7k
apache-2.0
254
LakeSoul is an end-to-end, realtime and cloud native Lakehouse framework with fast data ingestion, concurrent update and incremental data analytics on cloud storages for both BI and AI applications.
sql rust arrow flink spark velox python pytorch big-data datalake lakesoul lakehouse streaming datafusion postgresql vectorized huggingface
Created 2021-12-28
1,188 commits to main branch, last one 14 hours ago
kwai
blaze kwai
149
1.4k
apache-2.0
23
Blazing-fast query execution engine speaks Apache Spark language and has Arrow-DataFusion at its core.
spark big-data rust-lang datafusion
Created 2021-06-28
1,237 commits to master branch, last one 12 hours ago
apache
datafusion-comet apache
192
922
apache-2.0
55
Apache DataFusion Comet Spark Accelerator
rust arrow spark datafusion
Created 2024-01-15
730 commits to main branch, last one 5 hours ago
lakehq
sail lakehq
24
704
apache-2.0
10
LakeSail's computation framework with a mission to unify batch processing, stream processing, and compute-intensive (AI) workloads.
sql data rust arrow spark python pyspark big-data datafusion
Created 2023-12-21
342 commits to main branch, last one a day ago
paradedb
pg_analytics paradedb
21
522
postgresql
5
DuckDB-powered data lake analytics from Postgres
sql olap arrow duckdb iceberg parquet big-data columnar database datalake paradedb postgres analytics deltalake lakehouse datafusion postgresql object-storage lakehouse-platform realtime-analytics
This repository has been archived (exclude archived)
Created 2024-05-09
128 commits to dev branch, last one 12 days ago
splitgraph
seafowl splitgraph
14
481
apache-2.0
9
Analytical database for data-driven Web applications 🪶
api sql edge http rust database delta-rs datafusion delta-lake serverless visualization
Created 2022-07-04
1,448 commits to main branch, last one 2 months ago
ark-flow
arkflow ark-flow
13
433
apache-2.0
1
High-performance Rust stream processing engine, providing powerful data stream processing capabilities, supporting multiple input/output sources and processors.
sql flow rust kafka mysql tokio duckdb sqlite stream tokio-rs rust-lang datafusion postgresql
Created 2025-03-01
113 commits to main branch, last one 8 hours ago
kamu-data
kamu-cli kamu-data
14
311
other
16
Next-generation decentralized data lakehouse and a multi-party stream processing network
sql kamu flink spark jupyter open-data blockchain datafusion data-as-code data-science data-management open-data-fabric
Created 2019-05-19
1,777 commits to master branch, last one 3 days ago
JanKaul
iceberg-rust JanKaul
22
157
apache-2.0
8
Rust implementation of Apache Iceberg with integration for Datafusion
rust arrow iceberg datafusion
Created 2022-12-20
1,187 commits to main branch, last one 5 days ago
datafusion-contrib
datafusion-dft datafusion-contrib
13
142
apache-2.0
9
Batteries included CLI, TUI, and server implementations for DataFusion.
cli tui data arrow database datafusion
Created 2022-03-18
333 commits to main branch, last one 3 days ago
PRQL
prql-query PRQL
7
130
apache-2.0
4
Query and transform data with PRQL
csv data json prql query duckdb parquet database querydsl csv-files datafusion json-files postgresql csv-converter parquet-files parquet-tools parquet-viewer
This repository has been archived (exclude archived)
Created 2022-10-11
140 commits to main branch, last one 2 years ago
XiangpengHao
liquid-cache XiangpengHao
5
87
apache-2.0
5
10x lower latency for cloud-native DataFusion
arrow cache parquet datafusion object-store query-engine data-analytics
Created 2024-12-17
176 commits to main branch, last one 6 hours ago
datafusion-contrib
datafusion-java datafusion-contrib
14
76
apache-2.0
2
Java binding to Apache DataFusion
java arrow ballista datafusion
Created 2021-10-12
169 commits to main branch, last one about a month ago
duo-rs
duo duo-rs
7
73
mit
5
A lightweight Logging and Tracing observability solution for Rust, built with Apache Arrow, Apache Parquet and Apache DataFusion.
rust logging tracing datafusion apache-arrow observability apache-parquet
Created 2022-01-03
229 commits to master branch, last one 6 months ago
hw2499
etl-engine hw2499
14
70
unknown
3
etl engine 轻量级 跨平台 流批一体ETL引擎 数据抽取-转换-装载 ETL engine lightweight cross platform batch flow integration ETL engine data extraction transformation loading
cdc etl hive hbase kafka mysql redis influx oracle crontab elastic sqlite3 rocketmq sqlserver clickhouse datafusion etl-engine postgresql prometheus mysqlbinlog
Created 2022-04-21
239 commits to main branch, last one 4 months ago
datafusion-contrib
datafusion-objectstore-s3 datafusion-contrib
13
61
apache-2.0
10
S3 as an ObjectStore for DataFusion
rust datafusion
Created 2022-01-04
108 commits to main branch, last one 2 years ago
wheretrue
exon wheretrue
5
59
other
2
Exon is an OLAP query engine specifically for biology and life science applications.
ngs sql rust arrow datafusion proteomics bioinformatics
Created 2023-05-28
896 commits to main branch, last one 2 months ago
shauryashaurya
learn-data-munging shauryashaurya
21
48
mit
3
Notes on Data Engineering with Pandas, PySpark, Dask, Ray, Arrow DataFusion, Polars etc.
ray dask arrow numpy spark pandas polars jupyter pyspark datafusion dask-distributed data-engineering
Created 2022-06-19
515 commits to main branch, last one 8 days ago
metrico
influxdb3-community metrico
2
45
unknown
8
Community InfluxDB 3.0 "IOx" static builds + containers + Examples for Developers & Integrators. Experiment with low-cost storage, unlimited cardinality and FlightSQL APIs
iox flux musl rust arrow influx influxdb flightsql influxdb3 datafusion lineprotocol
Created 2023-05-20
201 commits to main branch, last one 17 days ago
baggiponte
awesome-pandas-alternatives baggiponte
3
44
mit
5
Awesome list of alternative dataframe libraries in Python.
sql cudf dask arrow pandas polars python awesome rapidsai datatable blazingsql datafusion apache-arrow awesome-list
Created 2021-11-21
13 commits to main branch, last one 2 years ago
biodatageeks
polars-bio biodatageeks
1
39
apache-2.0
2
Blazing-Fast Bioinformatic Operations on Python DataFrames
arrow pandas polars genomics rust-lang dataframes datafusion bioinformatics genomic-ranges genomic-intervals
Created 2024-11-26
88 commits to master branch, last one 5 days ago
splitgraph
seafowl-gcsfuse splitgraph
0
38
unknown
4
Scale to zero Seafowl hosting with Cloud Run
gcp faas rust seafowl severless datafusion
Created 2023-04-22
10 commits to master branch, last one about a year ago
treebee
elixir-arrow treebee
6
38
apache-2.0
5
Experimental Elixir bindings for Apache Arrow including Parquet and DataFusion
arrow parquet datafusion query-engine
Created 2021-03-20
62 commits to main branch, last one 4 years ago
datafusion-contrib
datafusion-materialized-views datafusion-contrib
2
27
apache-2.0
10
Incremental view maintenance & query rewriting for materialized views in DataFusion
sql rust arrow big-data datafusion materialized-views
Created 2024-12-23
22 commits to main branch, last one 5 days ago