6 results found Sort:

vaexio
vaex vaexio
589
8.2k
mit
143
Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀
hdf5 python bigdata pyarrow dataframe data-science tabular-data visualization machinelearning machine-learning memory-mapped-file
Created 2014-09-27
3,636 commits to master branch, last one 11 months ago
ibis-project
ibis ibis-project
548
4.5k
apache-2.0
82
the portable Python dataframe library
sql dask mssql mysql trino duckdb impala pandas polars python sqlite pyarrow pyspark bigquery database snowflake clickhouse datafusion postgresql sqlalchemy
Created 2015-04-17
8,288 commits to main branch, last one 10 hours ago
uber
petastorm uber
281
1.8k
apache-2.0
41
Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, a...
sysml parquet pyarrow pyspark pytorch tensorflow deep-learning parquet-files machine-learning
Created 2018-06-15
691 commits to master branch, last one 7 months ago
wheretrue
biobear wheretrue
7
141
mit
3
Work with bioinformatic files using Arrow, Polars, and/or DuckDB
arrow duckdb polars python biology pyarrow rust-bio samtools biopython bioinformatics
Created 2023-04-22
282 commits to main branch, last one 7 days ago
dacort
faker-cli dacort
4
70
mit
3
Command-line interface to quickly generate fake CSV and JSON data
aws csv json parquet pyarrow deltalake faker-provider
Created 2023-05-25
37 commits to main branch, last one 16 days ago
RandomFractals
chicago-crimes RandomFractals
4
37
agpl-3.0
4
Exploring Chicago crimes dataset with Jupyter notebooks, DuckDB, Malloy and new Panel/PyScript data and dashboard tools.
julia crimes duckdb malloy polars chicago parquet pyarrow large-csv malloydata jupyter-notebooks
Created 2022-10-09
243 commits to main branch, last one about a year ago