12 results found Sort:

matanolabs
matano matanolabs
109
1.5k
apache-2.0
21
Open source security data lake for threat hunting, detection & response, and cybersecurity analytics at petabyte scale on AWS
aws dfir rust siem cloud secops alerting big-data security serverless aws-security cloud-native cybersecurity log-analytics apache-iceberg cloud-security log-management security-tools threat-hunting detection-engineering
Created 2022-07-03
576 commits to main branch, last one 2 months ago
apache
incubator-xtable apache
166
1.0k
apache-2.0
27
Apache XTable (incubating) is a cross-table converter for lakehouse table formats that facilitates interoperability across data processing systems and query engines.
delta-lake apache-hudi apache-iceberg
Created 2023-07-21
302 commits to main branch, last one 9 days ago
datazip-inc
olake datazip-inc
54
565
apache-2.0
7
Fastest open-source tool for replicating Databases to Apache Iceberg or Data Lakehouse. ⚡ Efficient, quick and scalable data ingestion for real-time analytics. Supporting Postgres, MongoDB and MySQL
s3 cdc elt parquet database lakehouse replication data-pipeline apache-iceberg change-data-capture
Created 2024-10-15
196 commits to master branch, last one 7 days ago
cuebook
cuelake cuebook
28
285
apache-2.0
11
Use SQL to build ELT pipelines on a data lakehouse.
elt etl sql delta upsert datalake data-lake lakehouse pipelines spark-sql apache-spark data-pipeline data-transfer apache-iceberg data-ingestion data-engineering data-integration zeppelin-notebook incremental-updates
Created 2021-03-11
481 commits to main branch, last one 2 years ago
buster-so
buster buster-so
33
215
other
11
The open-source, AI-native data stack
ai data dbms database analytics lakehouse starrocks warehouse apache-iceberg business-intelligence
Created 2024-09-10
688 commits to main branch, last one a day ago
lhbench
lhbench lhbench
9
72
apache-2.0
2
Lakehouse storage system benchmark
cidr database benchmark lakehouse databricks delta-lake apache-hudi apache-iceberg
Created 2022-12-15
42 commits to main branch, last one 2 years ago
dominikhei
Local-Data-LakeHouse dominikhei
13
63
unknown
4
Sample Data Lakehouse deployed in Docker containers using Apache Iceberg, Minio, Trino and a Hive Metastore. Can be used for local testing.
minio trino data-lake lakehouse apache-iceberg data-lakehouse hive-metastore
Created 2023-04-04
15 commits to main branch, last one about a year ago
dacort
modern-data-lake-storage-layers dacort
29
48
cc0-1.0
2
Jupyter notebooks and AWS CloudFormation template to show how Hudi, Iceberg, and Delta Lake work
aws hudi iceberg amazon-emr delta-lake apache-hudi apache-iceberg
Created 2022-02-02
4 commits to main branch, last one 3 years ago
abeltavares
real-time-data-pipeline abeltavares
3
40
mit
1
📡 Real-time data pipeline with Kafka, Flink, Iceberg, Trino, MinIO, and Superset. Ideal for learning data systems.
s3 aws etl minio trino docker big-data lakehouse open-source apache-flink apache-kafka data-pipeline sql-analytics apache-iceberg real-time-data apache-superset data-engineering data-visualization streaming-analytics
Created 2025-01-12
6 commits to main branch, last one 2 months ago
aws-samples
transactional-datalake-using-apache-iceberg-on-aws-glue aws-samples
2
31
mit-0
3
Stream CDC into an Amazon S3 data lake in Apache Iceberg table format with AWS Glue Streaming and DMS
aws-dms aws-glue aws-athena apache-spark apache-iceberg
Created 2023-01-19
44 commits to main branch, last one about a month ago
bodo-ai
denali bodo-ai
2
26
apache-2.0
3
An open-source, community-driven REST catalog for Apache Iceberg!
go golang catalog iceberg apache-iceberg
Created 2024-06-18
6 commits to main branch, last one 8 months ago
aws-samples
monitoring-apache-iceberg-table-metadata-layer aws-samples
4
25
mit-0
4
Sample code to collect Apache Iceberg metrics for table monitoring
aws sam-cli aws-glue pyiceberg aws-lambda monitoring apache-spark data-quality apache-iceberg aws-cloudwatch
Created 2024-04-17
6 commits to main branch, last one 9 months ago