36 results found Sort:

risingwavelabs
risingwave risingwavelabs
599
7.3k
apache-2.0
80
Streaming database. Unified experience for real-time data ingestion, stream processing, and low-latency serving. Best-in-class performance and cost-efficiency. Supports SQL (Postgres-style) and Python...
etl sql rust flink kafka ksqldb python big-data database postgres analytics real-time postgresql cloud-native spark-streaming data-engineering materialized-view stream-processing real-time-analytics distributed-database
Created 2022-01-28
12,179 commits to main branch, last one 18 hours ago
Angel-ML
angel Angel-ML
1.6k
6.8k
other
447
A Flexible and Powerful Parameter Server for large-scale machine learning
model scala spark online-learning spark-streaming high-dimensional machine-learning parameter-server
Created 2017-04-25
2,963 commits to master branch, last one 2 years ago
lw-lin
CoolplaySpark lw-lin
1.4k
3.5k
unknown
441
酷玩 Spark: Spark 源代码解析、Spark 类库等
spark sparkcore apache-spark spark-streaming structured-streaming
Created 2015-12-03
138 commits to master branch, last one 2 years ago
LuckyZXL2016
Movie_Recommend LuckyZXL2016
1.0k
2.9k
mit
107
基于Spark的电影推荐系统，包含爬虫项目、web网站、后台管理系统以及spark推荐系统
hive mysql nginx scala hadoop scrapy ssm-maven spark-mllib spark-streaming
Created 2018-04-18
27 commits to master branch, last one 5 years ago
dotnet
spark dotnet
319
2.0k
mit
87
.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
emr tpch azure spark tpcds csharp dotnet fsharp bigdata analytics hdinsight microsoft spark-sql streaming databricks dotnet-core apache-spark dotnet-standard spark-streaming machine-learning
Created 2019-04-22
380 commits to main branch, last one 14 days ago
jacksu
utils4s jacksu
434
1.1k
unknown
129
scala、spark使用过程中，各种测试用例以及相关资料整理
akka scala spark breeze json4s scala-demo scala-spark spark-streaming
Created 2015-09-24
226 commits to master branch, last one 5 years ago
edp963
wormhole edp963
480
979
apache-2.0
103
Wormhole is a SPaaS (Stream Processing as a Service) Platform
wormhole spark-streaming stream-processing
Created 2017-09-05
6,922 commits to master branch, last one 4 years ago
microsoft
Mobius microsoft
212
938
mit
143
C# and F# language binding and extensions to Apache Spark
rdd spark csharp fsharp mobius bigdata dataset dstream dataframe eventhubs mapreduce streaming apache-spark near-real-time kafka-streaming spark-streaming
Created 2015-10-27
1,083 commits to master branch, last one 12 months ago
cdapio
cdap cdapio
344
763
other
96
An open source framework for building data analytic applications.
cdap java spark java-8 python dataset unified platform mapreduce middleware integration spark-streaming
Created 2014-08-02
52,928 commits to develop branch, last one 3 days ago
lw-lin
streaming-readings lw-lin
155
731
unknown
50
Streaming System 相关的论文读物
s4 spe flink heron storm drizzle dataflow millwheel streaming apache-spark spark-streaming streaming-engine stream-processing stream-processing-engine
Created 2017-01-24
3 commits to master branch, last one 7 years ago
harbby
sylph harbby
173
402
apache-2.0
46
Stream computing platform for bigdata
sql java flink sylph big-data streamsql spark-streaming
Created 2018-06-19
625 commits to main branch, last one 3 years ago
databrickslabs
dbldatagen databrickslabs
67
384
other
15
Generate relevant synthetic data quickly for your projects. The Databricks Labs synthetic data generator (aka `dbldatagen`) may be used to generate large simulated / synthetic data sets for test, POC...
faker spark python datagen pyspark deltalake databricks datagenerator datageneration synthetic-data data-generation spark-streaming delta-live-tables
Created 2019-07-23
300 commits to master branch, last one about a month ago
microsoft
data-accelerator microsoft
91
300
mit
31
Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsigh...
iot azure kafka react spark docker iothub nodejs big-data cosmosdb eventhub sparksql hdinsight spark-sql streaming apache-spark kafka-streams servicefabric streaming-data spark-streaming
Created 2019-03-14
961 commits to master branch, last one 18 days ago
paypal
gimel paypal
81
244
apache-2.0
36
Big Data Processing Framework - Unified Data API or SQL on Any Storage
jdbc gimel hbase kafka scala spark paypal python pyspark restapi big-data data-api teradata aerospike cassandra elasticsearch streaming-sql spark-streaming
Created 2018-04-04
171 commits to master branch, last one about a month ago
Azure
azure-event-hubs-spark Azure
175
235
apache-2.0
47
Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs
azure kafka scala spark apache stream bigdata connector eventhubs ingestion microsoft real-time streaming continuous databricks event-hubs apache-spark spark-streaming structured-streaming
Created 2015-09-04
377 commits to master branch, last one 2 years ago
databrickslabs
dqx databrickslabs
23
165
other
8
Databricks framework to validate Data Quality of pySpark DataFrames
dlt spark databricks data-quality data-profiling spark-streaming data-quality-checks data-quality-monitoring
Created 2024-04-23
64 commits to main branch, last one 3 days ago
hortonworks
streamline hortonworks
96
164
apache-2.0
685
StreamLine - Streaming Analytics
flink kafka storm real-time streaming kafka-streams spark-streaming
Created 2015-05-14
2,413 commits to master branch, last one 5 years ago
awantik
pyspark-learning awantik
126
157
unknown
18
Updated repository
spark pyspark spark-streaming
Created 2017-03-15
23 commits to master branch, last one 6 years ago
P7h
Spark-MLlib-Twitter-Sentiment-Analysis P7h
65
138
apache-2.0
11
:star2: :sparkles: Analyze and visualize Twitter Sentiment on a world map using Spark MLlib
scala spark spark-mllib visualization spark-streaming machine-learning naive-bayes-classification twitter-sentiment-analysis
Created 2016-08-18
14 commits to master branch, last one 7 years ago
qubole
kinesis-sql qubole
80
136
apache-2.0
13
Kinesis Connector for Structured Streaming
spark kinesis spark-streaming real-time-processing structured-streaming spark-structured-streaming
Created 2018-03-01
90 commits to master branch, last one 2 years ago
LearningJournal
Spark-Streaming-In-Python LearningJournal
155
121
mit
8
Apache Spark 3 - Structured Streaming Course Material
python bigdata pyspark big-data data-lake spark-sql apache-spark spark-streaming
Created 2020-07-21
29 commits to master branch, last one 4 years ago
abulbasar
pyspark-examples abulbasar
88
106
apache-2.0
6
Code examples on Apache Spark using python
spark apache python spark-streaming
Created 2017-08-09
138 commits to master branch, last one 2 years ago
snowch
movie-recommender-demo snowch
56
100
apache-2.0
11
This project walks through how you can create recommendations using Apache Spark machine learning. There are a number of jupyter notebooks that you can run on IBM Data Science Experience, and there a ...
dsx hive bokeh kafka redis spark hadoop bluemix cloudant notebook messagehub biginsights ibm-bluemix ibm-biginsights spark-streaming jupyter-notebook machine-learning collaborative-filtering python-flask-application alternating-least-squares
Created 2017-01-13
245 commits to master branch, last one 2 years ago
martandsingh
ApacheSpark martandsingh
63
95
unknown
12
This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which we need in our real life experience as a data engineer. We wil...
etl sql hive spark hadoop pyspark database datalake deltalake spark-sql databricks timetravel apachespark etl-pipeline data-analysis spark-streaming data-engineering
Created 2022-05-10
45 commits to master branch, last one 2 years ago
chermenin
spark-states chermenin
26
92
apache-2.0
8
Custom state store providers for Apache Spark
spark state apache stateful state-store apache-spark spark-streaming structured-streaming spark-structured-streaming
Created 2018-08-13
43 commits to master branch, last one 2 years ago
jmcmt87
spark_app_twitter jmcmt87
10
78
unknown
3
A data engineering project (Twitter monitor app)
s3 kafka altair pandas python mongodb pyspark spark-streaming
Created 2022-04-14
83 commits to main branch, last one 2 years ago
garystafford
kafka-connect-msk-demo garystafford
29
65
mit
3
For a series of posts on Amazon MSK, Amazon EKS, and Amazon EMR
aws kafka spark kubernetes kafka-connect spark-streaming
Created 2021-08-10
88 commits to main branch, last one 3 years ago
pathwaycom
pathway-benchmarks pathwaycom
6
63
mit
11
Benchmarks for data processing systems: Pathway, Spark, Flink, Kafka Streams
flink latency pathway pagerank streaming wordcount kafka-streams streaming-data spark-streaming benchmark-framework
Created 2023-04-17
102 commits to main branch, last one 7 months ago
dogukannulu
streaming_data_processing dogukannulu
26
59
unknown
2
Create a streaming data, transfer it to Kafka, modify it with PySpark, take it to ElasticSearch and MinIO
hdfs kafka minio spark docker hadoop kibana python airflow pyspark zookeeper elasticsearch spark-streaming
Created 2022-11-12
27 commits to main branch, last one about a year ago
dimajix
spark-training dimajix
66
53
unknown
5
Repository used for Spark Trainings
hive scala spark sqoop hadoop python pyspark spark-ml spark-training hadoop-training spark-streaming
Created 2015-12-28
358 commits to master branch, last one 3 years ago