37 results found Sort:

Angel-ML
angel Angel-ML
1.6k
6.7k
other
443
A Flexible and Powerful Parameter Server for large-scale machine learning
model scala spark online-learning spark-streaming high-dimensional machine-learning parameter-server
Created 2017-04-25
2,963 commits to master branch, last one 2 years ago
lw-lin
CoolplaySpark lw-lin
1.4k
3.5k
unknown
440
酷玩 Spark: Spark 源代码解析、Spark 类库等
spark sparkcore apache-spark spark-streaming structured-streaming
Created 2015-12-03
138 commits to master branch, last one 2 years ago
LuckyZXL2016
Movie_Recommend LuckyZXL2016
1.1k
2.9k
mit
106
基于Spark的电影推荐系统，包含爬虫项目、web网站、后台管理系统以及spark推荐系统
hive mysql nginx scala hadoop scrapy ssm-maven spark-mllib spark-streaming
Created 2018-04-18
27 commits to master branch, last one 6 years ago
dotnet
spark dotnet
325
2.1k
mit
84
.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
emr tpch azure spark tpcds csharp dotnet fsharp bigdata analytics hdinsight microsoft spark-sql streaming databricks dotnet-core apache-spark dotnet-standard spark-streaming machine-learning
Created 2019-04-22
384 commits to main branch, last one 19 days ago
jacksu
utils4s jacksu
433
1.1k
unknown
128
scala、spark使用过程中，各种测试用例以及相关资料整理
akka scala spark breeze json4s scala-demo scala-spark spark-streaming
Created 2015-09-24
226 commits to master branch, last one 6 years ago
edp963
wormhole edp963
478
978
apache-2.0
102
Wormhole is a SPaaS (Stream Processing as a Service) Platform
wormhole spark-streaming stream-processing
Created 2017-09-05
6,922 commits to master branch, last one 4 years ago
microsoft
Mobius microsoft
211
940
mit
141
C# and F# language binding and extensions to Apache Spark
rdd spark csharp fsharp mobius bigdata dataset dstream dataframe eventhubs mapreduce streaming apache-spark near-real-time kafka-streaming spark-streaming
Created 2015-10-27
1,083 commits to master branch, last one about a year ago
cdapio
cdap cdapio
345
771
other
95
An open source framework for building data analytic applications.
cdap java spark java-8 python dataset unified platform mapreduce middleware integration spark-streaming
Created 2014-08-02
53,044 commits to develop branch, last one 9 days ago
lw-lin
streaming-readings lw-lin
155
732
unknown
49
Streaming System 相关的论文读物
s4 spe flink heron storm drizzle dataflow millwheel streaming apache-spark spark-streaming streaming-engine stream-processing stream-processing-engine
Created 2017-01-24
3 commits to master branch, last one 8 years ago
harbby
sylph harbby
174
402
apache-2.0
45
Stream computing platform for bigdata
sql java flink sylph big-data streamsql spark-streaming
Created 2018-06-19
625 commits to main branch, last one 3 years ago
databrickslabs
dbldatagen databrickslabs
72
401
other
16
Generate relevant synthetic data quickly for your projects. The Databricks Labs synthetic data generator (aka `dbldatagen`) may be used to generate large simulated / synthetic data sets for test, POC...
faker spark python datagen pyspark deltalake databricks datagenerator datageneration synthetic-data data-generation spark-streaming delta-live-tables
Created 2019-07-23
303 commits to master branch, last one about a month ago
microsoft
data-accelerator microsoft
91
302
mit
29
Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsigh...
iot azure kafka react spark docker iothub nodejs big-data cosmosdb eventhub sparksql hdinsight spark-sql streaming apache-spark kafka-streams servicefabric streaming-data spark-streaming
Created 2019-03-14
961 commits to master branch, last one 3 months ago
databrickslabs
dqx databrickslabs
35
252
other
7
Databricks framework to validate Data Quality of pySpark DataFrames
dlt spark databricks data-quality data-profiling spark-streaming data-quality-checks data-quality-monitoring
Created 2024-04-23
102 commits to main branch, last one 17 days ago
paypal
gimel paypal
81
245
apache-2.0
35
Big Data Processing Framework - Unified Data API or SQL on Any Storage
jdbc gimel hbase kafka scala spark paypal python pyspark restapi big-data data-api teradata aerospike cassandra elasticsearch streaming-sql spark-streaming
Created 2018-04-04
171 commits to master branch, last one 4 months ago
Azure
azure-event-hubs-spark Azure
177
235
apache-2.0
45
Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs
azure kafka scala spark apache stream bigdata connector eventhubs ingestion microsoft real-time streaming continuous databricks event-hubs apache-spark spark-streaming structured-streaming
Created 2015-09-04
377 commits to master branch, last one 2 years ago
hortonworks
streamline hortonworks
96
164
apache-2.0
682
StreamLine - Streaming Analytics
flink kafka storm real-time streaming kafka-streams spark-streaming
Created 2015-05-14
2,413 commits to master branch, last one 5 years ago
awantik
pyspark-learning awantik
126
157
unknown
18
Updated repository
spark pyspark spark-streaming
Created 2017-03-15
23 commits to master branch, last one 6 years ago
P7h
Spark-MLlib-Twitter-Sentiment-Analysis P7h
65
139
apache-2.0
10
:star2: :sparkles: Analyze and visualize Twitter Sentiment on a world map using Spark MLlib
scala spark spark-mllib visualization spark-streaming machine-learning naive-bayes-classification twitter-sentiment-analysis
Created 2016-08-18
14 commits to master branch, last one 7 years ago
qubole
kinesis-sql qubole
80
136
apache-2.0
12
Kinesis Connector for Structured Streaming
spark kinesis spark-streaming real-time-processing structured-streaming spark-structured-streaming
Created 2018-03-01
90 commits to master branch, last one 2 years ago
LearningJournal
Spark-Streaming-In-Python LearningJournal
160
122
mit
7
Apache Spark 3 - Structured Streaming Course Material
python bigdata pyspark big-data data-lake spark-sql apache-spark spark-streaming
Created 2020-07-21
29 commits to master branch, last one 4 years ago
abulbasar
pyspark-examples abulbasar
91
107
apache-2.0
5
Code examples on Apache Spark using python
spark apache python spark-streaming
Created 2017-08-09
138 commits to master branch, last one 2 years ago
snowch
movie-recommender-demo snowch
56
99
apache-2.0
9
This project walks through how you can create recommendations using Apache Spark machine learning. There are a number of jupyter notebooks that you can run on IBM Data Science Experience, and there a ...
dsx hive bokeh kafka redis spark hadoop bluemix cloudant notebook messagehub biginsights ibm-bluemix ibm-biginsights spark-streaming jupyter-notebook machine-learning collaborative-filtering python-flask-application alternating-least-squares
Created 2017-01-13
245 commits to master branch, last one 2 years ago
martandsingh
ApacheSpark martandsingh
64
98
unknown
11
This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which we need in our real life experience as a data engineer. We wil...
etl sql hive spark hadoop pyspark database datalake deltalake spark-sql databricks timetravel apachespark etl-pipeline data-analysis spark-streaming data-engineering
Created 2022-05-10
45 commits to master branch, last one 2 years ago
chermenin
spark-states chermenin
26
92
apache-2.0
7
Custom state store providers for Apache Spark
spark state apache stateful state-store apache-spark spark-streaming structured-streaming spark-structured-streaming
Created 2018-08-13
45 commits to master branch, last one 2 months ago
jmcmt87
spark_app_twitter jmcmt87
10
82
unknown
3
A data engineering project (Twitter monitor app)
s3 kafka altair pandas python mongodb pyspark spark-streaming
Created 2022-04-14
83 commits to main branch, last one 2 years ago
pathwaycom
pathway-benchmarks pathwaycom
4
68
mit
11
Benchmarks for data processing systems: Pathway, Spark, Flink, Kafka Streams
flink latency pathway pagerank streaming wordcount kafka-streams streaming-data spark-streaming benchmark-framework
Created 2023-04-17
104 commits to main branch, last one about a month ago
garystafford
kafka-connect-msk-demo garystafford
29
66
mit
2
For a series of posts on Amazon MSK, Amazon EKS, and Amazon EMR
aws kafka spark kubernetes kafka-connect spark-streaming
Created 2021-08-10
88 commits to main branch, last one 3 years ago
dogukannulu
streaming_data_processing dogukannulu
25
60
unknown
1
Create a streaming data, transfer it to Kafka, modify it with PySpark, take it to ElasticSearch and MinIO
hdfs kafka minio spark docker hadoop kibana python airflow pyspark zookeeper elasticsearch spark-streaming
Created 2022-11-12
27 commits to main branch, last one about a year ago
zhonghuasheng
Projects zhonghuasheng
6
54
unknown
4
秒杀，音乐商店项目实战，Redis源码，推荐系统
jdbc mysql redis jquery springcloud spark-streaming jsp-servlet-mysql springcloudalibaba
Created 2019-05-22
583 commits to master branch, last one 2 years ago
dimajix
spark-training dimajix
66
53
unknown
4
Repository used for Spark Trainings
hive scala spark sqoop hadoop python pyspark spark-ml spark-training hadoop-training spark-streaming
Created 2015-12-28
358 commits to master branch, last one 3 years ago