Search Results - RepositoryStats

664

1.1k

other

129

MapReduce, Spark, Java, and Scala for Data Algorithms Book

java scala python mappers pyspark reducers mapreduce apache-spark partitioning apache-hadoop data-algorithms design-patterns hadoop-mapreduce machine-learning distributed-computing distributed-algorithms

Created 2014-08-06

787 commits to master branch, last one 5 months ago

CloudShuffleService bytedance

58

255

apache-2.0

9

Cloud Shuffle Service(CSS) is a general purpose remote shuffle solution for compute engines, including Spark/Flink/MapReduce.

flink spark hadoop-mapreduce

Created 2022-08-17

1 commits to main branch, last one 2 years ago

ctenopharyngodon-idella touero

0

140

apache-2.0

1

Use the MapReduce's Java interface to distributed crawle the data of Chinese universities and learn basic knowledge of hdfs.

java maven hadoop fastapi scraping mapreduce hadoop-mapreduce

Created 2023-04-10

77 commits to master branch, last one 5 months ago

big_data groda

26

72

mit

2

Tutorials on Big Data essentials: Hadoop, MapReduce, Spark. Explore a variety of tutorials and demonstrations on Big Data technologies, primarily in the form of Jupyter notebooks. Most notebooks are s...

mrjob spark bigtop docker hadoop bigdata pyspark big-data mapreduce spark-sql testdfsio hadoop-hdfs apache-spark apache-sedona hadoop-cluster mapreduce-bash gutenberg-ebooks hadoop-mapreduce jupyter-notebook

Created 2019-08-27

370 commits to master branch, last one 2 months ago

datapipelines-essentials-python vim89

38

53

apache-2.0

4

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformati...

etl xml spark hadoop python pyspark python3 big-data datalake spark-sql hadoop-hdfs xml-parsing apache-spark etl-pipeline data-pipeline etl-framework etl-components hadoop-mapreduce

Created 2019-11-16

15 commits to master branch, last one about a year ago