7 results found Sort:

seaweedfs
seaweedfs seaweedfs
2.3k
23.0k
apache-2.0
535
SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC activ...
s3 fuse hdfs posix seaweedfs kubernetes s3-storage cloud-drive hadoop-hdfs replication blob-storage erasure-coding object-storage tiered-file-system distributed-storage distributed-systems distributed-file-system
Created 2014-07-14
11,149 commits to master branch, last one a day ago
OBenner
data-engineering-interview-questions OBenner
412
1.2k
unknown
21
More than 2000+ Data engineer interview questions.
aws sql avro hive nifi azure flink flume hbase kafka spark hadoop impala airflow cassandra interview hadoop-hdfs data-structures data-engineering interview-questions
Created 2021-08-08
18 commits to master branch, last one 3 months ago
linkedin
dynamometer linkedin
34
131
bsd-2-clause
18
A tool for scale and performance testing of HDFS with a specific focus on the NameNode.
hdfs scale hadoop testing hdfs-dfs scale-up hadoop-hdfs testing-tools hadoop-framework performance-test hadoop-filesystem performance-metrics performance-testing performance-analysis
Created 2017-11-06
60 commits to master branch, last one 5 years ago
IBM
sparksql-for-hbase IBM
27
69
apache-2.0
31
Learn how to use Spark SQL and HSpark connector package to create / query data tables that reside in HBase region servers
sql hbase nosql spark ibmcode hadoop-hdfs apache-spark
Created 2017-08-31
107 commits to master branch, last one 5 years ago
groda
big_data groda
25
68
mit
4
Tutorials on Big Data essentials: Hadoop, MapReduce, Spark.
mrjob spark bigtop docker hadoop bigdata pyspark big-data mapreduce spark-sql testdfsio hadoop-hdfs apache-spark apache-sedona hadoop-cluster mapreduce-bash gutenberg-ebooks hadoop-mapreduce jupyter-notebook
Created 2019-08-27
351 commits to master branch, last one 15 days ago
vim89
datapipelines-essentials-python vim89
37
53
apache-2.0
6
Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformati...
etl xml spark hadoop python pyspark python3 big-data datalake spark-sql hadoop-hdfs xml-parsing apache-spark etl-pipeline data-pipeline etl-framework etl-components hadoop-mapreduce
Created 2019-11-16
15 commits to master branch, last one about a year ago
AhmetFurkanDEMIR
Data-Engineering-Project-with-HDFS-and-Kafka AhmetFurkanDEMIR
7
32
mit
2
Data Engineering Project with Hadoop HDFS and Kafka
data hdfs kafka docker hadoop python kafkaui pipline hdfs-dfs kafka-ui hadoop-hdfs hdfs-client data-engineer docker-compose kafka-consumer kafka-producer data-engineering hadoop-filesystem python-hdfs-client data-engineering-pipeline
Created 2023-11-04
3 commits to main branch, last one about a year ago