Search Results - RepositoryStats

2.2k

21.4k

apache-2.0

535

SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC activ...

s3 fuse hdfs posix seaweedfs kubernetes s3-storage cloud-drive hadoop-hdfs replication blob-storage erasure-coding object-storage tiered-file-system distributed-storage distributed-systems distributed-file-system

Created 2014-07-14

10,586 commits to master branch, last one 2 days ago

BigData-Notes heibaiying

4.2k

15.4k

unknown

443

大数据入门指南 :star:

hdfs hive yarn flume hbase kafka scala spark sqoop storm hadoop azkaban bigdata phoenix big-data mapreduce zookeeper

Created 2019-03-10

607 commits to master branch, last one about a year ago

ceph ceph

5.9k

13.4k

other

653

Ceph is a distributed object, block, and file storage platform

Created 2011-09-01

145,728 commits to main branch, last one 13 hours ago

juicefs juicedata

877

9.9k

apache-2.0

112

JuiceFS is a distributed POSIX file system built on top of Redis and S3.

go s3 hdfs posix redis golang bigdata storage filesystem cloud-native object-storage distributed-systems

Created 2021-01-08

3,205 commits to main branch, last one 23 hours ago

God-Of-BigData wangzhiwubigdata

3.1k

9.4k

unknown

332

专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...

hdfs hive flink flume hbase kafka spark hadoop azkaban bigdata zookeeper

Created 2019-02-14

405 commits to master branch, last one about a year ago

smart_open piskvorky

379

3.1k

mit

48

Utils for streaming large files (S3, HDFS, gzip, bz2...)

s3 bz2 boto file hdfs python webhdfs streaming gzip-stream hacktoberfest streaming-data

Created 2015-01-02

1,078 commits to develop branch, last one 23 days ago

TileDB TileDB-Inc

179

1.8k

mit

71

The Universal Storage Engine

s3 hdfs arrays tiledb dataframes dense-data s3-storage sparse-data data-science data-analysis sparse-arrays storage-engine scientific-computing

Created 2017-03-31

4,784 commits to dev branch, last one 22 hours ago

BigData-Interview water8394

442

1.6k

unknown

51

:dart: :star2:[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop/Hive/Spark/Flink/Hbase/Kafka/Zookeeper框架的面试题知识总结

hdfs yarn flink hbase kafka spark hadoop bigdata interview mapreduce interview-questions

Created 2019-08-12

80 commits to master branch, last one 3 years ago

hdfs colinmarc

339

1.4k

mit

38

A native go client for HDFS

go hdfs commandline

Created 2014-10-08

465 commits to master branch, last one 10 months ago

bigdata-growth collabH

331

1.3k

mit

30

大数据知识仓库涉及到数据仓库建模、实时计算、大数据、数据中台、系统设计、Java、算法等。

hdfs hive hudi kudu olap flink hbase kafka spark hadoop bigdata debezium mapreduce bigdatalearning

Created 2020-06-10

1,054 commits to master branch, last one 3 days ago

Addax wgzhao

288

1.1k

apache-2.0

32

Addax is a versatile open-source ETL tool that can seamlessly transfer data between various RDBMS and NoSQL databases, making it an ideal solution for data migration.

etl hdfs hive kudu datax excel mysql trino hadoop impala oracle database influxdb sqlserver clickhouse postgresql data-integrity

Created 2019-07-17

1,407 commits to master branch, last one a day ago

snakebite spotify

216

858

apache-2.0

129

A pure python HDFS client

hdfs python python-hdfs-client

This repository has been archived (exclude archived)

Created 2013-05-07

348 commits to master branch, last one 2 years ago

DevOps-Python-tools HariSekhon

335

736

mit

42

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Functions, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML...

Created 2015-10-27

5,331 commits to master branch, last one 4 days ago

kafka-connect-ui lensesio

131

496

other

25

Web tool for Kafka Connect |

s3 jms hdfs mqtt kafka redis twitter influxdb cassandra documentdb elasticsearch kafka-connect

Created 2016-12-02

197 commits to master branch, last one 2 years ago

kafka-connect-hdfs confluentinc

396

476

other

315

Kafka Connect HDFS connector

hdfs kafka hadoop big-data confluent streaming apache-kafka kafka-connector kafka-connect-hdfs

Created 2015-05-20

2,517 commits to master branch, last one 7 days ago

CloudEon dromara

93

396

apache-2.0

13

CloudEon uses Kubernetes to install and deploy open-source big data components, enabling the containerized operation of an open-source big data platform. This allows you to reduce your focus on underl...

hdfs yarn doris hadoop bigdata kubernetes cloudnative

Created 2023-01-29

883 commits to dev branch, last one about a month ago

bigdata_docker fabiogjardim

301

373

unknown

25

Big Data Ecosystem Docker

hue hdfs hive nifi hbase mongo mysql spark hadoop presto metabase zookeeper jupyter-notebook

Created 2020-03-20

82 commits to master branch, last one 2 years ago

storagetapper uber

67

335

mit

24

StorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service

s3 cdc etl avro hdfs json kafka mysql msgpack clickhouse postgresql

Created 2017-05-07

250 commits to master branch, last one 2 years ago

Spark-with-Python tirthajyoti

259

324

mit

10

Fundamentals of Spark with Python (using PySpark), code examples

sql hdfs mlib spark apache hadoop python pyspark big-data database analytics dataframe map-reduce apache-spark machine-learning parallel-computing distributed-computing

Created 2018-08-20

73 commits to master branch, last one 3 years ago

bigdata-file-viewer Eugene-Mark

54

282

gpl-2.0

3

A cross-platform (Windows, MAC, Linux) desktop application to view common bigdata binary format like Parquet, ORC, AVRO, etc. Support local file system, HDFS, AWS S3, Azure Blob Storage ,etc.

orc avro hdfs bigdata parquet

Created 2020-02-05

119 commits to master branch, last one 2 months ago

hdfs mtth

99

267

mit

15

API and command line interface for HDFS

cli hdfs python

Created 2014-03-10

367 commits to master branch, last one 7 months ago

wradlib wradlib

77

252

mit

25

weather radar data processing - python package

hdfs radar python sigmet xarray netcdf4 rainbow weather wradlib hacktoberfest

Created 2016-02-19

1,615 commits to main branch, last one 3 months ago

juicy-bigdata datawhalechina

36

242

unknown

5

🎉🎉🐳 Datawhale大数据处理导论教程 | 大数据技术方向的开篇课程🎉🎉

hdfs hive hbase spark hadoop bigdata mapreduce

Created 2021-11-18

322 commits to master branch, last one about a year ago

rumble RumbleDB

80

208

other

27

⛈️ RumbleDB 1.21.0 "Hawthorn blossom" 🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to d...

Created 2017-09-04

6,426 commits to master branch, last one 3 months ago

File-System-Paper hegongshan

19

193

unknown

3

Must-read Papers for File System (FS)

ceph hdfs file-system fragmentation fault-tolerance crash-consistency data-distribution local-file-system kernel-file-system metadata-management many-core-scalability user-space-file-system distributed-file-system

Created 2021-12-01

100 commits to main branch, last one 2 days ago

TileDB-Py TileDB-Inc

31

178

mit

32

Python interface to the TileDB storage engine

s3 hdfs array numpy python tiledb storage-manager

Created 2017-05-19

1,932 commits to dev branch, last one 2 days ago

dcos-commons mesosphere

175

157

apache-2.0

93

DC/OS SDK is a collection of tools, libraries, and documentation for easy integration of technologies such as Kafka, Cassandra, HDFS, Spark, and TensorFlow with DC/OS.