Distributed SQL transaction & query engine for data sharding, scaling, encryption, and more - on any database.

Apache Amoro (incubating) is a Lakehouse management system built on open data lake formats.

Sort files and pack them into partitions

KMeans Clustering using Spark on Uber's ride share data - Case Study (Big Data Analytics @uber)

TDengine is an open source, high-performance, cloud native time-series database optimized for Internet of Things (IoT), Connected Cars, Industrial IoT and DevOps.

𝗗𝗮𝘁𝗮, 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 & 𝗔𝗜. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. https://databend.com

A Cloud Native Batch System (Project under CNCF)

A general purpose Distributed Systems Framework

High Performance HTTP Sidecar Load Balancer

A robust web archive analytics toolkit

Upserts, Deletes And Incremental Processing on Big Data.

KDP(Kubernetes Data Platform) delivers a modern, hybrid and cloud-native data platform based on Kubernetes.

Apache Avro is a data serialization system.

Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.

A curated list of awesome big data frameworks, resources and other awesomeness. With repository stars⭐ and forks🍴

你想拥有‘上帝之眼’吗？你渴望力量吗？你希望一切信息尽在掌控吗？Hydra九头龙，保姆级为您打属于自己的造跨平台TB-PB级别个人数仓、搜索引擎（私人数据中心）。Hydra-面向云计算、多任务调度、MapReduce、通信、服务化、抽象化分布式操作系统——以实现小型爬虫搜索引擎为例。

JuiceFS is a distributed POSIX file system built on top of Redis and S3.

DailyNewsDriftCanada is a tool designed to analyze the sentiment of news headlines from various Canadian media outlets over time.

Code Library for My Blog

Possibly the fastest DataFrame-agnostic quality check library in town.