TopGit

Best Data open-source projects

Open-source data tools on GitHub — databases, ETL pipelines, analytics engines and dataframe libraries, ranked by stars and reviewed by TopGit.

Top Data repositories

M
macrozheng/mall

mall is a Java e-commerce reference project from developer macrozheng, pairing a customer storefront with a separate admin backend on one Spring Boot and MyBatis codebase. The README describes a modular layout, mall-admin, mall-portal, mall-search, mall-security, mall-mbg, and mall-common, plus a supporting stack of Elasticsearch, RabbitMQ, Redis, and MongoDB, deployed through Docker. A sister project, mall-swarm, offers a Spring Cloud Alibaba microservices version of the same system.

21884.5k29.8k
Java
# docker# elasticsearch# elk
29.8kRead review
D
doocs/advanced-java

Doocs Advanced Java (doocs/advanced-java) is a mostly Chinese-language collection of long-form Q&A documents on Java backend interview topics: message queues, Elasticsearch, Redis, database sharding, Dubbo, Zookeeper, Hystrix, and Spring Cloud microservices. Most of the writing traces back to one instructor, credited in the README as 中华石杉, organized by the Doocs community into linked markdown files. It is hosted on GitHub and is licensed CC-BY-SA-4.0.

51079.0k19.2k
Java
# advanced-java# distributed-search-engine# distributed-systems
19.2kRead review
E
elastic/elasticsearch

Elasticsearch is an open-source distributed search and analytics engine maintained at elastic/elasticsearch. Written in Java, it's described as the foundation of Elastic's Stack platform, combining full-text search, vector search, and a scalable data store in one system. The repository's GitHub metadata doesn't state a single clear license here, and the README excerpt doesn't spell one out either.

5.9k77.8k26.1k
Java
# elasticsearch# java# search-engine
26.1kRead review
G
grafana/grafana

Grafana is grafana/grafana on GitHub, an open-source project listed by GitHub as primarily TypeScript. It's licensed AGPL-3.0-only, with the repository noting Apache-2.0 exceptions documented separately in LICENSING.md. The project centers on turning metrics, logs, and traces from many backends into shared dashboards, alerts, and ad-hoc exploration views rather than storing that data itself.

3.1k3.4k76.1k14.5k
TypeScript
# alerting# analytics# business-intelligence
14.5kRead review
R
redis/redis

Redis is an in-memory data structure server at redis/redis, used as a cache, a broker for messaging, and an engine for querying documents and vectors, sitting in front of a primary database. Beyond simple key-value storage it natively supports hashes, lists, sets, sorted sets, JSON, streams, and vector sets, and the README documents Redis Open Source (renamed from Redis Community Edition at v8.0), plus the separately offered Redis Software and Redis Cloud.

9392.9k75.9k24.8k
C
# cache# caching# database
24.8kRead review
A
apache/superset

Apache Superset is a self-hosted, open source platform for exploring data and building dashboards, pairing a no-code chart builder with a full SQL Editor and a semantic layer for reusable metrics.

1.7k60274.2k18.1k
Python
# analytics# apache# apache-superset
18.1kRead review
A
Asabeneh/30-Days-Of-Python

30 Days Of Python is a free GitHub curriculum by Asabeneh Yetayeh that breaks Python into 30 daily topics, from syntax basics through web scraping, MongoDB, and building an API. The README pitches it at beginners and working professionals, admits the timeline realistically runs past 100 days, and points to a Telegram group and companion YouTube channel for support.

8618970.3k13.0k
Python
# 30-days-of-python# data# data-science
13.0kRead review
S
scikit-learn/scikit-learn

scikit-learn is an open-source Python module for machine learning at scikit-learn/scikit-learn, layered on top of SciPy and covering classification, regression, clustering, and dimensionality reduction. The project traces back to a 2007 Google Summer of Code effort by David Cournapeau, was known early on as scikits.learn, and is now kept up by a volunteer community with backing from several organizations, distributed under the BSD-3-Clause license.

3.5k2.1k66.9k27.3k
Python
# data-analysis# data-science# machine-learning
27.3kRead review
P
pandas-dev/pandas

pandas provides a labeled DataFrame for working with tabular data — the labeled DataFrame gives you alignment, indexing, and aggregation that raw NumPy arrays simply can't match without significant boilerplate. The API surface is enormous, covering everything from CSV reading to time-series resampling, and while the learning curve is real, the payoff is a coherent mental model for most data-wrangling tasks. It started at AQR Capital Management in 2008 and has been under active development since, with a mature ecosystem, extensive documentation on PyData.org, and community support through Stack Overflow, mailing lists, and Slack. It's not the fastest option for pure numerical compute (Polars and PyArrow handle large datasets with less overhead), but for exploratory analysis and prototyping in Python, the ecosystem depth and API consistency make pandas the practical default.

4.3k2.9k49.5k20.2k
Python
# alignment# data-analysis# data-science
20.2kRead review
P
prisma/prisma

Prisma ORM is a type-safe database toolkit for Node.js and TypeScript that generates a query builder from a schema you define. You describe your models in a `.prisma` schema, run `prisma generate`, and get a client where every query is typed against your tables — including the exact shape of partial selects.

4002.5k47.5k2.5k
TypeScript
# cockroachdb# database# javascript
2.5kRead review
A
abhigyanpatwari/GitNexus

A client-side knowledge-graph engine for codebases: a local CLI with MCP tools for AI agents, or a no-install Web UI at gitnexus.vercel.app, with a `gitnexus serve` bridge mode connecting the two.

17531145.5k5.0k
TypeScript
5.0kRead review
A
academic/awesome-datascience

This is a traditional awesome-list that curates hundreds of Data Science learning resources — from Python/R fundamentals through to deep learning, agents, and MLOps — organized into sections like tutorials, MOOCs, free courses, and a beginner roadmap. It covers the major bases (Pandas, NumPy, Scikit-Learn, data visualization) and goes broader into research tools, LLM/agent frameworks, and interview prep. The tradeoff: it's a link aggregator, not a course — there's no structure to follow beyond what you piece together yourself, and quality of linked resources varies. For someone who already knows what they're looking for, it's a decent bookmark collection; for a complete beginner following a curated path, the sheer volume can be overwhelming without a clear sequence.

333429.8k6.6k
# analytics# awesome-list# data-mining
6.6kRead review
T
typesense/typesense

Typesense is a fast, typo-tolerant in-memory search engine written in C++ that positions itself as a more developer-friendly alternative to Elasticsearch and a self-hostable replacement for Algolia. The core appeal is that it ships as a single binary with no runtime dependencies, making it straightforward to deploy locally or in production with a single command, and it handles fuzzy matching out-of-the-box without configuration. Beyond standard full-text search, it includes vector and hybrid search capabilities, geo search, faceting, filtering, synonyms, and a built-in RAG feature for conversational search over your own data. The API is clean and well-documented, with official clients for JavaScript, PHP, Python, and Ruby, plus a large ecosystem of community clients and framework integrations covering Laravel, Django, Ruby on Rails, Firebase, WordPress, and more. Benchmarks against real-world datasets show solid throughput (up to 250 concurrent queries per second on a 3-node cluster with 3 million products) and low latency (under 30ms average at scale), though the memory footprint scales with your data — plan for roughly 165MB per million records depending on field types and schema. The trade-off is that it lacks some enterprise-grade analytics features present in Algolia, and the GPL-3.0 license means server software usage is unrestricted, but modifications distributed to others must be open source. If you need search that just works without becoming a search infrastructure expert, Typesense is worth the install.

5886526.4k962
C++
# algolia# datastore# elasticsearch
962Read review
P
plotly/dash

Dash is Plotly's open-source Python framework for turning data-science code into web apps, pairing Python callbacks with Plotly.js charts and a React frontend on top of a Flask server. It's aimed at people who already work in pandas and Plotly and want a real, interactive UI around that work without adding a JavaScript build step to their workflow.

21754024.4k2.3k
Python
# ai# bioinformatics# charting
2.3kRead review
M
markusschanta/awesome-jupyter

Awesome Jupyter is a hand-picked list on GitHub, maintained by markusschanta, that links out to Jupyter-related runtimes, extensions, visualization libraries, and publishing tools rather than hosting any code. Entries are grouped into categories such as Runtimes/Frontends, Visualization, Testing, and JupyterLab Extensions, each with a short one-line description.

88164.7k463
# awesome# awesome-list# data-science
463Read review
M
malisper/pgrust

pgrust is a Postgres rewrite in Rust that aims to behave exactly like Postgres 18.3 rather than invent a new database. It uses the real Postgres regression suite as its correctness oracle, is disk compatible with Postgres, and can boot straight from an existing Postgres 18.3 data directory. The maintainer built it with Rust plus AI-assisted programming, and it's an experiment, not a production database.

3134.5k168
Rust
# ai-assisted-development# database# postgres
168Read review
C
CrunchyData/postgres-operator

PGO, short for the Postgres Operator, is Crunchy Data's open-source Kubernetes operator (Apache-2.0, written in Go, 4,436 GitHub stars) for running PostgreSQL clusters declaratively. It reconciles PostgresCluster custom resources into running clusters with Patroni-backed failover, pgBackRest disaster recovery, pgBouncer pooling, and pgMonitor observability wired in, and the standard install path layers on Crunchy Data's own Crunchy Postgres for Kubernetes distribution.

1111694.4k679
Go
# data-infrastructure# database# database-as-a-service
679Read review
C
crate/crate

CrateDB is an open-source, distributed SQL database maintained primarily by Crate.io at crate/crate, built on Lucene and licensed under Apache-2.0. The README describes it as PostgreSQL-compatible, designed to ingest and query large volumes of data — including time-series, full-text, and geospatial data — in near real-time across a horizontally scalable cluster.

1433324.4k606
Java
# analytics# big-data# cratedb
606Read review
C
ClickHouse/clickhouse-go

The ClickHouse Go Driver (ClickHouse/clickhouse-go) is the official Go client for ClickHouse, installed as github.com/ClickHouse/clickhouse-go/v2. It ships a native protocol interface for direct column encoding and a database/sql-compatible interface for existing Go tooling, both usable over TCP or HTTP transport, with connection pooling, failover, and compression handled by the library itself.

1683.3k670
Go
# analytics-database# clickhouse# database
670Read review
S
Snailclimb/interview-guide

InterviewGuide is an AI interview preparation platform combining resume analysis, mock interviews (text and voice), interview scheduling, and a RAG knowledge base. Built with Spring Boot 4.1, Java 25 virtual threads, Spring AI 2.0, PostgreSQL/pgvector, and Redis Stream for async processing. The frontend uses React 18 + TypeScript + Tailwind CSS. The project describes itself as very suitable as a learning and resume project. It is fully open-source under AGPL-3.0, with optional paid tutorials available for learners who want deeper architectural guidance.

12103.0k683
Java
# gradle# itext7# mapstruct
683Read review

Recently reviewed

M
markusschanta/awesome-jupyter

Awesome Jupyter is a hand-picked list on GitHub, maintained by markusschanta, that links out to Jupyter-related runtimes, extensions, visualization libraries, and publishing tools rather than hosting any code. Entries are grouped into categories such as Runtimes/Frontends, Visualization, Testing, and JupyterLab Extensions, each with a short one-line description.

88164.7k463
# awesome# awesome-list# data-science
463Read review
A
abhigyanpatwari/GitNexus

A client-side knowledge-graph engine for codebases: a local CLI with MCP tools for AI agents, or a no-install Web UI at gitnexus.vercel.app, with a `gitnexus serve` bridge mode connecting the two.

17531145.5k5.0k
TypeScript
5.0kRead review
M
malisper/pgrust

pgrust is a Postgres rewrite in Rust that aims to behave exactly like Postgres 18.3 rather than invent a new database. It uses the real Postgres regression suite as its correctness oracle, is disk compatible with Postgres, and can boot straight from an existing Postgres 18.3 data directory. The maintainer built it with Rust plus AI-assisted programming, and it's an experiment, not a production database.

3134.5k168
Rust
# ai-assisted-development# database# postgres
168Read review
S
Snailclimb/interview-guide

InterviewGuide is an AI interview preparation platform combining resume analysis, mock interviews (text and voice), interview scheduling, and a RAG knowledge base. Built with Spring Boot 4.1, Java 25 virtual threads, Spring AI 2.0, PostgreSQL/pgvector, and Redis Stream for async processing. The frontend uses React 18 + TypeScript + Tailwind CSS. The project describes itself as very suitable as a learning and resume project. It is fully open-source under AGPL-3.0, with optional paid tutorials available for learners who want deeper architectural guidance.

12103.0k683
Java
# gradle# itext7# mapstruct
683Read review
C
confluentinc/schema-registry

Confluent Schema Registry is a Java service that gives Kafka producers and consumers a shared, versioned source of truth for message schemas. It exposes a RESTful API for registering, fetching, and checking compatibility of Avro, JSON Schema, and Protobuf schemas, plus serializers that plug directly into Kafka clients. The core server is licensed under the Confluent Community License, a source-available license, while several client and serializer modules are split out under Apache 2.0.

2813912.5k1.2k
Java
# avro# avro-schema# confluent
1.2kRead review
C
ClickHouse/clickhouse-go

The ClickHouse Go Driver (ClickHouse/clickhouse-go) is the official Go client for ClickHouse, installed as github.com/ClickHouse/clickhouse-go/v2. It ships a native protocol interface for direct column encoding and a database/sql-compatible interface for existing Go tooling, both usable over TCP or HTTP transport, with connection pooling, failover, and compression handled by the library itself.

1683.3k670
Go
# analytics-database# clickhouse# database
670Read review
C
crate/crate

CrateDB is an open-source, distributed SQL database maintained primarily by Crate.io at crate/crate, built on Lucene and licensed under Apache-2.0. The README describes it as PostgreSQL-compatible, designed to ingest and query large volumes of data — including time-series, full-text, and geospatial data — in near real-time across a horizontally scalable cluster.

1433324.4k606
Java
# analytics# big-data# cratedb
606Read review
C
CrunchyData/postgres-operator

PGO, short for the Postgres Operator, is Crunchy Data's open-source Kubernetes operator (Apache-2.0, written in Go, 4,436 GitHub stars) for running PostgreSQL clusters declaratively. It reconciles PostgresCluster custom resources into running clusters with Patroni-backed failover, pgBackRest disaster recovery, pgBouncer pooling, and pgMonitor observability wired in, and the standard install path layers on Crunchy Data's own Crunchy Postgres for Kubernetes distribution.

1111694.4k679
Go
# data-infrastructure# database# database-as-a-service
679Read review

More Data repositories

A
apache/spark

Apache Spark - A unified analytics engine for large-scale data processing

3.5k46243.8k29.3k
Scala
# big-data# java# jdbc
29.3kRead review
D
duckdb/duckdb

DuckDB is an analytical in-process SQL database management system

80475740.1k3.5k
C++
# analytics# database# embedded-database
3.5kRead review
Y
YunaiV/ruoyi-vue-pro

🔥 官方推荐 🔥 RuoYi-Vue 全新 Pro 版本,优化重构所有功能。基于 Spring Boot + MyBatis Plus + Vue & Element 实现的后台管理系统 + 微信小程序,支持 RBAC 动态权限、数据权限、SaaS 多租户、Flowable 工作流、三方登录、支付、短信、商城、CRM、ERP、MES、IM、AI 大模型、IoT 物联网等功能。你的 ⭐️ Star ⭐️,是作者生发的动力!

405838.6k8.3k
Java
# flowable# mybatis-plus# mysql
8.3kRead review
T
typeorm/typeorm

TypeScript & JavaScript ORM for Node.js — supports PostgreSQL, MySQL, MariaDB, SQLite, SQL Server, Oracle, and more.

1.3k59736.6k6.7k
TypeScript
# active-record# cockroachdb# data-mapper
6.7kRead review
C
cockroachdb/cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.

1.1k8.2k32.4k4.1k
Go
# cockroachdb# database# distributed-database
4.1kRead review
S
sequelize/sequelize

Feature-rich ORM for modern Node.js and TypeScript, it supports PostgreSQL (with JSON and JSONB support), MySQL, MariaDB, SQLite, MS SQL Server, Snowflake, Oracle DB, DB2 and DB2 for IBM i.

1.3k1.1k30.4k4.3k
TypeScript
# db2-ibm-i# feature-rich# javascript
4.3kRead review
G
getredash/redash

Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.

54780128.7k4.6k
Python
# analytics# athena# bi
4.6kRead review
C
CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers

aka "Bayesian Methods for Hackers": An introduction to Bayesian methods + probabilistic programming with a computation/understanding-first, mathematics-second point of view. All in pure Python ;)

16020328.2k7.9k
Jupyter Notebook
# bayesian-methods# data-science# jupyter-notebook
7.9kRead review
O
OtterMind/Chat2DB

🔥🔥🔥 AI-driven database tool and SQL client, The hottest GUI client, supporting MySQL, Oracle, PostgreSQL, DB2, SQL Server, DB2, SQLite, H2, ClickHouse, and more.

7220727.8k3.0k
Java
# ai# bi# chatgpt
3.0kRead review
A
Automattic/mongoose

MongoDB object modeling designed to work in an asynchronous environment.

1.1k17527.5k4.0k
JavaScript
# mongo# mongodb# nodejs
4.0kRead review
S
sqlitebrowser/sqlitebrowser

Official home of the DB Browser for SQLite (DB4S) project. Previously known as "SQLite Database Browser" and "Database Browser for SQLite". Website at:

20983924.4k2.4k
C++
# c-plus-plus# cross-platform# database
2.4kRead review
P
pubkey/rxdb

The local-first database that runs on every JS runtime and replicates with your existing backend - no vendor, no lock-in - https://rxdb.info/

273723.3k1.2k
TypeScript
# angular# browser-database# couchdb
1.2kRead review

Explore other topics