apache/opendal — 5.5k★ on GitHub (Rust). Apache OpenDAL: One Layer, All Storage.
Snapshot summary built from the project's own GitHub metadata — there's no written TopGit review yet. The page will update automatically when a full review is published.
WHY NO REVIEW YET
TopGit writes full reviews for the most-starred, most-requested repositories. This page is a snapshot until then — see the READ ME tab for the original README in full.
Apache OpenDAL™ (/ˈoʊ.pən.dæl/, pronounced "OH-puhn-dal") is an Open Data Access Layer that keeps application storage code clean, from simple file reads and writes to streaming, byte-range reads, and concurrent uploads.
Use OpenDAL when your application talks to a single S3 bucket or local directory and you want one API for everyday operations, large transfers, and retries. Keep the same operator model when you test in memory, develop on local disk, or connect to 50+ storage services from Rust, Python, Node.js, Java, Go, and more.
Why OpenDAL
Clean code for a single backend: read, write, stat, list, copy, and delete through one Operator. Paths, metadata, and error kinds follow one model across operations.
Built for demanding I/O: stream large objects, read only the byte ranges you need, and upload parts concurrently on services with multipart or block writes. Add retries, timeouts, observability, and concurrency limits as layers.
Portable across environments and backends: use memory in tests, fs in local development, and object storage, file systems, databases, or key-value services in production. Capability checks show where services differ.
Proven in data-intensive systems: projects such as Dify, RAGFlow, LlamaIndex, Databend, RisingWave, GreptimeDB, and Lance build on OpenDAL. Read how ScopeDB accesses object storage with OpenDAL.
Bindings share the service model and follow their own language conventions, so available options and layers vary by binding. Each language guide lists what its binding supports.
Quickstart
Each quickstart uses the in-memory service, so it runs without credentials. The second snippet in each language shows the same API against S3.
Point the operator at S3, add retries, and read only what you need:
import opendal
from opendal.layers import RetryLayer
op = opendal.Operator("s3", bucket="my-bucket", region="us-east-1").layer(
RetryLayer(max_times=3)
)
# Read only the first 64 KiB of an object.
header = op.read("data/events.parquet", offset=0, size=64 * 1024)
# Stream a large object instead of loading it into memory.
total = 0
with op.open("data/events.parquet", "rb") as f:
while chunk := f.read(1024 * 1024):
total += len(chunk)
See the Python guide for async APIs, error handling, and production setup.
Rust
cargo add opendal
cargo add tokio --features full
use opendal::services;
use opendal::Operator;
use opendal::Result;
#[tokio::main]
async fn main() -> Result<()> {
let op = Operator::new(services::Memory::default())?;
op.write("hello.txt", "Hello, World!").await?;
let bytes = op.read("hello.txt").await?;
let meta = op.stat("hello.txt").await?;
op.delete("hello.txt").await?;
println!("read {} bytes, stat reports {} bytes", bytes.len(), meta.content_length());
Ok(())
}
Enable a service with its feature, for example cargo add opendal --features services-s3. This fragment reads a byte range and uploads a large object in concurrent parts:
use opendal::layers::RetryLayer;
use opendal::services::S3;
use opendal::Operator;
let op = Operator::new(S3::default().bucket("my-bucket").region("us-east-1"))?
.layer(RetryLayer::new().with_jitter());
// Read only the first 64 KiB of an object.
let header = op.read_with("data/events.parquet").range(0..64 * 1024).await?;
// Keep up to 4 part uploads in flight. `close` waits for them and commits the object.
let mut writer = op
.writer_with("archive/big.bin")
.chunk(8 * 1024 * 1024)
.concurrent(4)
.await?;
writer.write(first_part).await?;
writer.write(second_part).await?;
writer.close().await?;
See the Rust core guide and the API reference for streaming readers, conditional operations, and layers.
Configure credentials and other options as described in each service's documentation, for example S3.
Choose Your Language
Each binding has its own independent version number, which may differ from the Rust core version. Check the binding's own version when you look for updates or compatibility information.
Language
Package
Guide
API reference
Rust
opendal
Guide
docs.rs · dev
Python
opendal
Guide
dev
Node.js
opendal
Guide
dev
Java
opendal-java
Guide
javadoc.io · dev
Go
bindings/go
Guide
pkg.go.dev
Ruby
opendal
Guide
dev
C
Source
Guide
dev
C++
Source
Guide
dev
More bindings: .NET, D, Dart, Haskell, Lua, OCaml, PHP, Swift, and Zig.
Services and Layers
OpenDAL connects to 50+ services through one Operator. A selection by category:
See the services documentation for every service and its configuration keys.
Layers wrap an operator to add cross-cutting behavior: RetryLayer, TimeoutLayer, ConcurrentLimitLayer, ThrottleLayer, LoggingLayer, TracingLayer, MetricsLayer, PrometheusLayer, OtelMetricsLayer, MimeGuessLayer, RouteLayer, FoyerLayer, and more. These links point to the Rust core; each binding guide lists the layers it exposes.
Production Stories and Examples
ScopeDB on OpenDAL: sparse range reads, concurrent multipart writes, cloud authentication, and separate timeout and retry policies for reads and writes.
Who uses OpenDAL: projects and companies that build on OpenDAL.
Examples: Rust, Python, Node.js, and more.
Documentation
Website: https://opendal.apache.org
Vision: https://opendal.apache.org/vision
Rust release docs: https://docs.rs/opendal
Rust dev docs: https://opendal.apache.org/docs/rust/opendal/
Contribute
OpenDAL is an active open-source project. We are always open to people who want to use it or contribute to it. Here are some ways to go.
Start with Contributing Guide.
Submit Issues for bug report or feature requests.
Start Discussions for questions or ideas.
Talk to community directly at Discord.
Report security vulnerabilities to private mailing list
Branding
The first and most prominent mentions must use the full form: Apache OpenDAL™ of the name for any individual usage (webpage, handout, slides, etc.) Depending on the context and writing style, you should use the full form of the name sufficiently often to ensure that readers clearly understand the association of both the OpenDAL project and the OpenDAL software product to the ASF as the parent organization.
For more details, see the Apache Product Name Usage Guide.
License and Trademarks
Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0
Apache OpenDAL, OpenDAL, and Apache are either registered trademarks or trademarks of the Apache Software Foundation.
How does apache/opendal compare to other Data projects?
apache/opendal is tracked by TopGit in the Data category, with 5.5k GitHub stars and written in Rust. Browse the Data topic page on TopGit to compare it against similar projects by stars and activity.
Is apache/opendal open source?
Yes — apache/opendal ships under the Apache-2.0 license, which makes its source code freely readable (and, depending on license terms, forkable and reusable). Source: github.com/apache/opendal.
What else is in the Data space?
apache/opendal is tracked by TopGit under the Data category, alongside 7 GitHub-tagged topics. Trending and Topics pages list peer repositories of comparable stars and language.
What is apache/opendal?
apache/opendal (apache/opendal) is a Rust project on GitHub. From the project's own README: Apache OpenDAL: One Layer, All Storage.
Where do I read more about apache/opendal?
This TopGit page is a snapshot — the READ ME tab shows the project's own README content (links stripped, images preserved). The GitHub repository at github.com/apache/opendal is the definitive source.
Why is apache/opendal categorized under Data?
TopGit places apache/opendal in the Data category based on its GitHub topics and description (tagged: "azblob", "gcs", "hacktoberfest"). Categories are assigned from real repository metadata, not editorial guesswork.
Read full README in the tab above.
Is opendal worth your time?
ChatGPT, Claude and Perplexity can all read this page. Ask one of them what it makes of opendal.