Là một dự án mã nguồn mở, tikv/raft-engine đã đạt 642 sao trên GitHub, ngôn ngữ Rust. A persistent storage engine for Multi-Raft log
Tóm tắt dựng từ metadata GitHub của chính dự án — chưa có bài review TopGit. Trang sẽ tự động cập nhật khi bài review đầy đủ được xuất bản.
VÌ SAO CHƯA CÓ REVIEW
TopGit viết bài đầy đủ cho repo có nhiều sao nhất và được yêu cầu nhiều nhất. Trang này là snapshot trong thời gian chờ — xem README gốc ở tab READ ME.
Raft Engine is a persistent embedded storage engine with a log-structured design similar to bitcask. It is built for TiKV to store Multi-Raft logs.
Features
APIs for storing and retrieving protobuf log entries with consecutive indexes
Key-value storage for individual Raft Groups
Minimum write amplification
Collaborative garbage collection
Supports lz4 compression over log entries
Supports file system extension
Design
Raft Engine consists of two basic constructs: memtable and log file.
In memory, each Raft Group holds its own memtable, containing all the key value pairs and the file locations of all log entries. On storage, user writes are sequentially written to the active log file, which is periodically rotated below a configurable threshold. Different Raft Groups share the same log stream.
Write
Similar to RocksDB, Raft Engine provides atomic writes. Users can stash the changes into a log batch before submitting.
The writing of one log batch can be broken down into three steps:
Optionally compress the log entries
Write to log file
Apply to memtable
At step 2, to group concurrent requests, each writing thread must enter a queue. The first in line automatically becomes the queue leader, responsible for writing the entire group to the log file.
Both synchronous and non-sync writes are supported. When one write in a batch is marked synchronous, the batch leader will call fdatasync() after writing. This way, buffered data is guaranteed to be flushed out onto the storage.
After its data is written, each writing thread will proceed to apply the changes to memtable on their own.
Garbage Collection
After changes are applied to the local state machine, the corresponding log entries can be compacted from Raft Engine, logically. Because multiple Raft Groups share the same log stream, these truncated logs will punch holes in the log files. During garbage collection, Raft Engine scans for these holes and compacts log files to free up storage space. Only at this point, the unneeded log entries are deleted physically.
Raft Engine carries out garbage collection in a collaborative manner.
First, its timing is controlled by the user. Raft Engine consolidates and removes its log files only when the user voluntarily calls the purge_expired_files() routine. For reference, TiKV calls it every 10 seconds by default.
Second, it sends useful feedback to the user. Each time the GC routine is called, Raft Engine will examine itself and return a list of Raft Groups that hold particularly old log entries. Those log entries block the GC progress and should be compacted by the user.
Using this crate
Put this in your Cargo.toml:
[dependencies]
raft-engine = "0.4"
Available Cargo features:
scripting: Compiles with Rhai. This enables script debugging utilities including unsafe_repair.
nightly: Enables nightly-only features including test.
internals: Re-exports key components internal to Raft Engine. Enabled when building for docs.rs.
failpoints: Enables fail point testing powered by tikv/fail-rs.
swap: Use SwappyAllocator to limit the memory usage of Raft Engine. The memory budget can be configured with "memory-limit". Depending on the nightly feature.
See some basic use cases under the examples directory.
Contributing
Contributions are always welcome! Here are a few tips for making a PR:
All commits must be signed off (with git commit -s) to pass the DCO check.
Tests are automatically run against the changes, some of them can be run locally:
# run tests with nightly features
make
# run tests on stable toolchain
make WITH_STABLE_TOOLCHAIN=force
# filter a specific test case
make test EXTRA_CARGO_ARGS=<testname>
For changes that might induce performance effects, please quote the targeted benchmark results in the PR description. In addition to micro-benchmarks, there is a standalone stress test tool which you can use to demonstrate the system performance.
Trang TopGit này là một snapshot — tab "Readme" hiển thị nguyên văn README của repo (đã bỏ link, giữ ảnh). Repo GitHub ở github.com/tikv/raft-engine là nguồn chính thức.
tikv/raft-engine có phải mã nguồn mở không?
Có — tikv/raft-engine phát hành theo license Apache-2.0, nghĩa là mã nguồn mở để đọc, fork và (tùy license) tái sử dụng. Mã: github.com/tikv/raft-engine.
tikv/raft-engine có tag gì không?
Bản đồng bộ chưa ghi nhận topic GitHub nào cho tikv/raft-engine. GitHub topics hiển thị ở thanh bên phải trang repo — đó là nơi đáng kiểm tra nhất.
tikv/raft-engine có website riêng không?
TopGit chưa ghi nhận URL trang chủ cho tikv/raft-engine. Phần README ở tab phía trên thường có link demo, hoặc xem mô tả GitHub của repo.
tikv/raft-engine còn đang phát triển không?
Commit gần nhất trên tikv/raft-engine là 2 tháng trước (theo timestamp GitHub). Repo có 110 fork — một chỉ báo về mức độ quan tâm của cộng đồng.
tikv/raft-engine dùng license gì?
tikv/raft-engine phát hành theo license Apache-2.0. Nên mở file LICENSE trên GitHub để xác nhận — license metadata đôi khi lệch với thực tế dự án.
Đọc đầy đủ README ở tab phía trên.
Chưa chắc raft-engine có hợp với bạn?
Để ChatGPT, Claude hoặc Perplexity tìm hiểu giúp — bấm bên dưới và xem AI nói gì về raft-engine.