YBC - Yet another Blob Cache library.
This library implements fast in-process blob cache with persistence support.
===============================================================================
YBC features.
-
Huge amounts of data can be cached. Cache size may exceed available RAM size
by multiple orders of magnitude.
- Very large objects may be cached (up to 2^64 bytes). Think of media files
such as videos, audios and images.
- Very large cache size is supported (up to 2^64 items and up to 2^64 bytes).
In practice cache size is limited by free space on backing store,
not by available RAM size.
-
Persistence support. Cached objects survive process restart if the cache
is explicitly backed by files.
-
Data file layout is optimized for HDD and SSD devices. The code
prefers sequential I/O instead of random I/O. If random I/O is inevitable,
the code tries hard localizing it in the smallest possible memory region.
-
Automatic robust recovery from corrupted index files. Corruptions in data
files may be left unnoticed due to performance reasons - it may be quite
expensive validating multi-GB blobs on every access.
-
Built-in handling of dogpile effect (aka 'thundering herd').
-
Cached objects may be sharded among available backing store devices.
Such sharding may linearly increase cache performance if frequently accessed
items don't fit available physical RAM.
-
The library is thread-safe out of the box.
-
Concurrent atomic updates. It is safe updating an item from multiple threads
while other threads are reading old value for the item under the same key.
-
'Add transaction' support allows constructing object's value on the fly
without serializing it into a temporary buffer (aka 'zero-copy').
Uncommited transaction can be rolled back at any time.
Think of videos streamed directly into the cache without usage of temporary
buffers or complex objects requiring serialization from multiple distinct
places before storing them into the cache.
-
Readers and writers don't block each other while reading/writing data
from/into the cache. The speed is actually limited by hardware memory
bandwidth (if frequently accessed items fit RAM) or backing store
random I/O bandwidth (if requently accessed items don't fit RAM).
-
Instant invalidation of all items in the cache irregardless of cache size.
-
Optimization for multi-tiered memory hierarchy in modern CPUs. The code avoids
unnecessary random memory accesses and tightly packs frequently accessed data
in order to reduce working set size and increase CPU cache hit ratio.
-
The code avoids using dynamic memory allocations in frequently executed paths,
so cache performance is almost independent of the efficiency of the provided
malloc() implementation.
-
The code avoids using expensive system calls in frequently executed paths.
-
The code depends only on OS-supplied libraries.
-
The code can be easily ported to new platforms. All platform-specific
functions and types are hidden behind platform-independent wrappers.
-
Public interface (ybc.h) is designed with future versions' compatibility
in mind. It doesn't expose private structures' contents, so they can be freely
modified in the future versions of the library without breaking applications
dynamically linked against older versions.
-
Small library size (can be packed to 22Kb with -Os).
===============================================================================
Use-cases.
-
CDN cache.
-
File hosting cache.
-
Memcache-like shared cache.
-
Per-process local cache in web-servers.
-
Web-proxy or web-accelerator cache.
-
Web-browser cache.
-
Out-of-GC blob cache for programming languages with GC support.
================================================================================
Credits.
YBC design is inspired by Varnish's "Notes from the Architect" -
https://www.varnish-cache.org/trac/wiki/ArchitectNotes .
================================================================================
FAQ.
Q: Give me performance numbers!
A: Well, YBC achieves 25.8 Mops/s for get() calls and 5.8 Mops/s for set() calls
on my not-so-fast laptop with 2.50GHz Intel i5 CPU when running in 4 threads.
Q: Does YBC work with /dev/shm/ ?
A: Yes. Just create data and index files on /dev/shm . This will completely
eliminate page swapping at the cost of potential data loss on power failure
or computer reboot.
Q: Is this yet another boring cache implementation?
A: No. YBC is designed for modern OSes running on modern hardware. It takes
advantage of OS's and computer's hardware features, while simultaneouly
avoiding their weaknesses. This results in high performance and low resource
usage. Read 'Features' chapter for more details.
Q: OK, how to use it in my program?
A: 1. Investigate API provided by YBC at ybc.h . See tests/ folder
for examples on how to properly use the API.
2. Use this API in your program.
3. Build either object file (ybc-[32|64]-[debug|release].o) or library
(libybc-[debug|release].so) using the corresponding build target
in the provided Makefile.
4. Link the object file or library against your program.
5. ?????
6. PROFIT!!!
Take a look also at bindings/ folder. It contains YBC bindings for various
programming languages if you don't like programming in C. Though currently
only Go is supported :)
Other folders also worth investigation:
* libs/ folder contains additional libraries built on top of YBC.
* apps/ folder contains applications built on top of YBC. Currently
there are the following apps here with self-describing names:
* cdn-booster - caching HTTP proxy implementation on top
of Go bindings.
* cdn-booster-bench - benchmark tool for HTTP/1.1-compliant servers.
* memcached - memcache server implementation on top of Go bindings
for YBC. Unlike the original memcache server, it supports cache
sizes exceeding available RAM by multiple orders of magnitude.
It also provides cache persistence, so cached data survives server
restarts or server crashes.
* memcached-bench - benchmark tool for memcached servers.
Makefile already contains build targets for all these apps.
Q: Why recently added items may disappear from the cache, while their ttl isn't
expired yet?
A: Because YBC is a cache, not a storage. They serve different pruposes:
- Storage is used for items' persistence. It guarantees that added items
exist until they are explicitly deleted.
- Cache is used as a performance booster. It doesn't guarantee that added
items exist until they are explicitly deleted.
YBC may delete any item at any time due to various reasons, which depend
on implementation details. So always expect that the added item may disappear
at any time during subsequent requests.
Q: Can YBC cache small items, not large blobs?
A: Yes. YBC works well with small items - it even supports zero-byte key and
zero-byte value.
Q: Why YBC cannot adjust backing files' size on-demand? I.e. automatically
shrink files when the number of items in the cache is small and automatically
expand files when the number of items exceeds current files' capacity.
A: Because this is stupid idea due to the following reasons:
- If cache size isn't bound, then backing files may eat all the available
space on storage devices.
- Automatic file size adjustment may significantly increase file
fragmentation, which will lead to performance degradation.
- File size adjustment may lead to arbitrary long delays in requests'
serving.
- This will complicate the implementation, which may result in more bugs.
Q: What's the purpose of cache persistence?
A: Cache persistence allows skipping cache warm-up step ater the application
restart. This step can be very expensive and time-consuming for large caches
and slow backends. You can also make a 'golden' copy of a persistent cache
and start every node in the application cluster using their own copy
of the 'golden' cache, thus avoiding warm-up step on all nodes
in the cluster.
Q: How well YBC performance scales on multiple CPUs?
A: It scales perfectly if specially designed functions are used -
ybc_config_disable_overwrite_protection(), ybc_simple_set()
and ybc_simple_get().
Q: Are cache files platform-neutral?
A: No. Cache files are tightly tied to the platform they were created.
Cache files created on one platform (for example, x86) will be completely
broken when read on another platform (for example, x64). Though it is likely
they will appear as empty.
So copy cache files only between machines with identical platforms.
/ybc