aio-libs/async-lru

async-lru

:info: Simple lru cache for asyncio

.. image:: https://github.com/aio-libs/async-lru/actions/workflows/ci-cd.yml/badge.svg?event=push :target: https://github.com/aio-libs/async-lru/actions/workflows/ci-cd.yml?query=event:push :alt: GitHub Actions CI/CD workflows status

.. image:: https://img.shields.io/pypi/v/async-lru.svg?logo=Python&logoColor=white :target: https://pypi.org/project/async-lru :alt: async-lru @ PyPI

.. image:: https://codecov.io/gh/aio-libs/async-lru/branch/master/graph/badge.svg :target: https://codecov.io/gh/aio-libs/async-lru

.. image:: https://img.shields.io/matrix/aio-libs:matrix.org?label=Discuss%20on%20Matrix%20at%20%23aio-libs%3Amatrix.org&logo=matrix&server_fqdn=matrix.org&style=flat :target: https://matrix.to/#/%23aio-libs:matrix.org :alt: Matrix Room — #aio-libs:matrix.org

.. image:: https://img.shields.io/matrix/aio-libs-space:matrix.org?label=Discuss%20on%20Matrix%20at%20%23aio-libs-space%3Amatrix.org&logo=matrix&server_fqdn=matrix.org&style=flat :target: https://matrix.to/#/%23aio-libs-space:matrix.org :alt: Matrix Space — #aio-libs-space:matrix.org

Installation

.. code-block:: shell

pip install async-lru

Usage

This package is a port of Python's built-in functools.lru_cache <https://docs.python.org/3/library/functools.html#functools.lru_cache>_ function for asyncio <https://docs.python.org/3/library/asyncio.html>_. To better handle async behaviour, it also ensures multiple concurrent calls will only result in 1 call to the wrapped function, with all await\s receiving the result of that call when it completes.

.. code-block:: python

import asyncio

import aiohttp
from async_lru import alru_cache


@alru_cache(maxsize=32)
async def get_pep(num):
    resource = 'http://www.python.org/dev/peps/pep-%04d/' % num
    async with aiohttp.ClientSession() as session:
        try:
            async with session.get(resource) as s:
                return await s.read()
        except aiohttp.ClientError:
            return 'Not Found'


async def main():
    for n in 8, 290, 308, 320, 8, 218, 320, 279, 289, 320, 9991:
        pep = await get_pep(n)
        print(n, len(pep))

    print(get_pep.cache_info())
    # CacheInfo(hits=3, misses=8, maxsize=32, currsize=8)

    # closing is optional, but highly recommended
    await get_pep.cache_close()


asyncio.run(main())

TTL (time-to-live in seconds, expiration on timeout) is supported by accepting ttl configuration parameter (off by default):

.. code-block:: python

@alru_cache(ttl=5)
async def func(arg):
    return arg * 2

To prevent thundering herd issues when many cache entries expire simultaneously, you can add jitter to randomize the TTL for each entry:

.. code-block:: python

@alru_cache(ttl=3600, jitter=1800)
async def func(arg):
    return arg * 2

With ttl=3600, jitter=1800, each cache entry will have a random TTL between 3600 and 5400 seconds, spreading out invalidations over time.

The library supports explicit invalidation for specific function call by cache_invalidate():

.. code-block:: python

@alru_cache(ttl=5)
async def func(arg1, arg2):
    return arg1 + arg2

func.cache_invalidate(1, arg2=2)

The method returns True if corresponding arguments set was cached already, False otherwise.

To check whether a specific set of arguments is present in the cache without affecting hit/miss counters or LRU ordering, use cache_contains():

.. code-block:: python

@alru_cache(maxsize=32)
async def func(arg1, arg2):
    return arg1 + arg2

await func(1, arg2=2)

func.cache_contains(1, arg2=2)  # True
func.cache_contains(3, arg2=4)  # False

The method returns True if the result for the given arguments is cached, False otherwise.

Custom cache keys

By default the cache key is built from all arguments, like functools.lru_cache does. The key parameter accepts a callable that receives the same arguments as the wrapped function and returns the cache key, so arguments that do not affect the result can be excluded from it. This is an async-lru extension beyond the functools.lru_cache interface:

.. code-block:: python

@alru_cache(key=lambda db, query: query)
async def query_db(db, query):
    return await db.execute(query)

# Both calls share one cache entry despite the different connections.
await query_db(conn1, "SELECT ...")
await query_db(conn2, "SELECT ...")

The returned key must be hashable. cache_invalidate() and cache_contains() compute the key the same way, so they accept the full argument list as usual. For decorated methods the key callable receives the instance as its first argument. Passing typed=True together with key raises ValueError, since typed only affects the default key computation.

Limitations

Event Loop Affinity: alru_cache enforces that a cache instance is used with only one event loop. If you attempt to use a cached function from a different event loop than where it was first called, a RuntimeError will be raised:

.. code-block:: text

RuntimeError: alru_cache is not safe to use across event loops: this cache
instance was first used with a different event loop.
Use separate cache instances per event loop.

For typical asyncio applications using a single event loop, this is automatic and requires no configuration. If your application uses multiple event loops, create separate cache instances per loop:

.. code-block:: python

import threading

_local = threading.local()

def get_cached_fetcher():
    if not hasattr(_local, 'fetcher'):
        @alru_cache(maxsize=100)
        async def fetch_data(key):
            ...
        _local.fetcher = fetch_data
    return _local.fetcher

You can also reuse the logic of an already decorated function in a new loop by accessing __wrapped__:

.. code-block:: python

@alru_cache(maxsize=32)
async def my_task(x):
    ...

# In Loop 1:
# my_task() uses the default global cache instance

# In Loop 2 (or a new thread):
# Create a fresh cache instance for the same logic
cached_task_loop2 = alru_cache(maxsize=32)(my_task.__wrapped__)
await cached_task_loop2(x)

Security considerations

Cache keys are built only from explicit arguments. Like functools.lru_cache <https://docs.python.org/3/library/functools.html#functools.lru_cache>_, alru_cache derives its cache key solely from the positional and keyword arguments passed to the wrapped function. Implicit, request-scoped context — such as authentication headers, the current user or tenant, contextvars, thread/task locals, or module globals — is not part of the key and therefore not isolated between callers.

Because concurrent calls with the same key also share a single in-flight result (see the Usage section above), a value computed for one caller can be returned to another whenever their arguments are equal. In multi-tenant or multi-user services this can lead to cross-tenant data exposure if the cached coroutine's result depends on anything other than its explicit arguments.

To use alru_cache safely in these contexts:

• Make the cached coroutine a pure function of its arguments. Any value that affects the result — user_id, tenant_id, role, locale, feature flags, etc. — must be passed as an argument so it becomes part of the cache key, or use a separate cache instance per security domain.
• Avoid caching context-dependent functions. If a function reads request-scoped state from contextvars/globals rather than from its arguments, either refactor it to take that state explicitly or do not cache it.
• Consider typed=True when callers may pass multiple types. With the default typed=False, arguments that compare and hash equal share an entry (for example 1 and 1.0, or True and 1). Pass typed=True to key such arguments distinctly.
• Be wary of attacker-controlled key arguments. Objects with unusual __hash__ / __eq__ semantics can collide unexpectedly; only use trusted, well-behaved values as cache key components.

Benchmarks

async-lru uses CodSpeed <https://codspeed.io/>_ for performance regression testing.

To run the benchmarks locally:

.. code-block:: shell

pip install -r requirements-dev.txt
pytest --codspeed benchmark.py

The benchmark suite covers both bounded (with maxsize) and unbounded (no maxsize) cache configurations. Scenarios include:

• Cache hit
• Cache miss
• Cache fill/eviction (cycling through more keys than maxsize)
• Cache clear
• TTL expiry
• Cache invalidation
• Cache info retrieval
• Concurrent cache hits
• Baseline (uncached async function)

On CI, benchmarks are run automatically via GitHub Actions on Python 3.13, and results are uploaded to CodSpeed (if a CODSPEED_TOKEN is configured). You can view performance history and detect regressions on the CodSpeed dashboard.

Thanks

The library was donated by Ocean S.A. <https://ocean.io/>_

Thanks to the company for contribution.