[Core] implement redis cache mode

[vijaykramesh]

Why are these changes needed?

This adds a redis mode to the cache. This way I can have multiple processes in separate containers running the same application (that is using autogen) and they can share LLM cache (vs in the current disk cache implementation the SQLIte instance ends up being machine local and can't be easily shared across multiple containers/pods).

The actual redis caching is using pickling, same as the disk cache implementation uses. So the cache should be functionally equivalent to the disk cache version.

Docs added inline and then also agent_chat.md:

LLM Caching

Legacy Disk Cache

By default, you can specify a cache_seed in your llm_config in order to take advantage of a local DiskCache backed cache. This cache will be used to store the results of your LLM calls, and will be used to return results for the same input without making a call to the LLM. This is useful for saving on compute costs, and for speeding up inference.

assistant = AssistantAgent(
    "coding_agent",
    llm_config={
        "cache_seed": 42,
        "config_list": OAI_CONFIG_LIST,
        "max_tokens": 1024,
    },
)

Setting this cache_seed param to None will disable the cache.

Configurable Context Manager

A new configurable context manager allows you to easily turn on and off LLM cache, using either DiskCache or Redis. All LLM agents inside the context manager will use the same cache.

from autogen.cache.cache import Cache

with Cache.redis(cache_seed=42, redis_url="redis://localhost:6379/0") as cache_client:
    user.initiate_chat(assistant, message=coding_task, cache_client=cache_client)

with Cache.disk(cache_seed=42, cache_dir=".cache") as cache_client:
    user.initiate_chat(assistant, message=coding_task, cache_client=cache_client)

Here's an example of the new integration test running in CI (note I had to setup my fork to get it to run, I think it will only run when it is on main that is being merged into? - and in my fork the other tests fail due to my OAI_CONFIG_LIST not being correct.

Integration test coverage for the new code I added:

➜  autogen git:(vr/redis_cache) ✗ coverage run -a -m pytest test/agentchat/test_cache.py
=============================================================================================================== test session starts ===============================================================================================================
platform darwin -- Python 3.11.4, pytest-7.4.4, pluggy-1.3.0
rootdir: /Users/vijay/oss/autogen
configfile: pyproject.toml
plugins: Faker-19.3.0, anyio-3.7.1
collected 3 items


test/agentchat/test_cache.py ...                                                                                                                                                                                                            [100%]

=============================================================================================================== 3 passed in 47.20s ================================================================================================================

And then per PR feedback I added some unit tests for the cache implementations.

➜  autogen git:(vr/redis_cache) ✗ coverage run -a -m pytest test/cache
=============================================================================================================== test session starts ===============================================================================================================
platform darwin -- Python 3.11.4, pytest-7.4.4, pluggy-1.3.0
rootdir: /Users/vijay/oss/autogen
configfile: pyproject.toml
plugins: Faker-19.3.0, anyio-3.7.1
collected 14 items


test/cache/test_cache.py ....                                                                                                                                                                                                               [ 28%]
test/cache/test_disk_cache.py .....                                                                                                                                                                                                         [ 64%]
test/cache/test_redis_cache.py .....                                                                                                                                                                                                        [100%]

=============================================================================================================== 14 passed in 1.24s ================================================================================================================

Related issue number

Checks

• I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
• I've added tests (if relevant) corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed.

[vijaykramesh]

@microsoft-github-policy-service agree company="Regrello"

[davorrunje]

Redis cache and disk cache have different behaviors after being closed:
...
My thinking is that the cache should detaches itself from every agent instances once it exits the with context.

I agree. This will certainly cause problems in future if not unified.

[ekzhu]

@sonichi @vijaykramesh @davorrunje

Sorry I was incorrect to say that Redis and DiskCache will have different exit behaviors. I tested myself and there is no difference. Both caches will stay alive and re-opens once you use it again.

[ekzhu]

I pushed another commit to make sure a_run_chat has the same handling for cache as run_chat.