More documentation for Cache. Updated FAQ to include database Locked error (#1291)

* Update FAQ.md

Database Locked error in Azure Machine Learning compute instance

* Update website/docs/FAQ.md

Co-authored-by: Chi Wang <[email protected]>

* Update website/docs/FAQ.md

Co-authored-by: Chi Wang <[email protected]>

* update doc; add module import for Cache

* update doc

* nit

---------

Co-authored-by: Eric Zhu <[email protected]>
Co-authored-by: Chi Wang <[email protected]>

Author: 3 people

autogen/oai/__init__.py

@@ -8,6 +8,7 @@
     config_list_from_json,
     config_list_from_dotenv,
 )
+from autogen.cache.cache import Cache
 
 __all__ = [
     "OpenAIWrapper",
@@ -19,4 +20,5 @@
     "config_list_from_models",
     "config_list_from_json",
     "config_list_from_dotenv",
+    "Cache",
 ]

website/docs/FAQ.md

@@ -192,6 +192,34 @@ Please refer to https://microsoft.github.io/autogen/docs/reference/agentchat/con
 The "use_docker" arg in an agent's code_execution_config will be set to the name of the image containing the change after execution, when the conversation finishes.
 You can save that image name. For a new conversation, you can set "use_docker" to the saved name of the image to start execution there.
 
+## Database locked error
+
+When using VMs such as Azure Machine Learning compute instances,
+you may encounter a "database locked error". This is because the
+[LLM cache](./Use-Cases/agent_chat.md#cache)
+is trying to write to a location that the application does not have access to.
+
+You can set the `cache_path_root` to a location where the application has access.
+For example,
+
+```python
+from autogen import Cache
+
+with Cache.disk(cache_path_root="/tmp/.cache") as cache:
+    agent_a.initate_chat(agent_b, ..., cache=cache)
+```
+
+You can also use Redis cache instead of disk cache. For example,
+
+```python
+from autogen import Cache
+
+with Cache.redis(redis_url=...) as cache:
+    agent_a.initate_chat(agent_b, ..., cache=cache)
+```
+
+You can also disable the cache. See [here](./Use-Cases/agent_chat.md#llm-caching) for details.
+
 ## Agents are throwing due to docker not running, how can I resolve this?
 
 If running AutoGen locally the default for agents who execute code is for them to try and perform code execution within a docker container. If docker is not running, this will cause the agent to throw an error. To resolve this you have the below options:

website/docs/Use-Cases/agent_chat.md

@@ -287,15 +287,37 @@ By adopting the conversation-driven control with both programming language and n
 
 ### LLM Caching
 
-Since version 0.2.8, a configurable context manager allows you to easily configure LLM cache, using either DiskCache or Redis. All agents inside the context manager will use the same cache.
+Since version 0.2.8, a configurable context manager allows you to easily
+configure LLM cache, using either DiskCache or Redis. All agents inside the
+context manager will use the same cache.
 
 ```python
-from autogen.cache.cache import Cache
+from autogen import Cache
 
-with Cache.redis(cache_seed=42, redis_url="redis://localhost:6379/0") as cache:
+# Use Redis as cache
+with Cache.redis(redis_url="redis://localhost:6379/0") as cache:
     user.initiate_chat(assistant, message=coding_task, cache=cache)
 
-with Cache.disk(cache_seed=42, cache_dir=".cache") as cache:
+# Use DiskCache as cache
+with Cache.disk() as cache:
+    user.initiate_chat(assistant, message=coding_task, cache=cache)
+```
+
+You can vary the `cache_seed` parameter to get different LLM output while
+still using cache.
+
+```python
+# Setting the cache_seed to 1 will use a different cache from the default one
+# and you will see different output.
+with Cache.disk(cache_seed=1) as cache:
+    user.initiate_chat(assistant, message=coding_task, cache=cache)
+```
+
+By default DiskCache uses `.cache` for storage. To change the cache directory,
+set `cache_path_root`:
+
+```python
+with Cache.disk(cache_path_root="/tmp/autogen_cache") as cache:
     user.initiate_chat(assistant, message=coding_task, cache=cache)
 ```
 

website/docs/Use-Cases/enhanced_inference.md

@@ -168,33 +168,51 @@ Total cost: 0.00027
 
 ## Caching
 
-API call results are cached locally and reused when the same request is issued. This is useful when repeating or continuing experiments for reproducibility and cost saving.
+API call results are cached locally and reused when the same request is issued.
+This is useful when repeating or continuing experiments for reproducibility and cost saving.
 
 Starting version 0.2.8, a configurable context manager allows you to easily configure
 the cache, using either DiskCache or Redis.
-All `OpenAIWrapper` created inside the context manager can use the same cache through the constructor.
+All `OpenAIWrapper` created inside the context manager can use the same cache
+through the constructor.
 
 ```python
-from autogen.cache.cache import Cache
+from autogen import Cache
 
-with Cache.redis(cache_seed=42, redis_url="redis://localhost:6379/0") as cache:
+with Cache.redis(redis_url="redis://localhost:6379/0") as cache:
     client = OpenAIWrapper(..., cache=cache)
     client.create(...)
 
-with Cache.disk(cache_seed=42, cache_dir=".cache") as cache:
+with Cache.disk() as cache:
     client = OpenAIWrapper(..., cache=cache)
     client.create(...)
 ```
 
 You can also set a cache directly in the `create()` method.
 
 ```python
-client = OpenAIWrapper()
-with Cache.disk(cache_seed=42, cache_dir=".cache") as cache:
+client = OpenAIWrapper(...)
+with Cache.disk() as cache:
+    client.create(..., cache=cache)
+```
+
+You can vary the `cache_seed` parameter to get different LLM output while
+still using cache.
+
+```python
+# Setting the cache_seed to 1 will use a different cache from the default one
+# and you will see different output.
+with Cache.disk(cache_seed=1) as cache:
     client.create(..., cache=cache)
 ```
 
-You can control the randomness by setting the `cache_seed` parameter.
+By default DiskCache uses `.cache` for storage. To change the cache directory,
+set `cache_path_root`:
+
+```python
+with Cache.disk(cache_path_root="/tmp/autogen_cache") as cache:
+    client.create(..., cache=cache)
+```
 
 ### Turnning off cache
 
@@ -208,9 +226,16 @@ client = OpenAIWrapper(..., cache_seed=None)
 client.create(..., cache_seed=None)
 ```
 
-_NOTE_. openai v1.1 introduces a new param `seed`. The difference between autogen's `cache_seed` and openai's `seed` is that:
-* autogen uses local disk cache to guarantee the exactly same output is produced for the same input and when cache is hit, no openai api call will be made.
-* openai's `seed` is a best-effort deterministic sampling with no guarantee of determinism. When using openai's `seed` with `cache_seed` set to None, even for the same input, an openai api call will be made and there is no guarantee for getting exactly the same output.
+### Difference between `cache_seed` and openai's `seed` parameter
+
+openai v1.1 introduces a new param `seed`.
+The differences between autogen's `cache_seed` and openai's `seed`:
+    - autogen uses local disk cache to guarantee the exactly same output is produced
+    for the same input and when cache is hit, no openai api call will be made.
+    - openai's `seed` is a best-effort deterministic sampling with no guarantee
+    of determinism. When using openai's `seed` with `cache_seed` set to None,
+    even for the same input, an openai api call will be made and there is
+    no guarantee for getting exactly the same output.
 
 ## Error handling