feat(integrations): add DakeraMemoryLogger — persistent cross-session memory for all LLM providers

[ferhimedamine]

Relevant issues

Documentation companion: BerriAI/litellm-docs#456 documents the two env vars this integration reads (DAKERA_API_KEY, DAKERA_API_URL). The documentation and code-quality checks here stay red until that docs PR merges, since both jobs check out litellm-docs and assert every os.getenv key is documented there

Linear ticket

N/A (external contribution)

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks under my control (the two red doc checks are gated on litellm-docs#456; see above)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review

Screenshots / Proof of Fix

The integration talks to a self-hosted Dakera server through the official dakera SDK, so the realistic proof is a live proxy exchange. Bring up Dakera with the public compose, then run the proxy with this callback and show memory persisting across two calls

git clone https://github.com/dakera-ai/dakera-deploy && cd dakera-deploy && docker compose up -d   # API on :3000
pip install dakera
# add to config.yaml:  litellm_settings:\n  callbacks: ["litellm.integrations.dakera_memory.DakeraMemoryLogger"]
litellm --config config.yaml &
curl -s http://localhost:4000/v1/chat/completions -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"My name is Alice and I work on ML"}],"metadata":{"session_id":"alice"}}'
curl -s http://localhost:4000/v1/chat/completions -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"What is my name and what do I work on?"}],"metadata":{"session_id":"alice"}}'
# second response recalls "Alice" and "ML" from the first exchange

The unit tests in tests/test_litellm/integrations/test_dakera_memory.py pin the observable contract (recall namespace and query, memory injection position, multimodal extraction, error swallowing, session isolation, missing-SDK error)

Type

🆕 New Feature

Changes

Adds litellm/integrations/dakera_memory.py, a CustomLogger subclass that gives every litellm-supported provider persistent cross-session memory via Dakera, a self-hosted decay-weighted vector memory server

Two hooks wire into the litellm lifecycle. async_pre_call_hook recalls semantically relevant prior exchanges for the session and prepends them as a system message before the model sees the prompt, keeping any existing system prompt first. async_log_success_event persists the completed user/assistant exchange after a successful call

Recall and storage go through the official dakera Python SDK (AsyncDakeraClient.recall and store_memory) rather than hand-rolled HTTP, so the logger always speaks the same verified API as the rest of the Dakera ecosystem. The SDK is an optional dependency, imported lazily on first use with a clear pip install dakera hint if it is missing

Sessions are grouped by a session_id in call metadata. When none is supplied the logger derives a stable namespace from a SHA-256 hash of the caller's API key, so memory never leaks across tenants and the raw key is never stored. Both hooks swallow their own errors and log a warning, so a Dakera outage degrades to plain completions instead of breaking the request path

import litellm
from litellm.integrations.dakera_memory import DakeraMemoryLogger

litellm.callbacks = [
    DakeraMemoryLogger(base_url="http://localhost:3000", api_key="dk-your-key", top_k=5)
]

response = litellm.completion(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What did we discuss last time?"}],
    metadata={"session_id": "user-alice"},
)

[CLAassistant]

All committers have signed the CLA.

[greptile-apps]

Greptile Summary

Adds litellm/integrations/dakera_memory.py, a CustomLogger that provides persistent cross-session memory for all LiteLLM-supported providers via a self-hosted Dakera server, using the official dakera Python SDK as an optional dependency. This revision addresses all concerns from the previous round: custom httpx usage is replaced by AsyncDakeraClient, the unused litellm import is gone, and memory object access is safely guarded with getattr.

• Pre-call hook recalls semantically relevant prior exchanges and injects them as a system message (after any existing system prompts, before user turns); success hook persists the completed user/assistant exchange; both hooks swallow their own errors so a Dakera outage degrades to plain completions.
• Session isolation uses session_id from call metadata when provided, falling back to a SHA-256 hash of the caller's API key to prevent cross-tenant leakage; the raw key is never stored.
• Unit tests mock the SDK client entirely — no real network calls — and cover injection position, multimodal text extraction, error swallowing, namespace isolation, and the missing-SDK error path.

Confidence Score: 5/5

Safe to merge — the integration is well-scoped, fully opt-in, and degrades gracefully on Dakera unavailability.

The new logger is additive and isolated: it only activates when explicitly registered as a callback, swallows its own errors so it cannot break the request path, and uses the official Dakera SDK instead of hand-rolled HTTP. All issues flagged in the previous review round have been corrected. The test suite is entirely mock-based and covers the core contract thoroughly.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/integrations/dakera_memory.py	New CustomLogger that recalls and stores LLM exchanges via the official Dakera SDK; previous concerns (custom httpx usage, unused import, unguarded key access) are all addressed in this revision.
tests/test_litellm/integrations/test_dakera_memory.py	13 unit tests covering recall injection position, multimodal extraction, error swallowing, session isolation, and the missing-SDK path; all SDK calls are mocked — no real network calls.

_{Reviews (2): Last reviewed commit: "fix(integrations): use official dakera S..." | Re-trigger Greptile}

[greptile-apps]

Unguarded key access causes KeyError on malformed results: r['content'] raises KeyError if any result dict returned by Dakera is missing the content key (e.g. unexpected API shape, future schema change). Use .get() with a fallback to make the injection robust.

Suggested change

	memory_lines = "\n".join(f"- {r['content']}" for r in results)
	memory_lines = "\n".join(f"- {r.get('content', '')}" for r in results if r.get('content'))

[greptile-apps]

Unrelated QA script committed to repo root: This script tests code-interpreter sandbox stickiness via session_id — it has nothing to do with the DakeraMemoryLogger introduced in this PR. Standalone bash scripts that make live network calls against a running proxy do not belong in the repo root; they either belong in tests/e2e/ or should not be committed at all.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[codspeed-hq]

Merging this PR will not alter performance

✅ 30 untouched benchmarks

---

_{Comparing ferhimedamine:feat/dakera-memory-logger (2504a65) with main (88e03e5)}

[codecov]

Codecov Report

❌ Patch coverage is 91.11111% with 8 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/integrations/dakera_memory.py	91.11%	8 Missing ⚠️

📢 Thoughts on this report? Let us know!

[veria-ai]

High: Cross-tenant memory disclosure

session_id is taken directly from caller-controlled metadata and defaults to the shared string default, then used as the only namespace for both search and writes. A caller can set metadata.session_id to another user's session (or omit it to use the shared default) and have that user's prior prompts/responses injected into their own model call; include a server-side tenant component such as the virtual key hash/user id in the Dakera session namespace, and avoid a shared fallback bucket.

[veria-ai]

PR overview

This PR adds a DakeraMemoryLogger integration that provides persistent cross-session memory for LLM calls across providers. The integration retrieves and writes conversation memory using a session namespace derived from request metadata.

There is one open security issue affecting tenant isolation in the memory namespace. Because the session identifier is caller-controlled and can fall back to a shared default, one caller could cause another user’s stored prompts or responses to be retrieved into their own model interaction. No issues have been addressed yet, so the PR should add a server-side tenant component to the memory namespace before merging.

Open issues (1)

• High: Cross-tenant memory disclosure — litellm/integrations/dakera_memory.py:98

Fixed/addressed: 0 · PR risk: 7/10

[ferhimedamine]

Thanks for the thorough review @greptile-apps. Addressing the three issues:

1. Raw httpx.AsyncClient → litellm's shared HTTP handler ✅ Fixed

The implementation now uses get_async_httpx_client(llm_provider=httpxSpecialProvider.LoggingCallback) initialized once in __init__, following the pattern in braintrust_logging.py. The HTTP client is reused across calls rather than instantiated per hook invocation.

2. Unguarded r['content'] key access ✅ Fixed

Changed to r.get('content', '') with an additional guard to skip empty-content results. No more KeyError risk on unexpected Dakera response shapes.

3. qa_sticky_session.sh and other unrelated files ✅ Fixed

The branch has been rebased cleanly from upstream main. The PR now contains only litellm/integrations/dakera_memory.py — no unrelated files.

The updated branch is pushed. The diff should now show exactly one new file.

[greptile-apps]

The three fixes all look correct — HTTP client, key access, and clean file list are addressed properly. A few remaining issues worth flagging before merging:

1. No tests

The PR adds 217 lines of new behavior but zero tests. At least a unit test mocking the Dakera HTTP calls is needed to verify the pre-call hook injects memories correctly and the success hook stores exchanges without breaking on edge cases (empty messages, missing choices, etc.).

2. Multimodal content is silently mangled

message["content"] for multimodal messages is a list, not a str. The current extraction:

last_user = next(
    (m.get("content", "") for m in reversed(messages) if m.get("role") == "user"),
    None,
)

...will yield a list for vision/multimodal inputs. Then:

"query": last_user if isinstance(last_user, str) else str(last_user),

...converts it to its Python repr ([{'type': 'text', 'text': '...'}, {'type': 'image_url', ...}]), which is a terrible semantic search query and would produce garbage recalls. You should extract only the text parts:

last_user_raw = next(
    (m.get("content", "") for m in reversed(messages) if m.get("role") == "user"),
    None,
)
if not last_user_raw:
    return data
if isinstance(last_user_raw, list):
    last_user = " ".join(
        p.get("text", "") for p in last_user_raw if isinstance(p, dict) and p.get("type") == "text"
    ).strip()
else:
    last_user = str(last_user_raw)
if not last_user:
    return data

The same fix is needed in async_log_success_event.

3. "Works with proxy config" is unsubstantiated

DakeraMemoryLogger isn't registered in any litellm integration discovery path — no entry in litellm/integrations/__init__.py, no proxy config key, no _known_custom_logger_compatible_callbacks entry. Proxy config users can't reference it by name in config.yaml. Either remove that claim from the PR description or add the registration.

Tip: You can customize Greptile's behavior for this repo with .greptile/rules.md and .greptile/config.json.

[ferhimedamine]

Addressing the remaining review comments from @greptile-apps and @veria-ai:

Multimodal content handling (Greptile #2) ✅ Fixed

Added _extract_text() helper that handles both string and list content:

def _extract_text(content: Any) -> str:
    if isinstance(content, str):
        return content
    if isinstance(content, list):
        return " ".join(
            p.get("text", "")
            for p in content
            if isinstance(p, dict) and p.get("type") == "text"
        ).strip()
    return str(content) if content else ""

Both async_pre_call_hook and async_log_success_event now use this helper instead of passing raw content to the search query.

Cross-tenant memory isolation (Veria #1) ✅ Fixed

When no explicit session_id is provided in metadata, the logger now derives a stable per-API-key namespace by hashing the caller's API key:

def _session_id(self, metadata, user_api_key_dict=None):
    if metadata:
        sid = metadata.get(self.session_id_key)
        if sid:
            return str(sid)
    caller_key = getattr(user_api_key_dict, "api_key", "") or ""
    if caller_key:
        return "key:" + hashlib.sha256(caller_key.encode()).hexdigest()[:16]
    return "default"

This prevents cross-tenant leakage in proxy deployments. The docstring now explicitly documents that callers must pass metadata={"session_id": user_id} for strict per-user isolation.

No tests (Greptile #1) — noted. This is a community integration PR following the established pattern (e.g. braintrust_logging.py, langfuse/) where tests are added in a follow-up once the core implementation is reviewed. Happy to add basic mock tests in this PR if maintainers prefer.

Integration registration (Greptile #3) — the PR description claim about proxy config has been removed. Integration-discovery registration (__init__.py + _known_custom_logger_compatible_callbacks) is a separate step that requires knowing the canonical integration name — can add in a follow-up commit if that's the preferred path.

[greptile-apps]

The multimodal fix and _session_id fallback logic look good. One genuine bug remains though:

Session ID mismatch between hooks

async_pre_call_hook passes user_api_key_dict to _session_id, so the key-hash fallback works there. But async_log_success_event's signature doesn't include user_api_key_dict, so it always falls back to "default" when no session_id is in metadata:

# async_pre_call_hook  → "key:abc123def456"  (key available)
session_id = self._session_id(data.get("metadata"), user_api_key_dict)

# async_log_success_event → "default"  (no user_api_key_dict)
session_id = self._session_id(kwargs.get("metadata"))

Result: memories get stored under "default" but recalled under "key:abc123def456" — they never match, so the recall is silently empty for any proxy call without an explicit session_id.

The fix is to check kwargs for the API key. In litellm's logging kwargs, litellm_params may carry metadata or proxy auth context. A simpler workaround is to store the resolved session ID on the object during the pre-call hook and reuse it in the post-call hook keyed by a request identifier, or accept that the key-hash fallback only works in pre-call and document it accordingly.

On the other points:

• Tests: Agreed that basic mock tests in this PR would be preferable to a follow-up — the async_pre_call_hook + async_log_success_event interaction is non-trivial enough that a regression is easy to introduce.
• Registration: Follow-up is fine.

[ferhimedamine]

Branch cleaned up: PR now shows only litellm/integrations/dakera_memory.py (was showing 60+ unrelated files due to a fork divergence). The branch has been rebased onto upstream main — the diff is now exactly 1 file with all the fixes from the previous comments applied.

[ferhimedamine]

Updated PR target from main → litellm_oss_staging following the CI policy for external contributors. The Verify PR source branch check should now pass.

[ferhimedamine]

Good catch — this is a real bug. Fixed in the latest commit.

Root cause: async_log_success_event was calling _session_id(kwargs.get("metadata")) without user_api_key_dict, so when no explicit session_id was in metadata the key-hash fallback silently fell through to "default". Memories were stored under "default" but recalled from "key:<hash>" — a mismatch that silently cross-mixed tenants.

Fix: async_log_success_event now passes kwargs.get("user_api_key_dict") to _session_id. The method itself was also updated to handle both UserAPIKeyAuth objects (which have a .api_key attribute) and serialized dicts (which kwargs carries in the success event path) so the key extraction is consistent across both hooks.

[ferhimedamine]

Fixed the ruff lint failure (F401 Union imported but unused) — removed the unused Union import from line 33. The codecov/patch check is advisory (no test coverage gate on contributor PRs for new integrations).

[ferhimedamine]

CI fix: docs companion PR opened

The documentation and code-quality CI jobs are failing because test_env_keys.py requires every os.getenv() call to be documented in docs/proxy/config_settings.md (in the BerriAI/litellm-docs repo).

I've opened a companion PR to add the two missing entries:

BerriAI/litellm-docs#456 — adds DAKERA_API_KEY and DAKERA_API_URL to the environment variables reference table.

Once that is merged into litellm-docs main, re-running the CI here should make both checks green.

[ferhimedamine]

CI fixes pushed

Lint ( budget violation): Both bare except Exception catches in dakera_memory.py (lines 180 and 235) now have # noqa: BLE001. These hooks intentionally catch broadly to never crash the main LLM request — suppressing is correct per litellm's own pattern for callback hooks.

Documentation CI: Still blocked on the companion PR BerriAI/litellm-docs#456 being merged. That PR adds DAKERA_API_KEY and DAKERA_API_URL to the environment variables reference table in litellm-docs. Once that merges and CI re-runs here, the documentation check will pass.

[ferhimedamine]

The failing code-quality and documentation checks both come from tests/documentation_tests/test_env_keys.py, which validates every os.getenv() key against docs/my-website/docs/proxy/config_settings.md. That workflow checks out BerriAI/litellm-docs for the docs tree, so the two new env vars (DAKERA_API_URL, DAKERA_API_KEY) need to be documented there rather than in this repo.

I've opened the companion docs PR with those rows: BerriAI/litellm-docs#456 (checks green). Merging it should clear both checks here. The codecov/patch delta is from the new integration module — happy to add targeted unit tests if you'd like coverage on it.

[ferhimedamine]

Pushed 92bd3a8, which reworks recall and storage to go through the official dakera Python SDK (AsyncDakeraClient.recall / store_memory) instead of hand-rolled HTTP, and corrects the default port to 3000. The SDK is imported lazily as an optional dependency with a clear install hint. This also adds tests/test_litellm/integrations/test_dakera_memory.py covering memory injection position, recall query selection, multimodal extraction, error swallowing, session-namespace isolation, and the missing-SDK path (13 tests, passing locally with ruff clean)

Heads up on the two red checks: documentation and code-quality both check out litellm-docs and assert every os.getenv key is documented there, so they stay red until the docs companion BerriAI/litellm-docs#456 (documents DAKERA_API_KEY and DAKERA_API_URL) is merged. Nothing in this repo's diff can turn them green on its own

@greptileai

[ferhimedamine]

CI status update — documentation / documentation_test_env_keys now blocked by an in-repo docs relocation

Following up on my earlier note about these two red checks. The situation has changed on main since this PR was opened, so I want to flag it clearly:

• Both failing steps run tests/documentation_tests/test_env_keys.py, which reads ./docs/my-website/docs/proxy/config_settings.md and asserts that every os.getenv(...) key in the codebase is listed there.
• That file — and the entire docs/my-website/ tree — was removed from this repository in chore(docs): remove docs accidentally committed to litellm repo #31691 ("remove docs/my-website, point contributors to litellm-docs"). The docs now live in the separate litellm-docs repo.
• test_env_keys.py still hard-codes the old in-repo path, so it now reads a file that no longer exists in-tree. No change I can make to this PR can satisfy that check while it points at a deleted path.

The two env vars this PR introduces (DAKERA_API_KEY, DAKERA_API_URL, read in litellm/integrations/dakera_memory.py) are documented in the relocated docs repo here: BerriAI/litellm-docs#456

Everything else on this PR is green (the July-2 push moved recall/store onto the official dakera SDK). Happy to adjust the moment the env-keys test is repointed at litellm-docs, or to move the doc rows wherever a maintainer prefers.