litellm ryan march 20

[ryan-crabbe]

ryans branch

circle ci: https://app.circleci.com/pipelines/github/BerriAI/litellm?branch=litellm_ryan_march_20

feat: add control plane for multi-proxy worker management
feature Add Audit Log Export to External Callbacks
refactor: internal users page destructive action modal
fix: tags not displaying after creation
fix: followup secrets leaking in detailed debug
polish: add click-to-copy icon on User ID in internal users table
polish: remove required asterisks from v3 login form fields

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Mar 21, 2026 10:43pm

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing litellm_ryan_march_20 (3e27ff1) with main (1986f10)}

[greptile-apps]

Greptile Summary

This is a semi-daily integration branch bundling several features and polish fixes: a new High Availability Control Plane architecture (docs + React diagram component), Audit Log Export to External Callbacks (dispatching audit events to CustomLogger subclasses like S3), secret-redaction improvements in _logging.py, and several UI fixes (tags dropdown refresh, click-to-copy User ID, login form polish).

Key changes:

• Audit log callbacks: litellm.audit_log_callbacks is a new list (List[CALLBACK_TYPES]) that dispatches StandardAuditLogPayload to registered loggers fire-and-forget via asyncio.create_task. Callbacks fire before the DB write so they succeed even when the DB is unavailable. The S3Logger batches audit logs under an audit_logs/YYYY-MM-DD/ S3 prefix.
• Secret redaction: key-name patterns (master_key, database_url, access_token, etc.) are added to _build_secret_patterns() to catch secrets leaked inside dict/config repr strings.
• Tags fix: CreateKey component now sources tag options from the useTags API hook instead of the stale getPredefinedTags(data) prop-derived state.
• P1: _dispatch_audit_log_to_callbacks silently skips callbacks that are plain Callable types — a valid member of CALLBACK_TYPES — with no warning, making misconfiguration very hard to diagnose.
• P2: The useTags mock in create_key_button.test.tsx uses an array for data, but TagListResponse is Record<string, Tag>; the mock accidentally works due to Object.values() behavior on arrays.

Confidence Score: 3/5

• Generally safe to merge but has a silent-failure path for Callable-typed audit log callbacks that could confuse users in production.
• The audit log dispatch logic has a clear gap: CALLBACK_TYPES includes Callable, but _dispatch_audit_log_to_callbacks silently ignores any callable that isn't a CustomLogger instance. There are no runtime errors, but a user who registers a callable audit callback will see no invocation and no warning, making it extremely hard to debug. All other changes (docs, UI polish, secret redaction, S3 batching) look correct and are well-tested.
• litellm/proxy/management_helpers/audit_logs.py (silent callable drop) and ui/litellm-dashboard/src/components/organisms/create_key_button.test.tsx (wrong mock shape for TagListResponse)

Important Files Changed

Filename	Overview
litellm/proxy/management_helpers/audit_logs.py	Core audit log callback dispatch logic added; callable-type callbacks in CALLBACK_TYPES are silently dropped without any warning log, and DB write now happens after callback dispatch (intentionally).
litellm/integrations/s3_v2.py	Adds async_log_audit_log_event to S3Logger, correctly batching audit logs using the existing log_queue and s3BatchLoggingElement infrastructure.
litellm/proxy/proxy_server.py	Wires audit_log_callbacks config key into the proxy config loader; correctly warns when store_audit_logs is not set.
litellm/_logging.py	Adds key-name-based secret redaction patterns for common sensitive config keys like master_key, database_url, access_token, etc.
ui/litellm-dashboard/src/components/organisms/create_key_button.tsx	Replaces stale predefinedTags state with live useTags hook; Object.values() usage is correct since TagListResponse is Record<string, Tag>.
ui/litellm-dashboard/src/components/organisms/create_key_button.test.tsx	Adds useTags mock and tags dropdown test, but the mock uses an array shape instead of the correct Record<string, Tag> shape defined by TagListResponse.

Sequence Diagram

sequenceDiagram
    participant PS as proxy_server.py
    participant AL as audit_logs.py
    participant CB as _dispatch_audit_log_to_callbacks
    participant S3 as S3Logger
    participant CL as CustomLogger (subclass)
    participant DB as Prisma DB

    PS->>AL: create_audit_log_for_update(request_data)
    AL->>AL: check store_audit_logs & premium_user
    AL->>AL: serialize updated_values/before_value to JSON
    AL->>CB: await _dispatch_audit_log_to_callbacks(request_data)
    CB->>CB: _build_audit_log_payload(request_data)
    loop for each callback in audit_log_callbacks
        alt callback is CustomLogger instance
            CB-->>CL: asyncio.create_task(async_log_audit_log_event(payload))
        else callback is str (e.g. "s3_v2")
            CB->>CB: _resolve_audit_log_callback(name) [cached]
            CB-->>S3: asyncio.create_task(async_log_audit_log_event(payload))
        else callback is Callable
            CB->>CB: silently skipped (no warning logged)
        end
    end
    AL->>DB: prisma_client.db.litellm_auditlog.create(...)
    S3-->>S3: append to log_queue, flush if batch_size reached

Loading

Comments Outside Diff (1)

1. ui/litellm-dashboard/src/components/organisms/create_key_button.test.tsx, line 1653-1661 (link)

   Test mock uses wrong shape for TagListResponse

   TagListResponse is defined as Record<string, Tag> (a keyed object), not an array. The mock below returns an array, which accidentally works at runtime because Object.values([...]) on an array just returns the same elements — but the mock does not accurately represent the real data contract.

   If the component code changes to access the data by key (e.g., tagsData["production"]) rather than using Object.values(), this test will silently pass with the wrong mock shape while production breaks.

   The mock should match the real TagListResponse shape:

_{Last reviewed commit: "fix: resolve mypy ty..."}

[greptile-apps]

Stale callback cache on config reload

_audit_log_callback_cache is a module-level dict that is never cleared. In proxy_server.py, when audit_log_callbacks is re-configured (e.g., on a hot-reload), litellm.audit_log_callbacks is reset to [] and rebuilt — but _audit_log_callback_cache is not invalidated. This means any string-named callback like "s3_v2" will continue resolving to the old cached instance (with stale bucket name, credentials, etc.) after a config reload.

To fix, add a cache-clearing step whenever litellm.audit_log_callbacks is reconfigured. One approach is to expose a _clear_audit_log_callback_cache() helper and call it from proxy_server.py:

def _clear_audit_log_callback_cache() -> None:
    """Invalidate the resolved-callback cache (call on config reload)."""
    _audit_log_callback_cache.clear()

And in proxy_server.py, before rebuilding audit_log_callbacks:

elif key == "audit_log_callbacks":
    from litellm.proxy.management_helpers.audit_logs import _clear_audit_log_callback_cache
    _clear_audit_log_callback_cache()
    litellm.audit_log_callbacks = []
    ...

[greptile-apps]

Warning message omits the LITELLM_STORE_AUDIT_LOGS env var alternative

create_audit_log_for_update checks litellm.store_audit_logs or get_secret_bool("LITELLM_STORE_AUDIT_LOGS"), so users can enable audit logs via environment variable without touching litellm_settings. The current warning message only mentions store_audit_logs: true, which could cause confusion for users who rely on the env-var approach.

Suggested change

	else:
	verbose_proxy_logger.warning(
	"'audit_log_callbacks' is configured but 'store_audit_logs' is not enabled. "
	"Audit log callbacks will not fire until 'store_audit_logs: true' is added to litellm_settings."
	)
	verbose_proxy_logger.warning(
	"'audit_log_callbacks' is configured but 'store_audit_logs' is not enabled. "
	"Audit log callbacks will not fire until 'store_audit_logs: true' is added to litellm_settings "
	"or the LITELLM_STORE_AUDIT_LOGS environment variable is set to 'true'."
	)

[greptile-apps]

Missing word boundaries — key-name patterns can over-redact

The key-name patterns (e.g. access_token, refresh_token, webhook_url) are not anchored with word boundaries or negative lookbehinds. This means a key such as oauth_access_token, last_access_token, or my_webhook_url would produce a partial match, e.g.:

"last_access_token": "my-value"
# becomes → "last_REDACTED"

The regex matches access_token" (consuming the trailing " of the key name), then : ", then my-value — leaving a dangling "last_ prefix and a broken closing " in the log line.

Consider adding a word-boundary or negative lookbehind to anchor each key name:

r"(?<!['\"\w])(?:master_key|database_url|db_url|connection_string|"
r"private_key|signing_key|encryption_key|"
r"auth_token|access_token|refresh_token|"
r"slack_webhook_url|webhook_url|"
r"database_connection_string|"
r"huggingface_token|jwt_secret)"
r"""['\"]?\s*[:=]\s*['\"]?[^\s,'\"})\]{}>]+""",

Note: the existing api[_-]?key pattern already sidesteps this by requiring {8,} characters for the value. The new patterns have no such minimum length guard either.

[greptile-apps]

Callable callbacks silently dropped without warning

CALLBACK_TYPES is defined as Union[str, Callable, "CustomLogger"] (see litellm/__init__.py), meaning a user can legally add a plain callable to audit_log_callbacks. However, _dispatch_audit_log_to_callbacks only handles CustomLogger instances and strings — if a callback is a Callable, resolved stays None and the entry is silently skipped with no log message whatsoever.

This is a silent failure: a user who registers a callable expecting it to be invoked will see no error and no invocation. At minimum, an else branch with a warning is needed:

for callback in litellm.audit_log_callbacks:
    try:
        resolved: Optional[CustomLogger] = callback if isinstance(callback, CustomLogger) else None
        if isinstance(callback, str):
            resolved = _resolve_audit_log_callback(callback)
            if resolved is None:
                verbose_proxy_logger.warning(
                    "Could not resolve audit log callback: %s", callback
                )
                continue
        elif not isinstance(callback, CustomLogger):
            verbose_proxy_logger.warning(
                "Unsupported audit_log_callback type %s (%r) — only CustomLogger instances "
                "and string names are supported.",
                type(callback).__name__,
                callback,
            )
            continue

        if isinstance(resolved, CustomLogger):
            task = asyncio.create_task(resolved.async_log_audit_log_event(payload))
            task.add_done_callback(_audit_log_task_done_callback)
    except Exception as e:
        ...