Repository navigation
feat(azure_storage): upload audit logs to Azure Storage via audit_log_callbacks - #43838
Conversation
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
|
bugbot run |
|
|
|
1e97f3b to
dc12de3
Compare
|
bugbot run |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Merging this PR will not alter performance
|
dc12de3 to
56e21af
Compare
|
@greptileai please review the rebased head 56e21af (single commit on current main; the client expiry change is gone from the diff because main already fixed it in #43082) |
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 56e21af. Configure here.
e0d6ece to
3b73c6c
Compare
|
@greptileai please review the current head 3b73c6c, the account key upload path now shares one helper with the request log path |
…_callbacks Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
3b73c6c to
ee4c659
Compare
TLDR
Problem this solves:
audit_log_callbacks: ["azure_storage"]loads the logger but writes nothingHow it solves it:
AzureBlobStorageLogger.async_log_audit_log_eventuploads each audit event as JSONaudit_logs/YYYY-MM-DD/HH-MM-SS_<audit id>.jsonin the configured file systemUser Flow
Before: an admin who routes audit logs to Azure Storage finds the container empty
store_audit_logs: trueandaudit_log_callbacks: ["azure_storage"]and restarts the proxyAfter: the same actions produce one JSON file per audit event in Azure Storage
audit_logs/<today>/<time>_<audit id>.jsonfor each of the four rows, with the key value masked the same way the DB row isLinear ticket
Resolves LIT-6128
Files changed
litellm/integrations/azure_storage/azure_storage.pyasync_log_audit_log_eventplus_upload_json_to_file_path, which writes one file through the account key SDK client or the Entra ID DFS REST calls already used for request logs. The account key SDK sequence (create_file,append_data,flush_data) moves into_upload_bytes_with_account_keyso the request log path and the audit path share it instead of duplicating itAudit events are uploaded one file per event straight from the audit dispatcher task, not through the request log queue, so request log batches keep their current shape and file naming. Upload failures propagate to the dispatcher's task done callback, which already logs them. The audit payload reaching the callback is the
LiteLLM_AuditLogsmodel after itsmask_api_keysvalidator ran, sokeyfields arrive assk-r****yFFQand never as the raw virtual keyPre-Submission checklist
uv run pytest tests/unit/<your_test_file>.py -vScreenshots / Proof of Fix
Regime: a real Azure Storage account (ADLS Gen2, HNS on,
core.windows.net), real Postgres, the proxy started fromlitellm/proxy/proxy_cli.pywithPYTHONPATHset to the checkout under test. No mocks on the audit path (the only model is amock_responsemodel, used once to show request logs still upload). Before and After use the same config and the same four management calls against a fresh, empty file system per leg. Secrets live only in the proxy's environment, the curl lines below omit theAuthorizationheader and the raw generated key is shown as<new key>Shared config:
Shared commands (
$His-H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json"):Before (6fcf9ef, main without this PR)
Account key
Initialized Audit Log Callbacks - ['azure_storage']200 OK(POST /key/generate,POST /key/update,POST /key/<new key>/regenerate,POST /team/newin the access log)lit6128-main-keywith the DFS REST API (?resource=filesystem&recursive=true) returns no paths. The audit rows exist in Postgres, nothing reaches AzureAfter (ee4c659, the same single commit as 3b73c6c rebased onto main after #45845, the proof below was captured at 3b73c6c)
Account key
AZURE_STORAGE_ACCOUNT_KEYset, startup log printsInitialized Audit Log Callbacks - ['azure_storage']lit6128-pr2-keyshows one file per audit row{ "id": "30f941d4-6150-4d97-a861-4b6301d13a6c", "action": "created", "table_name": "LiteLLM_VerificationToken", "object_id": "1dd928d2744622ce4553d8214dbe62315cb85d7eaab640357d84bee2906c4754", "updated_values": {"key": "sk-r*****************yFFQ", "key_alias": "pr2-***-key", "team_id": null} }grep -c "Traceback"on the proxy log for this run returns 0Service principal (Entra ID, no account key set)
AZURE_STORAGE_TENANT_ID,AZURE_STORAGE_CLIENT_ID,AZURE_STORAGE_CLIENT_SECRETand noAZURE_STORAGE_ACCOUNT_KEY, so uploads go through the DFS REST pathlit6128-pr2-spshows one file per audit row{"id": "6f6d6050-6b4d-4364-bada-dfbbd023fe79", "action": "created", "table_name": "LiteLLM_TeamTable", "changed_by_api_key": "litellm_proxy_master_key"}grep -c "Traceback"on the proxy log returns 0Both listings and both payload reads in one terminal capture, with the count-only secret scrub of the proof text and both proxy logs at the bottom (account key, client secret, master key and every raw generated key all 0 hits):
Request logs keep working through the shared account key helper
curl -s $H localhost:4002/chat/completions -d '{"model":"mock-gpt","messages":[{"role":"user","content":"hi"}]}'returns{"id": "chatcmpl-384c5fbb-e5f3-4654-a316-3e2f7cedbfad", "content": "hello from mock"}AzureBlobStorageLogger - about to flush 1 eventsthenSuccessfully uploaded and wrote to 2026-10-10/chatcmpl-384c5fbb-e5f3-4654-a316-3e2f7cedbfad.jsonlit6128-pr2-keyreturns{"id": "chatcmpl-384c5fbb-e5f3-4654-a316-3e2f7cedbfad", "model": "openai/mock-gpt", "status": "success"}, so the request log file name and location are unchangedAdmin UI
1dd928d2...) are the four account key files above, the rows at 19:16:46 and 19:16:47 (object idf9664f4e...) are the four Entra ID filesThe dashboard has no control for choosing audit log destinations for any callback (no reference to
audit_log_callbacksunderui/litellm-dashboard/src, S3 audit export is also config only), so this PR adds noneType
New Feature
Caveats (if any)
Low
Link to Devin session: https://app.devin.ai/sessions/373eb10be9964560bd136af82e162bd9
Open in Devin Desktop: https://app.devin.ai/desktop/session/373eb10be9964560bd136af82e162bd9?variant=devin
Requested by: @yassin-berriai