Skip to content

feat: send litellm_metadata to Gray Swan Cygnal - #20297

Closed
agrattan0820 wants to merge 2 commits into
BerriAI:mainfrom
GraySwanAI:alexander/grayswan-litellm-metadata
Closed

feat: send litellm_metadata to Gray Swan Cygnal#20297
agrattan0820 wants to merge 2 commits into
BerriAI:mainfrom
GraySwanAI:alexander/grayswan-litellm-metadata

Conversation

@agrattan0820

Copy link
Copy Markdown
Contributor

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.
  • Branch creation CI run
    Link:

  • CI run for the last commit
    Link:

  • Merge / cherry-pick CI run
    Links:

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

@vercel

vercel Bot commented Feb 2, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Feb 2, 2026 9:59pm

Request Review

@agrattan0820 agrattan0820 changed the title feat: send litellm_metadata to Gray Swan Cygnal feat: send litellm_metadata to Gray Swan Cygnal Feb 2, 2026
@greptile-apps

greptile-apps Bot commented Feb 2, 2026

Copy link
Copy Markdown
Contributor

Greptile Overview

Greptile Summary

This PR adds the ability to send LiteLLM metadata (user identification and request context) to the Gray Swan Cygnal API endpoint, enabling better tracking and monitoring capabilities.

Key Changes

  • Modified _prepare_payload to accept an optional request_data parameter and call the new metadata extraction method
  • Added _extract_litellm_metadata method that extracts user identification fields (user_id, team_id, end_user_id, org_id), API key information (api_key_hash, api_key_alias, user_email), and request context (litellm_call_id, model, openai_user, request_route)
  • The metadata is included in the payload sent to Gray Swan's /cygnal/monitor endpoint under the litellm_metadata key

Impact

The change is backward compatible - metadata extraction only occurs when request_data is provided, and the litellm_metadata field is only added to the payload if metadata is found. This enables Gray Swan to receive user context without breaking existing functionality.

Confidence Score: 4/5

  • This PR is safe to merge with minimal risk - it adds optional metadata enrichment without changing core logic
  • The change is well-isolated and backward compatible. The new _extract_litellm_metadata method safely extracts metadata using dictionary .get() methods with fallbacks, avoiding KeyErrors. The metadata is only added to the payload when present, maintaining backward compatibility. The implementation follows existing patterns in the codebase and doesn't create new database objects or Router instances in the request path per the custom instructions.
  • No files require special attention

Important Files Changed

Filename Overview
litellm/proxy/guardrails/guardrail_hooks/grayswan/grayswan.py Added metadata extraction to include LiteLLM user identification and request context in Gray Swan API payload

Sequence Diagram

sequenceDiagram
    participant Client
    participant Guardrail as GraySwanGuardrail
    participant ExtractMetadata as _extract_litellm_metadata
    participant GraySwanAPI as Gray Swan /cygnal/monitor

    Client->>Guardrail: apply_guardrail(inputs, request_data)
    Guardrail->>Guardrail: Convert texts to messages
    Guardrail->>Guardrail: Get dynamic_body params
    Guardrail->>Guardrail: _prepare_payload(messages, dynamic_body, request_data)
    Guardrail->>ExtractMetadata: Extract metadata from request_data
    ExtractMetadata->>ExtractMetadata: Get litellm_call_id
    ExtractMetadata->>ExtractMetadata: Extract user_id, team_id, end_user_id, org_id
    ExtractMetadata->>ExtractMetadata: Extract api_key_hash, api_key_alias, user_email
    ExtractMetadata->>ExtractMetadata: Extract model, openai_user
    ExtractMetadata-->>Guardrail: Return litellm_metadata dict
    Guardrail->>Guardrail: Add litellm_metadata to payload
    Guardrail->>GraySwanAPI: POST /cygnal/monitor with enriched payload
    GraySwanAPI-->>Guardrail: Response with violation scores
    Guardrail-->>Client: Return processed result
Loading

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 file reviewed, no comments

Edit Code Review Agent Settings | Greptile

@agrattan0820

Copy link
Copy Markdown
Contributor Author

Duplicate of #19837

@agrattan0820
agrattan0820 deleted the alexander/grayswan-litellm-metadata branch February 3, 2026 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant