Add pyroscope for observability

[AlexsanderHamir]

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Type

🆕 New Feature

Changes

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Error		Feb 14, 2026 1:33am

[greptile-apps]

Greptile Overview

Greptile Summary

This PR adds optional Grafana Pyroscope continuous CPU profiling to the LiteLLM proxy, gated behind the LITELLM_ENABLE_PYROSCOPE environment variable. When enabled, it sends profiling data to a configured Pyroscope server. The implementation follows the existing pattern established by the Datadog tracer/profiler integration (_init_dd_tracer).

• Critical: pyroscope-io is not marked as optional = true in pyproject.toml, making it a required dependency for all litellm installations (including SDK-only users). This adds a native C extension (cffi) to every install and will fail on unsupported platforms.
• Missing documentation page: Both sidebars.js and config_settings.md reference proxy/pyroscope_profiling, but no corresponding .md file is included in the PR, resulting in broken doc links.
• No tests included: The PR checklist requires at least 1 test in tests/litellm/, but none are provided. A unit test for _init_pyroscope() (e.g., verifying it returns cleanly when disabled, raises on missing config) would be straightforward to add.
• Docstring inaccuracy: The method docstring claims PYROSCOPE_SERVER_ADDRESS defaults to http://localhost:4040 and PYROSCOPE_SAMPLE_RATE is a float, but the implementation requires the server address (no default) and parses sample rate as an integer.

Confidence Score: 2/5

• This PR has a dependency configuration bug that will affect all litellm installations and is missing required documentation and tests.
• The core Pyroscope integration logic is reasonable and follows existing patterns, but the missing optional = true flag on the pyroscope-io dependency is a significant packaging issue that would affect all users. Additionally, the missing documentation page will cause broken links, and no tests are provided as required by the contribution checklist.
• Pay close attention to pyproject.toml (missing optional flag) and docs/my-website/sidebars.js (broken doc reference).

Important Files Changed

Filename	Overview
pyproject.toml	pyroscope-io dependency is missing optional = true, making it a required dependency for all litellm installations instead of being proxy-only.
litellm/proxy/proxy_server.py	Adds _init_pyroscope() classmethod to ProxyStartupEvent. Logic is sound but logs an info-level message on every startup even when disabled, which is noisy. Missing documentation page referenced in sidebar and config docs.
poetry.lock	Adds pyroscope-io 0.8.16 to lock file. Reflects the incorrect optional = false from pyproject.toml. Also bumps Poetry version and litellm-proxy-extras version.
docs/my-website/docs/proxy/config_settings.md	Adds four new environment variable descriptions for Pyroscope configuration. References a documentation page (/proxy/pyroscope_profiling) that doesn't exist in this PR.
docs/my-website/sidebars.js	Adds proxy/pyroscope_profiling to sidebar under Alerting & Monitoring. The referenced doc page does not exist in this PR, which will cause a broken link.

Flowchart

flowchart TD
    A[Proxy Startup] --> B{LITELLM_ENABLE_PYROSCOPE=true?}
    B -- No --> C[Log disabled, return]
    B -- Yes --> D{import pyroscope}
    D -- ImportError --> E[Log warning: pyroscope-io not installed]
    D -- Success --> F{PYROSCOPE_APP_NAME set?}
    F -- No --> G[Raise ValueError - crash startup]
    F -- Yes --> H{PYROSCOPE_SERVER_ADDRESS set?}
    H -- No --> G
    H -- Yes --> I[Build configure_kwargs]
    I --> J{PYROSCOPE_SAMPLE_RATE set?}
    J -- Yes, valid --> K[Add sample_rate to kwargs]
    J -- Yes, invalid --> G
    J -- No --> L[pyroscope.configure]
    K --> L
    L --> M[Log success, profiling active]

Loading

_{Last reviewed commit: 0cbea29}

[greptile-apps]

_{5 files reviewed, 4 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Missing documentation page

The sidebar references proxy/pyroscope_profiling, and config_settings.md links to /proxy/pyroscope_profiling, but no corresponding documentation file (docs/my-website/docs/proxy/pyroscope_profiling.md) is included in this PR. This will cause a broken link in the docs site at build time or at runtime.

[greptile-apps]

Additional Comments (2)

litellm/proxy/proxy_server.py
Noisy info log when disabled

This logs at info level on every proxy startup, even though Pyroscope is disabled by default. The Datadog tracer (_init_dd_tracer) doesn't log anything when its feature flag is off. Consider changing this to debug level to avoid cluttering logs for the vast majority of users who don't use Pyroscope.

        if not get_secret_bool("LITELLM_ENABLE_PYROSCOPE", False):
            verbose_proxy_logger.debug(
                "LiteLLM: Pyroscope profiling is disabled (set LITELLM_ENABLE_PYROSCOPE=true to enable)."
            )
            return

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

---

litellm/proxy/proxy_server.py
Stale docstring

The docstring says PYROSCOPE_SERVER_ADDRESS (default http://localhost:4040) and PYROSCOPE_SAMPLE_RATE (float, no default), but the implementation requires PYROSCOPE_SERVER_ADDRESS (no default — raises ValueError if missing) and parses PYROSCOPE_SAMPLE_RATE as an integer via int(float(...)). The docstring should match the actual behavior.

        Sends profiles to PYROSCOPE_SERVER_ADDRESS (required, no default).
        Optional: PYROSCOPE_SAMPLE_RATE (integer, no default) to set the sample rate.

[AlexsanderHamir]

@greptileai

[greptile-apps]

Greptile Overview

Greptile Summary

Adds optional Grafana Pyroscope continuous CPU profiling to the LiteLLM proxy, gated behind the LITELLM_ENABLE_PYROSCOPE environment variable (off by default). The implementation follows the established pattern used by the Datadog tracer integration in ProxyStartupEvent.

• Adds _init_pyroscope() classmethod to ProxyStartupEvent with validation for required env vars (PYROSCOPE_APP_NAME, PYROSCOPE_SERVER_ADDRESS), optional PYROSCOPE_SAMPLE_RATE, and graceful ImportError handling when the package is not installed.
• Adds pyroscope-io as an optional dependency with sys_platform != 'win32' marker, included in the proxy extras group.
• Includes new documentation page (pyroscope_profiling.md) with setup instructions and environment variable reference.
• Adds mock-only unit tests covering disabled state, missing config validation, and sample rate parsing (missing ImportError branch coverage).
• Minor style issues: info-level log on every startup when disabled (should be debug), and extraneous blank lines in pyproject.toml.

Confidence Score: 4/5

• This PR is safe to merge — the feature is opt-in, off by default, and isolated to startup initialization with no impact on the request path.
• The implementation is clean, follows existing patterns (DD tracer), and is fully gated behind an env var. The dependency is correctly marked as optional in pyproject.toml. Only minor style issues found (log level, extra blank lines, missing test for ImportError path). The lock file inconsistency (optional=false) was already flagged in a previous review thread.
• Pay attention to poetry.lock — the optional = false inconsistency with pyproject.toml was noted in a prior review thread and should be resolved before merge.

Important Files Changed

Filename	Overview
litellm/proxy/proxy_server.py	Adds _init_pyroscope() classmethod to ProxyStartupEvent that optionally configures Pyroscope CPU profiling at startup. Clean implementation following the existing DD tracer pattern, with proper validation and ImportError handling. Minor concern about info-level logging when disabled.
pyproject.toml	Adds pyroscope-io as an optional dependency with sys_platform != 'win32' marker and includes it in the proxy extra group. Two extraneous blank lines introduced between sections.
poetry.lock	Lock file adds pyroscope-io 0.8.16 with optional = false which is inconsistent with pyproject.toml (optional = true). Lock file was regenerated with a different Poetry version (2.2.1 vs 2.1.4). Already flagged in previous review thread.
tests/test_litellm/proxy/test_pyroscope.py	Good set of mock-only unit tests covering disabled path, missing config validation, and sample rate parsing. Missing test for the ImportError branch when pyroscope-io is not installed.
docs/my-website/docs/proxy/pyroscope_profiling.md	New documentation page for Pyroscope profiling with clear setup instructions, environment variable reference, and platform notes.
docs/my-website/sidebars.js	Adds pyroscope_profiling entry to the monitoring sidebar section alongside alerting, pagerduty, and prometheus.
docs/my-website/docs/proxy/config_settings.md	Documents the four new Pyroscope-related environment variables in the config settings reference table.

Sequence Diagram

sequenceDiagram
    participant FastAPI as FastAPI Lifespan
    participant PSE as ProxyStartupEvent
    participant Env as Environment Variables
    participant Pyroscope as pyroscope-io

    FastAPI->>PSE: proxy_startup_event()
    PSE->>PSE: _init_pyroscope()
    PSE->>Env: get_secret_bool("LITELLM_ENABLE_PYROSCOPE")
    alt Disabled (default)
        Env-->>PSE: False
        PSE-->>PSE: return (log debug)
    else Enabled
        Env-->>PSE: True
        PSE->>Pyroscope: import pyroscope
        alt ImportError
            Pyroscope-->>PSE: ImportError
            PSE-->>PSE: log warning, continue
        else Imported OK
            PSE->>Env: PYROSCOPE_APP_NAME
            PSE->>Env: PYROSCOPE_SERVER_ADDRESS
            alt Missing required config
                PSE-->>FastAPI: raise ValueError (crash startup)
            else Config valid
                PSE->>Env: PYROSCOPE_SAMPLE_RATE (optional)
                PSE->>Pyroscope: pyroscope.configure(app_name, server_address, tags, sample_rate)
                Pyroscope-->>PSE: Profiling started
                PSE-->>PSE: log info
            end
        end
    end

Loading

_{Last reviewed commit: e604a04}

[greptile-apps]

_{7 files reviewed, 3 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Extraneous blank lines added

Two extra blank lines were introduced here. The rest of the file uses single blank lines between sections.

Suggested change

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

[greptile-apps]

Missing test for ImportError branch

The _init_pyroscope method has an except ImportError branch (line 5879 of proxy_server.py) that logs a warning when pyroscope-io is not installed but the feature is enabled. This code path is not covered by any test. Consider adding a test that patches sys.modules to raise ImportError for pyroscope when the feature is enabled, to verify the warning is logged and the proxy continues without crashing.

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}