Skip to content

ci(lint): run secret-scan pytest without syncing the project env - #35827

Open
mateo-berri wants to merge 1 commit into
litellm_internal_stagingfrom
litellm_secret_scan_no_project
Open

ci(lint): run secret-scan pytest without syncing the project env#35827
mateo-berri wants to merge 1 commit into
litellm_internal_stagingfrom
litellm_secret_scan_no_project

Conversation

@mateo-berri

@mateo-berri mateo-berri commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • secret-scan flakes by hitting its five minute job timeout
  • uv run --frozen first syncs 146 packages and builds the litellm wheel
  • the test itself needs only pytest and the stdlib

How it solves it:

  • run pytest with --no-project so nothing but pytest installs
  • environment setup drops from minutes to seconds

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Before, at 258d154 on PR #35807: the secret-scan job spent its whole budget installing an environment the test never uses, then died at the five minute cap before the single collected test ran (cancelled run)

20:55:04 Creating virtual environment at: .venv
20:55:04    Building litellm @ file:///home/runner/work/litellm/litellm
20:57:24       Built litellm @ file:///home/runner/work/litellm/litellm
20:57:24 Installed 146 packages in 224ms
20:57:27 collecting ... collected 1 item
20:57:28 ##[error]The operation was canceled.

After, at d7ef033, the same invocation CI now runs, from a shell with no project environment:

$ time uv run --no-project --with 'pytest==9.0.2' pytest tests/litellm/test_no_hardcoded_secrets.py -v
Installed 5 packages in 14ms
collecting ... collected 1 item
tests/litellm/test_no_hardcoded_secrets.py::test_no_hardcoded_basic_auth_secrets PASSED [100%]
============================== 1 passed in 1.35s ===============================
uv run --no-project --with 'pytest==9.0.2' pytest  -v  1.02s user 0.70s system 46% cpu 3.648 total

And the live end-to-end proof, since pull_request runs use the workflow file from the PR branch: this PR's own secret-scan check at d7ef033 passed in 1m8s, well clear of the five minute cap

Type

🚄 Infrastructure

Changes

tests/litellm/test_no_hardcoded_secrets.py walks the source tree with os.walk and regexes; it imports nothing from litellm, and no conftest exists anywhere on its path (repo root, tests/, or tests/litellm/). The job's uv run --frozen nevertheless synced the full project environment, and on a cold-cache runner building the litellm wheel alone took over two minutes, which is how a job capped at timeout-minutes: 5 got cancelled before its one test executed. Swapping --frozen for --no-project makes uv skip the project entirely and spin up an ephemeral environment containing just pytest, so the job no longer races its own timeout. The ggshield step after it already runs isolated through uv tool run and is untouched

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

@mateo-berri
mateo-berri requested a review from a team August 4, 2026 21:17
@greptile-apps

greptile-apps Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR speeds up the secret-scan CI job by running its self-contained pytest check outside the project environment.

  • Replaces uv run --frozen with uv run --no-project.
  • Retains the explicitly pinned pytest installation and leaves the ggshield scan unchanged.

Confidence Score: 5/5

The PR appears safe to merge because the targeted test is self-contained and remains runnable with the isolated pytest environment.

The changed invocation still provides pytest, preserves repository-root execution and pytest configuration discovery, and the test derives its source-tree scan path independently of the project environment.

Important Files Changed

Filename Overview
.github/workflows/test-linting.yml Runs the standalone secret-scan test in a minimal ephemeral pytest environment, avoiding an unnecessary full project synchronization without changing the test’s scan scope or dependencies.

Reviews (1): Last reviewed commit: "ci(lint): run secret-scan pytest without..." | Re-trigger Greptile

@codecov

codecov Bot commented Aug 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant