test(e2e): guard destructive spend-log truncate behind explicit opt-in - #33725
test(e2e): guard destructive spend-log truncate behind explicit opt-in#33725devin-ai-integration[bot] wants to merge 1 commit into
Conversation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
|
|
Greptile SummaryThis PR guards the destructive
Confidence Score: 4/5Safe to merge once the bob_the_builder.remediate scoping is resolved; the core opt-in guard and module restructuring are correct. The bob_the_builder.remediate(session) call has been accidentally pulled inside the E2E_RESET_SPEND_LOGS=1 early-return guard, silently disabling it in pipelines that don't set the new env-var. tests/e2e/conftest.py — the bob_the_builder.remediate call should be moved outside the E2E_RESET_SPEND_LOGS guard.
|
| Filename | Overview |
|---|---|
| tests/e2e/conftest.py | Adds E2E_RESET_SPEND_LOGS=1 opt-in guard and cleans up the sys.path hack, but now gates bob_the_builder.remediate behind the new env-var check as an unintended side-effect. |
| tests/e2e/e2e_db.py | New shared module that re-homes reset_spend_logs from the spend_tracking subdirectory; implementation is unchanged and the lazy psycopg import keeps startup cost minimal. |
| tests/e2e/quota_management/spend_tracking/spend_e2e_client.py | Removes the now-relocated reset_spend_logs function and trims the unused os import and all entry; no callers remain on the old path. |
Comments Outside Diff (1)
-
tests/e2e/conftest.py, line 121-135 (link)The
bob_the_builder.remediate(session)call is now gated behindE2E_RESET_SPEND_LOGS=1, but that was not its original behaviour. Previously it ran whenever any e2e test body had executed; now it only runs when the operator also opts in to the destructive spend-log reset. Any CI pipeline that doesn't set this env-var will silently stop running remediation after every e2e session.
Reviews (1): Last reviewed commit: "test(e2e): guard destructive spend-log t..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
Closing in favor of #33751, which supersedes this. Same fix (opt-in env guard on the destructive spend-log truncate plus dropping the sys.path hack via a top-level e2e_db module), and additionally extracts the guard into a dependency-injected pure helper with an injected-spy regression test, restores the bob_the_builder remediation gating, and includes a real-Postgres before/after proof |
Relevant issues
Linear ticket
Resolves LIT-4555
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
@greptileaito re-request a review after pushing changes)Screenshots / Proof of Fix
Harness-internal change with no live-proxy behavior to demonstrate; see the QA runbook below for the gates that were run
Type
🧹 Refactoring
✅ Test
Changes
tests/e2e/conftest.py'spytest_sessionfinishtruncatedLiteLLM_SpendLogsagainst whateverDATABASE_URLresolved to, gated only by "an e2e test body ran". Pointed at a shared or staging DB, a routine local run would wipe real spend data. It also reached the truncate helper through asys.path.insertintoquota_management/spend_tracking, a cross-suite import-by-path hack it then had to unwind in afinallyThis adds a real guard and requires an explicit opt-in before anything destructive happens. The truncate now runs only when
E2E_RESET_SPEND_LOGS=1is set, and the existing "a spend test actually ran" stash check stays as an additional necessary condition on top of the opt-in. Without the env var the session-finish hook returns early and leaves the DB untouchedThe
sys.pathhack is gone.reset_spend_logsmoved into a new shared moduletests/e2e/e2e_db.pythat sits alongsidee2e_config.pyandlifecycle.py, so it imports normally withfrom e2e_db import reset_spend_logsfrom both the top-level conftest and anywhere else. The implementation is unchanged (sameTRUNCATE TABLE "LiteLLM_SpendLogs", sameDATABASE_URLdefault).spend_e2e_client.pyno longer defines it or lists it in__all__; nothing else importedreset_spend_logsfrom that module, so there is no duplicated logic and no other caller relied on the old pathQA runbook
Harness-internal change; there is no new live-proxy behavior to reproduce, so the validation is the harness gates rather than a proxy call
make lint-e2e-basedpyright-> 0 errors, 0 warnings, 0 notestests/e2e,PYTHONPATH=. python -m coverage_registry.collector-> still runs and prints the coverage tabletests/e2e,pytest --collect-only -qq -p no:cacheprovider .-> collects with no import or fixture errorsmake pre-commit-> passes on the staged changesFinal Attestation
Link to Devin session: https://app.devin.ai/sessions/51a83116b250445cba40d78568532b6a
Requested by: @yassin-berriai