chore(sec): scope pip-audit to project-effective deps, not the whole venv#392
Conversation
…venv
Previously the workflow invoked `pip-audit --skip-editable`, which scans
every package in the CI venv — including bootstrap tooling (pip,
setuptools, wheel, virtualenv) that the server never imports at runtime.
The workflow's own comments already acknowledged this as the intent
("venv-bootstrap CVEs don't count against the project") and tried to
fix it by upgrading pip + setuptools before the scan. That is a no-op
when there is no fixed version published — exactly what happened on
2026-04-24 with pip 26.0.1 / CVE-2026-3219 (GHSA-58qw-9mgm-455v), which
reds every subsequent PR against main.
Fix: make the exclusion explicit by scope rather than by CVE allowlist.
`pip list --format freeze` captures the full resolved dep tree, a grep
strips the four bootstrap packages + the editable project, and
`pip-audit --disable-pip --no-deps -r scan-requirements.txt` scans the
rest. Every project dep (direct + transitive) plus pip-audit's own deps
stay in scope. If any of them picks up a CVE, the scan still fails.
Scope decisions are package-level and reviewable in the workflow.
CVE-level `--ignore-vuln` entries are explicitly called out as a last
resort in the workflow comment — they are how vuln backlogs accumulate.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
cmeans
left a comment
There was a problem hiding this comment.
[QA] QA Failed — CHANGELOG structural issues + stale PR-body scope claim
Scope logic is sound and empirically validated. Failing on CHANGELOG classification/structure + one doc-drift issue. All three are CHANGELOG/PR-body edits; no code changes needed.
Verification (all 5 PR-body steps pass)
Re-ran independently in this QA session, not just relying on CI:
- CI
auditpasses — run 24904283778, exit 0, "No known vulnerabilities found". - Scope count — CI log shows
91 scan-requirements.txt, excluded list is exactlymcp-awareness-server==0.18.2, pip==26.0.1, setuptools==82.0.1, virtualenv==21.2.4(wheel not installed on thesetup-python@v6runner, which is expected — grep just matches zero lines for it, no effect). - No CVE-level ignores — confirmed via
git diff main..HEAD: no--ignore-vuln, no.pip-audit-ignores. - Pip CVE out of scope (immediate unblock) — CI "Excluded" log lists
pip==26.0.1; scan does not report CVE-2026-3219. Exclusion is by package name, not by CVE ID. - Project-dep CVE would still fail — reproduced locally on a fresh Py3.12 venv (
pip install -e ".[dev]" + pip-audit==2.10.0):- Scoped run (bootstrap stripped) →
EXIT=0, "No known vulnerabilities found". - Same venv, pip-audit run without the grep (pip in scope) →
EXIT=1,pip 26.0.1 CVE-2026-3219reported. - Confirms the scope decision is load-bearing and inverts correctly.
- Scoped run (bootstrap stripped) →
- Grep anchoring —
^(pip|setuptools|wheel|virtualenv)==is safe againstpip-audit,pip-tools,setuptools-scm,setuptools-rust,virtualenv-clone, etc. No project or transitive dep in the current tree collides.
Findings
1. PR body scope block is stale (substantive — doc drift).
The ## Scope code block claims:
.github/workflows/pip-audit.yml | 36 ... --------
CHANGELOG.md | 3 +++
2 files changed, 31 insertions(+), 8 deletions(-)
Actual git diff main..HEAD --stat:
.github/workflows/pip-audit.yml | 58 ...---------
CHANGELOG.md | 3 ++-
2 files changed, 48 insertions(+), 13 deletions(-)
Delta is +17/-5. This was already stale on the closed #390 and carried over unedited. Fix: update the code block to the actual numbers.
2. CHANGELOG has two ### Changed sections under ## [Unreleased] (substantive — Keep-a-Changelog structure).
The PR renames the existing ### Security heading to ### Changed (to host the new pip-audit entry). But a separate ### Changed heading already existed below ### Added for the CONTRIBUTING.md expansion entries. Current structure under ## [Unreleased]:
### Fixed
### Changed ← renamed from Security; holds 9 prior security-tooling entries + new pip-audit entry
### Added
### Changed ← pre-existing; holds CONTRIBUTING.md section expansions
Keep-a-Changelog expects each category once per release. Two ### Changed sections under the same version header is a structural violation. Fix: consolidate into one ### Changed, or restore the original ### Security heading and put the new entry elsewhere (see #3).
3. CHANGELOG: the ### Security → ### Changed rename demotes a cluster of prior security-tooling entries (substantive).
The rename moves these entries out of ### Security and into ### Changed alongside the new pip-audit scope change:
- Semgrep static analysis + 3 custom rules (closes #363)
- Trivy Dockerfile + image CVE scan (closes #370)
- Original pip-audit workflow (closes #369)
- Gitleaks pre-commit + CI gate (closes #367)
- RLS property-based fuzz — R4 (closes #364, part of #359)
- RLS background-thread + pool coverage — R2 (closes #362, part of #359)
- RLS migration-safety test — R3 (closes #360, part of #359)
- Third-party GitHub Actions pinned to commit SHAs
- Extended RLS harness coverage — R1 (closes #361, part of #359)
All of these closed security-tracking issues and belong under ### Security. The new pip-audit-scope change is itself a security-adjacent fix (scoping a CVE scanner correctly). Fix options (either works): (a) keep ### Security and put the new pip-audit entry under it — cleanest, and naturally resolves finding #2; or (b) leave the new entry under ### Fixed (treating the scope bug as a fix), restoring ### Security intact for everything else.
Scope / design feedback (not blocking)
None. Design is tight; the empirical inversion test confirms the load-bearing choice.
Verdict: QA Failed. All three findings are CHANGELOG/PR-body edits — no workflow or code change needed. Once addressed, ping for round 2.
|
Applying QA Failed — 3 substantive CHANGELOG/PR-body findings in the review above. Core workflow change is correct and empirically validated; findings are docs-only edits. |
…r it Addresses QA findings #2 and #3 on PR #392: - Finding #2 (Keep-a-Changelog structure): the cherry-picked content renamed `### Security` to `### Changed` under `## [Unreleased]`, but `### Changed` already existed below `### Added` for the CONTRIBUTING.md section expansions. That left two `### Changed` headings under a single version — Keep-a-Changelog expects each category once per release. - Finding #3 (entries demoted): the rename pushed nine prior entries (Semgrep, trivy, pip-audit, gitleaks, four RLS harness items, third- party action pinning) out of `### Security` even though each of them closed a security-tracking issue. Those entries belong under Security. Fix: revert the heading to `### Security`. The new pip-audit-scope entry stays under that same heading — scoping a CVE scanner correctly IS a security-adjacent fix, so it belongs with the cluster. `## [Unreleased]` now has Fixed / Security / Added / Changed, each exactly once.
[Dev] Round 2 — addressing QA findingsThanks for the thorough verification. All three findings were doc-only edits; no workflow or code change needed. Finding resolutions1. PR body scope block stale — Fixed. Updated the Also updated the checklist line from 2. Two 3. Commit: Verification
Label transitionRemoving |
cmeans
left a comment
There was a problem hiding this comment.
[QA] Round 2 — Ready for QA Signoff
All three round-1 findings resolved with a CHANGELOG-only commit (1dc778d). Workflow file is byte-identical to round 1 — round-1 empirical validation carries over verbatim.
Fix verification
- Finding 1 (stale Scope block) — PR-body
## Scopenow reads2 files changed, 47 insertions(+), 12 deletions(-), matchinggit diff main..HEAD --statexactly. - Finding 2 (duplicate
### Changed) —[Unreleased]now has exactly one of each heading:### Fixed,### Security,### Added,### Changed. Keep-a-Changelog structure restored. - Finding 3 (Security entries demoted) —
### Securityheading restored intact; new pip-audit-scope entry placed under it alongside Semgrep/trivy/gitleaks/RLS/action-SHA-pinning entries. No reclassification of prior work.
Round-2 CI re-run (run 24904931414)
audit: SUCCESS, "No known vulnerabilities found"Packages in scan scope: 91 (unchanged from round 1)- Excluded:
mcp-awareness-server==0.18.2, pip==26.0.1, setuptools==82.0.1, virtualenv==21.2.4(unchanged) - All other checks green: CI (lint/typecheck/test 3.10–3.14), CodeQL, gitleaks, semgrep, QA Gate, codecov, CLA.
Carry-forward from round 1 (workflow file git diff d0f94ff..1dc778d -- .github/workflows/pip-audit.yml = 0 lines)
- Scope decision is load-bearing: local inverse test confirmed pip-in-scope → exit 1 with CVE-2026-3219, pip-excluded → exit 0.
- Grep anchoring
^(pip|setuptools|wheel|virtualenv)==is safe againstpip-audit,pip-tools,setuptools-scm/rust,virtualenv-clone, etc. - No CVE-level ignores; exclusion is by package name.
Verdict: Ready for QA Signoff. Zero new findings. Over to maintainer for QA Approved.
|
Applying Ready for QA Signoff — all 3 round-1 findings resolved in CHANGELOG-only commit |
Addresses QA round 1 findings on PR #393. Finding #1 (substantive — undercount): compose_summary used len(fired_intentions) for the summary line. fired_intentions is capped at 10 by generate_briefing, so any backlog of 11+ fired intentions silently reported as "10 intentions ready" regardless of the actual count. Reproduced in QA session with 15 seeded: summary said "10" while evaluation.intentions_fired correctly said 15. Impact at deploy time: the PR #392 deployer note flags 20+ accumulated fired handoffs on the production owner; every post-deploy briefing would have under-reported by 10. Fix: pull the total from briefing["evaluation"]["intentions_fired"], fall back to list length only when evaluation block is absent. Finding #2 (substantive — test coverage gap): the cap test asserted `sum(f.urgency == "high") == 3` which would also pass if the sort key flipped sign (high-urgency entries still in the top 10, just at slots 7-9). Replaced with a strict slot check — `[f.urgency for f in fired[:3]] == ["high", "high", "high"]` plus an `all(... "normal")` for slots 3-9. A sign flip on the urgency sort key now fails the test immediately. Also adds test_briefing_summary_reports_total_fired_count_not_capped as an explicit regression guard for finding #1 — seeds 15 fired, asserts the summary contains "15 intentions ready" and not "10 intentions ready".
…#394) ## Linked issue Fixes # none — version-stamp release, not tracked by a feature issue. ## Summary Version stamp release for v0.18.3 (patch, 0.18.2 → 0.18.3). Renames `[Unreleased]` → `[0.18.3] - 2026-04-24`, adds comparison link, bumps `pyproject.toml`. No code changes. Scope delta since v0.18.2 (13 commits, 1 runtime change): | Category | PRs | |---|---| | Runtime behavior (user-visible) | **#393** — briefing surfaces manually-fired intentions | | CI / security tooling (no runtime change) | #392 pip-audit scope fix, #386 Semgrep, #385 trivy, #382 pip-audit baseline, #380 gitleaks, #358 pinned action SHAs | | Test harness (no runtime change) | #379 R4 hypothesis-fuzz RLS, #377 R2 background-thread RLS, #373 R3 migration-safety RLS, #372 R1 extended RLS, #375 caplog flake fix | | Docs | #357 PR template + CONTRIBUTING expansion | Patch bump is correct: the one runtime change (#393) is a bug fix with additive briefing fields (`urgency`, `updated`) — no API break, no deprecations. ## Scope ``` CHANGELOG.md | 5 ++++- pyproject.toml | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-) ``` Version stamp only. Zero source code changes. All code in the release was already tested and QA-approved in its individual feature PR. ## AI-assistance disclosure - [ ] No AI used in producing this PR - [x] AI assisted with code generation (e.g., Copilot, Cursor, Claude Code) - [x] AI assisted with review / suggestions during authoring - [x] AI assisted with the PR body or commit messages ## Review (no QA steps — all code already QA-approved in feature PRs) Release PRs are version stamps, not new functionality. Reviewer checks: 1. - [x] `pyproject.toml` version bumped correctly (0.18.2 → 0.18.3). 2. - [x] `CHANGELOG.md` heading renamed `[Unreleased]` → `[0.18.3] - 2026-04-24` with today's date. 3. - [x] Empty `## [Unreleased]` section preserved above `[0.18.3]` for future work. 4. - [x] Comparison links at the bottom: `[Unreleased]` now points at `v0.18.3...HEAD`, new `[0.18.3]` link at `v0.18.2...v0.18.3`. 5. - [x] Scope delta table in this PR body matches `git log v0.18.2..release/v0.18.3 --oneline`. 6. - [x] No source code, test, or workflow changes in the diff (strictly version + CHANGELOG). ## Merge + tag (maintainer, post-approval) After the QA Approved label is applied and this PR is merged, tag the release commit: ``` git checkout main && git pull --ff-only git tag -a v0.18.3 -m "v0.18.3 — briefing surfaces manually-fired intentions" git push origin v0.18.3 ``` The `docker-publish.yml` workflow fires on tag push and publishes `mcp-awareness:0.18.3` + `mcp-awareness:latest`. Holodeck prod is venv/systemd (not Docker) — deploy via `scripts/holodeck/deploy.sh` on the operator workstation (git pull + pip + restart + HAProxy drain). `docker-compose.yaml` uses `:latest` so no update needed there. ## Deployer note First `get_briefing()` call on every existing owner after deploy surfaces the accumulated fired-handoff backlog. For the production owner that's 20+ entries since 2026-04-14. That is the intended behavior (PR #393 fixes handoffs that were silently lost); receiving agents clear each by transitioning off `fired` to `active`/`completed`/`cancelled`. ## Checklist - [x] `CHANGELOG.md` heading renamed and comparison links updated - [x] `pyproject.toml` version bumped - [x] `README.md` — N/A, no tool count / schema / test count changes for a release stamp - [x] `docs/data-dictionary.md` — N/A, no schema change - [x] `docker-compose.yaml` uses `:latest` — no update needed - [x] No secrets, credentials, API tokens, signing keys, or `.env` contents included - [x] I have read and will sign the [CLA](../CLA.md) via the `cla-assistant` bot Co-authored-by: cmeans-claude-dev[bot] <272174644+cmeans-claude-dev[bot]@users.noreply.github.com>
Linked issue
Fixes # none — replaces #390 (closed) with a fresh branch pushed correctly via the bot's HTTPS route so branch protection can proceed. Related: #391 tracks the install-time supply-chain layer this PR deliberately does not address.
Summary
pip-auditwas scanning the entire CI venv — including bootstrap tooling (pip,setuptools,wheel,virtualenv) that the server never imports at runtime. The workflow's own comments already called this out as the intent ("venv-bootstrap CVEs don't count against the project") and tried to fix it by upgradingpip + setuptoolsbefore the scan; that is a no-op the moment a CVE lands without an upstream fix (what happened to pip on 2026-04-24 via CVE-2026-3219 / GHSA-58qw-9mgm-455v, redding every subsequent PR againstmain). The earlier attempt on closed PR #390 introduced--ignore-vuln CVE-2026-3219as a short-term unblock; that pattern was rejected during review (rightly — it turns into a permanent bypass list, the "Enova-style backlog" failure mode). This PR scopes the scan explicitly instead:pip list --format freezecaptures the full resolved tree, a grep strips the four bootstrap packages + the editable project, andpip-audit --disable-pip --no-deps -r scan-requirements.txtscans the rest. Every project dep (direct + transitive) plus pip-audit's own deps stay in scope — if any picks up a CVE, the scan still fails. No CVE-level ignores.Scope
Single concern: scope the
pip-auditscan correctly. No source code, tests, deps, or other workflows touched. No CVE-level ignores added.AI-assistance disclosure
QA
Prerequisites
None — this is a CI-only change. All verification is through inspecting the
auditjob's output on this PR.Manual tests (via MCP tools)
(N/A — no MCP tool surface exercised by this PR. CI is the test harness.)
Verification steps for QA
auditcheck passes on this PR.Expected: the
auditjob reports "no known vulnerabilities found" and exits 0. TheWrite scan requirements filestep logs the scope (Packages in scan scope:with a count) and the excluded packages (pip, setuptools, wheel, virtualenv, mcp-awareness-server) — reviewer should confirm the excluded list is exactly those five and nothing else.The run log for
Write scan requirements fileshould show a scan count in the 100+ range (project + dev + pip-audit transitive deps). If the count is suspiciously small (e.g., <20), the grep stripped too much.Confirm the workflow diff contains no
--ignore-vulnflags and no.pip-audit-ignoresfile.On
main:auditis red on CVE-2026-3219 (pip). On this PR, the CVE should not appear in the scan output at all — pip is excluded by scope, not ignored by ID. Confirm the "Excluded" log line listspip==26.0.1(or whatever setup-python ships on that runner).Mentally simulate: pretend
requests(a project dep) got a new CVE. Under this scope, it would still appear inscan-requirements.txtand the scan would fail. The only way this PR could accidentally suppress such a finding is ifrequestswere caught by the grep, which it isn't (grep is anchored^(pip|setuptools|wheel|virtualenv)==). Confirm that anchoring.Checklist
CHANGELOG.mdentry added under[Unreleased]in Keep-a-Changelog format (under### Security)README.mdanddocs/data-dictionary.mdupdated if affected — N/A, CI-only change.envcontents included in the diffruff check,mypy, andpytestpass locally — N/A, no source changescla-assistantbot