From d0f94ffaa2982ae7b63fd1ff242f1b1330fbadb2 Mon Sep 17 00:00:00 2001 From: "cmeans-claude-dev[bot]" <272174644+cmeans-claude-dev[bot]@users.noreply.github.com> Date: Fri, 24 Apr 2026 12:18:03 -0500 Subject: [PATCH 1/2] chore(sec): scope pip-audit to project-effective deps, not the whole venv MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Previously the workflow invoked `pip-audit --skip-editable`, which scans every package in the CI venv — including bootstrap tooling (pip, setuptools, wheel, virtualenv) that the server never imports at runtime. The workflow's own comments already acknowledged this as the intent ("venv-bootstrap CVEs don't count against the project") and tried to fix it by upgrading pip + setuptools before the scan. That is a no-op when there is no fixed version published — exactly what happened on 2026-04-24 with pip 26.0.1 / CVE-2026-3219 (GHSA-58qw-9mgm-455v), which reds every subsequent PR against main. Fix: make the exclusion explicit by scope rather than by CVE allowlist. `pip list --format freeze` captures the full resolved dep tree, a grep strips the four bootstrap packages + the editable project, and `pip-audit --disable-pip --no-deps -r scan-requirements.txt` scans the rest. Every project dep (direct + transitive) plus pip-audit's own deps stay in scope. If any of them picks up a CVE, the scan still fails. Scope decisions are package-level and reviewable in the workflow. CVE-level `--ignore-vuln` entries are explicitly called out as a last resort in the workflow comment — they are how vuln backlogs accumulate. --- .github/workflows/pip-audit.yml | 58 ++++++++++++++++++++++++++------- CHANGELOG.md | 3 +- 2 files changed, 48 insertions(+), 13 deletions(-) diff --git a/.github/workflows/pip-audit.yml b/.github/workflows/pip-audit.yml index eab797f..d9d4e53 100644 --- a/.github/workflows/pip-audit.yml +++ b/.github/workflows/pip-audit.yml @@ -36,10 +36,9 @@ jobs: with: python-version: "3.12" - # Upgrade pip + setuptools first so venv-bootstrap CVEs don't count - # against the project. These packages aren't runtime deps — they're - # packaging tooling shipped with every venv — but pip-audit scans - # the whole environment, so we make sure they're current. + # Keep pip + setuptools current. Even though they're excluded from + # the scan scope below (see "Write scan requirements file"), a stale + # pip can break the install step itself. - name: Upgrade packaging tooling run: python -m pip install --upgrade pip setuptools @@ -49,13 +48,48 @@ jobs: - name: Install pip-audit run: pip install "pip-audit==2.10.0" - # Exit code 0 = no vulns, 1 = vulns found. Any finding fails the - # build — forces immediate triage. Known-accepted CVEs can be - # allowlisted with `--ignore-vuln CVE-ID` (add here with a comment - # explaining why; see CONTRIBUTING.md § "Security scanners"). + # pip-audit scans whatever Python environment it sees. The install + # steps above put bootstrap tooling (pip, setuptools, wheel, + # virtualenv) into the venv, but none of those are runtime deps of + # the mcp-awareness server — they're install-time tooling. A CVE + # in pip/setuptools/wheel cannot be reached by any request the + # server serves; the exposure path is install-time supply chain, + # addressed by Dependabot and signed commits, not by runtime-dep + # vulnerability scanning. # - # `--skip-editable` excludes the project itself (installed via - # `pip install -e`) from the scan; pip-audit would otherwise skip - # it with a warning since it isn't published on PyPI. + # Scope the scan explicitly: take the full installed set + # (`pip list --format freeze` — direct + transitive project deps + # plus CI tooling), remove only the bootstrap trio and the project + # itself, and pass the rest to `pip-audit -r`. pip-audit and its + # transitive deps stay in scope on purpose — if one of them picks + # up a CVE, we upgrade pip-audit (which we control) rather than + # hide the finding. No CVE-level ignores; scope decisions are + # package-level and reviewable. + - name: Write scan requirements file + run: | + pip list --format freeze \ + | grep -v -E '^(pip|setuptools|wheel|virtualenv)==' \ + | grep -v -E '^mcp-awareness-server==' \ + > scan-requirements.txt + echo "Packages in scan scope:" + wc -l scan-requirements.txt + echo "Excluded (bootstrap tooling + editable project):" + pip list --format freeze \ + | grep -E '^(pip|setuptools|wheel|virtualenv|mcp-awareness-server)==' \ + || true + + # Exit code 0 = no vulns, 1 = vulns found. Any finding fails the + # build — forces immediate triage. If a package in the scan scope + # picks up a CVE with no upstream fix, prefer a short-lived pin or + # a scope decision ("is this package actually reachable from the + # server's runtime path?") over adding a `--ignore-vuln` entry — + # CVE-level ignores tend to become forever-exemptions. + # --no-deps: scan exactly the packages listed (the requirements + # file already contains the full resolved tree from + # `pip list --format freeze`). + # --disable-pip: skip pip-audit's internal venv-creation + + # pip-install step for dep resolution (not needed since we're + # passing a fully-resolved list; faster and avoids a second + # bootstrap-pip round-trip on every CI run). - name: Run pip-audit - run: pip-audit --skip-editable + run: pip-audit --disable-pip --no-deps -r scan-requirements.txt diff --git a/CHANGELOG.md b/CHANGELOG.md index 4155113..65e3cb7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,7 +12,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **`alembic/env.py` no longer disables application loggers.** Python's `logging.config.fileConfig()` defaults to `disable_existing_loggers=True`, which silences every logger not listed in `alembic.ini` — including `mcp_awareness.postgres_store`. Production CLI alembic runs are one-shot processes so the silencing is invisible, but any long-lived Python process that calls alembic programmatically (tests invoking `alembic.command.upgrade`, server-side migration hooks, admin scripts) inherits a broken log surface for the rest of its lifetime. `alembic/env.py` now passes `disable_existing_loggers=False` to `fileConfig()`, preserving host-app logging. Surfaced by the R3 migration-safety test (first programmatic alembic consumer in the test suite): running `test_rls_migration_safety.py` before `tests/test_store.py::test_do_cleanup_logs_errors` silently dropped the error log record the store test asserts on, causing order-dependent failures that looked like a return of the [#374](https://github.com/cmeans/mcp-awareness/issues/374) flake. - **Flake in `tests/test_store.py::test_do_cleanup_logs_errors`.** Test was using the `caplog.at_level(...)` context manager plus `caplog.text` substring check. That pattern failed intermittently on CI (observed on Python 3.11 and 3.14) even though the code under test (`_do_cleanup`) is fully synchronous — the flake was in pytest's log-capture path, not in the production code. Rewritten to use `caplog.set_level(...)` at fixture scope and inspect `caplog.records` directly (logger name + level + message substring), which is sturdier across pytest/Python versions and produces a richer failure message when it does fire. Verified 10/10 consecutive local runs post-fix. Closes [#374](https://github.com/cmeans/mcp-awareness/issues/374). -### Security +### Changed +- **pip-audit now scans only project-effective deps (direct + transitive), not bootstrap tooling.** The workflow's stated intent was already to exclude venv-bootstrap packages (`pip`, `setuptools`, `wheel`, `virtualenv`) from counting against the project — they aren't runtime deps of the server, and a CVE in pip itself (e.g. [CVE-2026-3219 / GHSA-58qw-9mgm-455v](https://github.com/advisories/GHSA-58qw-9mgm-455v), pip's concatenated-archive-as-ZIP misclassification) is an install-time supply-chain concern addressed by Dependabot + signed commits, not by the runtime-dep scan. The previous implementation only tried to *upgrade* those packages before scanning, which is a no-op the moment a CVE with no upstream fix lands (exactly what CVE-2026-3219 did on 2026-04-24, failing every subsequent PR on `main`). The fix makes the exclusion explicit by scope, not by CVE allowlist: `.github/workflows/pip-audit.yml` now runs `pip list --format freeze` to capture the full resolved tree, strips the four bootstrap packages + the editable project, and passes the result to `pip-audit --disable-pip --no-deps -r scan-requirements.txt`. Every project dep (direct + transitive) plus the scanner's own deps stay in scope — if any of them picks up a CVE, the scan still fails. Scope decisions are package-level and reviewable in the workflow; CVE-level `--ignore-vuln` entries are explicitly called out in the workflow comment as a last resort (they become forever-exemptions the way ignored findings at big-company-size tend to). - **Semgrep static analysis in CI with three project-specific custom rules.** New `.github/workflows/semgrep.yml` runs the `semgrep/semgrep:1.161.0` container with the community rule packs `p/python` and `p/owasp-top-ten`, plus a `.semgrep/` directory carrying three custom rules written for this project: - `awareness-sql-missing-owner-id` (`.semgrep/awareness-sql-owner-scope.yml`) — every SQL statement that touches `entries`, `reads`, `actions`, or `embeddings` must include `owner_id` in its body. Statically enforces the SQL-filter layer that the R1–R4 RLS harness tests at runtime. Templated SQL files (`{where}`, `{order_by}` placeholders) are exempt since the Python caller injects `owner_id` as a parameter; deliberate cross-tenant statements can use a `semgrep:allow awareness-sql-missing-owner-id because ` escape hatch. - `no-credential-identifier-in-logs` (`.semgrep/no-token-in-logs.yml`) — rejects `logger.*` and `ctx.*` calls that interpolate variables named `token`, `bearer`, `authorization`, `password`, `secret`, `api_key`, etc. Preventive rule; zero current sites at landing but cheap insurance against a future accidental credential log. From 1dc778dd732ad1db62203480b78a34d9c8558947 Mon Sep 17 00:00:00 2001 From: "cmeans-claude-dev[bot]" <272174644+cmeans-claude-dev[bot]@users.noreply.github.com> Date: Fri, 24 Apr 2026 13:15:39 -0500 Subject: [PATCH 2/2] docs(changelog): restore ### Security heading and keep new entry under it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses QA findings #2 and #3 on PR #392: - Finding #2 (Keep-a-Changelog structure): the cherry-picked content renamed `### Security` to `### Changed` under `## [Unreleased]`, but `### Changed` already existed below `### Added` for the CONTRIBUTING.md section expansions. That left two `### Changed` headings under a single version — Keep-a-Changelog expects each category once per release. - Finding #3 (entries demoted): the rename pushed nine prior entries (Semgrep, trivy, pip-audit, gitleaks, four RLS harness items, third- party action pinning) out of `### Security` even though each of them closed a security-tracking issue. Those entries belong under Security. Fix: revert the heading to `### Security`. The new pip-audit-scope entry stays under that same heading — scoping a CVE scanner correctly IS a security-adjacent fix, so it belongs with the cluster. `## [Unreleased]` now has Fixed / Security / Added / Changed, each exactly once. --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 65e3cb7..a55ce08 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,7 +12,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **`alembic/env.py` no longer disables application loggers.** Python's `logging.config.fileConfig()` defaults to `disable_existing_loggers=True`, which silences every logger not listed in `alembic.ini` — including `mcp_awareness.postgres_store`. Production CLI alembic runs are one-shot processes so the silencing is invisible, but any long-lived Python process that calls alembic programmatically (tests invoking `alembic.command.upgrade`, server-side migration hooks, admin scripts) inherits a broken log surface for the rest of its lifetime. `alembic/env.py` now passes `disable_existing_loggers=False` to `fileConfig()`, preserving host-app logging. Surfaced by the R3 migration-safety test (first programmatic alembic consumer in the test suite): running `test_rls_migration_safety.py` before `tests/test_store.py::test_do_cleanup_logs_errors` silently dropped the error log record the store test asserts on, causing order-dependent failures that looked like a return of the [#374](https://github.com/cmeans/mcp-awareness/issues/374) flake. - **Flake in `tests/test_store.py::test_do_cleanup_logs_errors`.** Test was using the `caplog.at_level(...)` context manager plus `caplog.text` substring check. That pattern failed intermittently on CI (observed on Python 3.11 and 3.14) even though the code under test (`_do_cleanup`) is fully synchronous — the flake was in pytest's log-capture path, not in the production code. Rewritten to use `caplog.set_level(...)` at fixture scope and inspect `caplog.records` directly (logger name + level + message substring), which is sturdier across pytest/Python versions and produces a richer failure message when it does fire. Verified 10/10 consecutive local runs post-fix. Closes [#374](https://github.com/cmeans/mcp-awareness/issues/374). -### Changed +### Security - **pip-audit now scans only project-effective deps (direct + transitive), not bootstrap tooling.** The workflow's stated intent was already to exclude venv-bootstrap packages (`pip`, `setuptools`, `wheel`, `virtualenv`) from counting against the project — they aren't runtime deps of the server, and a CVE in pip itself (e.g. [CVE-2026-3219 / GHSA-58qw-9mgm-455v](https://github.com/advisories/GHSA-58qw-9mgm-455v), pip's concatenated-archive-as-ZIP misclassification) is an install-time supply-chain concern addressed by Dependabot + signed commits, not by the runtime-dep scan. The previous implementation only tried to *upgrade* those packages before scanning, which is a no-op the moment a CVE with no upstream fix lands (exactly what CVE-2026-3219 did on 2026-04-24, failing every subsequent PR on `main`). The fix makes the exclusion explicit by scope, not by CVE allowlist: `.github/workflows/pip-audit.yml` now runs `pip list --format freeze` to capture the full resolved tree, strips the four bootstrap packages + the editable project, and passes the result to `pip-audit --disable-pip --no-deps -r scan-requirements.txt`. Every project dep (direct + transitive) plus the scanner's own deps stay in scope — if any of them picks up a CVE, the scan still fails. Scope decisions are package-level and reviewable in the workflow; CVE-level `--ignore-vuln` entries are explicitly called out in the workflow comment as a last resort (they become forever-exemptions the way ignored findings at big-company-size tend to). - **Semgrep static analysis in CI with three project-specific custom rules.** New `.github/workflows/semgrep.yml` runs the `semgrep/semgrep:1.161.0` container with the community rule packs `p/python` and `p/owasp-top-ten`, plus a `.semgrep/` directory carrying three custom rules written for this project: - `awareness-sql-missing-owner-id` (`.semgrep/awareness-sql-owner-scope.yml`) — every SQL statement that touches `entries`, `reads`, `actions`, or `embeddings` must include `owner_id` in its body. Statically enforces the SQL-filter layer that the R1–R4 RLS harness tests at runtime. Templated SQL files (`{where}`, `{order_by}` placeholders) are exempt since the Python caller injects `owner_id` as a parameter; deliberate cross-tenant statements can use a `semgrep:allow awareness-sql-missing-owner-id because ` escape hatch.