OSV allowlist: scope each allowed lock to its reviewed advisories, bound to the lock's sha256 and reachability evidence

[seathatflowsinourveins]

Scope

• What this PR changes: the repo-wide OSV ignores' lock allowlist is scoped per advisory and bound to each lock's reviewed bytes.
  • Before, a path in IGNORE_ALLOWED_LOCKS exempted its lock from every repo-wide ignore, and a later change to the lock kept the exemption.
  • Reproduced: a temporary lock pinning nltk==3.10.3 and setuptools==80.10.2 reported both pins, and reported [] once its path was added.
• Why now: the GPT Researcher (Runtime worker: GPT Researcher on GPT-6 via OmniRoute (recipe, round 2) #426) and crawl4ai (Runtime worker: crawl4ai with GPT-6 extraction via OmniRoute (recipe, round 2) #428) runtime-worker locks pin nltk 3.10.3. Each gets its own entry, with its reachability receipt, in those PRs after this one lands.
• Base commit: 37739d32
• Head: f62c6263 (two commits: the change, then the review repair)
• Lane: lane:foundation
  • .github/osv-scanner.toml and tests/test_osv_lockfile_coverage.py belong to the foundation's GitHub-automation closure.
  • The test reads the trading lane's Lumibot lock as data, which docs/lanes.md assigns to the tool's lane. No trading file changes.
• Paths (4 files):
  • .github/osv-scanner.toml
  • tests/test_osv_lockfile_coverage.py
  • docs/decisions/2026-09-22-github-automation-closure.md (the Ignores bullet only)
  • manifests/evidence.json (re-registration per the hot-file protocol)

Changes

• IGNORE_ALLOWED_LOCKS is a mapping: path → {advisories, sha256, evidence}.

  • It has one entry, the Lumibot evaluation lock.
  • Both advisories are listed, as before.
  • Its sha256 a8dce0af… equals the lock_sha256 that its receipt (blueprints/us-equities/engine-trials/spy-one-zero-20260926/repository-checks.json) recorded.

• affected_pins(entries, ignore_ids, allowed) exempts an allowed lock only from the advisories its entry lists.

• digest_drift(allowed) reports any allowed lock whose bytes changed since its review.

  • .gitattributes * text=auto eol=lf keeps the digest the same on every host.

• Six new tests. Every allowed lock:

  • is a scanned inventory lockfile;
  • matches its reviewed digest;
  • names an existing evidence file;
  • lists only advisories that have an IGNORE_SCOPES entry and an active ignore.

  Two mutant tests cover a lock allowed for nltk only (it still reports setuptools 80.10.2) and a one-byte change to an allowed lock (caught).

• The nltk ignore's reason now names:

  • the affected APIs: TransitionParser, AveragedPerceptron.save/load, PerceptronTagger.save_to_json and the module function nltk.classify.maxent.save_maxent_params;
  • the unaffected guarded loader;
  • pathsec enforcement on by default;
  • the fix status and the expiry plan: relock every allowed lock onto the first release that ships the fixes, then delete the ignore.

  ignoreUntil and the setuptools ignore are unchanged.

• The decision record's Ignores bullet records the scoped, digest-bound rule (2026-09-27), the alternatives, the overturn condition and the evidence path.

SOTA sources

• osv-scanner v2.6.0 (e840a6e8):
  • internal/config/config.go L26-29, L38-50 and L104-112: ignores match by id and expiry only, never by path.
  • docs/configuration.md L23: one --config replaces every per-directory config.
  • That is why the path and digest scoping lives in the repository test.
• OSV GHSA-8mgp-746c-j5xp: modified 2026-09-02, last_affected 3.10.3, no fixed version (read again 2026-09-27 by the coordinator).
• nltk:
  • v3.10.3 nltk/pathsec.py L28: ENFORCE = True.
  • nltk/classify/maxent.py L1583: save_maxent_params is a module function.
• PyPI: the latest nltk is 3.10.3 (2026-08-12), and no later release exists. Coordinator read, 2026-09-27.
• nltk develop fixes: #3757 (merged 2026-08-20), #3759 (2026-08-18) and #3813 (2026-08-28), all into develop. Coordinator read via gh api, 2026-09-27.

Evidence-class table

Claim	Evidence class	Command / result
The old allowlist hid every advisory for a listed path	local integration check (semantic red)	Temporary lock with nltk==3.10.3 and setuptools==80.10.2: both pins reported while not allowed, [] once its path was added (in-process only)
The new tests need the new code	local integration test, fail-first	The six new tests on the old module: FAILED (errors=5), 1 passed (2 AttributeError, 2 NameError, 1 TypeError)
The suite passes	local integration test	python3 -m unittest tests.test_osv_lockfile_coverage tests.test_blind_checkout tests.test_workflow_security_coverage: Ran 77 tests … OK after the repair and the rebase onto 37739d32 (57 before this PR; the module has 30 tests, 16 existing and 14 new)
Each guard is needed	local mutation check	Six in-process mutations, each failing its test; the unmutated control passes
The real scan is unchanged	upstream tool run (osv-scanner v2.6.0)	Over the 47 inventory lockfiles after the repair: exit 0, "Filtered 3 vulnerabilities from output", "No issues found"; both ignores filtered with the updated reasons
Registry and push gate	local and native git push	scripts/validate.py passed after re-registration per docs/lanes.md. Each push went through #438's gate, the last on f62c6263, and returned OK
Independent verification	separate verifier run	All four acceptance commands re-run: exit 0, nothing refuted. It left two network facts unverified; the coordinator then checked both (Sources above)

Review round (one GPT-6 cross-family review, one repair)

A read-only review by gpt-6-astra at effort max returned DEFECTS: 2 major and 3 minor findings, each probed in memory. A Claude Opus 5.5 builder fixed all five in one repair round.

#	Finding	Disposition
1	major: affected pins escaped the parser: a backslash continuation inside a name, ==v80.10.2, ==83.0.0rc1	Fixed. Lines are joined per pip's requirements-file format, names are normalized per PEP 503, and a tracked package with any version that is not a plain release (^\d+(\.\d+)*$) fails closed. The builder also found setuptools>=70 (OSV reports it) and a comment on a continued line (pip drops the next line but OSV reads it); both now fail closed. No PEP 440 code was written by hand, because packaging is in neither CI nor the host's python3
2	major: -r includes were skipped, and an allowed lock's digest did not cover included bytes	Fixed. -r/--requirement/-c/--constraint includes are followed recursively with a cycle guard, as osv-scalibr's extractor does. An include that does not resolve fails closed. An allowed lock must be self-contained. No current inventory lock uses includes
3	minor: "load_from_json is not affected" was too categorical	Fixed. The reason quotes the advisory's "negative-control guarded helper" and notes that 3.10.3's perceptron.py L395-399 authorizes a caller-chosen directory
4	minor: "active" ignores meant only "configured"; an expired date still passed	Fixed. An ignore is active only while ignoreUntil > today (UTC), matching osv-scanner 2.6.0 (internal/config/config.go L149-157, ignoreUntil.After(time.Now())) and BurntSushi/toml v1.6.0 reading a bare date as local midnight. The decision records the deadline action if no fixed nltk ships before 2026-12-24
5	minor: the tests survived four mutations (read_text() digests, first lock only, a missing lock skipped, uv.lock skipped)	Fixed. New fixtures cover CRLF against LF bytes, two allowed locks where only the second drifts, a missing allowed lock and a vulnerable uv.lock

• Fail-first: each new test fails on ae6ddfac's code, or catches its named mutation where the old code was already correct (finding 5).
• Mutation pass: 19 named mutants, all caught, with the test file's sha256 unchanged before and after each run.
• Stricter guard on the real inventory: 659 requirements across 21 entries, 0 unparseable, and no current lock flagged.

Residual risks

• A trading-lane change to the Lumibot lock now fails test_every_allowed_lock_matches_its_reviewed_digest until its reachability is re-reviewed and the new sha256 is recorded. This is intended; the test message says what to do.
• The ignore still matches by id in every scanned lock, because osv-scanner has no per-path ignore. The repository test is the only path scope.
• The guard fails closed rather than parsing PEP 440. setuptools==83 or ==83.0 is reported as affected (integer tuples without zero-padding), and -e and abbreviated options are refused. A bare archive name such as nltk-3.10.3-py3-none-any.whl is not flagged, but OSV does not see nltk there either.
• Expiry is checked by UTC date, so a local run in another time zone can differ from OSV by a few hours.
• The repair had no second cross-family review (bounded loop: one review, one repair).
• The nltk ignore expires 2026-12-24. If no fixed release exists by then, the expiry plan has to be decided again.

Status

• Built by: a Claude Opus 5.5 builder in the resumed workflow wf_9dae7d7b-e15, with an independent verifier. The coordinator reviewed the diff, checked the network facts, committed, re-registered and pushed.
• Next: after merge, Runtime worker: GPT Researcher on GPT-6 via OmniRoute (recipe, round 2) #426 and Runtime worker: crawl4ai with GPT-6 extraction via OmniRoute (recipe, round 2) #428 each add their own lock entry, with its nltk reachability receipt as evidence.

🤖 Generated with Claude Code