test(analysis): govern real YouTube known-stem benchmark

AGENTS.md

@@ -3,6 +3,9 @@
 ## Project overview
 - BandScope is a local-first desktop app for rehearsal prep: a practical song view with likely harmony by section and by instrument or vocal role, form and groove cues, stems, playable ranges, simplification guidance, transposition or setup cues, part-overlap cues, visible confidence, and rehearsal priorities.
 - Authoritative delivery rules live in `ARCHITECTURE.md`, `docs/plans/`, and the root verification scripts.
+- Canonical product/technical requirements, ADRs, diagrams, and documentation sufficiency live in
+  `docs/PRD.md`, `docs/TRD.md`, `docs/adr/README.md`, `docs/architecture/diagrams.md`, and
+  `docs/documentation-coverage-matrix.md`.
 - Brand, tone, UX copy, and prioritization rules live in `docs/brand-story.md` and must be applied to PRDs, TRDs, UI copy, onboarding, empty states, and error messages.
 - App security rules live in `docs/security/app-security.md` and must be applied to file handling, URL intake, subprocesses, IPC, WebView usage, model loading, updates, logging, cache handling, and export behavior.
 - Dependency, SBOM, and supply-chain rules live in `docs/security/dependency-policy.md` and must be applied to dependency additions, GitHub Actions, releases, bundled binaries, and model artifacts.
@@ -59,9 +62,19 @@ This section applies to any agent (Claude, Codex, Cursor, opencode, ...) working
 - Frontend tests: `npm run test --workspaces --if-present`
 - Python tests: `uv run --project services/analysis-engine pytest --cov=src/bandscope_analysis --cov-report=term-missing --cov-fail-under=100`
 - Typecheck: `npm run typecheck --workspaces --if-present && uv run --project services/analysis-engine mypy src`
+- Known-stem offline contract: `uv run --project services/analysis-engine pytest services/analysis-engine/tests/test_youtube_stem_e2e.py -m 'not youtube_stem_e2e' -vv`
+- Known-stem live lane is explicit opt-in only; follow
+  `docs/engineering/youtube-known-stem-validation.md` and never claim a skipped or provider-failed
+  invocation passed.
 
 ## Architecture references
 - `ARCHITECTURE.md`
+- `docs/README.md`
+- `docs/PRD.md`
+- `docs/TRD.md`
+- `docs/adr/README.md`
+- `docs/architecture/diagrams.md`
+- `docs/documentation-coverage-matrix.md`
 - `docs/engineering/acceptance-criteria.md`
 - `docs/engineering/harness-engineering.md`
 - `docs/workflow/one-day-delivery-plan.md`
@@ -85,6 +98,7 @@ This section applies to any agent (Claude, Codex, Cursor, opencode, ...) working
 - Prefer practical, friendly, rehearsal-first wording over academic or authority-heavy language.
 - Do not reduce the product to a chord analyzer when form, timing, player coordination, playable ranges, simplification, and setup cues are the real rehearsal blockers.
 - Do not frame usability as a reason to accept weak analysis quality; BandScope should aim for both easy use and high accuracy.
+- Do not invent a parallel MIR product. #828 owns the #770 known-stem slice. Tempo Acc2 alone cannot accept rehearsal tempo; cite Schreiber, Urbano, & Müller (2020) for Acc1/Acc2, not Raffel (2014).
 
 ## Safety
 - Do not add network-dependent runtime paths for local analysis.

ARCHITECTURE.md

@@ -1,6 +1,16 @@
 # ARCHITECTURE.md
 
-Last updated: 2026-03-11
+Last updated: 2026-08-10
+
+## Documentation authority
+
+- Product requirements live in `docs/PRD.md`.
+- Technical requirements live in `docs/TRD.md`.
+- Decision status and supersession live in `docs/adr/README.md`.
+- Component, UML, deployment, and logical artifact views live in
+  `docs/architecture/diagrams.md`.
+- Sufficiency and requirement-to-evidence traceability live in
+  `docs/documentation-coverage-matrix.md`.
 
 ## Brand source
 
@@ -76,6 +86,57 @@ Last updated: 2026-03-11
 - Typical roles include bass, guitar, keyboard players, keyboard left hand, keyboard right hand, lead vocal, backing vocal, horns, strings, and other arrangement-carrying parts.
 - Shared contracts should be able to carry different harmonic guidance for simultaneous roles in the same section.
 
+## Source separation and model delivery
+
+- Production separation uses Demucs 4.0.1 `htdemucs` and returns exactly vocals, bass, drums, and
+  other for downstream local analysis. The retired `bandsplit-v1` profile is not a production
+  model.
+- Demucs random temporal shifts are disabled (`shifts=0`) so the same bytes and model produce
+  reproducible local analysis and benchmark evidence.
+- Model inference is local and fail-closed. A trusted provisioning step must place the exact
+  official weight artifact in a user-scoped cache before runtime; a missing artifact is never
+  fetched by the separator. The repository and release artifacts do not bundle the weights.
+- The exact signature, source URL, full SHA-256, byte size, distribution status, and model-rights
+  uncertainty are tracked in `supply-chain/supplemental-component-inventory.json` and ADR-0001.
+- The separator verifies a non-symlinked regular file's exact byte size and full SHA-256, then
+  passes those same verified bytes through PyTorch's `weights_only=True` restricted loader with an
+  exact reviewed global allowlist, strict model construction, and a serialized one-time cache. A
+  future artifact hash or allowlist change is executable-code review; model-rights/legal delivery
+  also remains a release blocker. The repository security owner must separately accept the residual
+  approved-pickle risk for the exact model hash/dependency lock, with expiry/re-review and rollback,
+  or approve a non-pickle replacement.
+- Current dependency markers exclude Demucs on macOS Intel; unsupported platforms must surface the
+  existing safe fallback rather than pretending to separate stems.
+- Quality claims are platform-scoped: every advertised OS/architecture needs an unchanged-candidate
+  pass, while every unproven artifact must exercise and advertise the fallback.
+
+## Known-stem validation boundary
+
+- The active known-stem branch crosses the production YouTube downloader and production separator,
+  while its reference loader, alignment, and metric utilities remain test-only.
+- It pins a creator-published vocal source and a separate finished master by exact hosts, byte
+  counts, full SHA-256 values, member, and member size; downloads and waveforms stay in test-owned
+  ephemeral storage.
+- The finished master proves candidate identity. YouTube-to-master and master-to-vocal global lags
+  are composed once before separation; predicted stems are never realigned. Quality requires
+  duration/identity checks, zero-mean SI-SDR improvement over the downloaded mixture. SI-SDR remains
+  the primary separation score (Le Roux et al., 2019). Harmony uses Odekerken/MIREX WCSR. Beat/onset
+  F-measure stays inside Chiu et al. (2025) ±70 ms. Tempo requires Schreiber, Urbano, & Müller
+  (2020) Acc1 and Acc2 together; Acc2 alone is forbidden, and Raffel (2014) is not an Acc1/Acc2
+  source. This branch also requires correct
+  vocal-stem assignment margin.
+- Deterministic metric/integrity/security contracts run in ordinary CI. Live network/model execution
+  is explicit opt-in and cannot be scheduled or made release-blocking until authorization and
+  calibration requirements in ADR-0002 are met.
+- The capability has no relational persistence. ADR-0003 and the logical artifact model in
+  `docs/architecture/diagrams.md` are authoritative instead of a physical ERD.
+- The planned `BenchmarkRun`/`BenchmarkEvidence` aggregate always binds candidate, fixture, model,
+  and sanitized toolchain provenance; identity and score blocks are stage-dependent. Persistence is
+  disabled until store/access/TTL/deletion controls are accepted.
+- Distinct user-facing import/model/decode/separation recovery states are planned under
+  PRD-KS-011/TRD-KS-013; the current benchmark failure taxonomy does not claim that product UX is
+  complete.
+
 ## Rehearsal outputs
 
 - Core rehearsal artifacts should include:

CHANGELOG.md

@@ -4,18 +4,75 @@
 
 ### Added
 
+- Added an opt-in real-YouTube/Demucs benchmark that verifies vocal separation against a
+  creator-published, SHA-256-pinned known stem without adding media files to the repository.
+- Added an independently pinned creator master for YouTube asset identity, full extracted-member
+  hashing, composed global offsets, calibrated provisional sentinels, and deterministic Demucs
+  inference (`shifts=0`).
+- Added canonical PRD, TRD, ADR, architecture/UML/logical-artifact diagrams, traceability, and
+  machine-checked documentation coverage for the known-stem quality boundary.
+- Replaced the retired FFT-era bandsplit model inventory with the exact htdemucs runtime artifact,
+  full SHA-256, byte size, delivery status, verified ffmpeg/ffprobe prerequisites, and release
+  blockers.
 - Name tonight's first playable range on the ready rehearsal map and tell the player to check that span on their instrument before the section.
 - Display the analyzed song tempo (BPM) as a badge in the rehearsal workspace.
 - 각 합주 역할(Role)별 개인 연습 진행도를 0~100% 범위로 기록 및 시각화할 수 있는 연습 진척도(`practiceProgress`) 트래커 기능 추가. UI 컨트롤(슬라이더 및 +/- 버튼)과 한/영 다국어 지원 포함.
 
 ### Changed
 
+- Lock rehearsal metric authority: Le Roux SI-SDR primary, Odekerken/MIREX WCSR, Chiu 2025 ±70 ms beat F-measure, Schreiber/Urbano/Müller Acc1+Acc2 with Acc2-alone forbidden, and Raffel 2014 not cited as an Acc1/Acc2 source. Tempo has no single primary metric; beat/onset admits F-measure only inside the 70 ms window.
 - Pinned npm `10.9.9` as the approved lockfile generator, activated it through Node-bundled Corepack before dependency consumption, and fail closed unless its bundled `tar` is at least `7.5.19`; primary CI still consumes the committed lock only through frozen `npm ci` validation, rejects mutable npm resolution in the lock gate, requires integrity evidence for public-registry lock entries, and preserves generator-sensitive root `@esbuild/*` peer metadata.
 
 ### Fixed
 
+- Rejected POSIX and Windows parent-directory segments at the YouTube download-output boundary
+  before the path reaches yt-dlp, returning a stable redacted failure without downloader execution.
+- Kept YouTube TLS verification enabled, using populated OS-managed CA roots when available and
+  retaining yt-dlp's maintained CA-bundle fallback when the system trust store is empty or fails.
+- Made htdemucs loading offline and fail-closed: the runtime accepts only the inventoried filename,
+  byte size, and full SHA-256, rejects filesystem identity races, and deserializes the verified
+  bytes with PyTorch's restricted `weights_only` loader, an exact reviewed global allowlist, strict
+  model construction, and serialized one-time caching rather than downloading a missing checkpoint.
+  Pre-open `lstat` and `open` failures stay redacted and close every obtained descriptor without a
+  None-check fallthrough, so a raced-away cache entry cannot skip the close or leak a path.
+- Verified exact platform-native sibling ffmpeg/ffprobe executable names and identities before any
+  live fixture access or yt-dlp invocation.
+- Isolated Numba's native-code cache for repository analysis commands so a stale or concurrently
+  compiled virtualenv cache cannot crash deterministic verification.
+- Reconciled stale CodeRabbit-gate wording with the canonical stable-check and review-equivalent
+  policy; qualifying evidence is now defined against the exact current head, and a rate-limited,
+  status-only, author, or predecessor review is not treated as completed review evidence.
+- Routed root npm/quickcheck Python entry points through a shared Node launcher that selects
+  `py -3`, `python3`, or `python` in a deterministic platform-specific order without masking
+  interpreter failures.
 - Upgraded the local score PDF parser to `pdfjs-dist` 6.2.108, pinned Undici 7.29.0 across the workspace, and constrained PDF loading to copied in-memory bytes with a same-origin bundled worker and npm-generated lock provenance.
 
+### Security Notes
+
+- Attack surface and trust boundary: YouTube URLs, response metadata, downloaded media, creator
+  fixtures, ffmpeg/ffprobe executables, and htdemucs checkpoint bytes remain untrusted until their
+  owning host, shape, size, filesystem identity, and full-hash allowlists pass.
+- Mitigations and failure behavior: TLS verification stays enabled; parent-directory segments are
+  rejected before the output template reaches yt-dlp; the complete ffmpeg/ffprobe path-and-hash
+  pair is verified before network fixture access; model loading is offline, same-byte, restricted
+  to `weights_only=True` plus the exact reviewed globals, and fails closed without an unrestricted
+  fallback.
+- Developer tooling: the cross-platform check launcher is repository-only, invokes only the fixed
+  `py`, `python3`, or `python` candidates with argument arrays and no shell, and propagates the first
+  available interpreter's failure instead of retrying past it.
+- Logging and privacy: raw media, model bytes, separated stems, credentials, and full local paths
+  are not retained in release evidence or emitted in bounded operator errors.
+- Test points: each candidate head must pass quickcheck, hosted SAST/Bandit/secret/security scans,
+  mutation tests for loader and allowlist bypasses, executable-identity rejection tests,
+  supply-chain verification, and the exact provisioned-model smoke test before merge.
+- Dependency and supply chain: no production dependency is added by this benchmark slice;
+  documentation policy checks pin `markdown-it-py 4.0.0` as a direct development dependency so
+  rendered Markdown—not lexical lookalikes—defines headings and tables. Canonical #783 is now
+  protected `develop` shipped truth; this branch inherits that JavaScript baseline rather than
+  duplicating or suppressing it. The supplemental inventory separately binds yt-dlp, ffmpeg/ffprobe,
+  and htdemucs to their declared delivery and integrity contracts.
+
+
 ## [0.1.3] - 2026-04-29
 
 ### Fixed

CLAUDE.md

@@ -33,6 +33,10 @@ npm run test        # JS workspace vitest suites + pytest with 100% coverage gat
 npm run build       # vite builds per workspace
 ```
 
+The canonical documentation graph starts at `docs/README.md`; product requirements, technical
+requirements, decision records, diagrams, and sufficiency are not replaceable by a PR body or old
+plan.
+
 Per-workspace and single-test:
 
 ```bash
@@ -53,7 +57,13 @@ Three layers, decoupled through shared contracts:
 
 - `apps/desktop` — Tauri 2 + Vite + React 19 shell (Tailwind 4, Base UI, Storybook). Feature screens live in `src/features/` (home, workspace, chords, ranges, player, settings). The ready workspace names tonight's first playable range and the next instrument check. `src/lib/analysis.ts` and `src/lib/job_runner.ts` call typed Tauri IPC commands, with a browser fallback that serves demo data when not running inside Tauri.
 - `apps/desktop/src-tauri/src/main.rs` — the Rust orchestration boundary. Tauri commands (`start_analysis_job`, `get_analysis_job_status`, `select_local_audio_source`, `import_youtube_url`) validate untrusted input (project IDs, file paths, URLs) and spawn the Python engine as a subprocess. There is no loopback HTTP listener and no network path for local analysis.
-- `services/analysis-engine` — Python package `bandscope_analysis` (librosa/numpy). Entry point `cli.py` reads a JSON job request on stdin and prints a structured job-status JSON envelope on stdout (`--progress-jsonl` streams progress lines). `api.py` orchestrates the pipeline across the `separation`, `sections`, `roles`, `chords`, `ranges`, `temporal`, `transcription`, and `youtube` modules.
+- `services/analysis-engine` — Python package `bandscope_analysis` (librosa/numpy). Entry point `cli.py` reads a JSON job request on stdin and prints a structured job-status JSON envelope on stdout (`--progress-jsonl` streams progress lines). `api.py` orchestrates the pipeline across the `separation`, `sections`, `roles`, `chords`, `ranges`, `temporal`, `transcription`, and `youtube` modules. Metric admission (#828 owns #770) uses Le Roux SI-SDR, Odekerken/MIREX WCSR, Chiu ±70 ms F-measure, and Schreiber/Urbano/Müller Acc1+Acc2; Acc2 alone is forbidden and Raffel 2014 is not an Acc1/Acc2 source.
+- Production source separation uses `htdemucs` on supported platforms. The exact runtime model
+  artifact is inventoried but not bundled; operators must provision it locally, and production
+  verifies its byte size and full SHA-256 before passing those same in-memory bytes through a
+  serialized `weights_only=True` loader with an exact reviewed global allowlist and strict model
+  construction. The active known-stem test crosses the production YouTube and separator boundaries;
+  see `docs/TRD.md` and the operator guide.
 
 Data flow: React UI → Tauri IPC command → Rust validation + Python subprocess over stdin/stdout → job status and progress events emitted back to the UI.
 
@@ -66,7 +76,7 @@ Supporting packages:
 ## Key conventions
 
 - Coverage is a hard gate: the Python engine requires 100% test coverage and 100% docstring coverage (Ruff `D100`–`D107` across `src`, `tests`, and repo scripts). Exported TypeScript declarations in `packages/shared-types` and `apps/desktop/src` require JSDoc with a description; `no-console` is an error.
-- Gitflow: `develop` is the default branch; `feature/*` targets `develop`, `main` is the protected release branch. Direct pushes to protected branches are not allowed, and every merge needs the required checks plus a passing CodeRabbit review (see `CONTRIBUTING.md` and `docs/repository/gitflow.md`).
+- Gitflow: `develop` is the default branch; `feature/*` targets `develop`, `main` is the protected release branch. Direct pushes to protected branches are not allowed, and every merge needs the stable checks and review-equivalent policy in `docs/security/github-required-checks.md`. CodeRabbit is requested by default and actionable findings must be addressed, but a stale or rate-limited hosted status is not review evidence.
 - The PR template (`.github/PULL_REQUEST_TEMPLATE.md`) requires a quickcheck confirmation, `Security Notes` (attack surface, trust boundary, mitigations, test points), a dependency/supply-chain checklist, and i18n impact.
 - i18n: the UI ships Korean and English locales (`apps/desktop/src/locales/ko`, `en`). Any user-visible string change must update both.
 - Documents under `docs/plans/` must include `Security Notes`; `scripts/checks/verify_security_notes.py` enforces this mechanically.

CONTRIBUTING.md

@@ -16,7 +16,9 @@ Read `docs/repository/gitflow.md` before opening a PR.
 ## Pull requests are mandatory
 
 - direct push to `main` or `develop` is not allowed
-- every protected-branch merge requires a passing `CodeRabbit` check
+- every protected-branch merge requires the stable checks and review-equivalent policy in
+  `docs/security/github-required-checks.md`; request CodeRabbit and address its current actionable
+  findings, but do not treat a stale or rate-limited hosted status as review evidence
 - all review conversations must be resolved before merge
 - required checks must stay green; do not bypass them
 

README.md

@@ -9,6 +9,10 @@ App security source of truth: `docs/security/app-security.md`
 Dependency and SBOM source of truth: `docs/security/dependency-policy.md`
 Cross-platform build policy source of truth: `docs/security/cross-platform-build-policy.md`
 GitHub bootstrap execution source of truth: `docs/workflow/github-bootstrap-execution-policy.md`
+Documentation authority index: `docs/README.md`
+Product requirements: `docs/PRD.md`
+Technical requirements: `docs/TRD.md`
+Architecture decisions and diagrams: `docs/adr/README.md`, `docs/architecture/diagrams.md`
 
 ## Public repository baseline