Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
4e5a252
test(analysis): add Tier 1 real-audio accuracy acceptance
cursoragent Aug 16, 2026
11e156d
fix(analysis): score C major acceptance from decoded WAV bytes
cursoragent Aug 16, 2026
d4282ba
test(accuracy): reject malformed report provenance
seonghobae Aug 16, 2026
f3ac87b
fix(accuracy): validate report provenance values
seonghobae Aug 16, 2026
1f375b2
docs(accuracy): record provenance validation hardening
seonghobae Aug 16, 2026
2da68dc
fix(changelog): preserve release history while recording provenance h…
seonghobae Aug 16, 2026
b778af6
docs(accuracy): document report provenance validation
seonghobae Aug 16, 2026
0f6386b
test(accuracy): prevent overlap double counting
seonghobae Aug 16, 2026
9cd0c76
fix(accuracy): union overlapping chord estimates
seonghobae Aug 16, 2026
82b275c
docs(changelog): record bounded chord recall
seonghobae Aug 16, 2026
05a6ce3
docs(accuracy): define overlap-safe recall
seonghobae Aug 16, 2026
5be801e
test(accuracy): reject non-finite tempo evidence
seonghobae Aug 16, 2026
0b4e73b
fix(accuracy): reject non-finite tempo evidence
seonghobae Aug 16, 2026
f1d039a
docs(accuracy): fail closed on non-finite tempo evidence
seonghobae Aug 16, 2026
3d41c00
docs(changelog): record finite tempo evidence
seonghobae Aug 16, 2026
59997be
test(accuracy): bound fixture timing inputs
seonghobae Aug 16, 2026
6e7da0b
fix(accuracy): bound non-finite fixture timing
seonghobae Aug 16, 2026
2b0bbbb
test(accuracy): reject non-finite chord timing evidence
seonghobae Aug 16, 2026
1302d41
fix(accuracy): fail closed on non-finite chord timing
seonghobae Aug 16, 2026
4cfdfab
docs(accuracy): document finite chord timing evidence
seonghobae Aug 16, 2026
4416299
docs(changelog): record finite chord timing guard
seonghobae Aug 16, 2026
a59085f
test(accuracy): fail closed without product version
seonghobae Aug 16, 2026
3ad3a52
fix(accuracy): require exact product version provenance
seonghobae Aug 16, 2026
65cb516
test(accuracy): align version provenance contract
seonghobae Aug 16, 2026
3856a9e
docs(changelog): record exact version provenance
seonghobae Aug 16, 2026
c5c1d23
docs(accuracy): require exact engine version provenance
seonghobae Aug 16, 2026
4dd0def
test(accuracy): reject unknown engine version evidence
seonghobae Aug 16, 2026
60ecd22
test(accuracy): cover inexact version provenance
seonghobae Aug 16, 2026
ebaa3f5
fix(accuracy): reject unknown version evidence
seonghobae Aug 16, 2026
db8412f
test(accuracy): reject malformed chord estimate intervals
seonghobae Aug 16, 2026
05929e2
fix(accuracy): reject malformed chord estimate intervals
seonghobae Aug 16, 2026
8e22c97
docs(changelog): record malformed interval rejection
seonghobae Aug 16, 2026
caaf7c7
docs(accuracy): document interval validity gate
seonghobae Aug 16, 2026
b9bb409
test(accuracy): reject padded report evidence strings
seonghobae Aug 16, 2026
eb3d304
fix(accuracy): reject padded report evidence strings
seonghobae Aug 16, 2026
6dcfc9a
docs(changelog): record exact accuracy-report strings
seonghobae Aug 16, 2026
3f2b820
test(accuracy): reject boolean metric evidence
seonghobae Aug 16, 2026
1f12562
fix(accuracy): reject boolean metric evidence
seonghobae Aug 16, 2026
8492f2c
docs(changelog): record boolean metric guard
seonghobae Aug 16, 2026
895f17b
docs(accuracy): document boolean evidence boundary
seonghobae Aug 16, 2026
49a1ce2
test(accuracy): reject boolean fixture inputs
seonghobae Aug 16, 2026
e83f3ed
fix(accuracy): reject boolean fixture inputs
seonghobae Aug 16, 2026
b8707ff
docs(changelog): record boolean fixture guards
seonghobae Aug 16, 2026
7f0c24c
docs(accuracy): document boolean fixture boundary
seonghobae Aug 16, 2026
e178558
test(accuracy): fail closed on malformed PCM evidence
seonghobae Aug 16, 2026
262adae
fix(accuracy): validate decoded PCM evidence
seonghobae Aug 16, 2026
e62af57
docs(changelog): record decoded PCM validation
seonghobae Aug 16, 2026
be49133
docs(accuracy): document decoded PCM trust boundary
seonghobae Aug 16, 2026
0af265c
fix(accuracy): preserve positive BPM error contract
seonghobae Aug 16, 2026
a2ebf30
test(accuracy): cover overflowing metric evidence
seonghobae Aug 16, 2026
85a2b99
test(accuracy): reject malformed decoded PCM evidence
seonghobae Aug 16, 2026
0c639e9
fix(accuracy): validate decoded PCM evidence
seonghobae Aug 16, 2026
39b72f6
docs(accuracy): harden decoded evidence contract
seonghobae Aug 16, 2026
a622723
docs(accuracy): record evidence validation repairs
seonghobae Aug 16, 2026
522fce9
test(accuracy): reject boolean metric evidence at report builder
seonghobae Aug 16, 2026
5a37841
fix(accuracy): validate metric evidence before coercion
seonghobae Aug 16, 2026
ddfdf76
test(accuracy): bind checksum and scoring to same bytes
seonghobae Aug 17, 2026
85ef78a
fix(accuracy): score the checksum-verified byte snapshot
seonghobae Aug 17, 2026
da7a2db
test(accuracy): cover checksum compatibility boundary
seonghobae Aug 17, 2026
b44af34
test(accuracy): reject overflowed fixture timing evidence
seonghobae Aug 17, 2026
bec9496
fix(accuracy): reject derived fixture timing overflow
seonghobae Aug 17, 2026
d2625a7
test(accuracy): reject zero-sample fixture evidence
seonghobae Aug 17, 2026
c1baa47
fix(accuracy): reject zero-sample fixture evidence
seonghobae Aug 17, 2026
5475670
docs(accuracy): record derived fixture timing guards
seonghobae Aug 17, 2026
77b7cb8
docs(changelog): record fixture timing evidence guards
seonghobae Aug 17, 2026
cf465aa
test(accuracy): reject unrepresentable click timing
seonghobae Aug 17, 2026
b0a0ac1
fix(accuracy): reject unrepresentable click timing
seonghobae Aug 17, 2026
7c93766
docs(changelog): record representable click timing guard
seonghobae Aug 17, 2026
1e8e7cf
docs(accuracy): define sample-representable click evidence
seonghobae Aug 17, 2026
32fb383
test(accuracy): align zero-length click acceptance
seonghobae Aug 17, 2026
e1a53a2
test(accuracy): restore acceptance suite before focused repair
seonghobae Aug 17, 2026
304684a
test(accuracy): fail closed on unrepresentable click pulse
seonghobae Aug 17, 2026
9bd3edb
fix(changelog): preserve released 0.1.4 history
seonghobae Aug 18, 2026
ebaf444
test(accuracy): reject multichannel WAV evidence
seonghobae Aug 18, 2026
c8e4711
test(accuracy): preserve stereo decode contract
seonghobae Aug 18, 2026
5df9fb1
docs(accuracy): align stereo decode evidence boundary
seonghobae Aug 18, 2026
8d32d7f
test(accuracy): bind tempo case identity to truth
seonghobae Aug 18, 2026
40f138e
fix(accuracy): bind tempo case id to truth
seonghobae Aug 18, 2026
fdb83ce
Merge remote-tracking branch 'origin/develop' into HEAD
seonghobae Aug 28, 2026
9144288
fix(accuracy): reject silent aliased click fixtures
seonghobae Aug 28, 2026
fa2abea
fix(accuracy): normalize numeric evidence guards
seonghobae Aug 29, 2026
b1ec3ef
fix(accuracy): bound fixture decode resources
seonghobae Aug 29, 2026
9d2fec2
fix(accuracy): apply header bounds to tempo scoring
seonghobae Aug 29, 2026
dbf88c1
fix(analysis): pin compatible numba runtime
seonghobae Aug 30, 2026
9f50f44
fix(analysis): use stable ARM64 numba runtime
seonghobae Aug 30, 2026
c20e907
Merge remote-tracking branch 'origin/cursor/bc-d6780991-1682-480d-b36…
seonghobae Aug 30, 2026
e11a0e3
Merge pull request #1064 from ContextualWisdomLab/codex/accuracy-boun…
seonghobae Aug 30, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ This section applies to any agent (Claude, Codex, Cursor, opencode, ...) working
## Architecture references
- `ARCHITECTURE.md`
- `docs/engineering/acceptance-criteria.md`
- `docs/doctoring/real-audio-accuracy-acceptance.md`
- `docs/engineering/harness-engineering.md`
- `docs/workflow/one-day-delivery-plan.md`
- `docs/workflow/pr-continuity.md`
Expand All @@ -85,6 +86,7 @@ This section applies to any agent (Claude, Codex, Cursor, opencode, ...) working
- Prefer practical, friendly, rehearsal-first wording over academic or authority-heavy language.
- Do not reduce the product to a chord analyzer when form, timing, player coordination, playable ranges, simplification, and setup cues are the real rehearsal blockers.
- Do not frame usability as a reason to accept weak analysis quality; BandScope should aim for both easy use and high accuracy.
- Do not claim harmony or tempo accuracy from mocked feature matrices. Tier 1 decoded-PCM fixtures and metrics live in `docs/doctoring/real-audio-accuracy-acceptance.md`.

## Safety
- Do not add network-dependent runtime paths for local analysis.
Expand Down
3 changes: 2 additions & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ARCHITECTURE.md

Last updated: 2026-03-11
Last updated: 2026-08-16

## Brand source

Expand Down Expand Up @@ -127,3 +127,4 @@ Last updated: 2026-03-11
- Supply-chain docs, workflow pinning, and lockfile verification are part of the default quickcheck path so dependency drift is caught early.
- Quickcheck and CI are expected to verify dependency review, audit, supplemental inventory, and SBOM baseline presence as part of bootstrap.
- Cross-platform build workflow presence and trigger coverage are part of the default supply-chain verification path.
- Real-audio accuracy acceptance is a separate buyer-facing gate from unit coverage. Tier 1 decoded-PCM fixtures, duration-weighted chord recall, and tempo Acc1 live in `docs/doctoring/real-audio-accuracy-acceptance.md` and `bandscope_analysis.accuracy`.
26 changes: 25 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,30 @@

### Added

- Add Tier 1 real-audio accuracy acceptance: a decoded C major WAV must recover `C`, and a 120 BPM click WAV must pass tempo Acc1, with checksum-fail-closed reports.

### Fixed

- Reject click-track accuracy fixtures whose beat interval or click pulse cannot be represented by at least one sample, preventing aliased or all-zero tempo evidence from entering acceptance runs.
- Reject click-track fixtures whose chosen pulse frequency aliases to a zero signal, so silent WAVs cannot become tempo acceptance evidence.
- Reject fixture timings whose derived sample count overflows or quantizes below one sample, and reject finite BPM inputs whose derived beat interval becomes non-finite, before they can drive real-audio accuracy allocation or loop authority.
- Reject malformed decoded-PCM accuracy evidence before chord recognition: the C-major acceptance buffer must be a non-empty, finite, floating-point, one-dimensional mono array and sample rates must be finite, positive, and non-Boolean.
- Reject non-numeric accuracy timing, tempo, fixture, and sample-rate evidence at the documented ValueError boundary instead of leaking NumPy or math TypeErrors.
- Reject accuracy-report metric integers whose conversion overflows floating-point evidence instead of letting an `OverflowError` escape the manifest validation boundary.
- Reject Boolean fixture duration, tempo, and sample-rate inputs so Python `True`/`False` values cannot become numeric real-audio acceptance authority or a 1 Hz WAV contract.
- Reject Boolean chord timing and tempo metric inputs so `True`/`False` cannot masquerade as numeric MIR acceptance evidence through Python's `bool`-as-`int` semantics.
- Reject blank or edge-whitespace-padded accuracy-report case IDs, metric names, and truth labels so portable acceptance evidence preserves exact registered identities.
- Reject empty or reversed chord-estimate intervals before duration-weighted recall so malformed timing evidence cannot be silently ignored by an accuracy acceptance score.
- Fail closed when accuracy-report provenance cannot resolve a non-empty product `VERSION`, instead of publishing an `unknown` engine version as valid evidence.
- Reject non-finite chord annotation and estimate timings before duration-weighted recall so NaN/Inf evidence cannot fabricate covered duration.
- Reject non-finite tempo estimates, ground-truth BPM values, and Acc1 tolerances as invalid accuracy evidence instead of recording them as ordinary misses.
- Select Numba 0.66.0 for ARM64 librosa beat tracking after the prior 0.62.1
and 0.63.1 resolutions reproducibly crashed on macOS arm64, while retaining
the 0.62.1 Intel resolution because the newer release has no macOS Intel wheel.
- Union overlapping matching chord-estimate intervals before duration-weighted recall so acceptance scores cannot double-count annotated time or exceed 100%.
- Reject malformed accuracy-report provenance, including non-hex SHA-256 text and non-finite metric values, before acceptance evidence is consumed.
- Bound accuracy fixture byte size, channel count, sample rate, and decoded duration before checksum staging or PCM allocation.
- Score the C major acceptance case from checksummed on-disk WAV bytes instead of the pre-write in-memory triad.
- Name tonight's first playable range on the ready rehearsal map and tell the player to check that span on their instrument before the section.
- Display the analyzed song tempo (BPM) as a badge in the rehearsal workspace.
- 각 합주 역할(Role)별 개인 연습 진행도를 0~100% 범위로 기록 및 시각화할 수 있는 연습 진척도(`practiceProgress`) 트래커 기능 추가. UI 컨트롤(슬라이더 및 +/- 버튼)과 한/영 다국어 지원 포함.
Expand Down Expand Up @@ -74,4 +98,4 @@

- `ChordsFeature` (코드 분석) 화면에서 각 파트(Role)의 `transpositionPlan`(이조/조옮김 계획)을 표시하는 기능을 추가했습니다.
- `RangesFeature` (음역대 분석) 화면에서 겹침 경고(Overlap warning) 외에 해당 파트의 채보(Transcription) 가능 노드 수를 요약하여 보여주는 기능을 추가했습니다.
- 신규 UI 요소에 대한 단위 테스트를 추가했습니다 (`apps/desktop/src/features/chords/index.test.tsx`, `apps/desktop/src/features/ranges/index.test.tsx`).
- 신규 UI 요소에 대한 단위 테스트를 추가했습니다 (`apps/desktop/src/features/chords/index.test.tsx`, `apps/desktop/src/features/ranges/index.test.tsx`).
1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,3 +73,4 @@ Supporting packages:
- Lockfiles (`package-lock.json`, `uv.lock`, `Cargo.lock`) are committed and must stay in sync; GitHub Actions are SHA-pinned. Adding a direct dependency requires the admission rationale defined in `AGENTS.md` and `docs/security/dependency-policy.md`.
- CI beyond quickcheck: `gate / ci / rust-check` (Tauri cargo check on macOS) and `build-baseline` Windows/macOS amd64+arm64 native builds are merge gates, alongside CodeQL, dependency-review, sbom, bandit, trivy, secret-scan, and security-audit workflows. Do not weaken or skip them.
- Version metadata lives in `VERSION`, the root `package.json`, and `CHANGELOG.md`; release flow is tag-driven (see `docs/operations/deploy-runbook.md`).
- Real-audio accuracy acceptance (decoded PCM, not mocked chroma) lives in `docs/doctoring/real-audio-accuracy-acceptance.md` and `bandscope_analysis.accuracy`.
161 changes: 161 additions & 0 deletions docs/doctoring/real-audio-accuracy-acceptance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
# Real-audio accuracy acceptance

Next action: before claiming a rehearsal result is accurate, run the Tier 1
decoded-PCM cases. A green unit suite that never opens a WAV is not buyer
evidence.

```bash
uv run --project services/analysis-engine pytest \
services/analysis-engine/tests/test_accuracy_acceptance.py \
services/analysis-engine/tests/test_accuracy_boolean_fixture_inputs.py \
services/analysis-engine/tests/test_accuracy_pcm_input_contract.py \
services/analysis-engine/tests/test_accuracy_manifest_version.py \
services/analysis-engine/tests/test_accuracy_metric_interval_contract.py \
services/analysis-engine/tests/test_accuracy_metric_overflow_contract.py
```

## Why this lock exists

BandScope sells rehearsal guidance. A buyer cannot distinguish “the job
succeeded” from “the engine heard the chord and tempo that are in the file”
until decoded PCM is scored against a known label (Raffel et al., 2014).

This layer is Tier 1 of issue #770: tiny, license-clean, deterministic WAV
fixtures generated in process. It does not claim genre coverage, perceptual
stem quality, or private-corpus readiness.

## Platform runtime compatibility

The decoded `click-120-bpm` case exercises `TemporalAnalyzer`'s production
`librosa.beat_track` path. On macOS arm64 with Python 3.12, the locked Numba
`0.62.1` and `0.63.1` runtimes reproducibly crashed inside librosa's Numba
gufunc during that case; `numba==0.66.0` completed the same path and the full
Tier 1 suite. The analysis engine therefore selects `numba==0.66.0` on ARM64
and retains `numba==0.62.1` on non-ARM targets because 0.66.0 does not publish
macOS Intel wheels required by the cross-platform build lane. The complete
`uv.lock` is regenerated by uv. This is a compatibility floor, not an accuracy
claim or a security exception.

Security Notes: the dependency change remains within the existing local-only
audio boundary. No network or subprocess authority is added; dependency
review, audit, SBOM, and cross-platform build gates remain required.

## Held cases

- `c-major-triad`: three seconds of C4+E4+G4 written to WAV, checksummed,
decoded from those bytes, and scored with duration-weighted chord recall.
Pass when recall of `C` is at least `0.70`. That floor is a BandScope Tier 1
tolerance. Matching estimate intervals are clipped to the annotation window
and unioned before duration is accumulated, so overlapping or duplicate
estimates cannot count the same annotated time twice or produce recall above
`1.0`. Annotation and estimate times must be finite non-Boolean numbers before
clipping, and each estimate interval must have a strictly increasing
start/end pair; Boolean, NaN, infinite, empty, or reversed timing is invalid
acceptance evidence and fails closed instead of being silently ignored or
allowed to fabricate covered duration. File-backed WAV evaluation accepts an
ordinary mono or multichannel WAV and deterministically downmixes decoded
multichannel samples to mono before the production recognizer runs. The direct
`evaluate_c_major_pcm` boundary is narrower: callers must provide a non-empty,
one-dimensional, finite floating-point mono PCM array and a finite positive
non-Boolean sample rate. Unresolved multichannel arrays passed directly,
integer arrays, NaN/infinite samples, or invalid rate evidence are rejected
rather than coerced at that direct evaluator boundary. The metric family is
WCSR/CSR (Odekerken et al., 2021; Raffel et al., 2014).
- `click-120-bpm`: eight seconds of 120 BPM clicks decoded by
`TemporalAnalyzer`. Pass when estimated tempo satisfies Acc1 at 4%
(Schreiber & Müller, 2020). Acc1 does not credit half-time or double-time.
Estimated BPM, true BPM, and the tolerance must all be finite non-Boolean
numbers; Boolean or non-finite metric inputs are invalid acceptance evidence
and fail closed rather than being recorded as an ordinary miss. This matters
in Python because `bool` is an integer subtype and would otherwise satisfy
ordinary numeric comparisons.
- Fixture generation and WAV serialization accept only finite positive
non-Boolean duration, BPM, and sample-rate evidence. In Python, `True` would
otherwise act as numeric `1`, which could create a one-second fixture, a
one-BPM click contract, or a one-Hz WAV while still looking type-compatible at
runtime. Derived fixture timing must also remain representable: the computed
fixture sample count, beat interval, and click-pulse width must each be finite
and must resolve to at least one sample at the requested rate. This prevents a
nominally valid high BPM from placing multiple beats onto the same sample and
rejects a low sample rate that produces an all-zero “click” fixture. Inputs
that overflow or undersample those derived quantities fail closed before
allocation, loop construction, or file serialization.
- Checksum mismatch fails closed on both file evaluators. Do not score a
tampered file as a pass.
- Machine-readable case reports are accepted only when the registered
provenance fields are present and typed, `audio_sha256` is exactly 64
hexadecimal characters, and `metric_value` is a finite numeric value.
Boolean, NaN, infinity, overflow-to-float, malformed digest, or missing-field
evidence fails closed rather than becoming a portable acceptance record.
- When a caller does not provide an explicit engine version, report creation
resolves the repository product `VERSION`. Missing or empty `VERSION`
provenance fails closed; `unknown` is not accepted as a substitute for the
exact engine version required by the accuracy evidence contract.

## Claim boundary

A passing case supports only the registered fixture, metric, engine version,
and tolerance. It does not establish universal musical correctness.

Keys-left, keys-right, and acoustic-guitar roles still carry arrangement
defaults (`C#`, `Emaj7`, `Eb`). Lead vocal harmony is the role that currently
reflects the `other` stem recognizer. Do not treat those defaults as
measured accuracy.

## References

Odekerken, D., Koops, H. V., & Volk, A. (2021). Improving audio chord
estimation by alignment and integration of crowd-sourced symbolic music.
*Transactions of the International Society for Music Information Retrieval,
4*(1), 141–155. https://doi.org/10.5334/tismir.81

Raffel, C., McFee, B., Humphrey, E. J., Salamon, J., Nieto, O., Liang, D., &
Ellis, D. P. W. (2014). MIR_EVAL: A transparent implementation of common MIR
metrics. In *Proceedings of the 15th International Society for Music
Information Retrieval Conference* (pp. 367–372).

Schreiber, H., & Müller, M. (2020). Music tempo estimation: Are we done yet?
*Transactions of the International Society for Music Information Retrieval,
3*(1), 111–125. https://doi.org/10.5334/tismir.43

## Security Notes

- Attack surface: generated WAV bytes, fixture duration/BPM/sample-rate inputs,
decoded PCM shape/dtype/finiteness/sample-rate evidence, SHA-256 digests,
recognizer segment timings, tempo estimates, product-version provenance, and
parsed case-report mappings passed into the accuracy acceptance path.
- Trust boundary: untrusted audio, runtime numeric evidence, decoded buffer
structure, recognizer output, and manifests; trusted repo-controlled fixture
definitions, true labels, metric definitions, registered floors, and the
repository product `VERSION`.
- Mitigations: no network, no shell, checksum fail-closed before C-major
decode and before tempo scoring, deterministic file-decoder downmix for
ordinary mono/stereo and bounded multichannel WAV input, a 100 MiB fixture
byte cap before checksum reads, an eight-channel cap, a 192 kHz sample-rate
cap, and a 15-minute decoded-duration cap, non-empty finite floating-point
one-dimensional mono PCM admission at the direct C-major evaluator boundary,
finite positive non-Boolean decoded sample-rate evidence, overlap-safe chord
duration, finite non-Boolean annotation/estimate timing, strictly increasing
estimate intervals, finite non-Boolean tempo metric inputs, finite positive
non-Boolean fixture duration/BPM/sample-rate inputs, finite derived fixture
sample count, beat interval, and click-pulse width with a one-sample minimum,
plus a non-zero click signal check,
strict SHA-256 syntax, finite-only report metric values including overflow
rejection, exact non-empty product-version provenance, bounded fixture
durations, and no copyrighted commercial recordings. Fixture paths are pytest
temp files; reports store SHA-256 and labels, not waveform bytes.
- Test points: deterministic digest, over-sized fixture rejection before digest
allocation, C major recall after file decode, ordinary stereo WAV downmix at
the file-decoder boundary, excessive channel/rate/duration rejection, direct decoded-PCM
empty/non-floating/non-finite/non-mono rejection and invalid sample-rate
rejection, overlapping matching intervals do not double-count annotation
duration, non-finite and Boolean chord annotation/estimate timing rejection,
empty/reversed estimate interval rejection, silence-on-disk vs in-memory
triad, 120 BPM Acc1, non-finite and Boolean tempo estimate / truth / tolerance
rejection, non-finite and Boolean fixture generation/WAV sample-rate
rejection, derived sample-count overflow and zero-sample rejection, aliased
zero-signal rejection, beat-interval overflow and sub-one-sample rejection,
sub-one-sample click-pulse
rejection, checksum mismatch through both file evaluators, malformed/non-hex
manifest provenance, NaN/infinity/overflow report rejection, missing/empty
product `VERSION` rejection, and silence must not pass as C major.
60 changes: 60 additions & 0 deletions docs/plans/2026-08-16-real-audio-accuracy-acceptance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Real-audio accuracy acceptance (Tier 1)

**Goal:** Give BandScope a buyer-readable accuracy gate: decoded PCM from a
known WAV must recover the expected chord or tempo, with a versioned report.

**Architecture:** A new `bandscope_analysis.accuracy` package generates
license-clean fixtures, scores them with documented MIR metrics, and emits a
strict case-report schema. Tests write real WAV files and call production
`ChordRecognizer` and `TemporalAnalyzer` helpers. Stem separation stays out
of this slice.

**Tech Stack:** NumPy, soundfile, librosa beat tracking, pytest.

## Security Notes

### Attack surface

Generated WAV files, SHA-256 digests, decoded PCM arrays, and JSON-shaped
accuracy reports.

### Trust boundary

Untrusted: any on-disk fixture bytes and any parsed report mapping.
Trusted: in-repo generators, metric definitions, and registered floors.

### Mitigations

- No network and no shell interpolation.
- Checksum mismatch raises before C-major decode and before tempo scoring.
- Fixture bytes are capped before hashing; WAV headers are bounded by channel,
sample-rate, and duration limits before decoded PCM allocation.
- Manifest parsing fails closed on missing or mistyped fields.
- Fixtures are short, synthetic, and license-clean.

### Test points

- Deterministic C major digest
- Duration-weighted C recall after file decode
- Silence on disk fails even when a C major array exists in memory
- 120 BPM Acc1 after file decode
- Checksum mismatch through both file evaluators
- Oversized, excessive-channel, excessive-rate, and excessive-duration input rejection
- Malformed report rejection
- Silence must not pass as C major

### Realistic threats

A tampered fixture or a missing digest could be scored as a pass and then
cited as release evidence.

### Logging and privacy

`TemporalAnalyzer` may log the fixture path. Tests use pytest `tmp_path`
names only. Case reports keep SHA-256, metric, and labels; they do not store
PCM.

### Remaining risk

Tier 1 does not cover Demucs stems, private commercial recordings, or
CPU/GPU numeric parity. Those remain later #770 tiers.
7 changes: 6 additions & 1 deletion services/analysis-engine/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,12 @@ requires-python = ">=3.12"
dependencies = [
"demucs>=4.0.1 ; sys_platform != 'darwin' or platform_machine == 'arm64'",
"librosa>=0.11.0",
"numba<0.67.0",
# librosa 0.11.0's beat tracker segfaults on macOS arm64 with numba 0.62.1
# and 0.63.1; 0.66.0 is the first tested release that keeps this path alive.
"numba==0.66.0 ; platform_machine == 'arm64'",
# Numba 0.66.0 has no macOS Intel wheel; retain the x86 package resolution
# for the mandatory macOS amd64 build lane.
"numba==0.62.1 ; platform_machine != 'arm64'",
"numpy>=1.26",
"soundfile>=0.13.1",
"urllib3>=2.7.0",
Expand Down
Loading
Loading