-
Notifications
You must be signed in to change notification settings - Fork 1
feat(analysis): enforce one canonical audio resource policy (#781) #985
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
seonghobae
wants to merge
51
commits into
develop
Choose a base branch
from
feat/canonical-audio-resource-policy-781
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
51 commits
Select commit
Hold shift + click to select a range
47a2e7c
feat(analysis): enforce one canonical audio resource policy (#781)
seonghobae 80c43f3
test: reject non-numeric decoded audio dtypes
seonghobae 706a80d
fix: fail closed on non-numeric decoded audio
seonghobae d104a40
docs: align audio policy rejection catalog
seonghobae bb62c80
test: preserve YouTube resource-policy reason codes
seonghobae c0b7422
fix: preserve YouTube policy rejection reasons
seonghobae 4c3d178
test: reject non-finite separator decoder output
seonghobae 713b1f2
test: keep separator policy regression type-safe
seonghobae b1b5bb2
fix: validate separator output before finite normalization
seonghobae ab2562f
test: keep empty chord buffers shape-invariant
seonghobae 37d0da0
test: define bounded audio metadata preflight contract
seonghobae 2f2fe18
feat: preflight untrusted audio container metadata
seonghobae a040f09
fix(analysis): fail closed cleanly on metadata probe errors
seonghobae d2cf204
test(analysis): keep metadata preflight lint-clean
seonghobae 98b70bc
Merge remote-tracking branch 'origin/develop' into HEAD
seonghobae 2ed5383
fix(analysis): close audio resource edge contracts
seonghobae 5d91c05
test(audio): reproduce pre-decode metadata admission gap
seonghobae 90681ef
fix(audio): preflight temporal source metadata before decode
seonghobae ad67cba
fix(audio): preflight transcription metadata before decode
seonghobae 18f8a9c
fix(audio): preflight separation source metadata before decode
seonghobae 5ae317e
test(audio): isolate transcription decode-duration contract
seonghobae e68d9a9
test(audio): isolate temporal decoder contracts from metadata admission
seonghobae 4e0d514
test(audio): isolate decoded separation policy contract
seonghobae 86a9719
test(audio): isolate separation decoder failure contracts
seonghobae 33d14e4
fix(audio): enforce desktop resource byte ceiling
seonghobae 0e2486a
fix(policy): fail closed on oversized metadata
seonghobae b8a6db4
refactor(policy): reuse rejection helper
seonghobae 669d1d0
fix(audio): preserve compressed path decoder support
seonghobae f3b040e
fix(audio): pin audioread runtime dependency
seonghobae 6bfe3c0
docs(audio): align metadata boundary notes
seonghobae 209cc2f
fix(audio): preserve compressed path decoder support
seonghobae ae58626
test(audio): require one bounded decode authority
seonghobae 0bacf79
fix(audio): centralize bounded PCM decode authority
seonghobae f71d46c
refactor(audio): route temporal decode through owned port
seonghobae f1a362e
test(audio): bind temporal regressions to decode port
seonghobae 9d0fbb2
refactor(audio): route bass decode through owned port
seonghobae a62329f
test(audio): bind bass transcription to decode port
seonghobae 2149b90
test(audio): require semantic decode port identifiers
seonghobae 05637b0
fix(audio): use semantic decode port identifiers
seonghobae 76ca943
test(audio): require semantic metadata identifiers
seonghobae a104929
fix(audio): use semantic metadata port identifiers
seonghobae bf98029
test(audio): lock semantic resource policy vocabulary
seonghobae cf42b0c
fix(audio): use semantic resource policy vocabulary
seonghobae ad3b32a
test(audio): follow semantic policy error contract
seonghobae ff309c6
fix(audio): follow semantic policy rejection contract
seonghobae b24a86b
fix(audio): consume semantic policy rejection fields
seonghobae e09720e
test(audio): follow semantic metadata rejection fields
seonghobae f616320
test(audio): adopt semantic policy vocabulary
seonghobae 539cb38
test(audio): follow semantic policy rejection field
seonghobae 37978b6
test(audio): align separator rejection contract
seonghobae 071c1c8
fix(audio): import decoder warnings from owning boundary
seonghobae File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,98 @@ | ||
| # Canonical audio resource policy | ||
|
|
||
| BandScope admits one rehearsal recording at a time. Every intake path, decoder, | ||
| and feature analyzer must apply the same versioned resource budget before | ||
| expensive work starts. | ||
|
|
||
| ## Published policy (version 1) | ||
|
|
||
| | Bound | Value | Why this number | | ||
| | --- | --- | --- | | ||
| | Encoded file bytes | 100 MiB inclusive | Existing temporal intake ceiling; long enough for a 15-minute stereo rehearsal capture without inviting decompression bombs. | | ||
| | Duration | 0.05 s through 15 minutes inclusive | Existing temporal and YouTube intake evidence. Not an invented five-minute cap. | | ||
| | Source sampling rate | 8 kHz through 192 kHz | Covers phone voice notes through high-rate interface captures. Feature DSP may resample after admission (bass pYIN at 22 050 Hz is allowed). | | ||
| | Target sampling rate | 44 100 Hz | Compact-disc PCM rate used by temporal analysis and stem separation. | | ||
| | Channel count | Mono or stereo | Rehearsal recordings are not multichannel session stems. | | ||
| | Decoded sample count | `15 × 60 × 44100` | Checked product of duration and target rate. | | ||
| | Decoded memory | sample count × 2 channels × 4 bytes | Float32 stereo estimate; overflow fails closed. | | ||
|
|
||
| ## Next action copy | ||
|
|
||
| Rejection copy is payload-free. It names the next rehearsal action and never | ||
| echoes paths, sizes, durations, or header bytes: | ||
|
|
||
| - Choose a shorter or smaller song file to start analysis. | ||
| - Choose a song shorter than 15 minutes to start analysis. | ||
| - Choose a shorter song file to start analysis. | ||
| - Choose a longer song file to start analysis. | ||
| - Choose a WAV, MP3, FLAC, or M4A file recorded at a standard sample rate. | ||
| - Choose a mono or stereo song file to start analysis. | ||
| - Choose another song file. This one could not be measured safely. | ||
| - Choose another song file. This one could not be read as audio. | ||
|
|
||
| Audit metadata records `policy_version` and `reason` on | ||
| `AudioResourcePolicyError`. Those fields stay off the user-facing string. | ||
|
|
||
| ## Validation order | ||
|
|
||
| 1. Encoded byte size, before open/decode, where the filesystem size is | ||
| trustworthy as an upper bound. | ||
| 2. YouTube metadata duration ceiling, before download. | ||
| 3. Decode with the canonical duration bound as a loader safety cap, not as a | ||
| silent shorter feature policy. | ||
| 4. Revalidate decoded arrays because container metadata is untrusted: layout, | ||
| sampling rate, sample count, wall-clock duration, and memory estimate. | ||
| 5. Feature DSP (chromagram hop, pYIN 22 050 Hz, Demucs split) runs only on an | ||
| admitted buffer. | ||
|
|
||
| Path-backed temporal and separation intake keeps the path form for librosa's | ||
| existing audioread fallback, so an M4A container that libsndfile cannot inspect | ||
| can still be admitted from local decoder metadata. File-like stem bytes retain | ||
| the libsndfile metadata contract. | ||
|
|
||
| ## Consumers | ||
|
|
||
| - `bandscope_analysis.audio_resource_policy` — versioned policy and validators | ||
| - `temporal.analyzer` — local file preflight and decoded revalidation | ||
| - `separation.audio_separator` — stem decode preflight and decoded revalidation | ||
| - `youtube.download_youtube_audio` — duration ceiling and 100 MiB encoded budget | ||
| - `transcription.api` — stem byte budget and 15-minute loader cap (no 120 s silent cap) | ||
| - `chords.chord_recognizer` — decoded revalidation at `recognize()` | ||
|
|
||
| Desktop Rust enforces this encoded-byte ceiling at local-file selection and | ||
| again when accepting the downloaded YouTube file into the app-owned cache. The | ||
| Python engine repeats the check before analysis as a defense-in-depth boundary. | ||
|
|
||
| ## Rollback | ||
|
|
||
| Revert this slice to restore feature-local limits (YouTube 50 MiB, bass | ||
| transcription 120 s, payload-bearing size errors). Do not leave a mix of | ||
| canonical validators and the old silent caps on the same branch. | ||
|
|
||
| ## Security Notes | ||
|
|
||
| - Attack surface: untrusted local files, YouTube containers, local decoder | ||
| fallback processes, decoder output, | ||
| and caller-supplied NumPy arrays. | ||
| - Trust boundary: this policy classifies resources only. It does not open | ||
| files, follow paths, or talk to the network. | ||
| - Mitigations: checked integer products, fail-closed non-finite metadata, | ||
| payload-free copy, decoded revalidation after untrusted headers, and the | ||
| existing fixed audioread decoder path only for already-resolved local files. | ||
| - Test points: inclusive ceilings, next-byte/next-millisecond rejections, | ||
| empty and malformed metadata, decoded expansion, overflow, provenance. | ||
| - Realistic threats: decompression bombs, huge channel counts, extreme | ||
| sampling rates, integer overflow in size conversions, inconsistent | ||
| feature-local caps that fail only after expensive work. | ||
| - Remaining risk: compressed-container preflight depends on an available local | ||
| decoder and duration still requires decoder metadata; GPU/VRAM budgets are | ||
| not part of policy version 1. | ||
|
|
||
| ## References | ||
|
|
||
| International Electrotechnical Commission. (1999). *Compact disc digital audio | ||
| system* (IEC 60908). Geneva, Switzerland: IEC. | ||
|
|
||
| National Institute of Standards and Technology. (2020). *Security and privacy | ||
| controls for information systems and organizations* (NIST Special Publication | ||
| 800-53 Rev. 5). https://doi.org/10.6028/NIST.SP.800-53r5 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
103 changes: 103 additions & 0 deletions
103
services/analysis-engine/src/bandscope_analysis/audio_decode.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,103 @@ | ||
| """Bounded PCM decode port for the Resource Admission & Decode context. | ||
|
|
||
| This module is the single analysis-engine boundary that may turn an admitted | ||
| container into PCM. The current adapter still delegates to ``librosa`` and is | ||
| therefore a transitional owner while #1129 removes the libsndfile-backed | ||
| runtime graph. Consumers must call this port rather than selecting decoder | ||
| fallbacks themselves. | ||
|
|
||
| Security Notes: | ||
| - Untrusted input: caller-authorized local audio handles/paths, container | ||
| metadata, decoder output, and third-party decoder exceptions. | ||
| - Trust boundary: metadata admission happens before PCM decode and decoded PCM | ||
| is revalidated before it can enter MIR/DSP consumers. | ||
| - Privacy: third-party decoder details are retained only as exception causes; | ||
| the surfaced resource-policy error is stable and payload-free. | ||
| - Dependency boundary: this adapter deliberately centralizes the legacy | ||
| librosa decoder so the commercial #1129 replacement has one owned seam. It | ||
| does not claim that libsndfile has already been removed. | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import warnings | ||
| from typing import cast | ||
|
|
||
| import librosa | ||
| import numpy as np | ||
| from numpy.typing import NDArray | ||
|
|
||
| from bandscope_analysis.audio_metadata import AudioSource, preflight_audio_metadata | ||
| from bandscope_analysis.audio_resource_policy import ( | ||
| DEFAULT_AUDIO_RESOURCE_POLICY, | ||
| AudioResourcePolicy, | ||
| AudioResourcePolicyError, | ||
| policy_rejection_message, | ||
| validate_decoded_audio, | ||
| ) | ||
|
|
||
| KNOWN_LIBROSA_NUMBA_WARNING_FILTERS = ( | ||
| (DeprecationWarning, r".*pkg_resources is deprecated.*", r".*librosa.*"), | ||
| (FutureWarning, r".*Numba.*", r".*numba.*"), | ||
| ) | ||
|
|
||
| AudioMonoArray = NDArray[np.float32] | ||
|
|
||
|
|
||
| def _malformed_decode_error() -> AudioResourcePolicyError: | ||
| """Build the stable payload-free decoder failure.""" | ||
| return AudioResourcePolicyError( | ||
| "malformed_header", policy_rejection_message("malformed_header") | ||
| ) | ||
|
|
||
|
|
||
| def decode_mono_audio( | ||
| audio_source: AudioSource, | ||
| *, | ||
| target_sample_rate_hz: int, | ||
| max_duration_seconds: float, | ||
| audio_resource_policy: AudioResourcePolicy = DEFAULT_AUDIO_RESOURCE_POLICY, | ||
| ) -> tuple[AudioMonoArray, int]: | ||
| """Admit and decode one source to bounded mono float32 PCM. | ||
|
|
||
| The audio source is first measured against the same resource policy that | ||
| later revalidates the decoded buffer. Decoder/provider detail never becomes | ||
| the surfaced error message. | ||
| """ | ||
| preflight_audio_metadata(audio_source, audio_resource_policy) | ||
|
|
||
| try: | ||
| with warnings.catch_warnings(): | ||
| warnings.filterwarnings( | ||
| "ignore", category=DeprecationWarning, module=r"^audioread" | ||
| ) | ||
| warnings.filterwarnings( | ||
| "ignore", category=FutureWarning, module=r"^audioread" | ||
| ) | ||
| for ( | ||
| warning_category, | ||
| warning_message, | ||
| warning_module, | ||
| ) in KNOWN_LIBROSA_NUMBA_WARNING_FILTERS: | ||
| warnings.filterwarnings( | ||
| "ignore", | ||
| category=warning_category, | ||
| message=warning_message, | ||
| module=warning_module, | ||
| ) | ||
| decoded_audio, sample_rate_hz = librosa.load( # type: ignore[arg-type] | ||
| audio_source, | ||
| sr=target_sample_rate_hz, | ||
| mono=True, | ||
| duration=max_duration_seconds, | ||
| ) | ||
| except Exception as decode_error: | ||
| raise _malformed_decode_error() from decode_error | ||
|
|
||
| try: | ||
| pcm_audio = np.ravel(np.asarray(decoded_audio, dtype=np.float32)) | ||
| except (TypeError, ValueError) as decode_error: | ||
| raise _malformed_decode_error() from decode_error | ||
|
|
||
| validate_decoded_audio(pcm_audio, sample_rate_hz, audio_resource_policy) | ||
| return cast(AudioMonoArray, pcm_audio), int(sample_rate_hz) |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.