docs(audio): document the audio effects system - #3420
Conversation
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
Restructured after review: audio has its own Studio groupThe first pass put one page in Studio / Edit. That was wrong for a reason worth stating: Studio now has an Audio group: The carve gets its own page because it is the feature people arrive for, and it is the one where the obvious fix (turn the music down) is the wrong one. Why not a new top-level tabThe tabs here are audience-scoped — Guides to learn, Studio for the app, Catalog for components, Developers for internals — not feature-scoped. Audio is spread across four tabs exactly the way color grading and captions are. A feature tab would be the only one of its kind, and it would either move the prompting chapter out of its Level 5 sequence and the reference page out of Developers, or list them twice. So the split happened inside Studio, where the app-surface pages belong, and the other two stayed with their audiences. Also in this pass
|
miguel-heygen
left a comment
There was a problem hiding this comment.
Strengths: docs/docs.json:216-225 gives Audio a task-shaped Studio group instead of overloading Edit, and each new Studio page ends with the required visible related links. studio/voiceover-carve.mdx:24-89 also explains the target-vs-source relationship and durable group form clearly.
Blocker — the promised complete registry is already incomplete at this exact head: reference/audio-effects.mdx:155 says fifteen effects and the tables omit pitchshift, but packages/core/src/audioFx.ts:509-535 ships it with semitones -12..12 and mix 0..1 (neither automatable). reference/audio-effects.mdx:237-245 says nineteen presets and studio/audio-effects.mdx:64-66 lists seven Character presets, while the same head has 22 total and ships chipmunk, giant, and monster at audioFxPresets.ts:336-357. Update counts, the registry/automation explanation, and both audience lists.
Blocker — the copyable HTML contradicts the parser warning directly above it: reference/audio-effects.mdx:27-31 correctly says carve.mjs only finds double-quoted attributes, then the examples at :45, :77, and :104 use single-quoted JSON attributes. Copying the chain example makes a later carve miss the existing chain and overwrite work; copying carve/automation can leave duplicate or unstripped attributes. Use double-quoted attributes with "/& as the page instructs. Also fix :18: data-audio-group is a plain id, not JSON, is ignored on video, and group-level chain/automation live on <hf-audio-group> too — all four do not simply go on audio/video as JSON.
Blocker — the automation contract is wrong on two axes: reference/audio-effects.mdx:86 and studio/audio-automation.mdx:50-52 say every lane is clip-relative, but group lanes are explicitly composition-time (webAudioTransport.ts:337-342; audioMixer.ts:1311-1344) because a group has no clip start. Split track and group clocks. reference/audio-effects.mdx:87 also says volume is 0–1; audioAutomation.ts:162-169 uses MAX_AUDIO_GAIN, which is +12 dB / about 3.981 (audioGain.ts:8-9). A copied boosted-volume lane is valid and should be documented.
Important current-main drift: the PR body says the CLI writes clip ids, but #3416 is now on main: carve.mjs records a safe shared voice group and refuses/falls back to ids when that group contains the bed or music/SFX. Refresh the command-line section (reference/audio-effects.mdx:338-364) and rebase so the docs describe the behavior that will ship beside them.
Audited: all five new task/prompt pages, the full reference page, navigation changes, and the touched guide handoffs; verified the cited wire/range/count claims against source.
Verdict: REQUEST CHANGES
Reasoning: The information architecture is good, but a reference advertised as exhaustive omits shipped effects/presets and gives copyable markup, time-base, and volume contracts that produce incorrect compositions.
— Magi
The effects feature had no documentation at all — the only mentions anywhere in docs/ were changelog entries. This adds the three pages the color-grading feature already has, one per audience, and closes a gap in the existing audio guide. - prompting/audio-effects — a new Level 5 chapter on asking for a mix in symptoms rather than in filters, with the voiceover carve as the headline, the level-before-depth check, groups, and the three requests that have no honest answer (de-essing, noise removal, tone matching). - studio/audio-effects — the rack, presets by symptom, the carve module, groups, the deliberately asymmetric mute/solo, automation lanes, and troubleshooting. - reference/audio-effects — the contract: all four attributes, every effect and parameter range, why some parameters cannot be automated, the 19 presets, the five jobs and five one-knob profiles, carve semantics, the group model, the render bus, preview/render parity, and the three lint rules. Also points the existing "duck music under important speech" advice in guides/voice-and-audio at the carve, which does that properly and was never named there. Facts verified against source rather than the shipped skill prose: the panel section is "Audio FX" (the neighbouring "Effects" section is visual effects, and an earlier draft of this page named the wrong one), MAX_AUTOMATION_POINTS is 512, fromPreset carries a preset id rather than a boolean, the leveller targets the track's own 80th percentile, group mute reaches the render while solo never leaves Studio, and the CLI carve currently writes clip ids. No screenshots in this pass, and no placeholders for them either — the pages carry concrete markup and tables instead. Screenshots of the rack, the carve module, and a group row would each earn their place later. mint validate and mint broken-links both pass.
Review feedback: the audio page did not belong in Studio / Edit. That group is already one task per page — canvas, timeline, animation, captions — and a single page covering the rack, the carve, groups, and automation lanes was four tasks bundled together and dropped in beside them. Studio now has an Audio group holding four task pages: - Effects and presets — the rack, presets by symptom, adding single effects in a working order, the one-knob controls, Even Out Levels - Voiceover carve — its own page, because it is the feature people come for - Groups, mute, and solo — including why mute reaches the export and solo cannot - Automation lanes — drawing envelopes, the shape menu, and which parameters cannot move at all Not a new top-level tab: the tabs here are audience-scoped (Guides, Studio, Catalog, Developers), so a feature tab would be the only one of its kind and would strand the prompting chapter out of its Level 5 sequence and the reference page out of Developers. Repointed the deep link in guides/voice-and-audio at the carve's own page. Lane interactions verified in source before documenting: the right-click menu offers Ramp up, Ramp down, Swell, Dip, and Simplify (which needs three points). Deliberately not documented: "clicking a lane label reveals it in the rack", which is not on main. mint validate and mint broken-links both pass.
All four verified in source before fixing; the review was right on every count. **The registry was incomplete.** `pitchshift` ships at `audioFx.ts:509-535` (`semitones` −12–12, `mix` 0–1, worklet-backed so neither automatable) and was missing entirely — sixteen effects, not fifteen. It also joins the worklet list, so five effects expose no automatable parameters rather than four. **Three presets were missing.** `chipmunk`, `giant`, and `monster` ship at `audioFxPresets.ts:336-357`, all built on `pitchshift`. Twenty-two presets, and Character holds ten. Fixed in both the reference table and the Studio list. **The copyable markup contradicted the warning above it.** The page said `carve.mjs` only finds double-quoted attributes and then gave three single-quoted examples — copying the chain example would make a later carve miss the existing chain and overwrite it. All three are now double-quoted with `"`, each followed by its unescaped reading so it stays legible. **The attribute table over-claimed.** `data-audio-group` is a plain id, not JSON, and is ignored on `<video>`; the other three also live on `<hf-audio-group>` for a group. The table now carries shape and valid host per attribute. **The automation contract was wrong on two axes.** A clip lane's `t` is clip-relative but a GROUP lane's is composition time, because a group has no `data-start` (`webAudioTransport.ts:337-342`, `audioMixer.ts:1311-1344`) — both pages now split the two clocks. And `volume` is not 0–1: the ceiling is `MAX_AUDIO_GAIN`, +12 dB or about 3.981 (`audioGain.ts:8-9`), so a boosting lane is valid and documented. **Current-main drift.** #3416 is merged, so the CLI now records the voices' shared group when it is safe and falls back to clip ids when that group contains the bed or a music/SFX member. Documented, including why neither refusal shows up on the run that writes it, and rebased onto main. mint validate and mint broken-links both pass.
b51889d to
f315077
Compare
|
All four blockers fixed at 1. The registry was incomplete — fixed
2. Three presets were missing — fixed
Fixed in the reference table and in the Studio page's Character list, with a line noting the three pitch presets change who is speaking rather than what they are speaking through. 3. The copyable markup contradicted the warning above it — fixedThis was the worst of the four, because following the page produced silent data loss: copy the chain example, and a later carve cannot see the chain it is supposed to preserve. All three JSON examples are now double-quoted with The attribute table also over-claimed. It now carries shape and valid host per attribute:
4. The automation contract was wrong on two axes — fixedTwo clocks. Confirmed in both paths: Volume is not 0–1. Current-main drift — fixed#3416 is merged, so the CLI section now describes what actually ships: it records the voices' shared group when that group is safe, and falls back to clip ids when the group contains the bed or a member classified music or SFX. I also documented the part that makes those two arrangements expensive to debug — neither shows up on the run that writes it, because the analysis sums the voices it detected, so the first pass looks correct however wrong the stored attribute is. Checks
Two things I did not change, and why:
|
miguel-heygen
left a comment
There was a problem hiding this comment.
Closed: the reference now has all 16 effects / 22 presets, correct escaped double-quoted examples, separate clip/group clocks, the +12 dB volume ceiling, and current #3416 group persistence/refusal behavior. Those prior blockers are fixed against source.
Blocker — the Studio add-effect label is still wrong: docs/studio/audio-effects.mdx:74-77 tells the reader the raw-effect menu groups effects under Space. The shipped menu is Filters, Dynamics, Non-linear, Time (propertyPanelFxAddMenu.tsx:22-28), and pitchshift also lives in that time group. A user following the task page will look for a group that does not exist. Rename it to Time and describe the family broadly enough to include pitch shift as well as delay/reverb/modulation.
Blocker — Pitch shift is still missing from the Studio automation warning: docs/studio/audio-automation.mdx:68-72 lists compressor, limiter, gate, and bitcrush as the worklet-backed effects with no automatable parameters, but pitchshift is the fifth and neither semitones nor mix is automatable (audioFx.ts:509-535). The reference was corrected at reference/audio-effects.mdx:252-256; the task page must make the same promise or it teaches users to create an inert lane.
Important: reference/audio-effects.mdx:25-27 says “the other three” JSON attributes live on <hf-audio-group>; that includes data-fx-carve grammatically and contradicts the table immediately above. Only data-fx-chain and data-automation are group metadata. Narrow the sentence.
Nit: the PR body still advertises 15 effects / 19 presets and clip-ids-only CLI behavior even though the final docs and current main have moved. Refresh it so the review summary matches what ships.
Verdict: REQUEST CHANGES
Reasoning: The wire/reference corrections are sound, but the Studio pages still name a nonexistent menu group and omit a shipped effect from the no-automation contract.
— Magi
…tchshift Second review round. All three findings were my own incomplete propagation — I corrected the reference for `pitchshift` last round and left the reader-facing pages behind it. **The add-menu family is `Time`, not `Space`.** `propertyPanelFxAddMenu.tsx:22-28` labels the four groups Filters / Dynamics / Non-linear / Time, and the time group holds pitchshift, delay, chorus, phaser, and reverb. The Studio page sent readers looking for a group that does not exist. It is now a table naming the family and its contents, and the reference's "Time — space and width" heading is retitled, since that description stopped covering the family the moment pitch shift joined it. Also from the same file: the menu offers the named jobs in place of a bare `peaking`, because picking `peaking` is picking a machine and leaving the real decision — which range — for afterwards. Worth saying on the task page. **Pitch shift was missing from both no-automation lists** that a reader actually follows — `studio/audio-automation` and, unflagged but the same defect, the prompting chapter. Five worklet effects in all four places now. Called out explicitly on the Studio page, because a rising pitch is exactly the thing someone reaches for a lane to do, and the lane will not report that it cannot. **Narrowed the group-metadata sentence.** "The other three are JSON, and on a group they live on `<hf-audio-group>`" swept in `data-fx-carve`, contradicting the table directly above it. Only `data-fx-chain` and `data-automation` are group metadata. mint validate and mint broken-links both pass.
|
All three fixed at 1. The add-menu family is
|
jrusso1020
left a comment
There was a problem hiding this comment.
Re-review at 158ced1c, additive to @miguel-heygen's two rounds. I confirmed its findings closed rather than re-deriving them as mine, then spent the pass on a mechanical source-diff of the tables this page promises are exhaustive, plus the nav and link surface.
Prior blocking concerns — all closed at this head. The four from 5000175366 (counts, single-quoted examples, the two clock/volume axes, the #3416 CLI drift) and both from 5000190813:
- The add-menu families are now a table at
studio/audio-effects.mdx:77-82reading Filters / Dynamics / Non-linear / Time, matchingGROUP_LABELatpropertyPanelFxAddMenu.tsx:23-28— and the Time row names delay, reverb, chorus, phaser and pitch shift, which is exactly the five effects carryinggroup: "time". The two correct uses of "Space" (the preset family at:66, reverb's knob at:117) were left alone, which is the right surgical outcome. - Pitch shift is now in the non-automatable set on all three pages that state it:
studio/audio-automation.mdx:70,prompting/audio-effects.mdx:131,reference/audio-effects.mdx:255. No page still says four. reference/audio-effects.mdx:25-28now says onlydata-fx-chainanddata-automationcan sit on an<hf-audio-group>and thatdata-fx-carvebelongs to a clip, which agrees with its own table two lines up.- Renaming
### Timefrom "space and width" to "where a track sits, and how it moves" (:227) was a good self-catch — the old subtitle stopped being true the moment pitch shift joined that group.
Strengths, and these are the load-bearing ones, because a reference advertised as exhaustive is only worth its diff against source:
- Every effect, parameter, range, default and
AUTOflag matchesHF_AUDIO_FXexactly. I extracted all 16 defs frompackages/core/src/audioFx.tsand compared field-by-field againstreference/audio-effects.mdx:188-236, including the params that don't read literally in source because they come from helpers (freq(),qParam(),gainDb()ataudioFx.ts:88-130).lowshelf20–2000 (200),highshelf500–20000 (4000),lowpass100–20000 (8000),peaking'sqdefault of 1 where everything else is 0.707,phaser.out_gain0–2,delay.feedback0.01–0.95,gain−60–12 — not one range, default or automatable flag is off. - The strength table at
:346-351is exact, including a rounding edge.carveProfile(audioCarve.ts:210-229) ismaxCutDb = 2 + s*16,bands = round(1 + s*6),duckDb = s*24. At the 0.25 default that is 6 dB,Math.round(2.5)→ 3 bands, and 6 dB of level room — the row's exact three numbers. At0it is one band withduckDb: 0, matching "spectral only, no level match at all"; at0.5the dip is 10 dB, which is also the figure the source's own comment uses for where a carve starts being heard as an effect. - All five jobs match
audioFxJobs.ts:52-84to the decimal (:298-304), and the claim at:309that every job also ships inside a preset at identical settings holds for all five —boom-tame200/−4/1.4,harsh-tame3200/−3/1.6, and Reduce Mud, Add Clarity and Reduce Boxiness insidevoice-cleanandvoice-broadcast. - The five one-knob profiles match
audioFxProfiles.tseffect-for-effect and label-for-label (:320-326), and the level-matched split — Evenness, Warmth, Space yes; Tightness, Crush no — is the same reason the source gives. - Both clocks are right at both ends:
webAudioTransport.ts:452passeselapsed: compositionTimefor a group lane, andaudioMixer.ts:1314-1316says the group WAV'st=0is composition time. The+12 dB ≈ 3.981ceiling isMAX_AUDIO_GAIN(audioGain.ts:8-9) andVOLUME_RANGE.maxis literally that constant — so "the same ceilingdata-volumeuses" is exactly true, not approximately. - The double-quote warning is precisely correct, which is easy to overstate and isn't:
attrOfisnew RegExp(`\\s${name}="([^"]*)"`, "i")atcarve.mjs:267, and the strip at:542is double-quote-only too, so a single-quoted attribute is invisible to both halves of the round-trip. Zero single-quoteddata-*attributes remain anywhere in the diff. viaX/viaYsupersedecurveonly when both are present — matchingaudioAutomation.ts:44-47("without them the segment falls back tocurve") rather than the looser reading.- The
sourcesparagraph at:411-418is the shipped #3416 semantics including the part that is easy to get wrong.groupSourceRefusalrefuses on bed-in-group and onmusic/sfxstrays, and deliberately permits avoiceorunknownstray — and the page states that carve-out explicitly instead of rounding it off to "any stray is unsafe". That is the subtlest line in that function and the page has it right. - Nav and links are clean: all 518 nav entries in
docs.jsonresolve to files at this head, and all 43 internal links and anchors across the eight changed pages resolve.
important — the CLI sample transcript at :398-404 is not output the program can produce. Three divergences from the two process.stdout.write calls in carve.mjs:
:401readscarve strength 0.25, but the real line is`carve strength ${args.strength}, ${voices.length} voice${…}`(carve.mjs:526) — it always carries a voice count.- The block has no
lanesline, butcarve.mjs:533always prints`lanes N carve[ + N kept]`as the fourth line of that report. :400readsvoice narration (only track left)— but"only track left"is the bed's detection reason (carve.mjs:334), never a voice's. A voice row'swhyis"name looks like a voice"or"plays over the bed"(:358).
Not copyable code, so not a blocker. But a reader diffing their real run against the page will think something went wrong, and (3) attributes a bed-only string to a voice row — in the one block on the page that teaches the detection model. Worth mentioning that this is the same trap I fell into on #3416: I quoted that why ternary off its label without reading which branch produces it.
nit — the 512-point cap is a silent truncation, and this page is scrupulous about silence everywhere else. :120 says "Maximum 512 points per lane"; audioAutomation.ts:326 returns out.slice(0, MAX_AUTOMATION_POINTS), so points past 512 are dropped on read with no error. The same paragraph already warns that a mistyped nodeId "costs you the envelope silently", so one clause in that voice would finish the thought.
nit — studio/audio-groups.mdx:59 inverts on a careless read. Under the column "In the export", mute's cell is "Included — muted audio is dropped from the render". "Included" describes the decision and "dropped" describes the audio, and for mute those two are opposites, so the cell can be parsed as "muted audio is included in the export". Solo's neighbouring "Ignored" is unambiguous because both readings agree there. reference/audio-effects.mdx:434 gets it right with "Reaches the export: Yes — muted members are dropped"; matching that phrasing would remove the ambiguity from the one table where reading it backwards ships a wrong mix.
nit — one PR-body count is off by one. The body says the five-worklet fact is "stated consistently on all four pages that mention it". It is stated consistently, which is the part that matters — but three pages mention it, not four (studio/audio-automation.mdx, prompting/audio-effects.mdx, reference/audio-effects.mdx); studio/audio-effects.mdx only links to the automation page. Cheap to fix while the body is open, and I flag body counts because on #3416 a wrong count of mine ended up quoted in the code.
Note, so it doesn't get filed as an omission later: there is a fourth audio_* lint code, audio_reactive_single_tween_per_group, but it lives in packages/lint/src/rules/gsap.ts:1515 and is about audio-reactive animation sampling, not the mix. "Three rules exist" is correctly scoped to the mix rules in media.ts — I checked rather than assumed, and the page is right.
CI at this head: no genuine failures. 12 success, 25 skipped on the docs-only path, Validate docs and Mintlify Deployment both green, Analyze (javascript-typescript) still running and CodeQL neutral. One bookkeeping note if you glance at the checks tab: there are two Format runs at this SHA and the red one is a cancelled superseded run — the live one is green. BLOCKED here is the review gate, not CI.
Audited: reference/audio-effects.mdx end-to-end and diffed against audioFx.ts, audioFxPresets.ts, audioFxJobs.ts, audioFxProfiles.ts, audioAutomation.ts, audioGain.ts, audioCarve.ts, audioGroups.ts, webAudioTransport.ts, audioMixer.ts, carve.mjs; studio/audio-effects.mdx, studio/audio-automation.mdx, studio/audio-groups.mdx in full; docs.json nav and every internal link in the diff. Trusting: studio/voiceover-carve.mdx, prompting/audio-effects.mdx, prompting/media-and-audio.mdx, guides/voice-and-audio.mdx read for handoffs and links but not re-derived against source; Studio interaction claims (the shape menu, Simplify's three-point minimum, the FX grouping offer) not exercised against a live build.
Verdict: COMMENT
Reasoning: Every prior blocker is closed, and the exhaustive tables are exact against source — I diffed all 16 effects, 22 presets, 5 jobs and 5 profiles field-by-field, and the carve CLI section matches shipped #3416 down to its subtlest carve-out. What is left is one unreproducible sample transcript and three nits, none of which should hold the PR. Not approving: @miguel-heygen's CHANGES_REQUESTED is still the live state and stamping is not mine to do here.
— Rames Jusso
The audio effects feature shipped with no documentation. Before this PR the only mentions of
data-fx-chain,data-fx-carve, ordata-audio-groupanywhere indocs/were changelog entries.This adds the same three-page shape the color-grading feature already has — one page per audience — with the Studio side split by task, plus a fix to the existing audio guide.
Pages
prompting/audio-effects(new Level 5 chapter)studio/audio-effectsstudio/voiceover-carvestudio/audio-groupsstudio/audio-automationreference/audio-effectsSplit by job, not by topic, so no list appears twice. Studio gets its own Audio group rather than overloading Edit — that group is already one task per page.
Not a new top-level tab: the tabs here are audience-scoped (Guides, Studio, Catalog, Developers), so a feature tab would be the only one of its kind and would strand the prompting chapter out of its Level 5 sequence and the reference page out of Developers.
Also fixed
guides/voice-and-audioadvised "duck music under important speech instead of reducing the whole track equally" — exactly what the carve does — and never named the feature. It now links to it.prompting/media-and-audiohands off to the new chapter instead of jumping to Level 6.Verified against source, not against prose
Two review rounds corrected real errors, all of them found by checking source rather than re-reading the page. What ships now:
pitchshift(audioFx.ts:509-535) — worklet-backed, so neither of its parameters is automatable.chipmunk,giant,monsterare built onpitchshift).propertyPanelFxAddMenu.tsx:22-28), and the menu offers the named jobs in place of a barepeaking.tis clip-relative, a group lane's is composition time, because a group has nodata-start(webAudioTransport.ts:337-342,audioMixer.ts:1311-1344).MAX_AUDIO_GAIN≈ 3.981, i.e. +12 dB (audioGain.ts:8-9), so a boosting lane is valid.", as the page's own warning requires — a single-quoted chain is invisible tocarve.mjs, which then overwrites work it could not see. Each example is followed by its unescaped reading.data-audio-groupis a plain id, not JSON, and ignored on<video>; onlydata-fx-chainanddata-automationcan also sit on an<hf-audio-group>.No screenshots
Skipped deliberately, with no placeholders left behind.
docs/AGENTS.mdtreats screenshots as product claims, and the pages carry real markup and parameter tables instead. Captures of the rack, the carve module, and a group row would each earn their place in a follow-up by someone who can verify the labels against a live build.Checks
mint validate— passed.mint broken-links— none. One callout maximum per page, no accordions, the Prompt Guide chapter ends with a single*Next:*line, each Studio page ends with## Related topics, and the reference page ends with neither — all perdocs/AGENTS.md.🤖 Generated with Claude Code