Skip to content

feat: bridge audio inputs for text-only models - #9807

Merged
diegosouzapw merged 8 commits into
release/v3.8.50from
feat/audio-bridge
Aug 10, 2026
Merged

diegosouzapw merged 8 commits into
release/v3.8.50from
feat/audio-bridge

Conversation

@diegosouzapw

@diegosouzapw diegosouzapw commented Aug 8, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • add conservative audio-capability resolution with registry, static, and synced-catalog precedence
  • add the priority-6 Audio Bridge guardrail, safe audio extraction, authenticated local transcription self-loop, cache/stats metadata, and partial-failure preservation
  • make the Modality Bridge Audio tab functional and source transcription models from the authenticated unified model catalog
  • translate the 12 new UI keys across all 42 non-English locales and document the runtime contract

Integration state

Parent PR #9782 is merged. This PR is now based directly on release/v3.8.50 and was rebuilt as the same eight audio-only commits on release SHA 4fc04a56045e0afb9b81b38ab8dc74fa9983e89d.

  • PR head: cdea9afd4223b8fc54c12e1601b8fe239706fbf8
  • diff: 67 paths, matching the pre-retarget audio diff
  • all 12 Audio Bridge values were structurally reconciled and preserved in all 43 locale catalogs
  • branch and remote head are clean and identical

Verification

Passing on the final retargeted head:

  • focused Audio Bridge, settings, capability, catalog-route, and glossary Node suites: 49/49
  • focused Modality Bridge UI suites: 3 files, 15/15
  • 17 Vision Bridge canary files plus repro fix(providers): Vision Bridge describe-model unreachable when no openai provider connected → raw image forwarded to non-vision model #8430: 138/138
  • npm run test:vitest: 37 files, 344/344
  • npm run typecheck:core
  • npm run lint
  • npm run check:cycles: 406 files, no cycles
  • all 42 locale validators
  • npm run i18n:sync-ui:dry -- --translate-markers: 0 additions, 0 translations
  • i18n coverage: 42/42 locales at 100% (11542/11542)
  • value-drift and zh-CN/zh-TW glossary checks
  • Prettier for all locale catalogs and git diff --check
  • Build App run 31352954886: success on the exact PR head, including artifact upload
  • Quality run 31353108429: change classification, merge integrity, Vitest, ESLint/ratchets, and unit shards 1/4 and 3/4 passed; shard 2/4 had only the inherited env/docs failure, while shard 4/4 had that same docs failure plus two stream assertions reproduced on the pure base; its duplicate advisory build remained queued when this description was refreshed

Base-red discrimination

The manual workflow_dispatch quality run exposed three inherited release reds. Each was reproduced on a disposable worktree at the pure base SHA 4fc04a56045e0afb9b81b38ab8dc74fa9983e89d; none of the affected paths is in this PR:

  1. Docs sync: .env.example contains OMNIROUTE_STRICT_SYSTEM_PROVIDERS, while docs/reference/ENVIRONMENT.md does not.
  2. File-size absolute fallback: the same four pre-existing violations in open-sse/handlers/videoGeneration.ts, open-sse/mcp-server/schemas/tools.ts, open-sse/mcp-server/server.ts, and open-sse/utils/stream.ts.
  3. Open-SSE typecheck ratchet: the same two TS2339 errors in open-sse/handlers/videoGeneration/falHandler.ts.
  4. Unit shard assertions: check-env-doc-sync and issue-7793-env-doc-sync-repro fail on the same missing env documentation. The two additional failures in stream-numeric-ids.test.ts and stream-utils.test.ts were run in isolation on both the PR head and the pure release SHA; both produced exactly 57/59 with the same replaced numeric ID and missing total_tokens assertion.

The file-size job used its documented absolute fallback because this was a manual dispatch and therefore had no pull-request base SHA. These reds are tracked as release-base health, including #9985; they are not assigned to the Audio Bridge diff.

Security and review reconciliation

The Standards and Spec reviews were reconciled before publication. Two findings were fixed: response-header CRLF hardening for both modality bridges and transcription-model discovery through the authenticated unified catalog. No remaining diff-owned code blocker was found.

Live Deepgram gate — merge blocked

Discovery was non-destructive and did not print credentials:

  • 177 provider connections inspected locally
  • 0 Deepgram connections and 0 active Deepgram connections with a usable key
  • no DEEPGRAM_* environment variable or repository Actions secret name

The required real external input_audio -> text-only chat proof cannot be executed in the current environment. Per Hard Rule #18, this PR must remain open and must not be merged until that live gate is recorded.

The single unblock is to add or expose one usable Deepgram connection locally. Then send a known input_audio clip to a text-only chat target and record the HTTP status, x-omniroute-modality-bridge header, and a non-sensitive response excerpt. No VPS deploy was performed.

Related

@diegosouzapw

Copy link
Copy Markdown
Collaborator Author

Final stacked-head validation for 689e1e0

No code blocker remains in this stacked diff. The single external gate remains the explicitly documented live Deepgram call, which cannot be run until one usable local Deepgram connection is exposed. No VPS deployment and no merge were performed.

@diegosouzapw

Copy link
Copy Markdown
Collaborator Author

⚠️ This PR branch is currently held by an active worktree session (Rule #22b). Keep as deferred until ownership is released.

⚠️ Review held: this PR branch is owned by an active parallel worktree session. Keep it in defer until ownership is released and re-run this PR assessment.

@diegosouzapw diegosouzapw changed the title feat: bridge audio inputs for text-only models [defer] feat: bridge audio inputs for text-only models Aug 9, 2026
@diegosouzapw diegosouzapw added merge-train-deferred PR ejetada do merge-train — triagem separada deferred-v3.8.50 Adiada para o ciclo v3.8.50 (validacao VPS, refactor, ou escopo grande) labels Aug 9, 2026
@diegosouzapw

Copy link
Copy Markdown
Collaborator Author

⚠️ This PR branch is currently held by an active worktree session (Rule #22b). Keep as deferred until ownership is released.

⚠️ Review held: this PR branch is owned by an active parallel worktree session. Keep it in defer until ownership is released and re-run this PR assessment.

1 similar comment
@diegosouzapw

Copy link
Copy Markdown
Collaborator Author

⚠️ This PR branch is currently held by an active worktree session (Rule #22b). Keep as deferred until ownership is released.

⚠️ Review held: this PR branch is owned by an active parallel worktree session. Keep it in defer until ownership is released and re-run this PR assessment.

@diegosouzapw diegosouzapw changed the title [defer] feat: bridge audio inputs for text-only models feat: bridge audio inputs for text-only models Aug 10, 2026
@diegosouzapw
diegosouzapw changed the base branch from feat/modality-bridge-page to release/v3.8.50 August 10, 2026 03:34
@diegosouzapw
diegosouzapw merged commit 4d0182d into release/v3.8.50 Aug 10, 2026
4 checks passed
@diegosouzapw

Copy link
Copy Markdown
Collaborator Author

Post-merge Deepgram live gate — passed on VPS .15

The required product gate was completed on the Local VPS against release f1eb0b835722ca744692834c84661f1bc8d564a6 plus the surgical runtime fix in #10229 (d4b8bd9cabe9ef35a6069bedf2a024acc996900a). No credential or sensitive request body is included here.

  • real WAV generated on VPS .15
  • chat target: openai/gpt-4.1-mini, verified text-only for audio
  • STT model: deepgram/nova-3, active provider credential
  • request path: real input_audio through /v1/chat/completions
  • HTTP status: 200
  • transparency header: x-omniroute-modality-bridge: audio->text;model=deepgram/nova-3;parts=1
  • sanitized response excerpt: OmniRoute Audio Bridge live verification number 739.
  • expected transcript match: true

The gate initially exposed a real runtime defect: Node native FormData was incompatible with the packaged Undici fetch installed by ProxyFetch, producing invalid multipart data in the self-loop. PR #10229 adds the TDD regression and explicit multipart Buffer serialization. The exact candidate was transactionally deployed and the gate above then passed.

Freeze evidence on VPS .15: 23/23 focused Node tests, 4/4 Audio Bridge UI Vitest tests, focused ESLint, typecheck:core, check:cycles, fragment validation and changelog anti-eat comparison all passed. See #10229 for the complete evidence and exact SHAs.

@diegosouzapw
diegosouzapw deleted the feat/audio-bridge branch August 13, 2026 08:01
@diegosouzapw diegosouzapw mentioned this pull request Aug 15, 2026
17 of 19 tasks
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
* feat(modality-bridge): resolve audio input capability

* feat(modality-bridge): resolve audio runtime settings

* feat(modality-bridge): add audio transcription helpers

* feat(modality-bridge): add Audio Bridge guardrail

* feat(dashboard): make Modality Bridge audio tab functional

* docs(guardrails): document Audio Bridge runtime

* fix(modality-bridge): harden audio catalog and response header

* chore(changelog): record audio modality bridge

---------

Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deferred-v3.8.50 Adiada para o ciclo v3.8.50 (validacao VPS, refactor, ou escopo grande) merge-train-deferred PR ejetada do merge-train — triagem separada

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant