diff --git a/changelog.d/features/9807-audio-modality-bridge.md b/changelog.d/features/9807-audio-modality-bridge.md new file mode 100644 index 000000000000..64d99490d22d --- /dev/null +++ b/changelog.d/features/9807-audio-modality-bridge.md @@ -0,0 +1 @@ +- **feat(modality bridge):** Transcribe chat audio for text-only models through the existing speech-to-text providers, with configurable limits, caching, runtime stats, and a dashboard self-test ([#9807](https://github.com/diegosouzapw/OmniRoute/pull/9807)) diff --git a/docs/security/GUARDRAILS.md b/docs/security/GUARDRAILS.md index ee802c70229f..a6e55119350d 100644 --- a/docs/security/GUARDRAILS.md +++ b/docs/security/GUARDRAILS.md @@ -7,7 +7,7 @@ lastUpdated: 2026-08-08 # Guardrails > **Source of truth:** `src/lib/guardrails/` -> **Last updated:** 2026-08-08 — v3.8.50 (Modality Bridge PR-2: dashboard settings page, live stats, test action, and media-provider shortcuts) +> **Last updated:** 2026-08-08 — v3.8.50 (Modality Bridge PR-3: Audio Bridge runtime and functional Audio settings tab) Guardrails enforce safety, policy, and content transformations at the boundary between OmniRoute and upstream providers. Each guardrail can inspect (and @@ -20,12 +20,13 @@ request. Blocking is an explicit decision (`block: true`), never an accident. ## Built-in Guardrails -The registry auto-loads four guardrails in priority order on import +The registry auto-loads five guardrails in priority order on import (see `registry.ts` → `registerDefaultGuardrails()`): | Priority | Name | Stage(s) | File | | -------- | ------------------- | -------------- | --------------------- | | `5` | `vision-bridge` | `preCall` | `visionBridge.ts` | +| `6` | `audio-bridge` | `preCall` | `audioBridge.ts` | | `10` | `pii-masker` | `pre` + `post` | `piiMasker.ts` | | `20` | `prompt-injection` | `preCall` | `promptInjection.ts` | | `95` | `credential-masker` | `pre` + `post` | `credentialMasker.ts` | @@ -113,8 +114,8 @@ The new `modalityBridge*` keys are Zod-validated in `updateSettingsSchema` `modalityBridgeVisionMode`, `modalityBridgeVisionModel`, `modalityBridgeVisionTaskAware`, `modalityBridgeVisionPrompt`, `modalityBridgeVisionTimeout`, `modalityBridgeVisionMaxImages`, the -`modalityBridgeCache*` trio, and the PR-3-reserved `modalityBridgeAudio*` -group. Migration `141_modality_bridge_settings.sql` copies existing legacy +`modalityBridgeCache*` trio, and the `modalityBridgeAudio*` group used by the +Audio Bridge. Migration `141_modality_bridge_settings.sql` copies existing legacy `visionBridge*` values to the matching new keys (idempotent, never overwrites an operator-set `modalityBridge*` value); the legacy keys stay accepted as a read fallback for one release cycle. @@ -130,8 +131,8 @@ swap is already visible in the response body's `model` field. `GET /api/modality-bridge/stats` (management auth, same tier as `GET /api/settings`) returns the in-memory per-modality counters -`{ bridged, cacheHits, failures, lastUsedAt }` for `vision` (and the -PR-3-reserved `audio`). Counters reset on process restart by design +`{ bridged, cacheHits, failures, lastUsedAt }` for `vision` and `audio`. +Counters reset on process restart by design (telemetry, not accounting). #### Dashboard configuration @@ -139,11 +140,12 @@ PR-3-reserved `audio`). Counters reset on process restart by design The dedicated dashboard page is `/dashboard/settings/modality-bridge`. Its URL-addressable `Vision`, `Audio`, and `Video` tabs preserve query parameters while switching the `tab` value. -The Vision tab is live: it exposes enablement, mode, model selection (including -the automatic default), task-aware prompting, advanced timeout/image/cache -limits, runtime counters, and a guarded sample request. Audio and Video are -explicit placeholders: Audio is reserved for PR-3, while Video remains tracked -in issue `#9760`. +The Vision tab exposes enablement, mode, model selection (including the automatic +default), task-aware prompting, advanced timeout/image/cache limits, runtime +counters, and a guarded sample request. The Audio tab is also live: it exposes +enablement, an STT-only model picker with Auto, timeout/max-clip limits, audio +counters, and an `input_audio` sample test. Video remains the explicit placeholder +tracked in issue `#9760`. The former Vision Bridge card under AI settings is a compatibility link to the new page; it no longer owns a second copy of the form. Media Providers also @@ -165,6 +167,64 @@ new mode/task-aware/cache defaults and the settings resolver live in `deps` constructor option so tests can inject fake `getSettings` and `callVisionModel` implementations. +### Audio Bridge (`audioBridge.ts`) — Modality Bridge PR-3 + +Intercepts audio-bearing chat requests before they reach a target that is not +known to accept audio input. It never reroutes the chat request: audio parts are +transcribed through the existing OpenAI-compatible multipart endpoint and the +chosen chat model continues with text transcripts. + +Flow: + +1. Resolve `supportsAudio` through `getResolvedModelCapabilities()`. Explicit + provider-registry metadata wins, then static model metadata, then synced + `modalities_input`. A declared input list without `audio` is `false`; no + capability evidence remains `null`. Both `false` and `null` activate the + conservative bridge, while `true` bypasses it. +2. Resolve `modalityBridgeAudio*` settings and extract spliceable top-level + audio parts from every message through the shared `detectMediaParts()` + detector. Supported wire shapes are OpenAI `input_audio`, `audio_url`, and + `source.media_type: "audio/*"`. Nested audio is detected for routing but not + removed by the splice path. Work is capped by `modalityBridgeAudioMaxClips`; + later parts stay untouched. +3. Honor a configured `provider/model`, or let `selectAudioBridgeModel()` walk + `AUDIO_TRANSCRIPTION_PROVIDERS` in stable catalog order and select the first + model with a usable active provider credential. +4. `callAudioTranscription()` converts base64/data-URI audio to a multipart + `file`, or downloads a remote `audio_url` through the public-only outbound + guard with DNS pinning and a 25 MB bound. It then POSTs the file and selected + model to the local `/v1/audio/transcriptions` self-loop, authenticated with + `resolveSelfLoopBearer()`. The existing transcription route performs normal + credential lookup, cooldown/rate-limit handling, and provider dispatch. +5. Successful calls replace their parts with `[Audio N]: `. Calls + run with `Promise.allSettled`: an individual failure preserves that original + audio part (#4012 contract). If every call fails and the target is proven + `supportsAudio === false`, the parts become + `[Audio N]: (unavailable — no STT provider connected)` (#8430 contract). For + an unknown target (`null`), an all-failure result stays untouched. A proven + text-only target with no usable STT credential receives the same explicit + stub without issuing a network call. + +Successful transcripts use the process-wide Modality Bridge LRU/TTL cache. The +key combines the audio reference, the stable `audio-transcription` operation +label, and selected STT model; failures are never cached. Audio attempts update +the shared `bridged`, `cacheHits`, `failures`, and `lastUsedAt` counters. +Transformed responses carry +`x-omniroute-modality-bridge: audio->text;model=;parts=`; untouched +requests do not receive an Audio Bridge segment. + +Runtime settings are DB-backed and Zod-validated: + +| Key | Default | Range | +| ----------------------------- | ------- | -------------- | +| `modalityBridgeAudioEnabled` | `true` | — | +| `modalityBridgeAudioModel` | `""` | Auto or STT ID | +| `modalityBridgeAudioTimeout` | `60000` | 1000–300000 | +| `modalityBridgeAudioMaxClips` | `3` | 1–10 | + +The shared cache remains controlled by `modalityBridgeCacheEnabled`, +`modalityBridgeCacheTtlMinutes`, and `modalityBridgeCacheMaxEntries`. + ### PII Masker (`piiMasker.ts`) Runs on **both** stages. @@ -370,8 +430,8 @@ Environment variables read by the built-in guardrails: | `PII_REDACTION_ENABLED` | `pii-masker` | When `true`, request PII is redacted (independent of injection mode). | | `PII_RESPONSE_SANITIZATION` / `_MODE` | `pii-masker` (downstream) | Controls response-side masker behavior. | -The Vision Bridge reads runtime config from the DB-backed settings store -(`getSettings()`), not env vars. The primary keys are +The Modality Bridge guardrails read runtime config from the DB-backed settings +store (`getSettings()`), not env vars. Vision's primary keys are `modalityBridgeVisionEnabled`, `modalityBridgeVisionMode`, `modalityBridgeVisionModel`, `modalityBridgeVisionTaskAware`, `modalityBridgeVisionPrompt`, `modalityBridgeVisionTimeout`, @@ -382,6 +442,11 @@ fallback; dashboard writes use the primary keys. Defaults and the fallback resolver live in `src/shared/constants/modalityBridgeDefaults.ts`, with legacy constants retained in `src/shared/constants/visionBridgeDefaults.ts`. +Audio uses `modalityBridgeAudioEnabled`, `modalityBridgeAudioModel`, +`modalityBridgeAudioTimeout`, and `modalityBridgeAudioMaxClips`, plus the shared +`modalityBridgeCache*` settings. Audio has no legacy-key fallback because these +keys were introduced with the Modality Bridge schema. + ## Custom Guardrails ```typescript @@ -419,9 +484,11 @@ Steps: Use `resetGuardrailsForTests()` between tests to start from a known state. Pass `{ registerDefaults: false }` to start with an empty registry and -register only the guardrails under test. The Vision Bridge guardrail accepts -dependency injection (`deps.getSettings`, `deps.callVisionModel`) so tests can -exercise the full flow without DB or network access. +register only the guardrails under test. Vision Bridge accepts dependency +injection (`deps.getSettings`, `deps.callVisionModel`); Audio Bridge exposes the +equivalent seams for settings, capabilities, STT model selection, credential +checks, and transcription. Tests can therefore exercise both flows without DB +or network access. ## See Also @@ -430,6 +497,7 @@ exercise the full flow without DB or network access. prompt-injection and PII masking - `src/shared/constants/visionBridgeDefaults.ts` — Vision Bridge defaults and forced-bridge model list +- `src/shared/constants/modalityBridgeDefaults.ts` — shared Vision/Audio runtime defaults - `docs/architecture/RESILIENCE_GUIDE.md` — orthogonal layer (circuit breaker, cooldowns) - `docs/reference/ENVIRONMENT.md` — full env var reference diff --git a/open-sse/config/providers/shared.ts b/open-sse/config/providers/shared.ts index 44cea91c0533..1198862b18ac 100644 --- a/open-sse/config/providers/shared.ts +++ b/open-sse/config/providers/shared.ts @@ -50,6 +50,7 @@ export interface RegistryModel { supportsReasoning?: boolean; supportedThinkingEfforts?: readonly string[]; supportsVision?: boolean; + supportsAudio?: boolean; supportsXHighEffort?: boolean; maxOutputTokens?: number; targetFormat?: string; diff --git a/src/app/(dashboard)/dashboard/settings/components/modalityBridge/ModalityBridgeAudioTab.tsx b/src/app/(dashboard)/dashboard/settings/components/modalityBridge/ModalityBridgeAudioTab.tsx new file mode 100644 index 000000000000..0674e191c7ad --- /dev/null +++ b/src/app/(dashboard)/dashboard/settings/components/modalityBridge/ModalityBridgeAudioTab.tsx @@ -0,0 +1,197 @@ +"use client"; + +import { useCallback, useEffect, useState } from "react"; +import { useTranslations } from "next-intl"; + +import { Card, ModelSelectField, Toggle } from "@/shared/components"; +import type { ApiModel } from "@/shared/components/ModelSelectField"; +import { + MODALITY_BRIDGE_DEFAULTS, + resolveAudioBridgeRuntimeSettings, +} from "@/shared/constants/modalityBridgeDefaults"; + +import ModalityBridgeAudioTestButton from "./ModalityBridgeAudioTestButton"; +import ModalityBridgeStatsRow from "./ModalityBridgeStatsRow"; + +interface AudioState { + modalityBridgeAudioEnabled: boolean; + modalityBridgeAudioModel: string; + modalityBridgeAudioTimeout: number; + modalityBridgeAudioMaxClips: number; +} + +function fromApi(data: Record): AudioState { + const runtime = resolveAudioBridgeRuntimeSettings(data); + return { + modalityBridgeAudioEnabled: runtime.enabled, + modalityBridgeAudioModel: runtime.model, + modalityBridgeAudioTimeout: runtime.timeoutMs, + modalityBridgeAudioMaxClips: runtime.maxClips, + }; +} + +function asSettingsRecord(value: unknown): Record { + return value && typeof value === "object" ? (value as Record) : {}; +} + +function clampNumber(raw: string, min: number, max: number, fallback: number): number { + const parsed = Number.parseInt(raw, 10); + return Math.min(max, Math.max(min, Number.isFinite(parsed) ? parsed : fallback)); +} + +export default function ModalityBridgeAudioTab() { + const t = useTranslations("settings"); + const [settings, setSettings] = useState(null); + const isSttModel = useCallback( + (model: ApiModel) => model.type === "audio" && model.subtype === "transcription", + [] + ); + + useEffect(() => { + let cancelled = false; + fetch("/api/settings") + .then((response) => (response.ok ? response.json() : null)) + .then((data: unknown) => { + if (!cancelled) setSettings(fromApi(asSettingsRecord(data))); + }) + .catch(() => { + if (!cancelled) setSettings(fromApi({})); + }); + return () => { + cancelled = true; + }; + }, []); + + const update = async (patch: Partial) => { + try { + const response = await fetch("/api/settings", { + method: "PATCH", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify(patch), + }); + if (response.ok) { + setSettings((previous) => (previous ? { ...previous, ...patch } : previous)); + } + } catch (error) { + console.error("Failed to update Audio Bridge settings:", error); + } + }; + + if (!settings) return null; + + const setLocal = (patch: Partial) => { + setSettings((previous) => (previous ? { ...previous, ...patch } : previous)); + }; + const commitNumber = ( + key: "modalityBridgeAudioTimeout" | "modalityBridgeAudioMaxClips", + raw: string, + min: number, + max: number, + fallback: number + ) => { + const value = clampNumber(raw, min, max, fallback); + setLocal({ [key]: value }); + void update({ [key]: value }); + }; + + return ( + +
+ void update({ modalityBridgeAudioEnabled: checked })} + label={t("modalityBridgeAudioEnabled")} + description={t("modalityBridgeAudioEnabledDesc")} + /> + + void update({ modalityBridgeAudioModel: value })} + className="text-sm" + /> + +
+ + {t("modalityBridgeAdvanced")} + +
+ setLocal({ modalityBridgeAudioTimeout: value })} + onBlur={(raw) => + commitNumber( + "modalityBridgeAudioTimeout", + raw, + 1000, + 300000, + MODALITY_BRIDGE_DEFAULTS.audioTimeoutMs + ) + } + /> + setLocal({ modalityBridgeAudioMaxClips: value })} + onBlur={(raw) => + commitNumber( + "modalityBridgeAudioMaxClips", + raw, + 1, + 10, + MODALITY_BRIDGE_DEFAULTS.audioMaxClips + ) + } + /> +
+
+ + + +
+
+ ); +} + +interface NumberFieldProps { + testId: string; + label: string; + min: number; + max: number; + value: number; + onChange: (value: number) => void; + onBlur: (raw: string) => void; +} + +function NumberField({ testId, label, min, max, value, onChange, onBlur }: NumberFieldProps) { + return ( + + ); +} diff --git a/src/app/(dashboard)/dashboard/settings/components/modalityBridge/ModalityBridgeAudioTestButton.tsx b/src/app/(dashboard)/dashboard/settings/components/modalityBridge/ModalityBridgeAudioTestButton.tsx new file mode 100644 index 000000000000..6961e06e6240 --- /dev/null +++ b/src/app/(dashboard)/dashboard/settings/components/modalityBridge/ModalityBridgeAudioTestButton.tsx @@ -0,0 +1,104 @@ +"use client"; + +import { useState } from "react"; +import { useTranslations } from "next-intl"; + +const SAMPLE_INPUT = { + model: "modality-bridge/self-test", + messages: [ + { + role: "user", + content: [ + { type: "text", text: "Transcribe this audio clip." }, + { + type: "input_audio", + input_audio: { + data: "UklGRiQAAABXQVZFZm10IBAAAAABAAEAQB8AAEAfAAABAAgAZGF0YQAAAAA=", + format: "wav", + }, + }, + ], + }, + ], +}; + +const DISABLED_GUARDRAILS = [ + "vision-bridge", + "pii-masker", + "prompt-injection", + "credential-masker", +]; + +function asRecord(value: unknown): Record | null { + return value && typeof value === "object" ? (value as Record) : null; +} + +function findAudioMeta(value: unknown): Record | null { + const body = asRecord(value); + if (!Array.isArray(body?.results)) return null; + for (const entry of body.results) { + const result = asRecord(entry); + if (result?.guardrail === "audio-bridge") return asRecord(result.meta); + } + return null; +} + +function readErrorMessage(value: unknown): string | null { + const error = asRecord(asRecord(value)?.error); + return typeof error?.message === "string" ? error.message : null; +} + +export default function ModalityBridgeAudioTestButton() { + const t = useTranslations("settings"); + const [running, setRunning] = useState(false); + const [result, setResult] = useState(null); + + const runTest = async () => { + setRunning(true); + setResult(null); + try { + const response = await fetch("/api/guardrails/test", { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ input: SAMPLE_INPUT, disabledGuardrails: DISABLED_GUARDRAILS }), + }); + const body: unknown = await response.json().catch(() => null); + if (!response.ok) throw new Error(readErrorMessage(body) ?? `HTTP ${response.status}`); + + const meta = findAudioMeta(body); + if (typeof meta?.clipsProcessed === "number" && meta.clipsProcessed >= 1) { + setResult( + t("modalityBridgeAudioTestOk", { + count: meta.clipsProcessed, + model: String(meta.sttModel ?? "unknown"), + }) + ); + } else { + setResult(t("modalityBridgeAudioTestNoop")); + } + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + setResult(t("modalityBridgeAudioTestError", { message })); + } finally { + setRunning(false); + } + }; + + return ( +
+ + {result && ( +

+ {result} +

+ )} +
+ ); +} diff --git a/src/app/(dashboard)/dashboard/settings/modality-bridge/page.tsx b/src/app/(dashboard)/dashboard/settings/modality-bridge/page.tsx index fcb3b6df5a06..14bc9a5f1ffd 100644 --- a/src/app/(dashboard)/dashboard/settings/modality-bridge/page.tsx +++ b/src/app/(dashboard)/dashboard/settings/modality-bridge/page.tsx @@ -5,6 +5,7 @@ import { usePathname, useRouter, useSearchParams } from "next/navigation"; import { useTranslations } from "next-intl"; import ModalityBridgeComingSoonTab from "@/app/(dashboard)/dashboard/settings/components/modalityBridge/ModalityBridgeComingSoonTab"; +import ModalityBridgeAudioTab from "@/app/(dashboard)/dashboard/settings/components/modalityBridge/ModalityBridgeAudioTab"; import ModalityBridgeVisionTab from "@/app/(dashboard)/dashboard/settings/components/modalityBridge/ModalityBridgeVisionTab"; type TabId = "vision" | "audio" | "video"; @@ -61,9 +62,7 @@ function ModalityBridgePageContent() {
{activeTab === "vision" && } - {activeTab === "audio" && ( - - )} + {activeTab === "audio" && } {activeTab === "video" && ( )} diff --git a/src/app/api/models/catalog/route.ts b/src/app/api/models/catalog/route.ts index 62909b77f1c8..500b2569b8d8 100644 --- a/src/app/api/models/catalog/route.ts +++ b/src/app/api/models/catalog/route.ts @@ -38,6 +38,7 @@ export async function GET(request: Request) { id: model.id, name: model.name || model.root || model.id, type: model.type || "chat", + ...(typeof model.subtype === "string" ? { subtype: model.subtype } : {}), custom: model.custom === true, ...(model.free === true ? { free: true } : {}), ...(model.capabilities ? { capabilities: model.capabilities } : {}), diff --git a/src/i18n/messages/ar.json b/src/i18n/messages/ar.json index 93bfb067ef2a..9384f4175801 100644 --- a/src/i18n/messages/ar.json +++ b/src/i18n/messages/ar.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "تجسير الفيديو (عينة الإطار + الترجمة) في قائمة الانتظار - راجع المشكلة #9760.", "modalityBridgeMovedTitle": "تم نقل Vision Bridge", "modalityBridgeMovedBody": "إعدادات Vision Bridge الآن موجودة في صفحة Modality Bridge المخصصة.", - "modalityBridgeMovedCta": "فتح إعدادات جسر الوضعية" + "modalityBridgeMovedCta": "فتح إعدادات جسر الوضعية", + "modalityBridgeAudioTitle": "جسر الصوت", + "modalityBridgeAudioDesc": "قم بنسخ الصوت باستخدام نموذج تحويل الكلام إلى نص قبل المتابعة مع نموذج النص المختار.", + "modalityBridgeAudioEnabled": "تفعيل جسر الصوت", + "modalityBridgeAudioEnabledDesc": "استبدل أجزاء الصوت بالنصوص عندما لا يمكن للنموذج المستهدف معالجة الصوت.", + "modalityBridgeAudioModel": "نموذج تحويل الكلام إلى نص", + "modalityBridgeAudioModelAuto": "تلقائي (أول مزود STT متصل)", + "modalityBridgeAudioMaxClips": "أقصى عدد لمقاطع الصوت لكل طلب", + "modalityBridgeAudioTestButton": "اختبر باستخدام صوت عينة", + "modalityBridgeAudioTestRunning": "اختبار الصوت…", + "modalityBridgeAudioTestOk": "جسر الصوت OK — تم نسخ {count} مقطع(ات) بواسطة {model}", + "modalityBridgeAudioTestNoop": "لم يتم تفعيل جسر الصوت (قد يدعم الهدف الصوت، لم يتم الاتصال بمزود STT، أو أن الجسر معطل)", + "modalityBridgeAudioTestError": "فشل اختبار الصوت: {message}" }, "contextRtk": { "title": "محرك آر تي كيه", diff --git a/src/i18n/messages/az.json b/src/i18n/messages/az.json index 7f1c6e9cb21d..73a0e5a07cc3 100644 --- a/src/i18n/messages/az.json +++ b/src/i18n/messages/az.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Video bridging (frame sampling + captioning) arxa planda var — məsələyə baxın #9760.", "modalityBridgeMovedTitle": "Vision Bridge köçürüldü", "modalityBridgeMovedBody": "Vision Bridge parametrləri indi xüsusi Modality Bridge səhifəsində mövcuddur.", - "modalityBridgeMovedCta": "Modallıq Körpüsü parametrlərini açın" + "modalityBridgeMovedCta": "Modallıq Körpüsü parametrlərini açın", + "modalityBridgeAudioTitle": "Səs Körpüsü", + "modalityBridgeAudioDesc": "Seçilmiş mətn modelinə davam etməzdən əvvəl səsdən mətbə çevirmək üçün nitqdən mətbə modelindən istifadə edin.", + "modalityBridgeAudioEnabled": "Səs Körpüsünü Aktivləşdirin", + "modalityBridgeAudioEnabledDesc": "Hədəf model audio emal edə bilmədikdə audio hissələri transkriptlərlə əvəz edin.", + "modalityBridgeAudioModel": "Nitqdan mətnə model", + "modalityBridgeAudioModelAuto": "Avtomatik (ilk qoşulmuş STT təminatçısı)", + "modalityBridgeAudioMaxClips": "Tələb başına maksimum audio kliplər", + "modalityBridgeAudioTestButton": "Nümunə audio ilə test edin", + "modalityBridgeAudioTestRunning": "Səsi test edir…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} klip(ler) {model} tərəfindən transkribasiya edildi", + "modalityBridgeAudioTestNoop": "Audio Bridge aktivləşmədi (hədəf audio dəstəkləyə bilər, STT provayderi qoşulmayıb, ya da körpü deaktivdir)", + "modalityBridgeAudioTestError": "Səs testi uğursuz oldu: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/bg.json b/src/i18n/messages/bg.json index f9cfd6e34faf..66a99eabb3f2 100644 --- a/src/i18n/messages/bg.json +++ b/src/i18n/messages/bg.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Видео свързване (извадка на кадри + надписи) е в списъка със задачи — вижте проблема #9760.", "modalityBridgeMovedTitle": "Vision Bridge преместен", "modalityBridgeMovedBody": "Настройките на Vision Bridge вече са налични на специализираната страница Modality Bridge.", - "modalityBridgeMovedCta": "Отворете настройките на Modality Bridge" + "modalityBridgeMovedCta": "Отворете настройките на Modality Bridge", + "modalityBridgeAudioTitle": "Аудио Мост", + "modalityBridgeAudioDesc": "Транскрибирайте аудио с модел за разпознаване на реч, преди да продължите с избрания текстов модел.", + "modalityBridgeAudioEnabled": "Активирайте Audio Bridge", + "modalityBridgeAudioEnabledDesc": "Заменете аудио частите с транскрипции, когато целевият модел не може да обработва аудио.", + "modalityBridgeAudioModel": "Модел за преобразуване на реч в текст", + "modalityBridgeAudioModelAuto": "Авто (първият свързан доставчик на STT)", + "modalityBridgeAudioMaxClips": "Максимален брой аудио клипове на заявка", + "modalityBridgeAudioTestButton": "Тест с примерен аудио файл", + "modalityBridgeAudioTestRunning": "Тестване на аудио…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} клип(а) транскрибирани от {model}", + "modalityBridgeAudioTestNoop": "Audio Bridge не беше активиран (целта може да поддържа аудио, няма свързан STT доставчик или моста е деактивиран)", + "modalityBridgeAudioTestError": "Тестът на звука не успя: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/bn.json b/src/i18n/messages/bn.json index 58146ad7d0fd..feb04f8abce7 100644 --- a/src/i18n/messages/bn.json +++ b/src/i18n/messages/bn.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "ভিডিও ব্রিজিং (ফ্রেম স্যাম্পলিং + ক্যাপশনিং) ব্যাকলগে রয়েছে — সমস্যা #9760 দেখুন।", "modalityBridgeMovedTitle": "ভিশন ব্রিজ স্থানান্তরিত হয়েছে", "modalityBridgeMovedBody": "ভিশন ব্রিজের সেটিংস এখন নির্দিষ্ট মডালিটি ব্রিজ পৃষ্ঠায় লাইভ।", - "modalityBridgeMovedCta": "মোডালিটি ব্রিজ সেটিংস খুলুন" + "modalityBridgeMovedCta": "মোডালিটি ব্রিজ সেটিংস খুলুন", + "modalityBridgeAudioTitle": "অডিও ব্রিজ", + "modalityBridgeAudioDesc": "নির্বাচিত টেক্সট মডেলের সাথে এগিয়ে যাওয়ার আগে একটি স্পিচ-টু-টেক্সট মডেলের সাহায্যে অডিও ট্রান্সক্রাইব করুন।", + "modalityBridgeAudioEnabled": "অডিও ব্রিজ সক্রিয় করুন", + "modalityBridgeAudioEnabledDesc": "লক্ষ্য মডেল অডিও প্রক্রিয়া করতে না পারলে অডিও অংশগুলি ট্রান্সক্রিপ্টের সাথে প্রতিস্থাপন করুন।", + "modalityBridgeAudioModel": "স্পিচ-টু-টেক্সট মডেল", + "modalityBridgeAudioModelAuto": "অটো (প্রথম সংযুক্ত STT প্রদানকারী)", + "modalityBridgeAudioMaxClips": "প্রতি অনুরোধে সর্বাধিক অডিও ক্লিপ", + "modalityBridgeAudioTestButton": "নমুনা অডিওর সাথে পরীক্ষা করুন", + "modalityBridgeAudioTestRunning": "অডিও পরীক্ষা করা হচ্ছে…", + "modalityBridgeAudioTestOk": "অডিও ব্রিজ ঠিক আছে — {count} ক্লিপ(গুলি) {model} দ্বারা ট্রান্সক্রাইব করা হয়েছে", + "modalityBridgeAudioTestNoop": "অডিও ব্রিজ সক্রিয় হয়নি (লক্ষ্য অডিও সমর্থন করতে পারে, কোন STT প্রদানকারী সংযুক্ত নয়, অথবা ব্রিজ নিষ্ক্রিয় রয়েছে)", + "modalityBridgeAudioTestError": "অডিও পরীক্ষা ব্যর্থ: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/cs.json b/src/i18n/messages/cs.json index bfb6f40bd7fd..7375412a77ad 100644 --- a/src/i18n/messages/cs.json +++ b/src/i18n/messages/cs.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Video bridging (frame sampling + captioning) je na backlogu — viz problém #9760.", "modalityBridgeMovedTitle": "Vision Bridge přesunuto", "modalityBridgeMovedBody": "Nastavení Vision Bridge nyní žije na vyhrazené stránce Modality Bridge.", - "modalityBridgeMovedCta": "Otevřít nastavení Modality Bridge" + "modalityBridgeMovedCta": "Otevřít nastavení Modality Bridge", + "modalityBridgeAudioTitle": "Audio Bridge", + "modalityBridgeAudioDesc": "Přepište audio pomocí modelu převodu řeči na text, než pokračujete s vybraným textovým modelem.", + "modalityBridgeAudioEnabled": "Povolit Audio Bridge", + "modalityBridgeAudioEnabledDesc": "Nahraďte audio části přepisy, když cílový model nemůže zpracovat audio.", + "modalityBridgeAudioModel": "Model pro převod řeči na text", + "modalityBridgeAudioModelAuto": "Auto (první připojený poskytovatel STT)", + "modalityBridgeAudioMaxClips": "Maximální počet audio klipů na požadavek", + "modalityBridgeAudioTestButton": "Test s ukázkovým zvukem", + "modalityBridgeAudioTestRunning": "Testování zvuku…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} klip(y) přepsány pomocí {model}", + "modalityBridgeAudioTestNoop": "Audio Bridge se nepodařilo aktivovat (cílové zařízení může podporovat audio, žádný poskytovatel STT není připojen, nebo je most deaktivován)", + "modalityBridgeAudioTestError": "Test zvuku selhal: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/da.json b/src/i18n/messages/da.json index 8c5d27f542fd..89246d2f4d7f 100644 --- a/src/i18n/messages/da.json +++ b/src/i18n/messages/da.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Video brokering (rammesampling + undertekster) er på backloggen — se problem #9760.", "modalityBridgeMovedTitle": "Vision Bridge flyttet", "modalityBridgeMovedBody": "Vision Bridge-indstillinger er nu live på den dedikerede Modality Bridge-side.", - "modalityBridgeMovedCta": "Åbn Modality Bridge-indstillinger" + "modalityBridgeMovedCta": "Åbn Modality Bridge-indstillinger", + "modalityBridgeAudioTitle": "Audio Bro", + "modalityBridgeAudioDesc": "Transskriber lyd med en tale-til-tekst model, før du fortsætter med den valgte tekstmodel.", + "modalityBridgeAudioEnabled": "Aktivér Audio Bridge", + "modalityBridgeAudioEnabledDesc": "Erstat lyddele med transkriptioner, når målmodellen ikke kan behandle lyd.", + "modalityBridgeAudioModel": "Tale-til-tekst model", + "modalityBridgeAudioModelAuto": "Auto (første tilsluttede STT-udbyder)", + "modalityBridgeAudioMaxClips": "Maksimalt antal lydklip pr. anmodning", + "modalityBridgeAudioTestButton": "Test med prøveaudio", + "modalityBridgeAudioTestRunning": "Tester lyd…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} klip(klip) transskriberet af {model}", + "modalityBridgeAudioTestNoop": "Audio Bridge blev ikke aktiveret (målet kan understøtte lyd, ingen STT-udbyder er tilsluttet, eller broen er deaktiveret)", + "modalityBridgeAudioTestError": "Lydtest mislykkedes: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/de.json b/src/i18n/messages/de.json index eb410b61073d..4fa1a92004e3 100644 --- a/src/i18n/messages/de.json +++ b/src/i18n/messages/de.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Video-Bridging (Frame-Sampling + Untertitelung) steht auf der Warteliste – siehe Issue #9760.", "modalityBridgeMovedTitle": "Vision Bridge verschoben", "modalityBridgeMovedBody": "Die Vision Bridge-Einstellungen sind jetzt auf der speziellen Modality Bridge-Seite verfügbar.", - "modalityBridgeMovedCta": "Modality Bridge-Einstellungen öffnen" + "modalityBridgeMovedCta": "Modality Bridge-Einstellungen öffnen", + "modalityBridgeAudioTitle": "Audio-Brücke", + "modalityBridgeAudioDesc": "Transkribiere Audio mit einem Sprach-zu-Text-Modell, bevor du mit dem gewählten Textmodell fortfährst.", + "modalityBridgeAudioEnabled": "Audio-Brücke aktivieren", + "modalityBridgeAudioEnabledDesc": "Ersetzen Sie Audioabschnitte durch Transkripte, wenn das Zielmodell Audio nicht verarbeiten kann.", + "modalityBridgeAudioModel": "Spracherkennungsmodell", + "modalityBridgeAudioModelAuto": "Auto (erster verbundener STT-Anbieter)", + "modalityBridgeAudioMaxClips": "Maximale Audio-Clips pro Anfrage", + "modalityBridgeAudioTestButton": "Testen mit Beispielaudio", + "modalityBridgeAudioTestRunning": "Audio wird getestet…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} Clip(s) von {model} transkribiert", + "modalityBridgeAudioTestNoop": "Audio Bridge wurde nicht aktiviert (das Ziel unterstützt möglicherweise Audio, kein STT-Anbieter ist verbunden oder die Brücke ist deaktiviert)", + "modalityBridgeAudioTestError": "Audiotest fehlgeschlagen: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/en.json b/src/i18n/messages/en.json index 101f08d936c8..20d53095cd93 100644 --- a/src/i18n/messages/en.json +++ b/src/i18n/messages/en.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Video bridging (frame sampling + captioning) is on the backlog — see issue #9760.", "modalityBridgeMovedTitle": "Vision Bridge moved", "modalityBridgeMovedBody": "Vision Bridge settings now live in the dedicated Modality Bridge page.", - "modalityBridgeMovedCta": "Open Modality Bridge settings" + "modalityBridgeMovedCta": "Open Modality Bridge settings", + "modalityBridgeAudioTitle": "Audio Bridge", + "modalityBridgeAudioDesc": "Transcribe audio with a speech-to-text model before continuing with the chosen text model.", + "modalityBridgeAudioEnabled": "Enable Audio Bridge", + "modalityBridgeAudioEnabledDesc": "Replace audio parts with transcripts when the target model cannot process audio.", + "modalityBridgeAudioModel": "Speech-to-text model", + "modalityBridgeAudioModelAuto": "Auto (first connected STT provider)", + "modalityBridgeAudioMaxClips": "Max audio clips per request", + "modalityBridgeAudioTestButton": "Test with sample audio", + "modalityBridgeAudioTestRunning": "Testing audio…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} clip(s) transcribed by {model}", + "modalityBridgeAudioTestNoop": "Audio Bridge did not activate (the target may support audio, no STT provider is connected, or the bridge is disabled)", + "modalityBridgeAudioTestError": "Audio test failed: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/es.json b/src/i18n/messages/es.json index 5a8e4533b276..0015b43e58a8 100644 --- a/src/i18n/messages/es.json +++ b/src/i18n/messages/es.json @@ -7650,6 +7650,13 @@ "modalityBridgeSubTabsAria": "Secciones del Puente de Modalidad", "modalityBridgeVisionTitle": "Puente de Visión", "modalityBridgeVisionDesc": "Describe imágenes con un modelo de visión y continúa con el modelo de texto elegido por el usuario.", + "modalityBridgeAudioTitle": "Puente de Audio", + "modalityBridgeAudioDesc": "Transcribe audio con un modelo de reconocimiento de voz antes de continuar con el modelo de texto elegido.", + "modalityBridgeAudioEnabled": "Habilitar Puente de Audio", + "modalityBridgeAudioEnabledDesc": "Reemplace las partes de audio con transcripciones cuando el modelo objetivo no pueda procesar audio.", + "modalityBridgeAudioModel": "Modelo de texto a voz", + "modalityBridgeAudioModelAuto": "Automático (primer proveedor STT conectado)", + "modalityBridgeAudioMaxClips": "Máx. clips de audio por solicitud", "modalityBridgeMode": "Modo", "modalityBridgeModeAuto": "Automático (recomendado)", "modalityBridgeModeAutoHint": "Heurística heredada: redirigir modelos individuales sin credenciales; describir de otra manera.", @@ -7680,6 +7687,11 @@ "modalityBridgeTestReroute": "El puente redirigió la solicitud a {model}", "modalityBridgeTestNoop": "El puente no se activó (el modelo puede soportar visión de forma nativa o el puente está deshabilitado)", "modalityBridgeTestError": "La prueba falló: {message}", + "modalityBridgeAudioTestButton": "Prueba con audio de muestra", + "modalityBridgeAudioTestRunning": "Probando audio…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} clip(s) transcritos por {model}", + "modalityBridgeAudioTestNoop": "Audio Bridge no se activó (el destino puede soportar audio, no hay proveedor de STT conectado, o el puente está deshabilitado)", + "modalityBridgeAudioTestError": "La prueba de audio falló: {message}", "modalityBridgeAudioComingSoon": "El puente de audio (voz → texto a través de /v1/audio/transcriptions) se incluirá en la próxima versión. Sus claves de configuración ya están reservadas.", "modalityBridgeVideoComingSoon": "El puenteo de video (muestreo de fotogramas + subtitulado) está en la lista de tareas pendientes — consulta el problema #9760.", "modalityBridgeMovedTitle": "Vision Bridge movido", diff --git a/src/i18n/messages/fa.json b/src/i18n/messages/fa.json index a685d117ba6a..92e730454fc8 100644 --- a/src/i18n/messages/fa.json +++ b/src/i18n/messages/fa.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "پل‌سازی ویدئو (نمونه‌برداری فریم + زیرنویس) در لیست کارها قرار دارد — به مشکل #9760 مراجعه کنید.", "modalityBridgeMovedTitle": "پل بینایی جابجا شد", "modalityBridgeMovedBody": "تنظیمات Vision Bridge اکنون در صفحه اختصاصی Modality Bridge فعال است.", - "modalityBridgeMovedCta": "تنظیمات پل مدالیته را باز کنید" + "modalityBridgeMovedCta": "تنظیمات پل مدالیته را باز کنید", + "modalityBridgeAudioTitle": "پل صوتی", + "modalityBridgeAudioDesc": "قبل از ادامه با مدل متن انتخاب شده، صدای خود را با یک مدل تبدیل گفتار به متن رونویسی کنید.", + "modalityBridgeAudioEnabled": "فعال‌سازی پل صوتی", + "modalityBridgeAudioEnabledDesc": "قسمت‌های صوتی را با متن‌ها جایگزین کنید زمانی که مدل هدف نمی‌تواند صوت را پردازش کند.", + "modalityBridgeAudioModel": "مدل تبدیل گفتار به متن", + "modalityBridgeAudioModelAuto": "خودکار (اولین ارائه‌دهنده STT متصل شده)", + "modalityBridgeAudioMaxClips": "حداکثر کلیپ‌های صوتی در هر درخواست", + "modalityBridgeAudioTestButton": "تست با صدای نمونه", + "modalityBridgeAudioTestRunning": "در حال آزمایش صدا…", + "modalityBridgeAudioTestOk": "پل صوتی OK — {count} کلیپ(ها) توسط {model} رونویسی شد", + "modalityBridgeAudioTestNoop": "پل صوتی فعال نشد (ممکن است هدف از صوت پشتیبانی کند، هیچ ارائه‌دهنده STT متصل نیست، یا پل غیرفعال است)", + "modalityBridgeAudioTestError": "آزمون صدا ناموفق بود: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/fi.json b/src/i18n/messages/fi.json index a969f27b86c8..21e33b2d9ef3 100644 --- a/src/i18n/messages/fi.json +++ b/src/i18n/messages/fi.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Videosillan (kehysnäytteenotto + tekstitys) toteuttaminen on odotuslistalla — katso ongelma #9760.", "modalityBridgeMovedTitle": "Vision Bridge siirretty", "modalityBridgeMovedBody": "Vision Bridge -asetukset ovat nyt käytettävissä omalla Modality Bridge -sivullaan.", - "modalityBridgeMovedCta": "Avaa Modality Bridge -asetukset" + "modalityBridgeMovedCta": "Avaa Modality Bridge -asetukset", + "modalityBridgeAudioTitle": "Äänisilta", + "modalityBridgeAudioDesc": "Transkriboi ääntä puheesta tekstiksi ennen kuin jatkat valitun tekstimallin kanssa.", + "modalityBridgeAudioEnabled": "Ota ääniyhteys käyttöön", + "modalityBridgeAudioEnabledDesc": "Korvaa ääniosat transkripteillä, kun kohdemalli ei voi käsitellä ääntä.", + "modalityBridgeAudioModel": "Puheesta tekstiksi -malli", + "modalityBridgeAudioModelAuto": "Auto (ensimmäinen yhdistetty STT-palveluntarjoaja)", + "modalityBridgeAudioMaxClips": "Maksimi äänitiedostot per pyyntö", + "modalityBridgeAudioTestButton": "Testaa näytteellä olevaa ääntä", + "modalityBridgeAudioTestRunning": "Testataan ääntä…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} klippiä transkriptoitu {model} avulla", + "modalityBridgeAudioTestNoop": "Audio Bridge ei aktivoitunut (kohde saattaa tukea ääntä, STT-toimittajaa ei ole liitetty tai silta on poistettu käytöstä)", + "modalityBridgeAudioTestError": "Äänitesti epäonnistui: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/fr.json b/src/i18n/messages/fr.json index d0a23dc6fb64..fcf7b9d320c7 100644 --- a/src/i18n/messages/fr.json +++ b/src/i18n/messages/fr.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Le pont vidéo (échantillonnage d'images + sous-titrage) est dans la liste des tâches — voir l'issue #9760.", "modalityBridgeMovedTitle": "Vision Bridge déplacé", "modalityBridgeMovedBody": "Les paramètres de Vision Bridge sont désormais disponibles sur la page dédiée à Modality Bridge.", - "modalityBridgeMovedCta": "Ouvrir les paramètres du pont de modalité" + "modalityBridgeMovedCta": "Ouvrir les paramètres du pont de modalité", + "modalityBridgeAudioTitle": "Pont Audio", + "modalityBridgeAudioDesc": "Transcrivez l'audio avec un modèle de reconnaissance vocale avant de continuer avec le modèle de texte choisi.", + "modalityBridgeAudioEnabled": "Activer le Pont Audio", + "modalityBridgeAudioEnabledDesc": "Remplacez les parties audio par des transcriptions lorsque le modèle cible ne peut pas traiter l'audio.", + "modalityBridgeAudioModel": "Modèle de reconnaissance vocale", + "modalityBridgeAudioModelAuto": "Auto (premier fournisseur STT connecté)", + "modalityBridgeAudioMaxClips": "Nombre maximum de clips audio par demande", + "modalityBridgeAudioTestButton": "Tester avec un audio d'exemple", + "modalityBridgeAudioTestRunning": "Test de l'audio…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} clip(s) transcrit(s) par {model}", + "modalityBridgeAudioTestNoop": "Le pont audio ne s'est pas activé (la cible peut prendre en charge l'audio, aucun fournisseur STT n'est connecté, ou le pont est désactivé)", + "modalityBridgeAudioTestError": "Échec du test audio : {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/gu.json b/src/i18n/messages/gu.json index 4a4f3ca7d0d3..169979167df7 100644 --- a/src/i18n/messages/gu.json +++ b/src/i18n/messages/gu.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "વિડિયો બ્રિજિંગ (ફ્રેમ નમૂનાકરણ + કેપ્શનિંગ) બેકલોગમાં છે — મુદ્દો #9760 જુઓ.", "modalityBridgeMovedTitle": "વિઝન બ્રિજ ખસેડાયો", "modalityBridgeMovedBody": "વિઝન બ્રિજની સેટિંગ્સ હવે સમર્પિત મોડાલિટી બ્રિજ પૃષ્ઠમાં જીવંત છે.", - "modalityBridgeMovedCta": "મોડાલિટી બ્રિજ સેટિંગ્સ ખોલો" + "modalityBridgeMovedCta": "મોડાલિટી બ્રિજ સેટિંગ્સ ખોલો", + "modalityBridgeAudioTitle": "ઓડિયો બ્રિજ", + "modalityBridgeAudioDesc": "ચૂંટાયેલા ટેક્સ્ટ મોડલ સાથે આગળ વધવા પહેલા બોલીને લખવા માટેના મોડલ સાથે ઓડિયોનું લખાણ બનાવો.", + "modalityBridgeAudioEnabled": "ઓડિયો બ્રિજ સક્રિય કરો", + "modalityBridgeAudioEnabledDesc": "જ્યારે લક્ષ્ય મોડેલ ઓડિયો પ્રક્રિયા કરી શકતું નથી ત્યારે ઓડિયો ભાગોને ટ્રાન્સક્રિપ્ટ સાથે બદલવા.", + "modalityBridgeAudioModel": "વાણી-થી-ટેક્સ્ટ મોડેલ", + "modalityBridgeAudioModelAuto": "ઓટો (પ્રથમ જોડાયેલ STT પ્રદાતા)", + "modalityBridgeAudioMaxClips": "પ્રતિ વિનંતી મહત્તમ ઓડિયો ક્લિપ્સ", + "modalityBridgeAudioTestButton": "નમૂના ઓડિયો સાથે પરીક્ષણ કરો", + "modalityBridgeAudioTestRunning": "ઓડિયોનું પરીક્ષણ કરી રહ્યું છે…", + "modalityBridgeAudioTestOk": "ઓડિયો બ્રિજ ઠીક છે — {count} ક્લિપ(ઓ) {model} દ્વારા લખવામાં આવ્યા", + "modalityBridgeAudioTestNoop": "ઓડિયો બ્રિજ સક્રિય થયો નથી (લક્ષ્ય ઓડિયોને સપોર્ટ કરી શકે છે, કોઈ STT પ્રદાતા જોડાયેલ નથી, અથવા બ્રિજ નિષ્ક્રિય છે)", + "modalityBridgeAudioTestError": "ઓડિયો પરીક્ષણ નિષ્ફળ ગયું: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/he.json b/src/i18n/messages/he.json index 1117b62ec0a4..e1012c3afece 100644 --- a/src/i18n/messages/he.json +++ b/src/i18n/messages/he.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "חיבור וידאו (דגימת פריימים + כתוביות) נמצא ברשימת המטלות — ראה בעיה #9760.", "modalityBridgeMovedTitle": "הגשר של Vision הוזז", "modalityBridgeMovedBody": "הגדרות Vision Bridge עכשיו חיות בדף Modality Bridge המוקדש.", - "modalityBridgeMovedCta": "פתח את הגדרות גשר המודאליות" + "modalityBridgeMovedCta": "פתח את הגדרות גשר המודאליות", + "modalityBridgeAudioTitle": "גשר שמע", + "modalityBridgeAudioDesc": "הקלד אודיו עם מודל דיבור לטקסט לפני שתמשיך עם מודל הטקסט הנבחר.", + "modalityBridgeAudioEnabled": "אפשר גשר שמע", + "modalityBridgeAudioEnabledDesc": "החלף חלקי שמע בטקסטים כאשר המודל המיועד אינו יכול לעבד שמע.", + "modalityBridgeAudioModel": "מודל דיבור לטקסט", + "modalityBridgeAudioModelAuto": "אוטומטי (ספק STT הראשון שמתחבר)", + "modalityBridgeAudioMaxClips": "מקסימום קטעי שמע לכל בקשה", + "modalityBridgeAudioTestButton": "בדוק עם שמע לדוגמה", + "modalityBridgeAudioTestRunning": "בודק אודיו…", + "modalityBridgeAudioTestOk": "גשר שמע בסדר — {count} קליפ(ים) שהוקלטו על ידי {model}", + "modalityBridgeAudioTestNoop": "Audio Bridge לא הופעל (ייתכן שהיעד תומך באודיו, אין ספק STT מחובר, או שהגשר מושבת)", + "modalityBridgeAudioTestError": "בדיקת שמע נכשלה: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/hi.json b/src/i18n/messages/hi.json index 10cd712a1055..bf3fa5108267 100644 --- a/src/i18n/messages/hi.json +++ b/src/i18n/messages/hi.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "वीडियो ब्रिजिंग (फ्रेम सैंपलिंग + कैप्शनिंग) बैकलॉग में है — मुद्दा #9760 देखें।", "modalityBridgeMovedTitle": "विजन ब्रिज स्थानांतरित किया गया", "modalityBridgeMovedBody": "विज़न ब्रिज सेटिंग्स अब समर्पित मोडालिटी ब्रिज पृष्ठ में लाइव हैं।", - "modalityBridgeMovedCta": "Modality Bridge सेटिंग्स खोलें" + "modalityBridgeMovedCta": "Modality Bridge सेटिंग्स खोलें", + "modalityBridgeAudioTitle": "ऑडियो ब्रिज", + "modalityBridgeAudioDesc": "चुने गए टेक्स्ट मॉडल के साथ जारी रखने से पहले ऑडियो को स्पीच-टू-टेक्स्ट मॉडल के साथ ट्रांसक्राइब करें।", + "modalityBridgeAudioEnabled": "ऑडियो ब्रिज सक्षम करें", + "modalityBridgeAudioEnabledDesc": "जब लक्षित मॉडल ऑडियो को संसाधित नहीं कर सकता है, तो ऑडियो भागों को ट्रांसक्रिप्ट के साथ बदलें।", + "modalityBridgeAudioModel": "स्पीच-टू-टेक्स्ट मॉडल", + "modalityBridgeAudioModelAuto": "ऑटो (पहला कनेक्टेड STT प्रदाता)", + "modalityBridgeAudioMaxClips": "प्रति अनुरोध अधिकतम ऑडियो क्लिप", + "modalityBridgeAudioTestButton": "नमूना ऑडियो के साथ परीक्षण करें", + "modalityBridgeAudioTestRunning": "ऑडियो का परीक्षण कर रहा है…", + "modalityBridgeAudioTestOk": "ऑडियो ब्रिज ठीक है — {count} क्लिप(स) {model} द्वारा ट्रांसक्राइब की गई", + "modalityBridgeAudioTestNoop": "ऑडियो ब्रिज सक्रिय नहीं हुआ (लक्ष्य ऑडियो का समर्थन कर सकता है, कोई STT प्रदाता कनेक्ट नहीं है, या ब्रिज अक्षम है)", + "modalityBridgeAudioTestError": "ऑडियो परीक्षण विफल: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/hu.json b/src/i18n/messages/hu.json index 4fb34eb2739b..9d38224677b0 100644 --- a/src/i18n/messages/hu.json +++ b/src/i18n/messages/hu.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "A videohíd (keretmintavételezés + feliratozás) a hátralékban van — lásd a #9760 számú problémát.", "modalityBridgeMovedTitle": "A Vision Bridge áthelyezve", "modalityBridgeMovedBody": "A Vision Bridge beállításai mostantól a dedikált Modality Bridge oldalon találhatók.", - "modalityBridgeMovedCta": "Nyisd meg a Modalitás Híd beállításait" + "modalityBridgeMovedCta": "Nyisd meg a Modalitás Híd beállításait", + "modalityBridgeAudioTitle": "Audio Híd", + "modalityBridgeAudioDesc": "Írja át a hangot egy beszédfelismerő modellel, mielőtt folytatná a választott szövegmodellel.", + "modalityBridgeAudioEnabled": "Audio Híd Engedélyezése", + "modalityBridgeAudioEnabledDesc": "Cseréld le a hangrészeket átiratokkal, amikor a célmodell nem tudja feldolgozni a hangot.", + "modalityBridgeAudioModel": "Beszédfelismerő modell", + "modalityBridgeAudioModelAuto": "Automatikus (első csatlakoztatott STT szolgáltató)", + "modalityBridgeAudioMaxClips": "Max audio clips per request", + "modalityBridgeAudioTestButton": "Teszt mintapéldány hanggal", + "modalityBridgeAudioTestRunning": "Audio tesztelése…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} klip(pek) átírása a {model} által", + "modalityBridgeAudioTestNoop": "Az Audio Bridge nem aktiválódott (a célpont támogathatja a hangot, nincs csatlakoztatva STT szolgáltató, vagy a híd le van tiltva)", + "modalityBridgeAudioTestError": "A hangteszt nem sikerült: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/id.json b/src/i18n/messages/id.json index e07abb08e2d2..b4825fb62412 100644 --- a/src/i18n/messages/id.json +++ b/src/i18n/messages/id.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Jembatan video (pengambilan bingkai + penulisan keterangan) ada di backlog — lihat masalah #9760.", "modalityBridgeMovedTitle": "Vision Bridge dipindahkan", "modalityBridgeMovedBody": "Pengaturan Vision Bridge sekarang tersedia di halaman Modality Bridge yang khusus.", - "modalityBridgeMovedCta": "Buka pengaturan Jembatan Modality" + "modalityBridgeMovedCta": "Buka pengaturan Jembatan Modality", + "modalityBridgeAudioTitle": "Jembatan Audio", + "modalityBridgeAudioDesc": "Transkripsikan audio dengan model pengenalan suara sebelum melanjutkan dengan model teks yang dipilih.", + "modalityBridgeAudioEnabled": "Aktifkan Jembatan Audio", + "modalityBridgeAudioEnabledDesc": "Ganti bagian audio dengan transkrip ketika model target tidak dapat memproses audio.", + "modalityBridgeAudioModel": "Model Ucapan ke Teks", + "modalityBridgeAudioModelAuto": "Auto (penyedia STT yang terhubung pertama)", + "modalityBridgeAudioMaxClips": "Maksimal klip audio per permintaan", + "modalityBridgeAudioTestButton": "Uji dengan audio sampel", + "modalityBridgeAudioTestRunning": "Menguji audio…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} klip telah ditranskripsi oleh {model}", + "modalityBridgeAudioTestNoop": "Audio Bridge tidak diaktifkan (target mungkin mendukung audio, tidak ada penyedia STT yang terhubung, atau jembatan dinonaktifkan)", + "modalityBridgeAudioTestError": "Pengujian audio gagal: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/in.json b/src/i18n/messages/in.json index 068f6ab2df51..bb9a288035fa 100644 --- a/src/i18n/messages/in.json +++ b/src/i18n/messages/in.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Jembatan video (pengambilan bingkai + penulisan keterangan) ada di backlog — lihat isu #9760.", "modalityBridgeMovedTitle": "Vision Bridge dipindahkan", "modalityBridgeMovedBody": "Pengaturan Vision Bridge sekarang tersedia di halaman Modality Bridge yang khusus.", - "modalityBridgeMovedCta": "Buka pengaturan Jembatan Modality" + "modalityBridgeMovedCta": "Buka pengaturan Jembatan Modality", + "modalityBridgeAudioTitle": "Jembatan Audio", + "modalityBridgeAudioDesc": "Transkripsikan audio dengan model ucapan-ke-teks sebelum melanjutkan dengan model teks yang dipilih.", + "modalityBridgeAudioEnabled": "Aktifkan Jembatan Audio", + "modalityBridgeAudioEnabledDesc": "Ganti bagian audio dengan transkrip ketika model target tidak dapat memproses audio.", + "modalityBridgeAudioModel": "Model Ucapan-ke-Teks", + "modalityBridgeAudioModelAuto": "Auto (penyedia STT yang terhubung pertama)", + "modalityBridgeAudioMaxClips": "Maksimal klip audio per permintaan", + "modalityBridgeAudioTestButton": "Uji dengan audio contoh", + "modalityBridgeAudioTestRunning": "Menguji audio…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} klip telah ditranskripsi oleh {model}", + "modalityBridgeAudioTestNoop": "Audio Bridge tidak diaktifkan (target mungkin mendukung audio, tidak ada penyedia STT yang terhubung, atau jembatan dinonaktifkan)", + "modalityBridgeAudioTestError": "Uji audio gagal: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/it.json b/src/i18n/messages/it.json index 1ba03c3cce03..dc7c13183322 100644 --- a/src/i18n/messages/it.json +++ b/src/i18n/messages/it.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Il bridging video (campionamento dei fotogrammi + sottotitolazione) è nella lista delle cose da fare — vedi il problema #9760.", "modalityBridgeMovedTitle": "Vision Bridge spostato", "modalityBridgeMovedBody": "Le impostazioni di Vision Bridge ora si trovano nella pagina dedicata a Modality Bridge.", - "modalityBridgeMovedCta": "Apri le impostazioni del Modality Bridge" + "modalityBridgeMovedCta": "Apri le impostazioni del Modality Bridge", + "modalityBridgeAudioTitle": "Audio Bridge", + "modalityBridgeAudioDesc": "Trascrivi l'audio con un modello di riconoscimento vocale prima di continuare con il modello di testo scelto.", + "modalityBridgeAudioEnabled": "Abilita Audio Bridge", + "modalityBridgeAudioEnabledDesc": "Sostituisci le parti audio con le trascrizioni quando il modello di destinazione non può elaborare l'audio.", + "modalityBridgeAudioModel": "Modello di riconoscimento vocale", + "modalityBridgeAudioModelAuto": "Auto (primo fornitore STT connesso)", + "modalityBridgeAudioMaxClips": "Numero massimo di clip audio per richiesta", + "modalityBridgeAudioTestButton": "Test con audio di esempio", + "modalityBridgeAudioTestRunning": "Testando l'audio…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} clip(s) trascritti da {model}", + "modalityBridgeAudioTestNoop": "Audio Bridge non è stato attivato (il target potrebbe supportare l'audio, nessun fornitore STT è connesso, oppure il bridge è disabilitato)", + "modalityBridgeAudioTestError": "Test audio non riuscito: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/ja.json b/src/i18n/messages/ja.json index c8e3ab75cd74..b6aacf0c630e 100644 --- a/src/i18n/messages/ja.json +++ b/src/i18n/messages/ja.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "ビデオブリッジ(フレームサンプリング + キャプショニング)はバックログにあります — イシュー #9760 を参照してください。", "modalityBridgeMovedTitle": "Vision Bridgeが移動しました", "modalityBridgeMovedBody": "Vision Bridgeの設定は、専用のModality Bridgeページに移動しました。", - "modalityBridgeMovedCta": "モダリティブリッジの設定を開く" + "modalityBridgeMovedCta": "モダリティブリッジの設定を開く", + "modalityBridgeAudioTitle": "オーディオブリッジ", + "modalityBridgeAudioDesc": "音声をテキストモデルで続行する前に、音声を音声認識モデルで文字起こししてください。", + "modalityBridgeAudioEnabled": "オーディオブリッジを有効にする", + "modalityBridgeAudioEnabledDesc": "ターゲットモデルが音声を処理できない場合は、音声部分をトランスクリプトに置き換えます。", + "modalityBridgeAudioModel": "音声認識モデル", + "modalityBridgeAudioModelAuto": "自動 (最初に接続された STT プロバイダー)", + "modalityBridgeAudioMaxClips": "リクエストごとの最大オーディオクリップ数", + "modalityBridgeAudioTestButton": "サンプル音声でテストする", + "modalityBridgeAudioTestRunning": "オーディオをテスト中…", + "modalityBridgeAudioTestOk": "オーディオブリッジ OK — {count} 件のクリップが {model} によって文字起こしされました", + "modalityBridgeAudioTestNoop": "Audio Bridgeはアクティブになりませんでした(ターゲットはオーディオをサポートしている可能性がありますが、STTプロバイダーが接続されていないか、ブリッジが無効になっています)", + "modalityBridgeAudioTestError": "オーディオテストに失敗しました: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/ko.json b/src/i18n/messages/ko.json index 3465d4dd4d94..29dea50b75d1 100644 --- a/src/i18n/messages/ko.json +++ b/src/i18n/messages/ko.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "비디오 브리징(프레임 샘플링 + 캡셔닝)은 백로그에 있습니다 — 이슈 #9760을 참조하세요.", "modalityBridgeMovedTitle": "비전 브리지 이동됨", "modalityBridgeMovedBody": "Vision Bridge 설정이 이제 전용 Modality Bridge 페이지에 있습니다.", - "modalityBridgeMovedCta": "모달리티 브리지 설정 열기" + "modalityBridgeMovedCta": "모달리티 브리지 설정 열기", + "modalityBridgeAudioTitle": "오디오 브리지", + "modalityBridgeAudioDesc": "선택한 텍스트 모델을 계속 진행하기 전에 음성을 텍스트로 변환하는 음성 인식 모델을 사용하세요.", + "modalityBridgeAudioEnabled": "오디오 브릿지 활성화", + "modalityBridgeAudioEnabledDesc": "대상 모델이 오디오를 처리할 수 없을 때 오디오 부분을 전사로 교체합니다.", + "modalityBridgeAudioModel": "음성 인식 모델", + "modalityBridgeAudioModelAuto": "자동 (첫 번째 연결된 STT 공급자)", + "modalityBridgeAudioMaxClips": "요청당 최대 오디오 클립 수", + "modalityBridgeAudioTestButton": "샘플 오디오로 테스트하기", + "modalityBridgeAudioTestRunning": "오디오 테스트 중…", + "modalityBridgeAudioTestOk": "오디오 브리지 확인 — {count} 클립이 {model}에 의해 전사되었습니다", + "modalityBridgeAudioTestNoop": "오디오 브리지가 활성화되지 않았습니다(대상이 오디오를 지원할 수 있지만 STT 공급자가 연결되어 있지 않거나 브리지가 비활성화되어 있습니다)", + "modalityBridgeAudioTestError": "오디오 테스트 실패: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/mr.json b/src/i18n/messages/mr.json index 181829d6e59a..c5883757827c 100644 --- a/src/i18n/messages/mr.json +++ b/src/i18n/messages/mr.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "व्हिडिओ ब्रिजिंग (फ्रेम सॅम्पलिंग + कॅप्शनिंग) बॅकलॉगमध्ये आहे — समस्या #9760 पहा.", "modalityBridgeMovedTitle": "व्हिजन ब्रिज हलवला", "modalityBridgeMovedBody": "व्हिजन ब्रिज सेटिंग्ज आता समर्पित मोडालिटी ब्रिज पृष्ठावर उपलब्ध आहेत.", - "modalityBridgeMovedCta": "मोडालिटी ब्रिज सेटिंग्ज उघडा" + "modalityBridgeMovedCta": "मोडालिटी ब्रिज सेटिंग्ज उघडा", + "modalityBridgeAudioTitle": "ऑडिओ ब्रिज", + "modalityBridgeAudioDesc": "चयनित टेक्स्ट मॉडेलसह पुढे जाण्यापूर्वी ऑडिओचे ट्रान्सक्रिप्शन करा.", + "modalityBridgeAudioEnabled": "ऑडिओ ब्रिज सक्षम करा", + "modalityBridgeAudioEnabledDesc": "लक्ष्य मॉडेल ऑडिओ प्रक्रिया करू शकत नाही तेव्हा ऑडिओ भागांचे स्थानांतर ट्रान्सक्रिप्टसह करा.", + "modalityBridgeAudioModel": "स्पीच-टू-टेक्स्ट मॉडेल", + "modalityBridgeAudioModelAuto": "ऑटो (पहिला कनेक्ट केलेला STT प्रदाता)", + "modalityBridgeAudioMaxClips": "प्रत्येक विनंतीसाठी कमाल ऑडिओ क्लिप्स", + "modalityBridgeAudioTestButton": "नमुना ऑडिओसह चाचणी करा", + "modalityBridgeAudioTestRunning": "ऑडिओची चाचणी करत आहे…", + "modalityBridgeAudioTestOk": "ऑडिओ ब्रिज ठीक आहे — {count} क्लिप(स) {model} द्वारे ट्रान्सक्राइब केल्या गेल्या", + "modalityBridgeAudioTestNoop": "ऑडिओ ब्रिज सक्रिय झाला नाही (लक्ष्य ऑडिओला समर्थन देऊ शकतो, कोणताही STT प्रदाता जोडलेला नाही, किंवा ब्रिज अक्षम आहे)", + "modalityBridgeAudioTestError": "ऑडिओ चाचणी अयशस्वी: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/ms.json b/src/i18n/messages/ms.json index fab7dc22c3b2..b5e6ebe48449 100644 --- a/src/i18n/messages/ms.json +++ b/src/i18n/messages/ms.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Jambatan video (pengambilan bingkai + penulisan) berada dalam senarai tunggu — lihat isu #9760.", "modalityBridgeMovedTitle": "Jambatan Visi dipindahkan", "modalityBridgeMovedBody": "Tetapan Vision Bridge kini tersedia di halaman Modality Bridge yang khusus.", - "modalityBridgeMovedCta": "Buka tetapan Jambatan Modality" + "modalityBridgeMovedCta": "Buka tetapan Jambatan Modality", + "modalityBridgeAudioTitle": "Jambatan Audio", + "modalityBridgeAudioDesc": "Transkrip audio dengan model pertuturan-ke-teks sebelum meneruskan dengan model teks yang dipilih.", + "modalityBridgeAudioEnabled": "Aktifkan Jambatan Audio", + "modalityBridgeAudioEnabledDesc": "Gantikan bahagian audio dengan transkrip apabila model sasaran tidak dapat memproses audio.", + "modalityBridgeAudioModel": "Model Pertuturan-ke-teks", + "modalityBridgeAudioModelAuto": "Auto (penyedia STT yang pertama disambungkan)", + "modalityBridgeAudioMaxClips": "Maksimum klip audio setiap permintaan", + "modalityBridgeAudioTestButton": "Uji dengan audio sampel", + "modalityBridgeAudioTestRunning": "Menguji audio…", + "modalityBridgeAudioTestOk": "Jambatan Audio OK — {count} klip telah ditranskripsikan oleh {model}", + "modalityBridgeAudioTestNoop": "Audio Bridge tidak diaktifkan (sasaran mungkin menyokong audio, tiada penyedia STT yang disambungkan, atau jambatan dinyahdayakan)", + "modalityBridgeAudioTestError": "Ujian audio gagal: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/nl.json b/src/i18n/messages/nl.json index 834c8aae6d09..436ce0fae035 100644 --- a/src/i18n/messages/nl.json +++ b/src/i18n/messages/nl.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Videobruggen (frame sampling + ondertiteling) staat op de backlog — zie probleem #9760.", "modalityBridgeMovedTitle": "Vision Bridge verplaatst", "modalityBridgeMovedBody": "De Vision Bridge-instellingen zijn nu beschikbaar op de speciale Modality Bridge-pagina.", - "modalityBridgeMovedCta": "Open Modality Bridge-instellingen" + "modalityBridgeMovedCta": "Open Modality Bridge-instellingen", + "modalityBridgeAudioTitle": "Audio Brug", + "modalityBridgeAudioDesc": "Transcribeer audio met een spraak-naar-tekstmodel voordat je verdergaat met het gekozen tekstmodel.", + "modalityBridgeAudioEnabled": "Audio Bridge Inschakelen", + "modalityBridgeAudioEnabledDesc": "Vervang audio-onderdelen door transcripties wanneer het doelformulier audio niet kan verwerken.", + "modalityBridgeAudioModel": "Spraak-naar-tekstmodel", + "modalityBridgeAudioModelAuto": "Auto (eerste verbonden STT-provider)", + "modalityBridgeAudioMaxClips": "Max audio clips per verzoek", + "modalityBridgeAudioTestButton": "Test met voorbeeldaudio", + "modalityBridgeAudioTestRunning": "Audio testen…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} clip(s) getranscribeerd door {model}", + "modalityBridgeAudioTestNoop": "Audio Bridge is niet geactiveerd (het doel kan audio ondersteunen, er is geen STT-provider verbonden, of de brug is uitgeschakeld)", + "modalityBridgeAudioTestError": "Audio-test mislukt: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/no.json b/src/i18n/messages/no.json index 58789e251415..9e8d1f7d487d 100644 --- a/src/i18n/messages/no.json +++ b/src/i18n/messages/no.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Videobroing (rammesampling + teksting) er på backloggen — se sak #9760.", "modalityBridgeMovedTitle": "Vision Bridge flyttet", "modalityBridgeMovedBody": "Innstillinger for Vision Bridge er nå tilgjengelige på den dedikerte Modality Bridge-siden.", - "modalityBridgeMovedCta": "Åpne Modality Bridge-innstillinger" + "modalityBridgeMovedCta": "Åpne Modality Bridge-innstillinger", + "modalityBridgeAudioTitle": "Lydbro", + "modalityBridgeAudioDesc": "Transkriber lyd med en tale-til-tekst-modell før du fortsetter med den valgte tekstmodellen.", + "modalityBridgeAudioEnabled": "Aktiver Lydbro", + "modalityBridgeAudioEnabledDesc": "Erstatt lydpartier med transkripsjoner når målmodellen ikke kan behandle lyd.", + "modalityBridgeAudioModel": "Tale-til-tekst-modell", + "modalityBridgeAudioModelAuto": "Auto (første tilkoblede STT-leverandør)", + "modalityBridgeAudioMaxClips": "Maksimalt antall lydklipp per forespørsel", + "modalityBridgeAudioTestButton": "Test med prøveaudio", + "modalityBridgeAudioTestRunning": "Tester lyd…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} klipp(e) transkribert av {model}", + "modalityBridgeAudioTestNoop": "Audio Bridge ble ikke aktivert (målet kan støtte lyd, ingen STT-leverandør er tilkoblet, eller broen er deaktivert)", + "modalityBridgeAudioTestError": "Lydtest mislyktes: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/phi.json b/src/i18n/messages/phi.json index 4fd30f026ca9..c11037806183 100644 --- a/src/i18n/messages/phi.json +++ b/src/i18n/messages/phi.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Ang video bridging (frame sampling + captioning) ay nasa backlog — tingnan ang isyu #9760.", "modalityBridgeMovedTitle": "Inilipat ang Vision Bridge", "modalityBridgeMovedBody": "Ang mga setting ng Vision Bridge ay ngayon nasa nakalaang pahina ng Modality Bridge.", - "modalityBridgeMovedCta": "Buksan ang mga setting ng Modality Bridge" + "modalityBridgeMovedCta": "Buksan ang mga setting ng Modality Bridge", + "modalityBridgeAudioTitle": "Audio Bridge", + "modalityBridgeAudioDesc": "I-transcribe ang audio gamit ang speech-to-text na modelo bago magpatuloy sa napiling text model.", + "modalityBridgeAudioEnabled": "I-enable ang Audio Bridge", + "modalityBridgeAudioEnabledDesc": "Palitan ang mga bahagi ng audio ng mga transcript kapag hindi kayang iproseso ng target na modelo ang audio.", + "modalityBridgeAudioModel": "Modelo ng pagsasalita sa teksto", + "modalityBridgeAudioModelAuto": "Auto (unang nakakonektang STT provider)", + "modalityBridgeAudioMaxClips": "Max na audio clip sa bawat kahilingan", + "modalityBridgeAudioTestButton": "Subukan gamit ang sample na audio", + "modalityBridgeAudioTestRunning": "Sinusubukan ang audio…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} clip(s) na na-transcribe ng {model}", + "modalityBridgeAudioTestNoop": "Hindi na-activate ang Audio Bridge (maaaring sinusuportahan ng target ang audio, walang nakakonektang STT provider, o naka-disable ang bridge)", + "modalityBridgeAudioTestError": "Nabigo ang pagsubok sa audio: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/pl.json b/src/i18n/messages/pl.json index f0604529d222..ad4c9526a24b 100644 --- a/src/i18n/messages/pl.json +++ b/src/i18n/messages/pl.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Mostkowanie wideo (próbkowanie klatek + napisy) jest na liście zadań — zobacz problem #9760.", "modalityBridgeMovedTitle": "Mostek Wizji przeniesiony", "modalityBridgeMovedBody": "Ustawienia Vision Bridge są teraz dostępne na dedykowanej stronie Modality Bridge.", - "modalityBridgeMovedCta": "Otwórz ustawienia Mostka Modalności" + "modalityBridgeMovedCta": "Otwórz ustawienia Mostka Modalności", + "modalityBridgeAudioTitle": "Mostek Audio", + "modalityBridgeAudioDesc": "Transkrybuj audio za pomocą modelu rozpoznawania mowy przed kontynuowaniem z wybranym modelem tekstowym.", + "modalityBridgeAudioEnabled": "Włącz Most Audio", + "modalityBridgeAudioEnabledDesc": "Zastąp części audio transkrypcjami, gdy docelowy model nie może przetworzyć audio.", + "modalityBridgeAudioModel": "Model rozpoznawania mowy na tekst", + "modalityBridgeAudioModelAuto": "Auto (pierwszy podłączony dostawca STT)", + "modalityBridgeAudioMaxClips": "Maksymalna liczba klipów audio na żądanie", + "modalityBridgeAudioTestButton": "Testuj z przykładowym dźwiękiem", + "modalityBridgeAudioTestRunning": "Testowanie dźwięku…", + "modalityBridgeAudioTestOk": "Most Audio OK — {count} klip(y) przetłumaczone przez {model}", + "modalityBridgeAudioTestNoop": "Mostek audio nie został aktywowany (docelowy może obsługiwać audio, żaden dostawca STT nie jest podłączony lub mostek jest wyłączony)", + "modalityBridgeAudioTestError": "Test audio nie powiódł się: {message}" }, "contextRtk": { "title": "Silnik RTK", diff --git a/src/i18n/messages/pt-BR.json b/src/i18n/messages/pt-BR.json index 325a6e3f1150..f8c2336f5c34 100644 --- a/src/i18n/messages/pt-BR.json +++ b/src/i18n/messages/pt-BR.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "A ponte de vídeo (amostragem de quadros + legendagem) está no backlog — veja a issue #9760.", "modalityBridgeMovedTitle": "Vision Bridge movido", "modalityBridgeMovedBody": "As configurações do Vision Bridge agora estão na página dedicada do Modality Bridge.", - "modalityBridgeMovedCta": "Abra as configurações do Modality Bridge" + "modalityBridgeMovedCta": "Abra as configurações do Modality Bridge", + "modalityBridgeAudioTitle": "Ponte de Áudio", + "modalityBridgeAudioDesc": "Transcreva o áudio com um modelo de reconhecimento de fala antes de continuar com o modelo de texto escolhido.", + "modalityBridgeAudioEnabled": "Ativar Ponte de Áudio", + "modalityBridgeAudioEnabledDesc": "Substitua as partes de áudio por transcrições quando o modelo de destino não puder processar áudio.", + "modalityBridgeAudioModel": "Modelo de fala para texto", + "modalityBridgeAudioModelAuto": "Automático (primeiro provedor STT conectado)", + "modalityBridgeAudioMaxClips": "Máx. de clipes de áudio por solicitação", + "modalityBridgeAudioTestButton": "Teste com áudio de amostra", + "modalityBridgeAudioTestRunning": "Testando áudio…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} clipe(s) transcrito(s) por {model}", + "modalityBridgeAudioTestNoop": "O Audio Bridge não foi ativado (o destino pode suportar áudio, nenhum provedor de STT está conectado ou a ponte está desativada)", + "modalityBridgeAudioTestError": "Teste de áudio falhou: {message}" }, "contextRtk": { "title": "Motor RTK", diff --git a/src/i18n/messages/pt.json b/src/i18n/messages/pt.json index 9fbfc1c65b2d..1f205173298f 100644 --- a/src/i18n/messages/pt.json +++ b/src/i18n/messages/pt.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "A ponte de vídeo (amostragem de quadros + legendagem) está na lista de tarefas — veja o problema #9760.", "modalityBridgeMovedTitle": "Vision Bridge movido", "modalityBridgeMovedBody": "As definições do Vision Bridge agora estão disponíveis na página dedicada do Modality Bridge.", - "modalityBridgeMovedCta": "Abrir as definições da Ponte de Modalidade" + "modalityBridgeMovedCta": "Abrir as definições da Ponte de Modalidade", + "modalityBridgeAudioTitle": "Ponte de Áudio", + "modalityBridgeAudioDesc": "Transcreva o áudio com um modelo de reconhecimento de fala antes de continuar com o modelo de texto escolhido.", + "modalityBridgeAudioEnabled": "Ativar Ponte de Áudio", + "modalityBridgeAudioEnabledDesc": "Substitua as partes de áudio por transcrições quando o modelo de destino não conseguir processar áudio.", + "modalityBridgeAudioModel": "Modelo de reconhecimento de voz", + "modalityBridgeAudioModelAuto": "Auto (primeiro fornecedor STT conectado)", + "modalityBridgeAudioMaxClips": "Máximo de clipes de áudio por pedido", + "modalityBridgeAudioTestButton": "Teste com áudio de amostra", + "modalityBridgeAudioTestRunning": "A testar áudio…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} clipe(s) transcrito(s) por {model}", + "modalityBridgeAudioTestNoop": "Audio Bridge não ativou (o destino pode suportar áudio, nenhum fornecedor de STT está conectado ou a ponte está desativada)", + "modalityBridgeAudioTestError": "Teste de áudio falhou: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/ro.json b/src/i18n/messages/ro.json index 2fa29eebaf44..e8d55b0685fc 100644 --- a/src/i18n/messages/ro.json +++ b/src/i18n/messages/ro.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Video bridging (frame sampling + captioning) este pe lista de așteptare — vezi problema #9760.", "modalityBridgeMovedTitle": "Vision Bridge mutat", "modalityBridgeMovedBody": "Setările Vision Bridge sunt acum disponibile pe pagina dedicată Modality Bridge.", - "modalityBridgeMovedCta": "Deschide setările Modality Bridge" + "modalityBridgeMovedCta": "Deschide setările Modality Bridge", + "modalityBridgeAudioTitle": "Punte Audio", + "modalityBridgeAudioDesc": "Transcrieți audio cu un model de recunoaștere vocală înainte de a continua cu modelul de text ales.", + "modalityBridgeAudioEnabled": "Activează Podul Audio", + "modalityBridgeAudioEnabledDesc": "Înlocuiți părțile audio cu transcrieri atunci când modelul țintă nu poate procesa audio.", + "modalityBridgeAudioModel": "Model de recunoaștere vocală", + "modalityBridgeAudioModelAuto": "Auto (primul furnizor STT conectat)", + "modalityBridgeAudioMaxClips": "Maxim număr de clipuri audio per cerere", + "modalityBridgeAudioTestButton": "Testați cu audio de probă", + "modalityBridgeAudioTestRunning": "Testare audio…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} clip-uri transcrise de {model}", + "modalityBridgeAudioTestNoop": "Audio Bridge nu s-a activat (ținta poate suporta audio, nu este conectat niciun furnizor STT sau podul este dezactivat)", + "modalityBridgeAudioTestError": "Testul audio a eșuat: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/ru.json b/src/i18n/messages/ru.json index 85f115679911..b0ccb57c9878 100644 --- a/src/i18n/messages/ru.json +++ b/src/i18n/messages/ru.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Видеомост (выборка кадров + субтитры) находится в очереди — см. задачу #9760.", "modalityBridgeMovedTitle": "Мост Видения перемещен", "modalityBridgeMovedBody": "Настройки Vision Bridge теперь находятся на отдельной странице Modality Bridge.", - "modalityBridgeMovedCta": "Открыть настройки моста модальности" + "modalityBridgeMovedCta": "Открыть настройки моста модальности", + "modalityBridgeAudioTitle": "Аудио Мост", + "modalityBridgeAudioDesc": "Транскрибируйте аудио с помощью модели распознавания речи перед тем, как продолжить с выбранной текстовой моделью.", + "modalityBridgeAudioEnabled": "Включить аудиомост", + "modalityBridgeAudioEnabledDesc": "Замените аудиочасти на транскрипты, когда целевая модель не может обрабатывать аудио.", + "modalityBridgeAudioModel": "Модель распознавания речи", + "modalityBridgeAudioModelAuto": "Авто (первый подключенный поставщик STT)", + "modalityBridgeAudioMaxClips": "Максимальное количество аудиоклипов на запрос", + "modalityBridgeAudioTestButton": "Тест с образцовым аудио", + "modalityBridgeAudioTestRunning": "Тестирование аудио…", + "modalityBridgeAudioTestOk": "Аудио мост ОК — {count} клип(ов) транскрибировано {model}", + "modalityBridgeAudioTestNoop": "Аудио мост не активирован (цель может поддерживать аудио, провайдер STT не подключен или мост отключен)", + "modalityBridgeAudioTestError": "Тест аудио не удался: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/sk.json b/src/i18n/messages/sk.json index 7d939a5de87a..baa43c2d3edb 100644 --- a/src/i18n/messages/sk.json +++ b/src/i18n/messages/sk.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Video bridging (frame sampling + captioning) je na backlogu — pozrite si problém #9760.", "modalityBridgeMovedTitle": "Vision Bridge presunutý", "modalityBridgeMovedBody": "Nastavenia Vision Bridge sú teraz dostupné na samostatnej stránke Modality Bridge.", - "modalityBridgeMovedCta": "Otvorenie nastavení Modality Bridge" + "modalityBridgeMovedCta": "Otvorenie nastavení Modality Bridge", + "modalityBridgeAudioTitle": "Audio Bridge", + "modalityBridgeAudioDesc": "Prepisujte zvuk pomocou modelu reč na text pred pokračovaním s vybraným textovým modelom.", + "modalityBridgeAudioEnabled": "Povoliť Audio Bridge", + "modalityBridgeAudioEnabledDesc": "Nahradiť audio časti prepisom, keď cieľový model nemôže spracovať audio.", + "modalityBridgeAudioModel": "Model na prevod reči na text", + "modalityBridgeAudioModelAuto": "Auto (prvý pripojený poskytovateľ STT)", + "modalityBridgeAudioMaxClips": "Maximálny počet audio klipov na požiadavku", + "modalityBridgeAudioTestButton": "Test s ukážkovým zvukom", + "modalityBridgeAudioTestRunning": "Testovanie zvuku…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} klip(y) prepisované pomocou {model}", + "modalityBridgeAudioTestNoop": "Audio Bridge sa nepodarilo aktivovať (cieľ môže podporovať audio, žiadny poskytovateľ STT nie je pripojený, alebo je most vypnutý)", + "modalityBridgeAudioTestError": "Test zvuku zlyhal: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/sv.json b/src/i18n/messages/sv.json index 7f08d920c0f6..a607cfc94fa8 100644 --- a/src/i18n/messages/sv.json +++ b/src/i18n/messages/sv.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Videobrobyggning (ramprovtagning + textning) ligger på backloggen — se ärende #9760.", "modalityBridgeMovedTitle": "Vision Bridge flyttad", "modalityBridgeMovedBody": "Inställningarna för Vision Bridge finns nu på den dedikerade Modality Bridge-sidan.", - "modalityBridgeMovedCta": "Öppna inställningar för Modality Bridge" + "modalityBridgeMovedCta": "Öppna inställningar för Modality Bridge", + "modalityBridgeAudioTitle": "Ljudbro", + "modalityBridgeAudioDesc": "Transkribera ljud med en tal-till-text-modell innan du fortsätter med den valda textmodellen.", + "modalityBridgeAudioEnabled": "Aktivera Ljudbro", + "modalityBridgeAudioEnabledDesc": "Ersätt ljuddelar med transkriptioner när målmodellen inte kan bearbeta ljud.", + "modalityBridgeAudioModel": "Tal-till-text-modell", + "modalityBridgeAudioModelAuto": "Auto (först ansluten STT-leverantör)", + "modalityBridgeAudioMaxClips": "Max ljudklipp per begäran", + "modalityBridgeAudioTestButton": "Testa med provljud", + "modalityBridgeAudioTestRunning": "Testar ljud…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} klipp transkriberade av {model}", + "modalityBridgeAudioTestNoop": "Audio Bridge aktiverades inte (målet kan stödja ljud, ingen STT-leverantör är ansluten, eller bron är inaktiverad)", + "modalityBridgeAudioTestError": "Ljudtestet misslyckades: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/sw.json b/src/i18n/messages/sw.json index ec8b508d9b90..d89af4f4e8a7 100644 --- a/src/i18n/messages/sw.json +++ b/src/i18n/messages/sw.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Kuhusisha video (uchukuaji wa fremu + uandishi wa maandiko) iko kwenye orodha ya kazi — angalia suala #9760.", "modalityBridgeMovedTitle": "Daraja la Maono limehamishwa", "modalityBridgeMovedBody": "Mipangilio ya Vision Bridge sasa ipo kwenye ukurasa maalum wa Modality Bridge.", - "modalityBridgeMovedCta": "Fungua mipangilio ya Daraja la Modality" + "modalityBridgeMovedCta": "Fungua mipangilio ya Daraja la Modality", + "modalityBridgeAudioTitle": "Daraja la Sauti", + "modalityBridgeAudioDesc": "Transcribe sauti kwa kutumia mfano wa hotuba hadi maandiko kabla ya kuendelea na mfano wa maandiko uliochaguliwa.", + "modalityBridgeAudioEnabled": "wezesha Daraja la Sauti", + "modalityBridgeAudioEnabledDesc": "Badilisha sehemu za sauti na nakala wakati mfano wa lengo hauwezi kushughulikia sauti.", + "modalityBridgeAudioModel": "Mfano wa sauti hadi maandiko", + "modalityBridgeAudioModelAuto": "Auto (mtoa STT wa kwanza kuunganishwa)", + "modalityBridgeAudioMaxClips": "Max sauti za sauti kwa ombi", + "modalityBridgeAudioTestButton": "Jaribu na sauti ya sampuli", + "modalityBridgeAudioTestRunning": "Kujaribu sauti…", + "modalityBridgeAudioTestOk": "Daraja la Sauti Sawa — {count} kipande(kipande) kilichotafsiriwa na {model}", + "modalityBridgeAudioTestNoop": "Daraja la Sauti halikufanya kazi (lengo linaweza kusaidia sauti, hakuna mtoa huduma wa STT aliyeunganishwa, au daraja limezuiliwa)", + "modalityBridgeAudioTestError": "Jaribio la sauti limefeli: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/ta.json b/src/i18n/messages/ta.json index 1e750916752d..0668824065ce 100644 --- a/src/i18n/messages/ta.json +++ b/src/i18n/messages/ta.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "வீடியோ பாலம் (படம் மாதிரி + தலைப்பு) பின்னணி பட்டியலில் உள்ளது — விவரம் #9760 ஐப் பார்க்கவும்.", "modalityBridgeMovedTitle": "விசன் ப்ரிட்ஜ் நகர்ந்தது", "modalityBridgeMovedBody": "Vision Bridge அமைப்புகள் தற்போது தனிப்பட்ட Modality Bridge பக்கத்தில் உள்ளன.", - "modalityBridgeMovedCta": "மோடாலிட்டி பாலம் அமைப்புகளை திறக்கவும்" + "modalityBridgeMovedCta": "மோடாலிட்டி பாலம் அமைப்புகளை திறக்கவும்", + "modalityBridgeAudioTitle": "ஆடியோ பாலம்", + "modalityBridgeAudioDesc": "தேர்ந்தெடுக்கப்பட்ட உரை மாதிரியை தொடர்வதற்கு முன், உரையாடலை உரை-க்கு-உரை மாதிரியுடன் மாற்றவும்.", + "modalityBridgeAudioEnabled": "ஆடியோ பாலத்தை இயக்கவும்", + "modalityBridgeAudioEnabledDesc": "இலக்கு மாதிரி ஒலியை செயலாக்க முடியாத போது ஒலிப் பகுதிகளை உரைபடுத்தல்களால் மாற்றவும்.", + "modalityBridgeAudioModel": "உரை-இல்-எழுத்து மாதிரி", + "modalityBridgeAudioModelAuto": "தானியங்கி (முதலில் இணைக்கப்பட்ட STT வழங்குநர்)", + "modalityBridgeAudioMaxClips": "ஒரு கோரிக்கைக்கு அதிகபட்ச ஆடியோ கிளிப்புகள்", + "modalityBridgeAudioTestButton": "சாம்பிள் ஒலியுடன் சோதனை செய்கின்றது", + "modalityBridgeAudioTestRunning": "ஒலியை சோதிக்கிறேன்…", + "modalityBridgeAudioTestOk": "ஆடியோ பாலம் சரி — {count} கிளிப்(கள்) {model} மூலம் உரைமாற்றம் செய்யப்பட்டது", + "modalityBridgeAudioTestNoop": "ஆடியோ பாலம் செயல்படுத்தப்படவில்லை (இலக்கு ஆடியோவை ஆதரிக்கலாம், STT வழங்குநர் இணைக்கப்படவில்லை, அல்லது பாலம் முடக்கப்பட்டுள்ளது)", + "modalityBridgeAudioTestError": "ஒலி சோதனை தோல்வியடைந்தது: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/te.json b/src/i18n/messages/te.json index 39ca6eaa71d5..05499427ff8f 100644 --- a/src/i18n/messages/te.json +++ b/src/i18n/messages/te.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "వీడియో బ్రిడ్జింగ్ (ఫ్రేమ్ నమూనా + శీర్షికలు) బ్యాక్‌లాగ్‌లో ఉంది — సమస్య #9760ని చూడండి.", "modalityBridgeMovedTitle": "విజన్ బ్రిడ్జ్ మోవ్డ్", "modalityBridgeMovedBody": "విజన్ బ్రిడ్జ్ సెట్టింగ్స్ ఇప్పుడు ప్రత్యేకమైన మోడాలిటీ బ్రిడ్జ్ పేజీలో లైవ్ ఉన్నాయి.", - "modalityBridgeMovedCta": "మోడాలిటీ బ్రిడ్జ్ సెట్టింగ్స్‌ను తెరవండి" + "modalityBridgeMovedCta": "మోడాలిటీ బ్రిడ్జ్ సెట్టింగ్స్‌ను తెరవండి", + "modalityBridgeAudioTitle": "ఆడియో బ్రిడ్జ్", + "modalityBridgeAudioDesc": "ఎంచుకున్న టెక్స్ట్ మోడల్‌తో కొనసాగడానికి ముందు స్పీచ్-టు-టెక్స్ట్ మోడల్‌తో ఆడియోను ట్రాన్స్‌క్రైబ్ చేయండి.", + "modalityBridgeAudioEnabled": "ఆడియో బ్రిడ్జ్‌ను ప్రారంభించండి", + "modalityBridgeAudioEnabledDesc": "లక్ష్య మోడల్ ఆడియోను ప్రాసెస్ చేయలేకపోతే ఆడియో భాగాలను ట్రాన్స్క్రిప్ట్‌లతో మార్చండి.", + "modalityBridgeAudioModel": "స్పీచ్-టు-టెక్స్ట్ మోడల్", + "modalityBridgeAudioModelAuto": "ఆటో (మొదటి కనెక్ట్ అయిన STT ప్రొవైడర్)", + "modalityBridgeAudioMaxClips": "ప్రతి అభ్యర్థనకు గరిష్ట ఆడియో క్లిప్‌లు", + "modalityBridgeAudioTestButton": "నమూనా ఆడియోతో పరీక్షించండి", + "modalityBridgeAudioTestRunning": "ఆడియోని పరీక్షిస్తున్నాము…", + "modalityBridgeAudioTestOk": "ఆడియో బ్రిడ్జ్ బాగుంది — {count} క్లిప్(లు) {model} ద్వారా ట్రాన్స్‌క్రైబ్ చేయబడ్డాయి", + "modalityBridgeAudioTestNoop": "ఆడియో బ్రిడ్జ్ చెల్లించబడలేదు (లక్ష్యము ఆడియోను మద్దతు ఇవ్వవచ్చు, ఎలాంటి STT ప్రొవైడర్ కనెక్ట్ చేయబడలేదు, లేదా బ్రిడ్జ్ అచేతనంగా ఉంది)", + "modalityBridgeAudioTestError": "ఆడియో పరీక్ష విఫలమైంది: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/th.json b/src/i18n/messages/th.json index 3e8d03dc7fbc..3257b0b5e56c 100644 --- a/src/i18n/messages/th.json +++ b/src/i18n/messages/th.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "การเชื่อมโยงวิดีโอ (การสุ่มตัวอย่างเฟรม + การใส่คำบรรยาย) อยู่ในรายการที่รอการดำเนินการ — ดูปัญหา #9760.", "modalityBridgeMovedTitle": "Vision Bridge ถูกย้ายแล้ว", "modalityBridgeMovedBody": "การตั้งค่า Vision Bridge ตอนนี้อยู่ในหน้า Modality Bridge ที่จัดเตรียมไว้เฉพาะ.", - "modalityBridgeMovedCta": "เปิดการตั้งค่า Modality Bridge" + "modalityBridgeMovedCta": "เปิดการตั้งค่า Modality Bridge", + "modalityBridgeAudioTitle": "สะพานเสียง", + "modalityBridgeAudioDesc": "ถอดเสียงออดิโอด้วยโมเดลการแปลงคำพูดเป็นข้อความก่อนที่จะดำเนินการต่อด้วยโมเดลข้อความที่เลือก", + "modalityBridgeAudioEnabled": "เปิดใช้งาน Audio Bridge", + "modalityBridgeAudioEnabledDesc": "แทนที่ส่วนเสียงด้วยข้อความเมื่อโมเดลเป้าหมายไม่สามารถประมวลผลเสียงได้.", + "modalityBridgeAudioModel": "โมเดลแปลงเสียงเป็นข้อความ", + "modalityBridgeAudioModelAuto": "อัตโนมัติ (ผู้ให้บริการ STT ที่เชื่อมต่อครั้งแรก)", + "modalityBridgeAudioMaxClips": "คลิปเสียงสูงสุดต่อคำขอ", + "modalityBridgeAudioTestButton": "ทดสอบด้วยเสียงตัวอย่าง", + "modalityBridgeAudioTestRunning": "กำลังทดสอบเสียง…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} คลิป(ส์) ถูกถอดความโดย {model}", + "modalityBridgeAudioTestNoop": "Audio Bridge ไม่ได้เปิดใช้งาน (เป้าหมายอาจรองรับเสียง, ไม่มีผู้ให้บริการ STT ที่เชื่อมต่ออยู่, หรือสะพานถูกปิดใช้งาน)", + "modalityBridgeAudioTestError": "การทดสอบเสียงล้มเหลว: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/tr.json b/src/i18n/messages/tr.json index 62ca2a4517e2..f0c04ae4f2f1 100644 --- a/src/i18n/messages/tr.json +++ b/src/i18n/messages/tr.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Video köprüleme (çerçeve örnekleme + altyazı) beklemede — sorun #9760'a bakın.", "modalityBridgeMovedTitle": "Vision Bridge taşındı", "modalityBridgeMovedBody": "Vision Bridge ayarları artık özel Modality Bridge sayfasında yer alıyor.", - "modalityBridgeMovedCta": "Modality Bridge ayarlarını açın" + "modalityBridgeMovedCta": "Modality Bridge ayarlarını açın", + "modalityBridgeAudioTitle": "Ses Köprüsü", + "modalityBridgeAudioDesc": "Seçilen metin modeliyle devam etmeden önce sesi metne dönüştürme modelini kullanarak sesi transkribe et.", + "modalityBridgeAudioEnabled": "Ses Köprüsünü Etkinleştir", + "modalityBridgeAudioEnabledDesc": "Hedef model ses işleyemediğinde ses parçalarını transkriptlerle değiştir.", + "modalityBridgeAudioModel": "Konuşmadan Metne Modeli", + "modalityBridgeAudioModelAuto": "Otomatik (ilk bağlı STT sağlayıcısı)", + "modalityBridgeAudioMaxClips": "Her istekteki maksimum ses klibi", + "modalityBridgeAudioTestButton": "Örnek ses ile test et", + "modalityBridgeAudioTestRunning": "Ses testi yapılıyor…", + "modalityBridgeAudioTestOk": "Ses Köprüsü Tamam — {count} klip {model} tarafından transkribe edildi", + "modalityBridgeAudioTestNoop": "Ses Köprüsü etkinleştirilemedi (hedef ses desteği verebilir, STT sağlayıcısı bağlı değil veya köprü devre dışı)", + "modalityBridgeAudioTestError": "Ses testi başarısız oldu: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/uk-UA.json b/src/i18n/messages/uk-UA.json index 1931707ada18..80c645c83101 100644 --- a/src/i18n/messages/uk-UA.json +++ b/src/i18n/messages/uk-UA.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Відеоміст (вибірка кадрів + субтитрування) знаходиться в черзі — див. питання #9760.", "modalityBridgeMovedTitle": "Міст Візії переміщено", "modalityBridgeMovedBody": "Налаштування Vision Bridge тепер доступні на окремій сторінці Modality Bridge.", - "modalityBridgeMovedCta": "Відкрити налаштування моста модальності" + "modalityBridgeMovedCta": "Відкрити налаштування моста модальності", + "modalityBridgeAudioTitle": "Аудіо Міст", + "modalityBridgeAudioDesc": "Транскрибуйте аудіо за допомогою моделі розпізнавання мови перед продовженням з обраною текстовою моделлю.", + "modalityBridgeAudioEnabled": "Увімкнути Аудіо Міст", + "modalityBridgeAudioEnabledDesc": "Замініть аудіо частини на транскрипти, коли цільова модель не може обробити аудіо.", + "modalityBridgeAudioModel": "Модель розпізнавання мови", + "modalityBridgeAudioModelAuto": "Авто (перший підключений постачальник STT)", + "modalityBridgeAudioMaxClips": "Максимальна кількість аудіокліпів на запит", + "modalityBridgeAudioTestButton": "Тестування з тестовим аудіо", + "modalityBridgeAudioTestRunning": "Тестування аудіо…", + "modalityBridgeAudioTestOk": "Аудіо міст OK — {count} кліп(и) транскрибовано {model}", + "modalityBridgeAudioTestNoop": "Аудіоміст не активувався (можливо, ціль підтримує аудіо, жоден постачальник STT не підключений, або міст вимкнено)", + "modalityBridgeAudioTestError": "Тест аудіо не вдався: {message}" }, "contextRtk": { "title": "Двигун RTK", diff --git a/src/i18n/messages/ur.json b/src/i18n/messages/ur.json index 5bc9e630a078..bb7cc417067f 100644 --- a/src/i18n/messages/ur.json +++ b/src/i18n/messages/ur.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "ویڈیو برجنگ (فریم سیمپلنگ + کیپشننگ) بیک لاگ میں ہے — مسئلہ #9760 دیکھیں۔", "modalityBridgeMovedTitle": "ویژن برج منتقل ہوگیا", "modalityBridgeMovedBody": "ویژن برج کی ترتیبات اب مخصوص موڈالیٹی برج صفحے میں موجود ہیں۔", - "modalityBridgeMovedCta": "موڈالیٹی برج کی ترتیبات کھولیں" + "modalityBridgeMovedCta": "موڈالیٹی برج کی ترتیبات کھولیں", + "modalityBridgeAudioTitle": "آڈیو برج", + "modalityBridgeAudioDesc": "آڈیو کو تقریر سے متن ماڈل کے ساتھ نقل کریں اس سے پہلے کہ منتخب کردہ متن ماڈل کے ساتھ جاری رکھیں۔", + "modalityBridgeAudioEnabled": "آڈیو برج فعال کریں", + "modalityBridgeAudioEnabledDesc": "جب ہدف ماڈل آڈیو کو پروسیس نہیں کر سکتا تو آڈیو کے حصے کو تحریری شکل سے تبدیل کریں۔", + "modalityBridgeAudioModel": "تقریر سے متن ماڈل", + "modalityBridgeAudioModelAuto": "خودکار (پہلا منسلک STT فراہم کنندہ)", + "modalityBridgeAudioMaxClips": "فی درخواست زیادہ سے زیادہ آڈیو کلپس", + "modalityBridgeAudioTestButton": "نمونہ آڈیو کے ساتھ ٹیسٹ کریں", + "modalityBridgeAudioTestRunning": "آڈیو کی جانچ ہو رہی ہے…", + "modalityBridgeAudioTestOk": "آڈیو برج ٹھیک ہے — {count} کلپ(ز) کی نقل {model} کے ذریعے کی گئی", + "modalityBridgeAudioTestNoop": "آڈیو برج فعال نہیں ہوا (ہو سکتا ہے کہ ہدف آڈیو کی حمایت کرتا ہو، کوئی STT فراہم کنندہ منسلک نہیں ہے، یا برج غیر فعال ہے)", + "modalityBridgeAudioTestError": "آڈیو ٹیسٹ ناکام: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/vi.json b/src/i18n/messages/vi.json index 0a52f43c79fd..0ddf5bf97082 100644 --- a/src/i18n/messages/vi.json +++ b/src/i18n/messages/vi.json @@ -7684,7 +7684,19 @@ "modalityBridgeVideoComingSoon": "Video bridging (frame sampling + captioning) đang ở trong danh sách chờ — xem vấn đề #9760.", "modalityBridgeMovedTitle": "Vision Bridge đã được di chuyển", "modalityBridgeMovedBody": "Cài đặt Vision Bridge hiện đã có trên trang Modality Bridge chuyên dụng.", - "modalityBridgeMovedCta": "Mở cài đặt Cầu Modality" + "modalityBridgeMovedCta": "Mở cài đặt Cầu Modality", + "modalityBridgeAudioTitle": "Cầu Nối Âm Thanh", + "modalityBridgeAudioDesc": "Chuyển đổi âm thanh thành văn bản bằng mô hình nhận diện giọng nói trước khi tiếp tục với mô hình văn bản đã chọn.", + "modalityBridgeAudioEnabled": "Bật Cầu Nối Âm Thanh", + "modalityBridgeAudioEnabledDesc": "Thay thế các phần âm thanh bằng bản sao khi mô hình mục tiêu không thể xử lý âm thanh.", + "modalityBridgeAudioModel": "Mô hình chuyển đổi giọng nói thành văn bản", + "modalityBridgeAudioModelAuto": "Tự động (nhà cung cấp STT được kết nối đầu tiên)", + "modalityBridgeAudioMaxClips": "Số lượng clip âm thanh tối đa cho mỗi yêu cầu", + "modalityBridgeAudioTestButton": "Kiểm tra với âm thanh mẫu", + "modalityBridgeAudioTestRunning": "Đang kiểm tra âm thanh…", + "modalityBridgeAudioTestOk": "Audio Bridge OK — {count} clip(s) đã được chuyển ngữ bởi {model}", + "modalityBridgeAudioTestNoop": "Audio Bridge không được kích hoạt (mục tiêu có thể hỗ trợ âm thanh, không có nhà cung cấp STT nào được kết nối, hoặc cầu nối bị vô hiệu hóa)", + "modalityBridgeAudioTestError": "Kiểm tra âm thanh không thành công: {message}" }, "contextRtk": { "title": "RTK Engine", diff --git a/src/i18n/messages/zh-CN.json b/src/i18n/messages/zh-CN.json index bed23abc1a1c..640878867b77 100644 --- a/src/i18n/messages/zh-CN.json +++ b/src/i18n/messages/zh-CN.json @@ -7650,6 +7650,13 @@ "modalityBridgeSubTabsAria": "模态桥部分", "modalityBridgeVisionTitle": "视觉桥", "modalityBridgeVisionDesc": "使用视觉模型描述图像,然后继续使用用户选择的文本模型。", + "modalityBridgeAudioTitle": "音频桥接", + "modalityBridgeAudioDesc": "在继续使用所选文本模型之前,使用语音转文本模型转录音频。", + "modalityBridgeAudioEnabled": "启用音频桥接", + "modalityBridgeAudioEnabledDesc": "当目标模型无法处理音频时,用转录文本替换音频部分。", + "modalityBridgeAudioModel": "语音转文本模型", + "modalityBridgeAudioModelAuto": "自动(首次连接的STT提供者)", + "modalityBridgeAudioMaxClips": "每个请求的最大音频剪辑数", "modalityBridgeMode": "模式", "modalityBridgeModeAuto": "自动(推荐)", "modalityBridgeModeAutoHint": "遗留启发式:在没有凭据的情况下重新路由单个模型;否则请描述。", @@ -7680,6 +7687,11 @@ "modalityBridgeTestReroute": "桥接将请求重新路由到 {model}", "modalityBridgeTestNoop": "桥接未激活(模型可能原生支持视觉或桥接已禁用)", "modalityBridgeTestError": "测试失败:{message}", + "modalityBridgeAudioTestButton": "使用示例音频进行测试", + "modalityBridgeAudioTestRunning": "测试音频…", + "modalityBridgeAudioTestOk": "音频桥接正常 — {count} 个剪辑由 {model} 转录", + "modalityBridgeAudioTestNoop": "音频桥未激活(目标可能支持音频,但未连接STT提供者,或桥接已禁用)", + "modalityBridgeAudioTestError": "音频测试失败:{message}", "modalityBridgeAudioComingSoon": "音频桥接(语音 → 文本通过 /v1/audio/transcriptions)将在下一个版本中发布。其设置键已被保留。", "modalityBridgeVideoComingSoon": "视频桥接(帧采样 + 字幕)在待办事项中 — 请参阅问题 #9760。", "modalityBridgeMovedTitle": "Vision Bridge 已移动", diff --git a/src/i18n/messages/zh-TW.json b/src/i18n/messages/zh-TW.json index ce6857e97408..5da0bee0252c 100644 --- a/src/i18n/messages/zh-TW.json +++ b/src/i18n/messages/zh-TW.json @@ -7650,6 +7650,13 @@ "modalityBridgeSubTabsAria": "模態橋接部分", "modalityBridgeVisionTitle": "視覺橋接", "modalityBridgeVisionDesc": "使用視覺模型描述圖像,然後繼續使用用戶選擇的文本模型。", + "modalityBridgeAudioTitle": "音訊橋接", + "modalityBridgeAudioDesc": "在繼續使用選定的文本模型之前,請使用語音轉文本模型轉錄音頻。", + "modalityBridgeAudioEnabled": "啟用音訊橋接", + "modalityBridgeAudioEnabledDesc": "當目標模型無法處理音頻時,將音頻部分替換為文字稿。", + "modalityBridgeAudioModel": "語音轉文字模型", + "modalityBridgeAudioModelAuto": "自動(第一個連接的 STT 提供者)", + "modalityBridgeAudioMaxClips": "每個請求的最大音頻片段數量", "modalityBridgeMode": "模式", "modalityBridgeModeAuto": "自動(建議)", "modalityBridgeModeAutoHint": "舊版啟發式:在沒有憑證的情況下重新路由單個模型;否則請描述。", @@ -7680,6 +7687,11 @@ "modalityBridgeTestReroute": "橋接將請求重新路由到 {model}", "modalityBridgeTestNoop": "橋接未啟用(模型可能原生支持視覺或橋接已禁用)", "modalityBridgeTestError": "測試失敗:{message}", + "modalityBridgeAudioTestButton": "使用範例音訊測試", + "modalityBridgeAudioTestRunning": "測試音訊…", + "modalityBridgeAudioTestOk": "音訊橋接正常 — {count} 個片段由 {model} 轉錄", + "modalityBridgeAudioTestNoop": "音頻橋接未啟用(目標可能支持音頻,未連接STT提供者,或橋接已禁用)", + "modalityBridgeAudioTestError": "音訊測試失敗:{message}", "modalityBridgeAudioComingSoon": "音訊橋接(語音 → 文字透過 /v1/audio/transcriptions)將在下一個版本中發佈。其設定鍵已經被保留。", "modalityBridgeVideoComingSoon": "視頻橋接(幀取樣 + 字幕)在待辦事項中 — 請參見問題 #9760。", "modalityBridgeMovedTitle": "Vision Bridge 已移動", diff --git a/src/lib/guardrails/audioBridge.ts b/src/lib/guardrails/audioBridge.ts new file mode 100644 index 000000000000..d2e856c027b9 --- /dev/null +++ b/src/lib/guardrails/audioBridge.ts @@ -0,0 +1,137 @@ +import { getSettings as defaultGetSettings } from "@/lib/db/settings"; +import { getResolvedModelCapabilities } from "@/lib/modelCapabilities"; +import { resolveAudioBridgeRuntimeSettings } from "@/shared/constants/modalityBridgeDefaults"; + +import { BaseGuardrail, type GuardrailContext, type GuardrailResult } from "./base"; +import { + callAudioTranscription as defaultCallAudioTranscription, + extractAudioParts, + replaceAudioParts, + selectAudioBridgeModel, + type AudioCredentialCheck, + type AudioPart, + type AudioTranscriptionConfig, +} from "./audioBridgeHelpers"; +import { bridgeCacheKey, getSharedBridgeCacheFor } from "./modalityBridge/bridgeCache"; +import { recordBridgeUse } from "./modalityBridge/bridgeStats"; + +export interface AudioBridgeDependencies { + getSettings?: () => Promise>; + getCapabilities?: (model: string) => { supportsAudio: boolean | null }; + hasUsableCredentials?: AudioCredentialCheck; + selectModel?: (configuredModel: string) => Promise; + callTranscription?: (part: AudioPart, config: AudioTranscriptionConfig) => Promise; +} + +type AudioBridgeBody = { + model?: string; + messages?: Array<{ role?: string; content?: unknown }>; + [key: string]: unknown; +}; + +export class AudioBridgeGuardrail extends BaseGuardrail { + name = "audio-bridge"; + priority = 6; + + private readonly deps: AudioBridgeDependencies; + + constructor(options?: { enabled?: boolean; deps?: AudioBridgeDependencies }) { + super("audio-bridge", { priority: 6, enabled: options?.enabled }); + this.deps = options?.deps ?? {}; + } + + async preCall(payload: unknown, context: GuardrailContext): Promise> { + if (!this.enabled || context.disabledGuardrails?.includes("audio-bridge")) { + return { block: false }; + } + + const body = payload as AudioBridgeBody; + const model = context.model || body?.model; + if (!model || !Array.isArray(body?.messages) || body.messages.length === 0) { + return { block: false }; + } + + const getSettings = this.deps.getSettings ?? defaultGetSettings; + let persisted: Record = {}; + try { + persisted = await getSettings(); + } catch { + // Database settings are optional during early boot; defaults remain safe. + } + const runtime = resolveAudioBridgeRuntimeSettings(persisted); + if (!runtime.enabled) return { block: false }; + + const audioParts = extractAudioParts(body.messages); + if (audioParts.length === 0) return { block: false }; + + const capabilities = (this.deps.getCapabilities ?? getResolvedModelCapabilities)(model); + if (capabilities.supportsAudio === true) return { block: false }; + + const limitedParts = audioParts.slice(0, runtime.maxClips); + const startedAt = Date.now(); + const configuredModel = runtime.model || "auto"; + const sttModel = this.deps.selectModel + ? await this.deps.selectModel(configuredModel) + : await selectAudioBridgeModel(configuredModel, this.deps.hasUsableCredentials); + if (!sttModel) { + if (capabilities.supportsAudio !== false) return { block: false }; + const stubs = limitedParts.map( + (_part, index) => `[Audio ${index + 1}]: (unavailable — no STT provider connected)` + ); + for (const _part of limitedParts) recordBridgeUse("audio", { failure: true }); + return { + block: false, + modifiedPayload: replaceAudioParts(body, limitedParts, stubs), + meta: { + clipsProcessed: stubs.length, + processingTimeMs: Date.now() - startedAt, + sttModel: "unavailable", + }, + }; + } + + const callTranscription = this.deps.callTranscription ?? defaultCallAudioTranscription; + const cache = runtime.cacheEnabled ? getSharedBridgeCacheFor(runtime) : null; + const settled = await Promise.allSettled( + limitedParts.map(async (part, index) => { + const key = cache ? bridgeCacheKey(part.ref, "audio-transcription", sttModel) : null; + const cached = key && cache ? cache.get(key) : undefined; + const transcript = + cached ?? + (await callTranscription(part, { model: sttModel, timeoutMs: runtime.timeoutMs })); + if (cached === undefined && key && cache) cache.set(key, transcript); + recordBridgeUse("audio", { cacheHit: cached !== undefined }); + return `[Audio ${index + 1}]: ${transcript}`; + }) + ); + + const transcripts = settled.map((result, index): string | null => { + if (result.status === "fulfilled") return result.value; + const message = + result.reason instanceof Error ? result.reason.message : String(result.reason); + context.log?.warn?.("AUDIO_BRIDGE", `Failed to transcribe audio ${index + 1}: ${message}`); + recordBridgeUse("audio", { failure: true }); + return null; + }); + if ( + capabilities.supportsAudio === false && + transcripts.every((transcript) => transcript === null) + ) { + for (let index = 0; index < transcripts.length; index++) { + transcripts[index] = `[Audio ${index + 1}]: (unavailable — no STT provider connected)`; + } + } + const clipsProcessed = transcripts.filter((value) => value !== null).length; + if (clipsProcessed === 0) return { block: false }; + + return { + block: false, + modifiedPayload: replaceAudioParts(body, limitedParts, transcripts), + meta: { + clipsProcessed, + processingTimeMs: Date.now() - startedAt, + sttModel, + }, + }; + } +} diff --git a/src/lib/guardrails/audioBridgeHelpers.ts b/src/lib/guardrails/audioBridgeHelpers.ts new file mode 100644 index 000000000000..1124a3393018 --- /dev/null +++ b/src/lib/guardrails/audioBridgeHelpers.ts @@ -0,0 +1,239 @@ +import { AUDIO_TRANSCRIPTION_PROVIDERS } from "@omniroute/open-sse/config/audioRegistry.ts"; +import { detectMediaParts } from "@omniroute/open-sse/utils/mediaParts"; + +import { getRuntimePorts } from "@/lib/runtime/ports"; +import { fetchRemoteImage } from "@/shared/network/remoteImageFetch"; +import { resolveSelfLoopBearer } from "@/shared/middleware/chatBodyAdmission"; + +import { hasUsableCredentialsForModel } from "./visionBridgeCredentials"; + +export type AudioCredentialCheck = (model: string) => Promise; + +export interface AudioPart { + messageIndex: number; + partIndex: number; + ref: string; + shape: "input_audio" | "audio_url" | "audio_source"; + format?: string; +} + +export interface AudioTranscriptionConfig { + model: string; + timeoutMs: number; +} + +export interface AudioTranscriptionDependencies { + fetchImpl?: typeof fetch; + fetchRemote?: ( + url: string, + options: { signal: AbortSignal } + ) => Promise<{ buffer: Buffer; contentType: string; url: string }>; + getPort?: () => number; + getBearer?: () => string; +} + +const AUDIO_FORMAT_MIME: Record = { + aac: "audio/aac", + flac: "audio/flac", + m4a: "audio/mp4", + mp3: "audio/mpeg", + mp4: "audio/mp4", + ogg: "audio/ogg", + opus: "audio/opus", + wav: "audio/wav", + webm: "audio/webm", +}; + +const AUDIO_MIME_FORMAT: Record = { + "audio/aac": "aac", + "audio/flac": "flac", + "audio/mp4": "mp4", + "audio/mpeg": "mp3", + "audio/mp3": "mp3", + "audio/ogg": "ogg", + "audio/opus": "opus", + "audio/wav": "wav", + "audio/wave": "wav", + "audio/webm": "webm", + "audio/x-wav": "wav", +}; + +type AudioMessage = { role?: string; content?: unknown }; + +function formatFromRef(ref: string): string | undefined { + const dataMime = /^data:([^;,]+)/i.exec(ref)?.[1]?.toLowerCase(); + if (dataMime) return AUDIO_MIME_FORMAT[dataMime]; + try { + const pathname = new URL(ref).pathname; + const extension = pathname.includes(".") ? pathname.split(".").pop()?.toLowerCase() : undefined; + return extension && AUDIO_FORMAT_MIME[extension] ? extension : undefined; + } catch { + return undefined; + } +} + +function formatForDetectedPart( + messages: ReadonlyArray, + messageIndex: number, + partIndex: number, + ref: string +): string | undefined { + const content = messages[messageIndex]?.content; + const raw = Array.isArray(content) ? content[partIndex] : null; + if (!raw || typeof raw !== "object") return formatFromRef(ref); + const part = raw as Record; + const inputAudio = part.input_audio as Record | undefined; + if (typeof inputAudio?.format === "string" && inputAudio.format.trim()) { + return inputAudio.format.trim().toLowerCase(); + } + const source = part.source as Record | undefined; + if (typeof source?.media_type === "string") { + return AUDIO_MIME_FORMAT[source.media_type.toLowerCase()] ?? formatFromRef(ref); + } + return formatFromRef(ref); +} + +/** Extract only spliceable top-level audio parts from every message. */ +export function extractAudioParts( + messages: ReadonlyArray | undefined | null +): AudioPart[] { + if (!Array.isArray(messages)) return []; + return detectMediaParts(messages) + .filter( + (part) => + part.kind === "audio" && + !part.nested && + part.ref.length > 0 && + (part.shape === "input_audio" || + part.shape === "audio_url" || + part.shape === "audio_source") + ) + .map((part) => ({ + messageIndex: part.messageIndex, + partIndex: part.partIndex, + ref: part.ref, + shape: part.shape as AudioPart["shape"], + format: formatForDetectedPart(messages, part.messageIndex, part.partIndex, part.ref), + })); +} + +/** Replace successful transcripts in place; null results preserve the original audio. */ +export function replaceAudioParts( + body: TBody, + parts: readonly AudioPart[], + transcripts: readonly (string | null)[] +): TBody { + const result = structuredClone(body); + for (let index = 0; index < parts.length && index < transcripts.length; index++) { + const transcript = transcripts[index]; + if (transcript === null) continue; + const part = parts[index]; + const content = result.messages?.[part.messageIndex]?.content; + if (!Array.isArray(content) || part.partIndex >= content.length) continue; + content[part.partIndex] = { type: "text", text: transcript }; + } + return result; +} + +function listTranscriptionModels(): string[] { + const models: string[] = []; + for (const [providerId, provider] of Object.entries(AUDIO_TRANSCRIPTION_PROVIDERS)) { + for (const model of provider.models) { + models.push(model.id.startsWith(`${providerId}/`) ? model.id : `${providerId}/${model.id}`); + } + } + return models; +} + +/** Select a configured STT model, or the first catalog model with usable credentials. */ +export async function selectAudioBridgeModel( + configuredModel: string, + hasUsableCredentials: AudioCredentialCheck = hasUsableCredentialsForModel +): Promise { + const fixed = configuredModel.trim(); + if (fixed && fixed !== "auto") { + return (await hasUsableCredentials(fixed)) === false ? null : fixed; + } + + for (const candidate of listTranscriptionModels()) { + if ((await hasUsableCredentials(candidate)) === true) return candidate; + } + return null; +} + +/** Send one audio part through OmniRoute's existing multipart transcription route. */ +export async function callAudioTranscription( + part: AudioPart, + config: AudioTranscriptionConfig, + deps: AudioTranscriptionDependencies = {} +): Promise { + const controller = new AbortController(); + const timeout = setTimeout(() => controller.abort(), config.timeoutMs); + try { + let bytes: Buffer; + let detectedMime: string | undefined; + const dataUri = /^data:([^;,]+);base64,(.+)$/is.exec(part.ref); + if (dataUri) { + detectedMime = dataUri[1].toLowerCase(); + bytes = Buffer.from(dataUri[2], "base64"); + } else if (/^https?:\/\//i.test(part.ref)) { + const fetchRemote = + deps.fetchRemote ?? + ((url: string, options: { signal: AbortSignal }) => + fetchRemoteImage(url, { + guard: "public-only", + maxBytes: 25 * 1024 * 1024, + pinDns: true, + signal: options.signal, + timeoutMs: config.timeoutMs, + })); + const remote = await fetchRemote(part.ref, { signal: controller.signal }); + bytes = remote.buffer; + detectedMime = remote.contentType.split(";", 1)[0]?.trim().toLowerCase(); + } else { + bytes = Buffer.from(part.ref, "base64"); + } + + const configuredFormat = part.format?.trim().toLowerCase(); + const format = + configuredFormat || (detectedMime ? AUDIO_MIME_FORMAT[detectedMime] : undefined) || "wav"; + const mime = + (detectedMime?.startsWith("audio/") ? detectedMime : undefined) ?? + AUDIO_FORMAT_MIME[format] ?? + "application/octet-stream"; + const file = new Blob([Uint8Array.from(bytes)], { type: mime }); + const form = new FormData(); + form.set("file", file, `audio.${format.replace(/[^a-z0-9]/g, "") || "wav"}`); + form.set("model", config.model); + + const port = (deps.getPort ?? (() => getRuntimePorts().port))(); + const bearer = (deps.getBearer ?? resolveSelfLoopBearer)(); + const response = await (deps.fetchImpl ?? fetch)( + `http://localhost:${port}/v1/audio/transcriptions`, + { + method: "POST", + signal: controller.signal, + headers: { + Accept: "application/json", + Authorization: `Bearer ${bearer}`, + }, + body: form, + } + ); + if (!response.ok) { + throw new Error(`Audio transcription failed (${response.status})`); + } + const data = (await response.json()) as { text?: unknown }; + if (typeof data.text !== "string") { + throw new Error("Audio transcription returned an invalid response"); + } + return data.text.trim(); + } catch (error) { + if (error instanceof Error && error.name === "AbortError") { + throw new Error("Audio transcription timed out"); + } + throw error; + } finally { + clearTimeout(timeout); + } +} diff --git a/src/lib/guardrails/modalityBridge/bridgeCache.ts b/src/lib/guardrails/modalityBridge/bridgeCache.ts index e0a10429e6c4..f4efb797fa3e 100644 --- a/src/lib/guardrails/modalityBridge/bridgeCache.ts +++ b/src/lib/guardrails/modalityBridge/bridgeCache.ts @@ -77,6 +77,8 @@ export function getSharedBridgeCache(ttlMs: number, maxEntries: number): BridgeC * bridge (vision, audio) goes through here so the minutes→ms conversion can * never diverge between callers and thrash the singleton on each request. */ -export function getSharedBridgeCacheFor(settings: VisionBridgeRuntimeSettings): BridgeCache { +export function getSharedBridgeCacheFor( + settings: Pick +): BridgeCache { return getSharedBridgeCache(settings.cacheTtlMinutes * 60_000, settings.cacheMaxEntries); } diff --git a/src/lib/guardrails/modalityBridge/bridgeStats.ts b/src/lib/guardrails/modalityBridge/bridgeStats.ts index 052c73fabe0a..b511ced37dcc 100644 --- a/src/lib/guardrails/modalityBridge/bridgeStats.ts +++ b/src/lib/guardrails/modalityBridge/bridgeStats.ts @@ -48,6 +48,12 @@ interface GuardrailMetaEntry { meta?: Record | null; } +function headerModelToken(value: unknown): string { + return String(value ?? "unknown") + .slice(0, 200) + .replace(/[^A-Za-z0-9._~/-]/g, "_"); +} + /** Response header value for a describe-bridged request; null when untouched. */ export function buildModalityBridgeHeader(results: GuardrailMetaEntry[]): string | null { const segments: string[] = []; @@ -59,12 +65,16 @@ export function buildModalityBridgeHeader(results: GuardrailMetaEntry[]): string !meta.rerouted ) { segments.push( - `image->text;model=${String(meta.visionModel ?? "unknown")};parts=${meta.imagesProcessed}` + `image->text;model=${headerModelToken(meta.visionModel)};parts=${meta.imagesProcessed}` ); } - if (r.guardrail === "audio-bridge" && typeof meta.clipsProcessed === "number") { + if ( + r.guardrail === "audio-bridge" && + typeof meta.clipsProcessed === "number" && + !meta.rerouted + ) { segments.push( - `audio->text;model=${String(meta.sttModel ?? "unknown")};parts=${meta.clipsProcessed}` + `audio->text;model=${headerModelToken(meta.sttModel)};parts=${meta.clipsProcessed}` ); } } diff --git a/src/lib/guardrails/registry.ts b/src/lib/guardrails/registry.ts index 531539ab17f3..774e47014af4 100644 --- a/src/lib/guardrails/registry.ts +++ b/src/lib/guardrails/registry.ts @@ -7,6 +7,7 @@ import { import { PIIMaskerGuardrail } from "./piiMasker"; import { PromptInjectionGuardrail } from "./promptInjection"; import { VisionBridgeGuardrail } from "./visionBridge"; +import { AudioBridgeGuardrail } from "./audioBridge"; import { CredentialMaskerGuardrail } from "./credentialMasker"; /** @@ -286,6 +287,7 @@ export function registerDefaultGuardrails() { if (defaultGuardrailsRegistered) return guardrailRegistry; guardrailRegistry.register(new VisionBridgeGuardrail()); + guardrailRegistry.register(new AudioBridgeGuardrail()); guardrailRegistry.register(new PIIMaskerGuardrail()); guardrailRegistry.register(new CredentialMaskerGuardrail()); guardrailRegistry.register(new PromptInjectionGuardrail()); diff --git a/src/lib/modelCapabilities.ts b/src/lib/modelCapabilities.ts index 700d4adfd973..de36dc4d80a6 100644 --- a/src/lib/modelCapabilities.ts +++ b/src/lib/modelCapabilities.ts @@ -121,6 +121,7 @@ export interface ResolvedModelCapabilities { supportsThinking: boolean | null; supportsTools: boolean | null; supportsVision: boolean | null; + supportsAudio: boolean | null; supportsMaxTokens: boolean; attachment: boolean | null; structuredOutput: boolean | null; @@ -539,6 +540,25 @@ function resolveVisionCapability( return null; } +/** + * Resolve whether a chat model accepts audio input. + * + * Explicit catalog metadata wins. Synced input modalities are authoritative + * only when they contain at least one declared modality; an empty list means + * that no source knows the answer and remains `null` so Audio Bridge can act + * conservatively. + */ +export function resolveAudioCapability( + spec: Pick | undefined, + registryModel: { supportsAudio?: boolean } | null, + modalitiesInput: readonly string[] +): boolean | null { + if (typeof registryModel?.supportsAudio === "boolean") return registryModel.supportsAudio; + if (typeof spec?.supportsAudio === "boolean") return spec.supportsAudio; + if (modalitiesInput.length === 0) return null; + return modalitiesInput.some((entry) => String(entry).toLowerCase().includes("audio")); +} + /** * Issue #6524: an operator-set `max_output_tokens` capability override (see * `src/lib/db/modelCapabilityOverrides.ts`) is the manual escape hatch for a @@ -719,6 +739,7 @@ export function getResolvedModelCapabilities( lookupKey, customVisionOverride ); + const supportsAudio = resolveAudioCapability(spec, registryModel, modalitiesInput); // #8250: when resolve promoted vision over a contradictory attachment=false, // expose attachment=true so catalog / Vision Bridge / clients see one verdict. @@ -736,6 +757,7 @@ export function getResolvedModelCapabilities( supportsThinking, supportsTools, supportsVision, + supportsAudio, supportsMaxTokens: heuristicMaxTokens(lookupKey), attachment, structuredOutput: synced?.structured_output ?? null, diff --git a/src/shared/components/ModelSelectField.tsx b/src/shared/components/ModelSelectField.tsx index 1e5667b0c87d..50986153a720 100644 --- a/src/shared/components/ModelSelectField.tsx +++ b/src/shared/components/ModelSelectField.tsx @@ -4,10 +4,12 @@ import { useEffect, useState } from "react"; import Select from "./Select"; import Input from "./Input"; -interface ApiModel { +export interface ApiModel { provider: string; model: string; fullModel?: string; + type?: string; + subtype?: string; } export interface ModelSelectFieldProps { @@ -21,6 +23,10 @@ export interface ModelSelectFieldProps { allowCustom?: boolean; /** Let operators select the empty-value placeholder (for Auto/default semantics). */ allowEmpty?: boolean; + /** Optional catalog predicate, e.g. restrict the picker to STT models. */ + modelFilter?: (model: ApiModel) => boolean; + /** Model API to read. The unified catalog includes specialty audio/video surfaces. */ + modelSource?: "available" | "catalog"; className?: string; } @@ -29,6 +35,35 @@ interface FetchState { options: { value: string; label: string }[]; } +function asRecord(value: unknown): Record | null { + return value && typeof value === "object" ? (value as Record) : null; +} + +function readCatalogModels(value: unknown): ApiModel[] { + const catalog = asRecord(asRecord(value)?.catalog); + if (!catalog) return []; + + const models: ApiModel[] = []; + for (const [provider, rawBucket] of Object.entries(catalog)) { + const bucket = asRecord(rawBucket); + if (!Array.isArray(bucket?.models)) continue; + for (const rawModel of bucket.models) { + const model = asRecord(rawModel); + const id = typeof model?.id === "string" ? model.id : ""; + if (!id) continue; + const providerPrefix = `${provider}/`; + models.push({ + provider, + model: id.startsWith(providerPrefix) ? id.slice(providerPrefix.length) : id, + fullModel: id.startsWith(providerPrefix) ? id : `${providerPrefix}${id}`, + type: typeof model?.type === "string" ? model.type : undefined, + subtype: typeof model?.subtype === "string" ? model.subtype : undefined, + }); + } + } + return models; +} + /** * hidePaid-aware model picker (#6540). Loads options from `GET /api/models` * (already filters by `hidePaidModels`) instead of a static catalog. Falls @@ -46,18 +81,27 @@ export default function ModelSelectField({ ariaLabel, allowCustom = true, allowEmpty = false, + modelFilter, + modelSource = "available", className, }: ModelSelectFieldProps) { const [state, setState] = useState({ status: "loading", options: [] }); useEffect(() => { let cancelled = false; - fetch("/api/models") + const endpoint = modelSource === "catalog" ? "/api/models/catalog" : "/api/models"; + fetch(endpoint) .then((res) => (res.ok ? res.json() : Promise.reject(new Error("fetch failed")))) .then((data) => { if (cancelled) return; - const models: ApiModel[] = Array.isArray(data?.models) ? data.models : []; - const options = models.map((m) => { + const models: ApiModel[] = + modelSource === "catalog" + ? readCatalogModels(data) + : Array.isArray(data?.models) + ? data.models + : []; + const filteredModels = modelFilter ? models.filter(modelFilter) : models; + const options = filteredModels.map((m) => { const full = m.fullModel || `${m.provider}/${m.model}`; return { value: full, label: full }; }); @@ -69,7 +113,7 @@ export default function ModelSelectField({ return () => { cancelled = true; }; - }, []); + }, [modelFilter, modelSource]); if (state.status === "error" && allowCustom) { return ( diff --git a/src/shared/constants/modalityBridgeDefaults.ts b/src/shared/constants/modalityBridgeDefaults.ts index 336d45b2800b..4fd98b9d615d 100644 --- a/src/shared/constants/modalityBridgeDefaults.ts +++ b/src/shared/constants/modalityBridgeDefaults.ts @@ -34,6 +34,16 @@ export interface VisionBridgeRuntimeSettings { cacheMaxEntries: number; } +export interface AudioBridgeRuntimeSettings { + enabled: boolean; + model: string; + timeoutMs: number; + maxClips: number; + cacheEnabled: boolean; + cacheTtlMinutes: number; + cacheMaxEntries: number; +} + // Typed candidate pickers: a stored value of the wrong type (e.g. the string // "off" in a boolean field) is skipped so the next candidate/default wins. function pickBoolean(...values: unknown[]): boolean | undefined { @@ -83,3 +93,22 @@ export function resolveVisionBridgeRuntimeSettings( pickNumber(s.modalityBridgeCacheMaxEntries) ?? MODALITY_BRIDGE_DEFAULTS.cacheMaxEntries, }; } + +/** Resolve persisted Audio Bridge settings with the shared PR-1 defaults. */ +export function resolveAudioBridgeRuntimeSettings( + settings: Record | null | undefined +): AudioBridgeRuntimeSettings { + const s = settings ?? {}; + return { + enabled: pickBoolean(s.modalityBridgeAudioEnabled) ?? MODALITY_BRIDGE_DEFAULTS.audioEnabled, + model: pickString(s.modalityBridgeAudioModel) ?? MODALITY_BRIDGE_DEFAULTS.audioModel, + timeoutMs: pickNumber(s.modalityBridgeAudioTimeout) ?? MODALITY_BRIDGE_DEFAULTS.audioTimeoutMs, + maxClips: pickNumber(s.modalityBridgeAudioMaxClips) ?? MODALITY_BRIDGE_DEFAULTS.audioMaxClips, + cacheEnabled: + pickBoolean(s.modalityBridgeCacheEnabled) ?? MODALITY_BRIDGE_DEFAULTS.cacheEnabled, + cacheTtlMinutes: + pickNumber(s.modalityBridgeCacheTtlMinutes) ?? MODALITY_BRIDGE_DEFAULTS.cacheTtlMinutes, + cacheMaxEntries: + pickNumber(s.modalityBridgeCacheMaxEntries) ?? MODALITY_BRIDGE_DEFAULTS.cacheMaxEntries, + }; +} diff --git a/src/shared/constants/modelSpecs.ts b/src/shared/constants/modelSpecs.ts index c9f46e67d42c..d15fe50425ed 100644 --- a/src/shared/constants/modelSpecs.ts +++ b/src/shared/constants/modelSpecs.ts @@ -15,6 +15,7 @@ export interface ModelSpec { supportsThinking?: boolean; supportsTools?: boolean; supportsVision?: boolean; + supportsAudio?: boolean; // Model defaults to adaptive thinking and REJECTS an explicit `thinking.type:"disabled"` // (upstream returns 400). Used to normalize the request when a combo/route substitutes // this model after the client already chose `disabled`. See issue #3554. diff --git a/tests/unit/audio-bridge-settings.test.ts b/tests/unit/audio-bridge-settings.test.ts new file mode 100644 index 000000000000..3dcfe736cb9c --- /dev/null +++ b/tests/unit/audio-bridge-settings.test.ts @@ -0,0 +1,56 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + MODALITY_BRIDGE_DEFAULTS, + resolveAudioBridgeRuntimeSettings, +} from "../../src/shared/constants/modalityBridgeDefaults.ts"; + +test("Audio Bridge runtime settings use the PR-1 defaults", () => { + assert.deepEqual(resolveAudioBridgeRuntimeSettings({}), { + enabled: MODALITY_BRIDGE_DEFAULTS.audioEnabled, + model: MODALITY_BRIDGE_DEFAULTS.audioModel, + timeoutMs: MODALITY_BRIDGE_DEFAULTS.audioTimeoutMs, + maxClips: MODALITY_BRIDGE_DEFAULTS.audioMaxClips, + cacheEnabled: MODALITY_BRIDGE_DEFAULTS.cacheEnabled, + cacheTtlMinutes: MODALITY_BRIDGE_DEFAULTS.cacheTtlMinutes, + cacheMaxEntries: MODALITY_BRIDGE_DEFAULTS.cacheMaxEntries, + }); +}); + +test("Audio Bridge runtime settings accept typed persisted values", () => { + assert.deepEqual( + resolveAudioBridgeRuntimeSettings({ + modalityBridgeAudioEnabled: false, + modalityBridgeAudioModel: "deepgram/nova-3", + modalityBridgeAudioTimeout: 12_000, + modalityBridgeAudioMaxClips: 2, + modalityBridgeCacheEnabled: false, + modalityBridgeCacheTtlMinutes: 5, + modalityBridgeCacheMaxEntries: 20, + }), + { + enabled: false, + model: "deepgram/nova-3", + timeoutMs: 12_000, + maxClips: 2, + cacheEnabled: false, + cacheTtlMinutes: 5, + cacheMaxEntries: 20, + } + ); +}); + +test("Audio Bridge runtime settings ignore wrong-typed persisted values", () => { + const settings = resolveAudioBridgeRuntimeSettings({ + modalityBridgeAudioEnabled: "false", + modalityBridgeAudioModel: 42, + modalityBridgeAudioTimeout: "12000", + modalityBridgeAudioMaxClips: null, + }); + + assert.equal(settings.enabled, MODALITY_BRIDGE_DEFAULTS.audioEnabled); + assert.equal(settings.model, MODALITY_BRIDGE_DEFAULTS.audioModel); + assert.equal(settings.timeoutMs, MODALITY_BRIDGE_DEFAULTS.audioTimeoutMs); + assert.equal(settings.maxClips, MODALITY_BRIDGE_DEFAULTS.audioMaxClips); +}); diff --git a/tests/unit/guardrails/audioBridge.test.ts b/tests/unit/guardrails/audioBridge.test.ts new file mode 100644 index 000000000000..5c0a7a20ea44 --- /dev/null +++ b/tests/unit/guardrails/audioBridge.test.ts @@ -0,0 +1,279 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + AudioBridgeGuardrail, + type AudioBridgeDependencies, +} from "../../../src/lib/guardrails/audioBridge.ts"; +import { + registerDefaultGuardrails, + resetGuardrailsForTests, +} from "../../../src/lib/guardrails/registry.ts"; +import { buildModalityBridgeHeader } from "../../../src/lib/guardrails/modalityBridge/bridgeStats.ts"; + +const audioPayload = () => ({ + model: "example/text-only", + messages: [ + { + role: "user", + content: [ + { type: "input_audio", input_audio: { data: "UklGRg==", format: "wav" } }, + { type: "text", text: "What was said?" }, + ], + }, + ], +}); + +const twoAudioPayload = () => ({ + model: "example/text-only", + messages: [ + { + role: "user", + content: [ + { type: "input_audio", input_audio: { data: "UklGRjE=", format: "wav" } }, + { type: "audio_url", audio_url: { url: "data:audio/wav;base64,UklGRjI=" } }, + ], + }, + ], +}); + +function createGuardrail(overrides: Partial = {}) { + return new AudioBridgeGuardrail({ + deps: { + getSettings: async () => ({ + modalityBridgeAudioEnabled: true, + modalityBridgeAudioModel: "deepgram/nova-3", + }), + getCapabilities: () => ({ supportsAudio: false }), + selectModel: async () => "deepgram/nova-3", + callTranscription: async () => "hello from the clip", + ...overrides, + }, + }); +} + +test("AudioBridgeGuardrail has the approved name and priority", () => { + const guardrail = createGuardrail(); + assert.equal(guardrail.name, "audio-bridge"); + assert.equal(guardrail.priority, 6); +}); + +test("native audio-capable targets bypass transcription", async () => { + let calls = 0; + const guardrail = createGuardrail({ + getCapabilities: () => ({ supportsAudio: true }), + callTranscription: async () => { + calls += 1; + return "should not run"; + }, + }); + + const result = await guardrail.preCall(audioPayload(), {}); + assert.equal(calls, 0); + assert.equal(result.modifiedPayload, undefined); +}); + +test("disabled settings and per-request disable both bypass transcription", async () => { + let calls = 0; + const disabledBySetting = createGuardrail({ + getSettings: async () => ({ modalityBridgeAudioEnabled: false }), + callTranscription: async () => { + calls += 1; + return "should not run"; + }, + }); + const enabled = createGuardrail({ + callTranscription: async () => { + calls += 1; + return "should not run"; + }, + }); + + assert.equal((await disabledBySetting.preCall(audioPayload(), {})).modifiedPayload, undefined); + assert.equal( + ( + await enabled.preCall(audioPayload(), { + disabledGuardrails: ["audio-bridge"], + }) + ).modifiedPayload, + undefined + ); + assert.equal(calls, 0); +}); + +test("text-only targets receive the STT transcript in place of audio", async () => { + const guardrail = createGuardrail(); + const result = await guardrail.preCall(audioPayload(), {}); + const modified = result.modifiedPayload as ReturnType; + + assert.deepEqual(modified.messages[0].content[0], { + type: "text", + text: "[Audio 1]: hello from the clip", + }); + assert.equal(modified.messages[0].content[1].text, "What was said?"); + assert.equal(result.meta?.clipsProcessed, 1); + assert.equal(result.meta?.sttModel, "deepgram/nova-3"); + assert.equal(typeof result.meta?.processingTimeMs, "number"); +}); + +test("all STT failures become explicit stubs for a proven text-only target", async () => { + const guardrail = createGuardrail({ + callTranscription: async () => { + throw new Error("no STT connection"); + }, + }); + + const result = await guardrail.preCall(twoAudioPayload(), {}); + const modified = result.modifiedPayload as ReturnType; + assert.deepEqual( + modified.messages[0].content.map((part) => ("text" in part ? part.text : null)), + [ + "[Audio 1]: (unavailable — no STT provider connected)", + "[Audio 2]: (unavailable — no STT provider connected)", + ] + ); + assert.equal(result.meta?.clipsProcessed, 2); +}); + +test("missing STT credentials become stubs for a proven text-only target", async () => { + let calls = 0; + const guardrail = createGuardrail({ + selectModel: async () => null, + callTranscription: async () => { + calls += 1; + return "should not run"; + }, + }); + + const result = await guardrail.preCall(audioPayload(), {}); + const modified = result.modifiedPayload as ReturnType; + assert.equal(calls, 0); + assert.deepEqual(modified.messages[0].content[0], { + type: "text", + text: "[Audio 1]: (unavailable — no STT provider connected)", + }); + assert.equal(result.meta?.sttModel, "unavailable"); +}); + +test("partial STT failure preserves only the failed audio part", async () => { + const original = twoAudioPayload(); + const guardrail = createGuardrail({ + getSettings: async () => ({ + modalityBridgeAudioEnabled: true, + modalityBridgeAudioModel: "deepgram/nova-3", + modalityBridgeCacheEnabled: false, + }), + callTranscription: async (part) => { + if (part.partIndex === 0) throw new Error("first failed"); + return "second succeeded"; + }, + }); + + const result = await guardrail.preCall(original, {}); + const modified = result.modifiedPayload as ReturnType; + assert.deepEqual(modified.messages[0].content[0], original.messages[0].content[0]); + assert.deepEqual(modified.messages[0].content[1], { + type: "text", + text: "[Audio 2]: second succeeded", + }); + assert.equal(result.meta?.clipsProcessed, 1); +}); + +test("unknown target capability preserves audio when every STT call fails", async () => { + const original = audioPayload(); + original.messages[0].content[0].input_audio.data = "dW5rbm93bi1hdWRpbw=="; + const snapshot = structuredClone(original); + const guardrail = createGuardrail({ + getCapabilities: () => ({ supportsAudio: null }), + getSettings: async () => ({ + modalityBridgeAudioEnabled: true, + modalityBridgeAudioModel: "deepgram/nova-3", + modalityBridgeCacheEnabled: false, + }), + callTranscription: async () => { + throw new Error("temporary STT failure"); + }, + }); + + const result = await guardrail.preCall(original, {}); + assert.equal(result.modifiedPayload, undefined); + assert.deepEqual(original, snapshot, "the input object must not be mutated"); +}); + +test("successful transcripts are reused from the shared cache", async () => { + let calls = 0; + const payload = audioPayload(); + payload.messages[0].content[0].input_audio.data = "Y2FjaGUtdW5pcXVl"; + const guardrail = createGuardrail({ + callTranscription: async () => { + calls += 1; + return "cached transcript"; + }, + }); + + await guardrail.preCall(payload, {}); + await guardrail.preCall(payload, {}); + assert.equal(calls, 1); +}); + +test("maxClips limits work without dropping later audio parts", async () => { + const original = twoAudioPayload(); + const guardrail = createGuardrail({ + getSettings: async () => ({ + modalityBridgeAudioEnabled: true, + modalityBridgeAudioModel: "deepgram/nova-3", + modalityBridgeAudioMaxClips: 1, + modalityBridgeCacheEnabled: false, + }), + }); + + const result = await guardrail.preCall(original, {}); + const modified = result.modifiedPayload as ReturnType; + assert.deepEqual(modified.messages[0].content[0], { + type: "text", + text: "[Audio 1]: hello from the clip", + }); + assert.deepEqual(modified.messages[0].content[1], original.messages[0].content[1]); +}); + +test("default registry places Audio Bridge after Vision Bridge", () => { + resetGuardrailsForTests({ registerDefaults: false }); + const names = registerDefaultGuardrails() + .list() + .map((guardrail) => guardrail.name); + assert.deepEqual(names.slice(0, 2), ["vision-bridge", "audio-bridge"]); + resetGuardrailsForTests(); +}); + +test("audio transparency header is emitted only for transformed clips", () => { + assert.equal( + buildModalityBridgeHeader([ + { + guardrail: "audio-bridge", + meta: { clipsProcessed: 2, sttModel: "deepgram/nova-3" }, + }, + ]), + "audio->text;model=deepgram/nova-3;parts=2" + ); + assert.equal( + buildModalityBridgeHeader([ + { + guardrail: "audio-bridge", + meta: { clipsProcessed: 2, sttModel: "deepgram/nova-3", rerouted: true }, + }, + ]), + null + ); + assert.equal( + buildModalityBridgeHeader([ + { + guardrail: "audio-bridge", + meta: { + clipsProcessed: 1, + sttModel: "deepgram/nova-3\r\nx-injected: yes", + }, + }, + ]), + "audio->text;model=deepgram/nova-3__x-injected__yes;parts=1" + ); +}); diff --git a/tests/unit/guardrails/audioBridgeHelpers.test.ts b/tests/unit/guardrails/audioBridgeHelpers.test.ts new file mode 100644 index 000000000000..89ee5444de92 --- /dev/null +++ b/tests/unit/guardrails/audioBridgeHelpers.test.ts @@ -0,0 +1,188 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + callAudioTranscription, + extractAudioParts, + replaceAudioParts, + selectAudioBridgeModel, + type AudioPart, +} from "../../../src/lib/guardrails/audioBridgeHelpers.ts"; + +test("fixed STT model is honored when its credential is usable", async () => { + const checked: string[] = []; + const selected = await selectAudioBridgeModel("deepgram/nova-2", async (model) => { + checked.push(model); + return true; + }); + + assert.equal(selected, "deepgram/nova-2"); + assert.deepEqual(checked, ["deepgram/nova-2"]); +}); + +test("fixed STT model is rejected when its credential is unavailable", async () => { + assert.equal(await selectAudioBridgeModel("deepgram/nova-2", async () => false), null); +}); + +test("auto selects the first catalog STT model with a usable credential", async () => { + const selected = await selectAudioBridgeModel( + "auto", + async (model) => model === "deepgram/nova-3" + ); + + assert.equal(selected, "deepgram/nova-3"); +}); + +test("input_audio is posted as multipart to the authenticated transcription self-loop", async () => { + let capturedUrl = ""; + let capturedInit: RequestInit | undefined; + const part: AudioPart = { + messageIndex: 0, + partIndex: 0, + ref: Buffer.from("RIFF test audio").toString("base64"), + shape: "input_audio", + format: "wav", + }; + + const transcript = await callAudioTranscription( + part, + { model: "deepgram/nova-3", timeoutMs: 1_000 }, + { + fetchImpl: async (input, init) => { + capturedUrl = String(input); + capturedInit = init; + return Response.json({ text: "hello from audio" }); + }, + getPort: () => 3210, + getBearer: () => "internal-test-key", + } + ); + + assert.equal(transcript, "hello from audio"); + assert.equal(capturedUrl, "http://localhost:3210/v1/audio/transcriptions"); + assert.equal(capturedInit?.method, "POST"); + assert.equal(new Headers(capturedInit?.headers).get("authorization"), "Bearer internal-test-key"); + + const form = capturedInit?.body as FormData; + assert.equal(form.get("model"), "deepgram/nova-3"); + const file = form.get("file") as File; + assert.equal(file.name, "audio.wav"); + assert.equal(file.type, "audio/wav"); + assert.equal(Buffer.from(await file.arrayBuffer()).toString(), "RIFF test audio"); +}); + +test("audio extraction and replacement cover the full history without dropping failed clips", () => { + const body = { + model: "text-only/model", + messages: [ + { + role: "user", + content: [ + { type: "input_audio", input_audio: { data: "UklGRg==", format: "wav" } }, + { type: "text", text: "first" }, + ], + }, + { + role: "user", + content: [ + { type: "audio_url", audio_url: { url: "data:audio/mpeg;base64,SUQz" } }, + { source: { media_type: "audio/ogg", data: "T2dnUw==" } }, + { + type: "text", + nested: { type: "input_audio", input_audio: { data: "bmVzdGVk", format: "wav" } }, + }, + ], + }, + ], + }; + + const parts = extractAudioParts(body.messages); + assert.deepEqual( + parts.map(({ messageIndex, partIndex, shape, format }) => ({ + messageIndex, + partIndex, + shape, + format, + })), + [ + { messageIndex: 0, partIndex: 0, shape: "input_audio", format: "wav" }, + { messageIndex: 1, partIndex: 0, shape: "audio_url", format: "mp3" }, + { messageIndex: 1, partIndex: 1, shape: "audio_source", format: "ogg" }, + ] + ); + + const replaced = replaceAudioParts(body, parts, ["[Audio 1]: hello", null, "[Audio 3]: bye"]); + assert.deepEqual(replaced.messages[0].content[0], { type: "text", text: "[Audio 1]: hello" }); + assert.deepEqual( + replaced.messages[1].content[0], + body.messages[1].content[0], + "a failed transcription must preserve the original audio part" + ); + assert.deepEqual(replaced.messages[1].content[1], { type: "text", text: "[Audio 3]: bye" }); + assert.deepEqual( + replaced.messages[1].content[2], + body.messages[1].content[2], + "nested audio is not a spliceable top-level part" + ); +}); + +test("audio_url data URIs are decoded before multipart upload", async () => { + let uploaded: File | null = null; + await callAudioTranscription( + { + messageIndex: 0, + partIndex: 0, + ref: "data:audio/mpeg;base64,SUQz", + shape: "audio_url", + format: "mp3", + }, + { model: "deepgram/nova-3", timeoutMs: 1_000 }, + { + fetchImpl: async (_input, init) => { + uploaded = (init?.body as FormData).get("file") as File; + return Response.json({ text: "ok" }); + }, + getPort: () => 3210, + getBearer: () => "internal-test-key", + } + ); + + assert.ok(uploaded); + assert.equal(uploaded.type, "audio/mpeg"); + assert.equal(Buffer.from(await uploaded.arrayBuffer()).toString(), "ID3"); +}); + +test("remote audio_url uses the guarded remote fetch before self-loop upload", async () => { + let fetchedUrl = ""; + let uploaded: File | null = null; + await callAudioTranscription( + { + messageIndex: 0, + partIndex: 0, + ref: "https://media.example.test/clip.ogg", + shape: "audio_url", + format: "ogg", + }, + { model: "deepgram/nova-3", timeoutMs: 1_000 }, + { + fetchRemote: async (url) => { + fetchedUrl = url; + return { + buffer: Buffer.from("OggS remote audio"), + contentType: "audio/ogg", + url, + }; + }, + fetchImpl: async (_input, init) => { + uploaded = (init?.body as FormData).get("file") as File; + return Response.json({ text: "ok" }); + }, + getPort: () => 3210, + getBearer: () => "internal-test-key", + } + ); + + assert.equal(fetchedUrl, "https://media.example.test/clip.ogg"); + assert.ok(uploaded); + assert.equal(Buffer.from(await uploaded.arrayBuffer()).toString(), "OggS remote audio"); +}); diff --git a/tests/unit/modality-bridge-audio-i18n.test.ts b/tests/unit/modality-bridge-audio-i18n.test.ts new file mode 100644 index 000000000000..891e6fb1e334 --- /dev/null +++ b/tests/unit/modality-bridge-audio-i18n.test.ts @@ -0,0 +1,28 @@ +import assert from "node:assert/strict"; +import fs from "node:fs"; +import test from "node:test"; + +const messages = JSON.parse( + fs.readFileSync(new URL("../../src/i18n/messages/en.json", import.meta.url), "utf8") +) as { settings?: Record }; + +test("English settings catalog contains every Audio Bridge UI string", () => { + const expected = [ + "modalityBridgeAudioTitle", + "modalityBridgeAudioDesc", + "modalityBridgeAudioEnabled", + "modalityBridgeAudioEnabledDesc", + "modalityBridgeAudioModel", + "modalityBridgeAudioModelAuto", + "modalityBridgeAudioMaxClips", + "modalityBridgeAudioTestButton", + "modalityBridgeAudioTestRunning", + "modalityBridgeAudioTestOk", + "modalityBridgeAudioTestNoop", + "modalityBridgeAudioTestError", + ]; + + for (const key of expected) { + assert.equal(typeof messages.settings?.[key], "string", `settings.${key}`); + } +}); diff --git a/tests/unit/model-alias-route.test.ts b/tests/unit/model-alias-route.test.ts index 4682bdd25f3c..5517defc440d 100644 --- a/tests/unit/model-alias-route.test.ts +++ b/tests/unit/model-alias-route.test.ts @@ -11,6 +11,7 @@ process.env.JWT_SECRET = process.env.JWT_SECRET || "model-alias-route-jwt"; const core = await import("../../src/lib/db/core.ts"); const modelsDb = await import("../../src/lib/db/models.ts"); +const providersDb = await import("../../src/lib/db/providers.ts"); const localDb = await import("../../src/lib/localDb.ts"); const route = await import("../../src/app/api/models/alias/route.ts"); const catalogRoute = await import("../../src/app/api/models/catalog/route.ts"); @@ -86,6 +87,15 @@ test("model alias route requires a dashboard session when management auth is ena }); test("api models catalog route reuses the unified catalog diagnostics headers", async () => { + await providersDb.createProviderConnection({ + provider: "deepgram", + authType: "apikey", + name: "deepgram-audio-catalog", + apiKey: "dg-test", + isActive: true, + testStatus: "active", + }); + v1Catalog.__resetCatalogBuilderRunsForTest(); const response = await catalogRoute.GET( new Request("http://localhost/api/models/catalog", { headers: { "x-request-id": "req-model-catalog-1" }, @@ -98,6 +108,11 @@ test("api models catalog route reuses the unified catalog diagnostics headers", assert.match(response.headers.get("X-Model-Catalog-Version") || "", /^model-metadata-v1:/); assert.equal(typeof body.catalog, "object"); assert.equal(typeof body.catalogVersion, "string"); + const nova = body.catalog.deepgram.models.find( + (model: { id?: string }) => model.id === "deepgram/nova-3" + ); + assert.equal(nova.type, "audio"); + assert.equal(nova.subtype, "transcription"); }); test("v1 models catalog emits diagnostics headers alongside the OpenAI-compatible list", async () => { diff --git a/tests/unit/model-capabilities-audio.test.ts b/tests/unit/model-capabilities-audio.test.ts new file mode 100644 index 000000000000..8c3327e40a3f --- /dev/null +++ b/tests/unit/model-capabilities-audio.test.ts @@ -0,0 +1,21 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +import { resolveAudioCapability } from "../../src/lib/modelCapabilities.ts"; + +test("explicit registry audio capability wins", () => { + assert.equal(resolveAudioCapability(undefined, { supportsAudio: true }, []), true); +}); + +test("explicit static audio capability is used when registry is silent", () => { + assert.equal(resolveAudioCapability({ supportsAudio: false }, null, []), false); +}); + +test("synced input modalities identify audio-capable and text-only models", () => { + assert.equal(resolveAudioCapability(undefined, null, ["text", "audio"]), true); + assert.equal(resolveAudioCapability(undefined, null, ["text"]), false); +}); + +test("missing audio capability evidence stays unknown", () => { + assert.equal(resolveAudioCapability(undefined, null, []), null); +}); diff --git a/tests/unit/ui/modality-bridge-audio-tab.test.tsx b/tests/unit/ui/modality-bridge-audio-tab.test.tsx new file mode 100644 index 000000000000..3b6691024bb1 --- /dev/null +++ b/tests/unit/ui/modality-bridge-audio-tab.test.tsx @@ -0,0 +1,184 @@ +// @vitest-environment jsdom +import React, { act } from "react"; +import { createRoot, type Root } from "react-dom/client"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +import ModalityBridgeAudioTab from "@/app/(dashboard)/dashboard/settings/components/modalityBridge/ModalityBridgeAudioTab"; + +vi.mock("next-intl", () => ({ + useTranslations: () => (key: string) => key, +})); + +const roots: Array<{ root: Root; el: HTMLDivElement }> = []; + +async function waitFor(predicate: () => boolean, label: string): Promise { + const startedAt = Date.now(); + while (!predicate()) { + if (Date.now() - startedAt > 2000) throw new Error(`Timed out waiting for: ${label}`); + await act(async () => { + await new Promise((resolve) => setTimeout(resolve, 10)); + }); + } +} + +describe("ModalityBridgeAudioTab", () => { + let fetchMock: ReturnType; + + beforeEach(() => { + (globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true; + fetchMock = vi.fn(async (input: RequestInfo | URL, init?: RequestInit) => { + const url = String(input); + if (url.includes("/api/models/catalog")) { + return Response.json({ + catalog: { + deepgram: { + provider: "Deepgram", + models: [ + { id: "deepgram/nova-3", type: "audio", subtype: "transcription" }, + { id: "deepgram/aura", type: "audio", subtype: "speech" }, + ], + }, + openai: { + provider: "OpenAI", + models: [{ id: "openai/gpt-5.6", type: "chat" }], + }, + }, + }); + } + if (url.includes("/api/models")) return Response.json({ models: [] }); + if (url.includes("/api/modality-bridge/stats")) { + return Response.json({ + vision: { bridged: 0, cacheHits: 0, failures: 0, lastUsedAt: null }, + audio: { bridged: 4, cacheHits: 2, failures: 1, lastUsedAt: null }, + }); + } + if (url.includes("/api/guardrails/test")) { + return Response.json({ + blocked: false, + results: [ + { + guardrail: "audio-bridge", + meta: { clipsProcessed: 1, sttModel: "deepgram/nova-3" }, + }, + ], + payload: {}, + }); + } + if (url.includes("/api/settings")) { + if (init?.method === "PATCH") return Response.json({}); + return Response.json({ + modalityBridgeAudioEnabled: true, + modalityBridgeAudioModel: "deepgram/nova-3", + modalityBridgeAudioTimeout: 12_000, + modalityBridgeAudioMaxClips: 2, + }); + } + return Response.json({}); + }); + vi.stubGlobal("fetch", fetchMock); + }); + + afterEach(() => { + for (const { root, el } of roots.splice(0)) { + act(() => root.unmount()); + el.remove(); + } + vi.unstubAllGlobals(); + }); + + async function render(): Promise { + const el = document.createElement("div"); + document.body.appendChild(el); + const root = createRoot(el); + await act(async () => { + root.render(); + }); + roots.push({ root, el }); + await waitFor( + () => el.querySelector('[data-testid="modality-bridge-audio-timeout"]') !== null, + "audio settings" + ); + return el; + } + + it("shows only transcription models and exposes selectable Auto", async () => { + const el = await render(); + expect(fetchMock).toHaveBeenCalledWith("/api/models/catalog"); + await waitFor( + () => + Array.from(el.querySelectorAll("option")).some( + (option) => option.value === "deepgram/nova-3" + ), + "STT models" + ); + const optionValues = Array.from(el.querySelectorAll("option")).map((option) => option.value); + expect(optionValues).toContain(""); + expect(optionValues).toContain("deepgram/nova-3"); + expect(optionValues).not.toContain("deepgram/aura"); + expect(optionValues).not.toContain("openai/gpt-5.6"); + expect(el.textContent).toContain("4 modalityBridgeStatsBridged"); + }); + + it("PATCHes only the new audio setting when toggled", async () => { + const el = await render(); + const toggle = el.querySelector('[role="switch"]') as HTMLButtonElement; + await act(async () => toggle.click()); + + await waitFor( + () => fetchMock.mock.calls.some(([, init]) => init?.method === "PATCH"), + "audio PATCH" + ); + const call = fetchMock.mock.calls.find(([, init]) => init?.method === "PATCH"); + const body = JSON.parse(String(call?.[1]?.body)) as Record; + expect(body).toEqual({ modalityBridgeAudioEnabled: false }); + expect(body).not.toHaveProperty("visionBridgeEnabled"); + }); + + it("clamps the audio timeout before PATCHing", async () => { + const el = await render(); + const timeout = el.querySelector( + '[data-testid="modality-bridge-audio-timeout"]' + ) as HTMLInputElement; + act(() => { + const setter = Object.getOwnPropertyDescriptor( + window.HTMLInputElement.prototype, + "value" + )?.set; + setter?.call(timeout, "500"); + timeout.dispatchEvent(new Event("input", { bubbles: true })); + }); + await act(async () => { + timeout.dispatchEvent(new FocusEvent("focusout", { bubbles: true })); + await new Promise((resolve) => setTimeout(resolve, 0)); + }); + + await waitFor( + () => + fetchMock.mock.calls.some(([, init]) => { + if (init?.method !== "PATCH") return false; + return JSON.parse(String(init.body)).modalityBridgeAudioTimeout === 1000; + }), + "clamped audio timeout" + ); + }); + + it("runs the Audio Bridge self-test with an input_audio part", async () => { + const el = await render(); + const button = Array.from(el.querySelectorAll("button")).find((candidate) => + candidate.textContent?.includes("modalityBridgeAudioTestButton") + ); + await act(async () => button?.click()); + await waitFor( + () => el.textContent?.includes("modalityBridgeAudioTestOk") ?? false, + "audio self-test result" + ); + + const call = fetchMock.mock.calls.find(([url]) => String(url).includes("/api/guardrails/test")); + const body = JSON.parse(String(call?.[1]?.body)) as { + input?: { messages?: Array<{ content?: Array<{ type?: string }> }> }; + }; + expect(body.input?.messages?.[0]?.content?.some((part) => part.type === "input_audio")).toBe( + true + ); + }); +}); diff --git a/tests/unit/ui/modality-bridge-page.test.tsx b/tests/unit/ui/modality-bridge-page.test.tsx index 74d4a4f4f45a..d0b056001727 100644 --- a/tests/unit/ui/modality-bridge-page.test.tsx +++ b/tests/unit/ui/modality-bridge-page.test.tsx @@ -38,6 +38,10 @@ vi.mock( "@/app/(dashboard)/dashboard/settings/components/modalityBridge/ModalityBridgeVisionTab", () => ({ default: () =>
vision
}) ); +vi.mock( + "@/app/(dashboard)/dashboard/settings/components/modalityBridge/ModalityBridgeAudioTab", + () => ({ default: () =>
audio
}) +); const roots: Array<{ root: Root; el: HTMLDivElement }> = []; @@ -71,7 +75,8 @@ describe("Modality Bridge settings page", () => { navigation.search = "tab=audio"; const el = renderPage(); expect(el.querySelectorAll('[role="tab"]')).toHaveLength(3); - expect(el.textContent).toContain("modalityBridgeAudioComingSoon"); + expect(el.querySelector('[data-testid="audio-tab"]')).toBeTruthy(); + expect(el.textContent).not.toContain("modalityBridgeAudioComingSoon"); expect(el.querySelector('[role="tab"][aria-selected="true"]')?.textContent).toContain( "modalityBridgeAudioTab" );