feat(audio): select a transcription backend and carry the transcript to the model

docs/api/README.md

@@ -1263,6 +1263,9 @@ console.log(result.content);
 - [PDFProviderConfig](type-aliases/PDFProviderConfig.md)
 - [PDFProcessorOptions](type-aliases/PDFProcessorOptions.md)
 - [AudioProviderConfig](type-aliases/AudioProviderConfig.md)
+- [AudioTranscriptionProvider](type-aliases/AudioTranscriptionProvider.md)
+- [AudioTranscriptionSelection](type-aliases/AudioTranscriptionSelection.md)
+- [AudioTranscriptionOutcome](type-aliases/AudioTranscriptionOutcome.md)
 - [AudioProcessorOptions](type-aliases/AudioProcessorOptions.md)
 - [VideoProcessorOptions](type-aliases/VideoProcessorOptions.md)
 - [OfficeProcessorOptions](type-aliases/OfficeProcessorOptions.md)

docs/api/type-aliases/AudioProcessorOptions.md

@@ -40,7 +40,7 @@ Language code for transcription (e.g., 'en', 'es', 'fr')
 
 > `optional` **prompt?**: `string`
 
-Context or prompt to guide transcription accuracy
+OpenAI/Whisper-only context prompt to guide transcription accuracy; ignored by Google and Azure
 
 ---
 

docs/api/type-aliases/AudioTranscriptionOutcome.md

@@ -0,0 +1,51 @@
+[**NeuroLink API Reference**](../README.md)
+
+---
+
+[NeuroLink API Reference](../README.md) / AudioTranscriptionOutcome
+
+# Type Alias: AudioTranscriptionOutcome
+
+> **AudioTranscriptionOutcome** = `object`
+
+What a transcription attempt produced (#409/#416).
+
+`transcriptionSkippedReason` is present on exactly the attempts that yielded
+no transcript, so a caller can tell "no speech in this audio" from "no
+backend was reachable".
+
+## Properties
+
+### transcript
+
+> **transcript**: `string` \| `undefined`
+
+---
+
+### hasTranscript
+
+> **hasTranscript**: `boolean`
+
+---
+
+### transcriptionProvider
+
+> **transcriptionProvider**: `string` \| `undefined`
+
+---
+
+### transcriptionLanguage
+
+> **transcriptionLanguage**: `string` \| `undefined`
+
+---
+
+### transcriptionDuration
+
+> **transcriptionDuration**: `number` \| `undefined`
+
+---
+
+### transcriptionSkippedReason
+
+> **transcriptionSkippedReason**: `string` \| `undefined`

docs/api/type-aliases/AudioTranscriptionProvider.md

@@ -0,0 +1,17 @@
+[**NeuroLink API Reference**](../README.md)
+
+---
+
+[NeuroLink API Reference](../README.md) / AudioTranscriptionProvider
+
+# Type Alias: AudioTranscriptionProvider
+
+> **AudioTranscriptionProvider** = `"openai"` \| `"google"` \| `"azure"`
+
+A transcription backend `AudioProcessor` can drive (#413).
+
+Deliberately narrower than `AudioProcessorOptions.provider`, which stays a
+free-form `string` on the public surface for backward compatibility: a
+caller's spelling is normalised onto this union (accepting aliases such as
+"whisper" for "openai") and an unrecognised one is reported rather than
+silently ignored.

docs/api/type-aliases/AudioTranscriptionSelection.md

@@ -0,0 +1,15 @@
+[**NeuroLink API Reference**](../README.md)
+
+---
+
+[NeuroLink API Reference](../README.md) / AudioTranscriptionSelection
+
+# Type Alias: AudioTranscriptionSelection
+
+> **AudioTranscriptionSelection** = \{ `provider`: [`AudioTranscriptionProvider`](AudioTranscriptionProvider.md); `label`: `string`; `reason?`: `undefined`; \} \| \{ `provider?`: `undefined`; `label?`: `undefined`; `reason`: `string`; \}
+
+Outcome of choosing a transcription backend (#413).
+
+A union rather than a nullable provider so the "nothing usable" case is
+forced to carry the reason — the failure mode this replaces was every
+unavailable backend collapsing into an indistinguishable empty result.

docs/api/type-aliases/FileProcessingResult.md

@@ -220,3 +220,40 @@ Clip length in seconds, when the processor could measure it.
 Surfaced here so the native-delivery gate does not have to re-probe a
 container the processor has already opened. Absent when probing failed
 — an unknown duration, not a zero-length clip.
+
+#### duration?
+
+> `optional` **duration?**: `number`
+
+Media duration in seconds, read from the container header.
+
+Sourced from the demuxed stream rather than from the transcription
+backend, so it is present for every audio file — including one that was
+never transcribed because no provider was configured.
+
+#### language?
+
+> `optional` **language?**: `string`
+
+Language of the transcript, as reported by the transcription provider
+(BCP-47 or ISO-639-1 depending on the backend), or the language the
+caller pinned via `AudioProcessorOptions.language`.
+
+Absent when the file was not transcribed — its absence therefore means
+"unknown", never "not speech".
+
+#### transcriptionLength?
+
+> `optional` **transcriptionLength?**: `number`
+
+Character count of the transcript.
+
+`0` is a meaningful value distinct from `undefined`: it means a provider
+ran and returned an empty transcript (silence, music, no speech), where
+`undefined` means no transcription was attempted at all.
+
+#### transcriptionProvider?
+
+> `optional` **transcriptionProvider?**: `string`
+
+Which backend produced the transcript, e.g. "openai-whisper".

docs/api/type-aliases/GenerateOptions.md

@@ -269,6 +269,56 @@ and the inline copies had already drifted apart in what they documented.
 
 ---
 
+### audioOptions?
+
+> `optional` **audioOptions?**: `object`
+
+Audio transcription options for attached audio files (#413/#440).
+
+An attached audio file is transcribed automatically when a backend is
+configured; this is only needed to override which one, or to help it.
+Without it the first configured backend wins, in the order OpenAI
+(Whisper), Google, Azure.
+
+#### provider?
+
+> `optional` **provider?**: `string`
+
+Transcription backend: "openai" (aliases "whisper", "openai-whisper"),
+"google" or "azure". An unavailable or unrecognised choice is never
+swapped for another backend: no transcript is produced, and the
+selection reason is logged as a warning.
+
+#### transcriptionModel?
+
+> `optional` **transcriptionModel?**: `string`
+
+Transcription model, e.g. "whisper-1". Backend-specific.
+
+#### language?
+
+> `optional` **language?**: `string`
+
+Language hint, e.g. "en". Improves accuracy on non-English speech.
+
+#### prompt?
+
+> `optional` **prompt?**: `string`
+
+OpenAI/Whisper-only context prompt to bias transcription (proper nouns,
+jargon). Ignored by Google and Azure.
+
+#### Example
+
+```typescript
+await neurolink.generate({
+  input: { text: "Summarise this call", files: ["./call.mp3"] },
+  audioOptions: { provider: "openai", language: "en" },
+});
+```
+
+---
+
 ### tts?
 
 > `optional` **tts?**: [`TTSOptions`](TTSOptions.md)

docs/api/type-aliases/ProcessedAudio.md

@@ -123,6 +123,29 @@ Extracted ID3/Vorbis/APE tags
 
 > `optional` **transcriptionProvider?**: `string`
 
+### transcriptionLanguage?
+
+> `optional` **transcriptionLanguage?**: `string`
+
+Language the transcription backend reported for the speech (#409), when
+it reported one.
+
+Not always a _detected_ language: when the backend's response carries no
+language (Whisper's `verbose_json` usually does; Google/Azure responses
+vary), this falls back to the caller-requested `options.language` instead
+of going unset. A caller that must distinguish "the backend detected X"
+from "X is just what was asked for" cannot do so from this field alone.
+
+### transcriptionDuration?
+
+> `optional` **transcriptionDuration?**: `number`
+
+Audio duration in seconds as measured by the transcription backend (#409).
+
+Kept apart from `metadata.duration`, which comes from the container
+header: the two disagree on a file with a broken or absent header, and the
+header is the one that is available without a transcription call.
+
 ### transcriptionSkippedReason?
 
 > `optional` **transcriptionSkippedReason?**: `string`

docs/api/type-aliases/StreamOptions.md

@@ -213,6 +213,42 @@ module's own default, unchanged unless a caller opts in here.
 
 ---
 
+### audioOptions?
+
+> `optional` **audioOptions?**: `object`
+
+Audio transcription options for attached audio files (#413/#440).
+
+Mirrors `GenerateOptions.audioOptions`. `stream()` spreads its options
+rather than rebuilding them field by field, so declaring it here is what
+lets it reach the message builder.
+
+#### provider?
+
+> `optional` **provider?**: `string`
+
+Backend: "openai" (aliases "whisper"), "google" or "azure".
+
+#### transcriptionModel?
+
+> `optional` **transcriptionModel?**: `string`
+
+Transcription model, e.g. "whisper-1". Backend-specific.
+
+#### language?
+
+> `optional` **language?**: `string`
+
+Language hint, e.g. "en".
+
+#### prompt?
+
+> `optional` **prompt?**: `string`
+
+OpenAI/Whisper-only context prompt to bias transcription; ignored by Google and Azure.
+
+---
+
 ### tts?
 
 > `optional` **tts?**: [`TTSOptions`](TTSOptions.md)

docs/api/type-aliases/TextGenerationOptions.md

@@ -679,6 +679,42 @@ module's own default, unchanged unless a caller opts in here.
 
 ---
 
+### audioOptions?
+
+> `optional` **audioOptions?**: `object`
+
+Audio transcription options for attached audio files (#413/#440).
+
+Mirrors `GenerateOptions.audioOptions`; declared here because
+`buildGenerateTextOptions` rebuilds options field by field, so anything
+missing from this type cannot reach the message builder at all.
+
+#### provider?
+
+> `optional` **provider?**: `string`
+
+Backend: "openai" (aliases "whisper"), "google" or "azure".
+
+#### transcriptionModel?
+
+> `optional` **transcriptionModel?**: `string`
+
+Transcription model, e.g. "whisper-1". Backend-specific.
+
+#### language?
+
+> `optional` **language?**: `string`
+
+Language hint, e.g. "en".
+
+#### prompt?
+
+> `optional` **prompt?**: `string`
+
+OpenAI/Whisper-only context prompt to bias transcription; ignored by Google and Azure.
+
+---
+
 ### enableSummarization?
 
 > `optional` **enableSummarization?**: `boolean`

package.json

@@ -248,6 +248,7 @@
     "pre-push": "pnpm run check:deps && pnpm run build && pnpm run test:provider-structure && pnpm run test:model-manifests",
     "check:all": "pnpm run lint && pnpm run format --check && pnpm run validate && pnpm run validate:commit",
     "test:file-formats": "pnpm exec tsx test/continuous-test-suite-file-formats.ts",
+    "test:audio-transcription": "pnpm exec tsx test/continuous-test-suite-audio-transcription.ts",
     "test:multimodal": "pnpm run test:file-formats && pnpm run test:multimodal:sdk && pnpm run test:video-native && pnpm run test:video-frames",
     "test:multimodal:sdk": "pnpm exec tsx test/continuous-test-suite-multimodal-sdk.ts",
     "test:video-frames": "pnpm exec tsx test/continuous-test-suite-video-frames.ts",

src/cli/loop/optionsSchema.ts

@@ -28,6 +28,7 @@ export const textGenerationOptionsSchema: Record<
     | "csvOptions"
     | "pdfOptions"
     | "imageOptions"
+    | "audioOptions" // Complex object (#413); SDK-only via GenerateOptions.audioOptions — no --audio-* flags exist yet, not `/set`
     | "videoOptions" // Complex object, not set via simple CLI commands
     | "officeOptions"
     | "tts"

src/lib/core/modules/MessageBuilder.ts

@@ -131,6 +131,10 @@ export class MessageBuilder {
             csvOptions: options.csvOptions,
             pdfOptions: options.pdfOptions,
             imageOptions: options.imageOptions,
+            // The audio option bag is carried the same way: omitted here, a
+            // caller's transcription backend, model and language never reach
+            // AudioProcessor and it silently falls back to auto-selection.
+            audioOptions: options.audioOptions,
             // Same reason as csvOptions: the keyframe budget and the
             // transcription flag are read downstream by the detector, and
             // a reconstruction that omits them silently restores defaults.
@@ -300,6 +304,10 @@ export class MessageBuilder {
             csvOptions: options.csvOptions,
             pdfOptions: options.pdfOptions,
             imageOptions: options.imageOptions,
+            // The audio option bag is carried the same way: omitted here, a
+            // caller's transcription backend, model and language never reach
+            // AudioProcessor and it silently falls back to auto-selection.
+            audioOptions: options.audioOptions,
             // Same reason as csvOptions: the keyframe budget and the
             // transcription flag are read downstream by the detector, and
             // a reconstruction that omits them silently restores defaults.

src/lib/neurolink.ts

@@ -5801,6 +5801,10 @@ Current user's request: ${currentInput}`;
       csvOptions: options.csvOptions,
       pdfOptions: options.pdfOptions,
       imageOptions: options.imageOptions,
+      // #413/#440: the audio option bag belongs in the same list. Left out
+      // here it was accepted by the public type and then silently discarded
+      // before the message builder, like every field above.
+      audioOptions: options.audioOptions,
       // videoOptions belongs in that same list and was never in it, so no SDK
       // caller's video settings have ever reached a provider -- frames and
       // quality included, not just the native-video flags added alongside.