Add OpenAI Whisper Audio Transcription Support

[Trynax]

Related to #376 - Support for Voice Models

Implementation

Adds comprehensive OpenAI Whisper audio transcription support to Echo with whisper-1 and whisper-large-v3 models.

Key Features

• ✅ Audio transcription and translation
• ✅ Complete TypeScript SDK support
• ✅ Example components for Next.js and Vite

Architecture

const formData = new FormData();
formData.append('file', audioBlob);
formData.append('model', 'whisper-1');

const response = await fetch(`${ECHO_BASE_URL}/audio/transcriptions`, {
  method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'X-App-Id': appId },
  body: formData
});

Files Added

• OpenAIAudioProvider.ts - Server-side audio processing
• Audio model definitions and SDK integration
• Example components and comprehensive tests
• Enhanced provider factory and validation

Tests fail(model not found) against production API (expected - needs server deployment)

Provides clean foundation for future voice features (#376) and resolves previous merge conflict issues (#406).

[vercel]

@Trynax is attempting to deploy a commit to the Merit Systems Team on Vercel.

A member of the Team first needs to authorize it.

[rsproule]

High level:

• try to use the external providers as much as possible for all types
• should test that this works with the sdks directly instead of doing raw fetch
• the examples look like they have updated the frontend, but i do not see the backend portion for this
• document ur testing strategy (i know running this locally is a PITA, lmk if you run into problems here, we are working on making this easier)

[rsproule]

we should not be rebuilding our own client here. we can use the actual open ai client like here

echo/packages/app/server/src/clients/openai-image-formdata-client.ts

Lines 11 to 15 in ca3ea92

	const client = new OpenAI({
	apiKey: process.env.ECHO_API_KEY || '',
	baseURL: 'http://localhost:3070',
	});

[rsproule]

use OpenAI's types directly

[rsproule]

audioResponse.duration alr exists

[rsproule]

https://platform.openai.com/docs/guides/speech-to-text#streaming

streaming does have some support

[rsproule]

use the ai-sdk for all of this stuff in here like the other smoke tests

[rsproule]

we can just iterate over these like the other tests, no need to write each manually

[rsproule]

this is not a thing. i think this is slopped up

[Trynax]

Yoo @rsproule

I've addressed all the feedback
Followed the image and text testing strategy but the tests fail with 422 "model not supported" errors, which I assume is expected since the backend hasn't been deployed yet right?

[rsproule]

Yoo @rsproule

I've addressed all the feedback Followed the image and text testing strategy but the tests fail with 422 "model not supported" errors, which I assume is expected since the backend hasn't been deployed yet right?

You should be able to point the tests at a local version of the backend which has your changes in it. You will need to fill it some portion of the .env (we are making this easier to run locally, i know its a pain rn)