Add context-aware transcription and dynamic ChatGPT models - #1016
Merged
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughThe OpenAI plugin adds context-aware file and realtime transcription, configurable live delay, account-scoped ChatGPT model discovery and caching, new settings and localized text, expanded tests, and a manifest version update to 1.3.0. ChangesOpenAI transcription and ChatGPT model enhancement
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant OpenAIPlugin
participant TranscriptionClient
participant RealtimeSession
participant CodexModelsEndpoint
OpenAIPlugin->>TranscriptionClient: submit context-aware file transcription
OpenAIPlugin->>RealtimeSession: create configured live transcription
RealtimeSession-->>OpenAIPlugin: return transcript result
OpenAIPlugin->>CodexModelsEndpoint: fetch account-scoped ChatGPT models
CodexModelsEndpoint-->>OpenAIPlugin: return model catalog
Possibly related PRs
Suggested labels: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
SeoFood
changed the title
SeoFood
marked this pull request as ready for review
July 28, 2026 19:57
coderabbitai
Bot
added
area: dictation
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (2)
TypeWhisperPluginSDK/Plugins/OpenAIPlugin/Tests/OpenAIPluginTests.swift (1)
257-257: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueSwiftLint flags the
String(decoding:as:)conversions (Lines 257, 309, 362, 363, 402, 561).
optional_data_string_conversionprefers the failable initializer; consider a small helper so the rule is satisfied in one place.♻️ Suggested helper
private static func bodyString(_ data: Data) throws -> String { try XCTUnwrap(String(bytes: data, encoding: .utf8)) }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@TypeWhisperPluginSDK/Plugins/OpenAIPlugin/Tests/OpenAIPluginTests.swift` at line 257, Replace the flagged String(decoding:as:) conversions in OpenAIPluginTests with a shared throwing bodyString helper that uses String(bytes:encoding:) and XCTUnwrap; update all listed request-body assertions to call this helper while preserving their existing error propagation.Source: Linters/SAST tools
TypeWhisperPluginSDK/Plugins/OpenAIPlugin/OpenAIPlugin.swift (1)
707-709: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winCapability decisions are re-derived from hard-coded model IDs in three places.
OpenAITranscriptionModelCapabilityis the intended single source of truth for transport, context-aware hints, and realtime behavior, but three sites bypass it and compare against specific model IDs, so adding a model to the capability table would silently produce wrong requests or a missing control.
TypeWhisperPluginSDK/Plugins/OpenAIPlugin/OpenAIPlugin.swift#L707-L709: replace themodelID == gptLiveTranscribeModelIDcheck withOpenAITranscriptionModelCapability.capability(for: modelID)?.usesContextAwareHints ?? false.TypeWhisperPluginSDK/Plugins/OpenAIPlugin/OpenAIPlugin.swift#L2084-L2093: passcapability.modelInfo.idintocontextAwareFileTranscriptionClient.transcribe(...)instead of letting the client hard-codegptTranscribeModelIDat Line 800.TypeWhisperPluginSDK/Plugins/OpenAIPlugin/OpenAIPlugin.swift#L3197-L3210: gate the live delay picker ontranscriptionModelIsRealtime+transcriptionModelUsesContextAwareHintsrather thanselectedModel == liveModelId.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@TypeWhisperPluginSDK/Plugins/OpenAIPlugin/OpenAIPlugin.swift` around lines 707 - 709, Use OpenAITranscriptionModelCapability as the single source of truth for all capability decisions: in TypeWhisperPluginSDK/Plugins/OpenAIPlugin/OpenAIPlugin.swift:707-709, update usesContextAwareHints to call capability(for:) and return usesContextAwareHints with a false fallback; at 2084-2093, pass capability.modelInfo.id to contextAwareFileTranscriptionClient.transcribe; at 3197-3210, gate the live delay picker using transcriptionModelIsRealtime and transcriptionModelUsesContextAwareHints instead of selectedModel and a hard-coded live model ID.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@TypeWhisperPluginSDK/Plugins/OpenAIPlugin/Localizable.xcstrings`:
- Around line 148-163: Update the transcription context field in
OpenAIPlugin.swift around TextField("Describe the recording topic, setting, or
relevant context.", ...) so its label resolves from the plugin localization
bundle instead of the main app bundle. Use an explicitly bundled localized Text
or LocalizedStringResource, or pass the resolved localized string into
TextField, while preserving the existing binding and vertical-axis behavior.
In `@TypeWhisperPluginSDK/Plugins/OpenAIPlugin/OpenAIPlugin.swift`:
- Around line 3045-3060: Update OpenAIChatGPTModel.isVisibleInPicker to use a
denylist: return false only when visibility lowercases to "hide" or "hidden",
and true for nil and all other values, including "list".
---
Nitpick comments:
In `@TypeWhisperPluginSDK/Plugins/OpenAIPlugin/OpenAIPlugin.swift`:
- Around line 707-709: Use OpenAITranscriptionModelCapability as the single
source of truth for all capability decisions: in
TypeWhisperPluginSDK/Plugins/OpenAIPlugin/OpenAIPlugin.swift:707-709, update
usesContextAwareHints to call capability(for:) and return usesContextAwareHints
with a false fallback; at 2084-2093, pass capability.modelInfo.id to
contextAwareFileTranscriptionClient.transcribe; at 3197-3210, gate the live
delay picker using transcriptionModelIsRealtime and
transcriptionModelUsesContextAwareHints instead of selectedModel and a
hard-coded live model ID.
In `@TypeWhisperPluginSDK/Plugins/OpenAIPlugin/Tests/OpenAIPluginTests.swift`:
- Line 257: Replace the flagged String(decoding:as:) conversions in
OpenAIPluginTests with a shared throwing bodyString helper that uses
String(bytes:encoding:) and XCTUnwrap; update all listed request-body assertions
to call this helper while preserving their existing error propagation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 1d13c044-6841-4aac-82c9-d0041d3f00bd
📒 Files selected for processing (4)
TypeWhisperPluginSDK/Plugins/OpenAIPlugin/Localizable.xcstringsTypeWhisperPluginSDK/Plugins/OpenAIPlugin/OpenAIPlugin.swiftTypeWhisperPluginSDK/Plugins/OpenAIPlugin/Tests/OpenAIPluginTests.swiftTypeWhisperPluginSDK/Plugins/OpenAIPlugin/manifest.json
This was referenced Jul 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
OpenAI announced two new transcription models:
gpt-transcribeandgpt-live-transcribe. This PR adds both models to the OpenAI plugin using the existing plugin SDK contracts and keeps the host compatibility floor unchanged.The local development smoke also exposed that ChatGPT Login's Refresh button returned a hard-coded model snapshot instead of requesting the account-specific Codex catalog. The current Codex catalog includes GPT-5.6 Sol, Terra, and Luna, so this PR now uses the same dynamic
/backend-api/codex/models?client_version=...contract as Codex. Future visible models can therefore appear after refresh without another plugin update.Official references:
What changed
gpt-transcribeas the default only when no model selection has been persisted; existing selections remain unchanged.gpt-transcriberequests through/v1/audio/transcriptionswith M4A-to-WAV fallback, context, sanitized ordered keywords, ordered language hints, and detected-language parsing.gpt-live-transcribeto the existing WebSocket transcription path with context, keywords, language hints, and configurable delay.gpt-realtime-whisperand existing REST model payloads unchanged.whisper-1before any network request.ChatGPT-Account-ID, then honor server visibility and priority.1.3.0while keepingminHostVersion: 1.5.0and SDK compatibilityv1.Test plan
swift test --package-path TypeWhisperPluginSDK --filter OpenAIPluginTests(39 tests)xcodebuild -project TypeWhisper.xcodeproj -scheme OpenAIPlugin -configuration Release CODE_SIGNING_ALLOWED=NO build/Users/marco/Projects/typewhisper-dev-tools/build-typewhisper-plugin-dev.sh --run OpenAIPlugin /Users/marco/.codex/worktrees/c7bc/typewhisper-mac1.3.0.v1.5.0host tag.gpt-5.6-sol,gpt-5.6-terra,gpt-5.6-luna,gpt-5.5,gpt-5.4,gpt-5.4-mini, andgpt-5.3-codex-spark.gpt-transcribeAPI smoke with context, multiple language hints, and dictionary terms.gpt-live-transcribeAPI smoke with delaylow, partial updates, and a final committed result.whisper-1translation and verify a GPT transcription model is blocked locally withTranslate requires Whisper 1.The real transcription API checks remain required before merge. Merge and plugin release are separate approval gates.
Summary by CodeRabbit