Skip to content

Fix OpenAI file transcription finalization - #1030

Merged
SeoFood merged 2 commits into
mainfrom
seofood/openai-plugin-1.3.1-finalization
Jul 29, 2026
Merged

SeoFood merged 2 commits into
mainfrom
seofood/openai-plugin-1.3.1-finalization

Conversation

@SeoFood

@SeoFood SeoFood commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • buffer file-based OpenAI transcription sessions while preserving throttled live previews
  • always finalize the complete buffered audio when dictation stops
  • release OpenAIPlugin 1.3.1 with matching manifest and bundle versions

Root cause

The OpenAI plugin advertised live transcription globally, but gpt-transcribe rejected native live session creation because it is a file model. The host then promoted the last batch preview when live finalization returned no result, so audio after the final preview could be lost.

User impact

Dictation with GPT Transcribe now retains words spoken after the last preview, including endings after a pause, without requiring a new host version.

Test plan

  • swift test --package-path TypeWhisperPluginSDK --filter OpenAIPluginTests
  • /Users/marco/Projects/typewhisper-dev-tools/build-typewhisper-plugin-dev.sh --clean --run OpenAIPlugin /Users/marco/.codex/worktrees/8e5d/typewhisper-mac
  • Real dictation smoke with GPT Transcribe, a 2.2-second pause, and a trailing marker; confirmed the full ending and usedLiveResult=true.

Summary by CodeRabbit

  • New Features
    • Added live transcription support for file-based transcription models with buffered audio.
    • Provides incremental preview/transcription progress during recording, with orderly preview generation.
  • Bug Fixes
    • Translation requests are now validated consistently across transcription modes.
    • Preview/transcription error handling improved so final transcription can still complete.
  • Tests
    • Expanded file-model live transcription coverage, including buffering, incremental updates, and re-entrant preview behavior.
  • Chores
    • Updated the OpenAI plugin version to 1.3.1.

@coderabbitai

coderabbitai Bot commented Jul 29, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: ca5a98a8-5cc0-4dbc-aee5-6469e9b11603

📥 Commits

Reviewing files that changed from the base of the PR and between 014b536 and 7e680a1.

📒 Files selected for processing (2)
  • TypeWhisperPluginSDK/Plugins/OpenAIPlugin/OpenAIPlugin.swift
  • TypeWhisperPluginSDK/Plugins/OpenAIPlugin/Tests/OpenAIPluginTests.swift
🚧 Files skipped from review as they are similar to previous changes (1)
  • TypeWhisperPluginSDK/Plugins/OpenAIPlugin/OpenAIPlugin.swift

📝 Walkthrough

Walkthrough

The OpenAI plugin now supports buffered live transcription for non-realtime file models, including previews, finalization, cancellation, explicit capability routing, expanded tests, and version updates to 1.3.1.

Changes

OpenAI file transcription

Layer / File(s) Summary
Buffered file transcription session
TypeWhisperPluginSDK/Plugins/OpenAIPlugin/OpenAIPlugin.swift
Adds an actor that buffers audio, emits RMS-based preview transcriptions, finalizes complete transcriptions, supports cancellation, and marks transcription capabilities Sendable.
Capability-based session routing
TypeWhisperPluginSDK/Plugins/OpenAIPlugin/OpenAIPlugin.swift
Routes non-realtime capabilities to the file session, preserves realtime connections, validates translations first, and supports explicit transcription overrides.
Behavior coverage and version alignment
TypeWhisperPluginSDK/Plugins/OpenAIPlugin/Tests/OpenAIPluginTests.swift, TypeWhisperPluginSDK/Plugins/OpenAIPlugin/manifest.json, TypeWhisper.xcodeproj/project.pbxproj, TypeWhisperPluginSDK/Plugins/OpenAIPlugin/OpenAIPlugin.swift
Tests buffered final, preview, and re-entrant behavior and updates plugin, build, and fallback versions to 1.3.1.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant OpenAIFileTranscriptionSession
  participant OpenAIPlugin
  participant OpenAIAPI
  Client->>OpenAIPlugin: createLiveTranscriptionSession(capability)
  OpenAIPlugin->>OpenAIFileTranscriptionSession: create file-model session
  Client->>OpenAIFileTranscriptionSession: appendAudio(chunks)
  OpenAIFileTranscriptionSession->>OpenAIPlugin: request preview transcription
  OpenAIPlugin->>OpenAIAPI: submit buffered audio
  OpenAIAPI-->>Client: preview progress
  Client->>OpenAIFileTranscriptionSession: finish()
  OpenAIFileTranscriptionSession->>OpenAIPlugin: request final transcription
  OpenAIPlugin->>OpenAIAPI: submit complete audio
  OpenAIAPI-->>Client: final transcription
Loading

Possibly related PRs

Suggested labels: enhancement, area: dictation

Poem

A bunny hears the buffers flow,
Preview words begin to grow.
Full audio hops to OpenAI,
Final text lands crisp and spry.
Version three-one, carrots high!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly matches the main change: fixing OpenAI file transcription finalization.
Description check ✅ Passed The description covers summary, root cause, user impact, and test plan, though it does not use the exact checkbox template.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch seofood/openai-plugin-1.3.1-finalization

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot added area: dictation Related to recording and transcription enhancement New feature or request labels Jul 29, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@TypeWhisperPluginSDK/Plugins/OpenAIPlugin/OpenAIPlugin.swift`:
- Around line 1073-1083: Update the preview flow in appendAudio around
shouldRequestPreview and transcribe to prevent overlapping requests: add an
in-flight guard, mark it before awaiting transcribe, and clear it on every
completion path. While a request is active, retain/coalesce newly appended audio
so the latest preview is evaluated and requested after completion rather than
starting concurrent requests. Add a test covering a delayed transcription and
confirming requests remain serialized.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 589cfc08-8904-428a-bb65-63a0c4411e2f

📥 Commits

Reviewing files that changed from the base of the PR and between bea4f2d and 014b536.

📒 Files selected for processing (4)
  • TypeWhisper.xcodeproj/project.pbxproj
  • TypeWhisperPluginSDK/Plugins/OpenAIPlugin/OpenAIPlugin.swift
  • TypeWhisperPluginSDK/Plugins/OpenAIPlugin/Tests/OpenAIPluginTests.swift
  • TypeWhisperPluginSDK/Plugins/OpenAIPlugin/manifest.json

Comment thread TypeWhisperPluginSDK/Plugins/OpenAIPlugin/OpenAIPlugin.swift Outdated
@SeoFood
SeoFood merged commit 2947284 into main Jul 29, 2026
13 checks passed
@SeoFood
SeoFood deleted the seofood/openai-plugin-1.3.1-finalization branch July 29, 2026 16:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: dictation Related to recording and transcription enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant