Skip to content

feat(voice): add Codex subscription billing to native GPT-Live - #108940

Open
TheSmokeDev wants to merge 4 commits into
NousResearch:mainfrom
TheSmokeDev:smokedev/native-live-followup
Open

TheSmokeDev wants to merge 4 commits into
NousResearch:mainfrom
TheSmokeDev:smokedev/native-live-followup

Conversation

@TheSmokeDev

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds an explicit ChatGPT/Codex subscription option to the native GPT-Live mode merged in #108137. Set voice.gpt_live.auth: subscription to use the Hermes host's existing Codex OAuth sign-in. API billing remains the default; subscription authentication or transport failures never select the API route or chained STT/TTS.

The server keeps the OAuth bearer and matching account identity private. The desktop uses the subscription WebRTC event format and returns Hermes replies through the corresponding context-append protocol. Hermes remains the agent, with the conversation's existing model, tools, memory and approvals.

Related Issue

Builds on the native GPT-Live mode merged in #108137.

Type of Change

  • New feature (non-breaking explicit billing option)

Changes Made

  • Reuse hermes_cli.auth_codex.resolve_codex_runtime_credentials; keep API and subscription model/voice settings separate.
  • Add the Codex WebRTC request/SDP response format, real call/delegation IDs, captured transcript fragments and 500-byte UTF-8 context appends. Do not follow authenticated redirects or expose provider error bodies.
  • Fetch authoritative voice status before opening an engine. Unknown or unavailable subscription status fails closed; resolved API mode retains its existing chained fallback.
  • Ignore repeated delegation IDs and label subscription transcript timing as local arrival so the existing five-minute context window still expires.
  • Document configuration and the subscription access requirement. No new dependency, plugin, tool, worker stack or login flow.

How to Test

  1. scripts/run_tests.sh tests/tools/test_voice_live_subscription.py tests/tui_gateway/test_voice_live_delegation.py -j 1 -q12 passed on Windows, Python 3.11.13.
  2. Desktop targeted regression: cd apps/desktop && npx vitest run src/lib/voice-live-subscription.test.ts --project ui. The same committed test file passed 3 cases locally with existing Vitest 4.1.10 and a temporary Node/module-alias configuration. It exercises the actual session/start-selection code with fake media and mocked network boundaries. The full desktop workspace configuration/build/typecheck was not run locally because this worktree has no installed workspace dependencies.
  3. Red-on-base proof: the 7 new Python cases called the API resolver on the unmodified base; the renderer cases found duplicate dispatch, no subscription event handling, and discarded auth metadata. The five-minute transcript case also failed before its arrival-time fix. Ruff and git diff --check pass.
  4. Manual provider/microphone test remains to be run: sign in through hermes auth, select subscription mode, start native Desktop voice, delegate a request to Hermes, hear the answer, mute/unmute and close. Repeat with explicit API mode. No live provider or microphone calls were made for this PR's local tests.

Checklist

  • Read the contribution guide and searched overlapping PRs.
  • Conventional, signed-off SmokeDev commit; one native Live feature.
  • Focused behavioral regressions and existing API/delegation regressions pass.
  • Updated operating docs and cli-config.yaml.example.
  • Considered cross-platform behavior: browser WebRTC and existing Python OAuth/HTTP dependencies; no shell or OS-specific runtime changes.
  • Full Python/desktop suites and live provider/microphone smoke test.

Protocol Evidence

Wire shapes were verified against OpenAI Codex commit c4017a87aacc7558002b7cb510025e967c1d765e: WebRTC call exchange, frameless events, context appends/history, and subscription model/headers.

Signed-off-by: SmokeDev degensmoke@gmail.com

Signed-off-by: SmokeDev <degensmoke@gmail.com>
@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/desktop Electron desktop app (apps/desktop/*) comp/tools Tool registry, model_tools, toolsets tool/tts Text-to-speech and transcription area/config Config system, migrations, profiles area/auth Authentication, OAuth, credential pools provider/openai OpenAI / Codex Responses API labels Sep 12, 2026
@kvnloo

kvnloo commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Comment assumes head 31df979d718bf76fa9baf05f344dc4a590edbb93; void if moved.

KEEP — Native GPT-Live Codex subscription billing

CHECK

One-click close / ordering

Landing-evidence

  • When merged, please reply with the merge commit SHA on main (not “completed”).

@whyyagswhy

Copy link
Copy Markdown
Contributor

Independent current-main and live-provider verification for head 31df979d718bf76fa9baf05f344dc4a590edbb93.

One rebase fix is required: after replaying this PR onto NousResearch/hermes-agent@784d5c3f9c, Desktop tsc --noEmit fails because notifyError now requires its localized fallback/title argument. I opened the one-line fix against the author's branch: TheSmokeDev#17

After that fix:

  • current-main TypeScript, focused ESLint, Windows-footgun check, compatibility-pointer check, and full Desktop production build pass;
  • 206 related Python web/auth/voice tests pass;
  • 8 focused current-main Desktop tests pass;
  • current OpenAI Codex main (4701aa4b4239c70063ab6f2fcb835324f9c109f4) still matches this PR's gpt-live-1-codex, /backend-api/codex/realtime/calls, quicksilver=v2, AVAS, frameless events, and 500-byte context-append contracts.

I also ran the missing live subscription canary with an existing Codex OAuth sign-in, without refreshing or changing the auth store:

  1. Real Chromium negotiated a WebRTC answer and opened oai-events.
  2. Synthesized speech produced 56 characters of input_transcript.added fragments, a provider-issued delegation.created, output_transcript.added, and clean session.closed.
  3. The returned renderer payload contained no token, account, or authorization fields.

This closes the PR body's manual provider/protocol uncertainty for the subscription handshake, transcription, and delegation legs. It does not claim a human microphone/speaker UX check.

Signed-off-by: SmokeDev <degensmoke@gmail.com>
@l0cka

l0cka commented Sep 18, 2026

Copy link
Copy Markdown

Tested this feature replayed onto 77ecc72: 331 related Python tests passed (23 skipped), 9 focused Desktop tests passed, and Desktop typechecks/production build passed. Found a subscription context-expiry edge case during silence; submitted the minimal two-file fix and red/green regression to your feature branch: TheSmokeDev#18 . Two broader voice-prefs tests also fail on unmodified main and were left untouched. No new live-provider or human mic validation of this PR is claimed.

@alt-glitch alt-glitch added the comp/cli CLI entry point, hermes_cli/, setup wizard label Sep 18, 2026
Co-authored-by: Daniel Alkurdi <13148507+l0cka@users.noreply.github.com>
@TheSmokeDev

Copy link
Copy Markdown
Contributor Author

@l0cka merged via TheSmokeDev#18 — head is now 9a3f694. Clean find: arrival-clocked fragments never aged out during silence, so the window anchor froze. Thanks for shipping the red/green regression with the fix.

@alt-glitch alt-glitch added sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades labels Sep 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/auth Authentication, OAuth, credential pools area/config Config system, migrations, profiles comp/cli CLI entry point, hermes_cli/, setup wizard comp/desktop Electron desktop app (apps/desktop/*) comp/tools Tool registry, model_tools, toolsets P3 Low — cosmetic, nice to have provider/openai OpenAI / Codex Responses API sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data tool/tts Text-to-speech and transcription type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants