Skip to content

Inject host environment via transport hints#23779

Merged
noanflaherty merged 3 commits into
mainfrom
noanflaherty/host-env-transport-hints
Apr 6, 2026
Merged

Inject host environment via transport hints#23779
noanflaherty merged 3 commits into
mainfrom
noanflaherty/host-env-transport-hints

Conversation

@noanflaherty
Copy link
Copy Markdown
Contributor

@noanflaherty noanflaherty commented Apr 6, 2026

Summary

When the assistant runs on managed platform and uses host_bash, it guesses the user's home directory from their display name instead of knowing the actual macOS username. This adds hostHomeDir and hostUsername to transport metadata (macOS only, via discriminated union), removes iOS from host proxy setup, and injects the interface ID and macOS host environment into transport hints every turn.

PRs merged into feature branch

Part of plan: host-env-transport-hints.md


Open with Devin

@noanflaherty noanflaherty self-assigned this Apr 6, 2026
@noanflaherty
Copy link
Copy Markdown
Contributor Author

Self-review is starting. Results will be posted here when complete.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dbffd9a83b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

// Set the proxy BEFORE updateClient so updateClient's call to
// hostBashProxy.updateSender targets the correct (new) proxy.
if (sourceInterface === "macos" || sourceInterface === "ios") {
if (sourceInterface === "macos") {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Clear stale host proxies for iOS turns during active runs

After removing iOS from proxy setup, this branch no longer reinitializes host proxies for iOS, but it also does not clear existing proxies when a message arrives while conversation.isProcessing() is true. In that case, a conversation that previously ran on macOS can retain stale host proxies, and queued iOS turns still re-enable proxy availability in assistant/src/daemon/conversation-process.ts (sourceInterface === "macos" || "ios"), allowing host_bash/computer-use to route through the old macOS proxy instead of the intended non-host path. Please clear proxies for iOS even during active runs (or stop restoring proxy availability for iOS in dequeue logic) to keep the new interface boundary consistent.

Useful? React with 👍 / 👎.

@noanflaherty noanflaherty merged commit 5474e61 into main Apr 6, 2026
13 checks passed
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

⚠️ 1 issue in files not directly in the diff

⚠️ Incomplete iOS removal: queue drain path still restores proxy availability and preactivates computer-use for iOS (assistant/src/daemon/conversation-process.ts:304)

The PR consistently removed || sourceInterface === "ios" from proxy creation in conversations.ts:304, server.ts:1099, and conversation-routes.ts:1120, but missed the same pattern in assistant/src/daemon/conversation-process.ts:304. When a queued message from an iOS client is drained, the code still calls conversation.restoreProxyAvailability() and conversation.addPreactivatedSkillId("computer-use"). Since proxies are no longer created for iOS, restoreProxyAvailability() is a harmless no-op, but addPreactivatedSkillId("computer-use") will load the computer-use skill for iOS conversations — making computer-use tools available to the assistant even though no host proxy exists to execute them. The comment on line 298 also still references "ios".

View 3 additional findings in Devin Review.

Open in Devin Review

@noanflaherty noanflaherty deleted the noanflaherty/host-env-transport-hints branch April 6, 2026 16:20
noanflaherty added a commit that referenced this pull request Apr 6, 2026
* refactor: discriminated union for transport metadata, remove iOS proxy setup (#23776)

* feat: inject interface ID and macOS host environment into transport hints (#23777)

* feat: send hostHomeDir and hostUsername from macOS client (#23778)
noanflaherty added a commit that referenced this pull request Apr 6, 2026
* Increase teleport import timeout from 2 to 5 minutes (#23749)

* increase teleport import timeout from 2 to 5 minutes

* fix: update platform import timeout error message to say 5 minutes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Update billing tab copy: referral subtitle, remove earning cap note, move credit info to card subtitle (#23751)

* fix(macos): always collapse thinking blocks by default (#23750)

Thinking blocks were auto-expanding during streaming, showing a wall of
text. Remove the auto-expand logic so blocks always start collapsed.
Users can still manually expand them. The header already shows
"Thinking..." vs "Thought process" as a streaming indicator.

Closes LUM-729

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ashlee@vellum.ai <ashlee@vellum.ai>

* [LUM-684/LUM-726] Fix dictation crash: pass nil format to installTap (#23754)

* Fix dictation crash: pass nil format to installTap, consolidate audio engine calls

Pass nil for the format parameter in AVAudioNode.installTap(onBus:bufferSize:format:block:)
so AVAudioEngine uses its own internal hardware format, which is always self-consistent.
This prevents NSInternalInconsistencyException crashes caused by format.sampleRate != hwFormat.sampleRate
when the cached format from outputFormat(forBus:) diverges from the engine's internal hardware
format after audio route changes (Bluetooth, USB mic, AirPods mode switch).

AudioEngineController.swift:
- installTapAndStart() now passes nil instead of explicit format to installTap
- Removed 6 now-unused methods: inputNodeFormat(), installTap(bufferSize:format:block:),
  removeTap(), prepare(), start(), prepareAndStart()

OpenAIVoiceService.swift:
- startRecording(): replaced separate inputNodeFormat/installTap/prepare/start chain
  with single installTapAndStart() call
- startBargeInMonitor(): same migration to installTapAndStart()
- Removed error-path removeTap() call (handled internally by installTapAndStart)

Resolves: LUM-684, LUM-726
Co-Authored-By: tkheyfets <timur@vellum.ai>

* fix: use explicit block: parameter in guard statements for installTapAndStart

Swift doesn't support trailing closure syntax with guard statements,
causing compilation errors. Use explicit block: parameter label instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: tkheyfets <timur@vellum.ai>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: replace contrast buttons with primary style (#23753)

Remove all production usages of .contrast button style in favor of .primary.
Fixes white-on-white button visibility issues in chat composer.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Inject host environment via transport hints (#23779)

* refactor: discriminated union for transport metadata, remove iOS proxy setup (#23776)

* feat: inject interface ID and macOS host environment into transport hints (#23777)

* feat: send hostHomeDir and hostUsername from macOS client (#23778)

* fix: remove iOS from proxy restoration in conversation-process.ts (#23782)

---------

Co-authored-by: Carson Shaar <carson.s.shaar@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ashlee@vellum.ai <ashlee@vellum.ai>
Co-authored-by: tkheyfets <timur@vellum.ai>
Co-authored-by: Tirman Sidhu <tirmansidhu@gmail.com>
dvargasfuertes pushed a commit that referenced this pull request Apr 6, 2026
* Increase teleport import timeout from 2 to 5 minutes (#23749)

* increase teleport import timeout from 2 to 5 minutes

* fix: update platform import timeout error message to say 5 minutes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Update billing tab copy: referral subtitle, remove earning cap note, move credit info to card subtitle (#23751)

* fix(macos): always collapse thinking blocks by default (#23750)

Thinking blocks were auto-expanding during streaming, showing a wall of
text. Remove the auto-expand logic so blocks always start collapsed.
Users can still manually expand them. The header already shows
"Thinking..." vs "Thought process" as a streaming indicator.

Closes LUM-729

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ashlee@vellum.ai <ashlee@vellum.ai>

* [LUM-684/LUM-726] Fix dictation crash: pass nil format to installTap (#23754)

* Fix dictation crash: pass nil format to installTap, consolidate audio engine calls

Pass nil for the format parameter in AVAudioNode.installTap(onBus:bufferSize:format:block:)
so AVAudioEngine uses its own internal hardware format, which is always self-consistent.
This prevents NSInternalInconsistencyException crashes caused by format.sampleRate != hwFormat.sampleRate
when the cached format from outputFormat(forBus:) diverges from the engine's internal hardware
format after audio route changes (Bluetooth, USB mic, AirPods mode switch).

AudioEngineController.swift:
- installTapAndStart() now passes nil instead of explicit format to installTap
- Removed 6 now-unused methods: inputNodeFormat(), installTap(bufferSize:format:block:),
  removeTap(), prepare(), start(), prepareAndStart()

OpenAIVoiceService.swift:
- startRecording(): replaced separate inputNodeFormat/installTap/prepare/start chain
  with single installTapAndStart() call
- startBargeInMonitor(): same migration to installTapAndStart()
- Removed error-path removeTap() call (handled internally by installTapAndStart)

Resolves: LUM-684, LUM-726
Co-Authored-By: tkheyfets <timur@vellum.ai>

* fix: use explicit block: parameter in guard statements for installTapAndStart

Swift doesn't support trailing closure syntax with guard statements,
causing compilation errors. Use explicit block: parameter label instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: tkheyfets <timur@vellum.ai>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: replace contrast buttons with primary style (#23753)

Remove all production usages of .contrast button style in favor of .primary.
Fixes white-on-white button visibility issues in chat composer.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Inject host environment via transport hints (#23779)

* refactor: discriminated union for transport metadata, remove iOS proxy setup (#23776)

* feat: inject interface ID and macOS host environment into transport hints (#23777)

* feat: send hostHomeDir and hostUsername from macOS client (#23778)

* fix: remove iOS from proxy restoration in conversation-process.ts (#23782)

---------

Co-authored-by: Carson Shaar <carson.s.shaar@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ashlee@vellum.ai <ashlee@vellum.ai>
Co-authored-by: tkheyfets <timur@vellum.ai>
Co-authored-by: Tirman Sidhu <tirmansidhu@gmail.com>
dvargasfuertes pushed a commit that referenced this pull request Apr 6, 2026
* Release v0.6.1

* Cherry-pick fixes for v0.6.1 (#23785)

* Increase teleport import timeout from 2 to 5 minutes (#23749)

* increase teleport import timeout from 2 to 5 minutes

* fix: update platform import timeout error message to say 5 minutes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Update billing tab copy: referral subtitle, remove earning cap note, move credit info to card subtitle (#23751)

* fix(macos): always collapse thinking blocks by default (#23750)

Thinking blocks were auto-expanding during streaming, showing a wall of
text. Remove the auto-expand logic so blocks always start collapsed.
Users can still manually expand them. The header already shows
"Thinking..." vs "Thought process" as a streaming indicator.

Closes LUM-729

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ashlee@vellum.ai <ashlee@vellum.ai>

* [LUM-684/LUM-726] Fix dictation crash: pass nil format to installTap (#23754)

* Fix dictation crash: pass nil format to installTap, consolidate audio engine calls

Pass nil for the format parameter in AVAudioNode.installTap(onBus:bufferSize:format:block:)
so AVAudioEngine uses its own internal hardware format, which is always self-consistent.
This prevents NSInternalInconsistencyException crashes caused by format.sampleRate != hwFormat.sampleRate
when the cached format from outputFormat(forBus:) diverges from the engine's internal hardware
format after audio route changes (Bluetooth, USB mic, AirPods mode switch).

AudioEngineController.swift:
- installTapAndStart() now passes nil instead of explicit format to installTap
- Removed 6 now-unused methods: inputNodeFormat(), installTap(bufferSize:format:block:),
  removeTap(), prepare(), start(), prepareAndStart()

OpenAIVoiceService.swift:
- startRecording(): replaced separate inputNodeFormat/installTap/prepare/start chain
  with single installTapAndStart() call
- startBargeInMonitor(): same migration to installTapAndStart()
- Removed error-path removeTap() call (handled internally by installTapAndStart)

Resolves: LUM-684, LUM-726
Co-Authored-By: tkheyfets <timur@vellum.ai>

* fix: use explicit block: parameter in guard statements for installTapAndStart

Swift doesn't support trailing closure syntax with guard statements,
causing compilation errors. Use explicit block: parameter label instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: tkheyfets <timur@vellum.ai>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: replace contrast buttons with primary style (#23753)

Remove all production usages of .contrast button style in favor of .primary.
Fixes white-on-white button visibility issues in chat composer.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Inject host environment via transport hints (#23779)

* refactor: discriminated union for transport metadata, remove iOS proxy setup (#23776)

* feat: inject interface ID and macOS host environment into transport hints (#23777)

* feat: send hostHomeDir and hostUsername from macOS client (#23778)

* fix: remove iOS from proxy restoration in conversation-process.ts (#23782)

---------

Co-authored-by: Carson Shaar <carson.s.shaar@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ashlee@vellum.ai <ashlee@vellum.ai>
Co-authored-by: tkheyfets <timur@vellum.ai>
Co-authored-by: Tirman Sidhu <tirmansidhu@gmail.com>

* [skip ci] Cherry-pick fixes for v0.6.1 (#23820)

* revert: disable Teleport feature flag by default (#23744) (#23815)

* fix: replace auxWhite-on-primaryBase with VButton across the app (#23802)

* fix: use VButton for inline surface action buttons

Replace raw Button with manual color functions in InlineSurfaceRouter
with the design system VButton component. The manual buttonForeground
used VColor.auxWhite (always #FFFFFF) against VColor.primaryBase which
resolves to #FDFDFC in dark mode, producing invisible white-on-white
text.

Closes LUM-730

Co-Authored-By: ashlee@vellum.ai <ashlee@vellum.ai>

* fix: replace auxWhite-on-primaryBase with VButton in additional locations

FileUploadSurfaceView: Upload/Cancel buttons used raw Button with
VColor.auxWhite on VColor.primaryBase — white-on-white in dark mode.
Replaced with VButton(.primary) and VButton(.outlined).

JITPermissionView: Permission buttons used the same auxWhite pattern.
Replaced with VButton(.primary/.outlined, isFullWidth: true).

ImproveExperienceStepView: ToS checkbox checkmark used auxWhite on
primaryBase fill. Changed to VColor.contentInset which adapts per
color scheme.

ChatGallerySection: Gallery demo of surface action pills mirrored
the old buggy pattern. Updated to use VButton so the gallery
accurately represents production rendering.

Co-Authored-By: ashlee@vellum.ai <ashlee@vellum.ai>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ashlee@vellum.ai <ashlee@vellum.ai>

* Make dictation engine start non-blocking with audio route resilience (#23811)

* Make dictation engine start non-blocking and improve audio resilience

- Add installTapAndStartAsync to AudioEngineController for non-blocking
  engine start using Swift concurrency (withCheckedContinuation)
- Extract installTapAndStartImpl to share logic between sync/async paths
- Listen for AVAudioEngineConfigurationChange to re-prewarm inputNode
  after Bluetooth device connect/disconnect and AirPods mode switches
- Restructure VoiceInputManager.beginRecording() to show recording UI
  and play activation chime immediately, then start engine async via Task
- Move DictationContextCapture off the critical path: engine starts
  concurrently on its audio queue while context capture runs on main
- Add SFSpeechRecognizer transient unavailability retry (recreate if
  isAvailable returns false after sleep/wake or heavy use)
- Handle edge case where PTT is released before async engine start
  completes (stopRecordingForDictation cleans up directly)

Co-Authored-By: tkheyfets <timur@vellum.ai>

* Tear down engine when async startup outlives recording session

When PTT is released before installTapAndStartAsync completes, the
isRecording guard now stops and removes the tap if the engine started
successfully, preventing the mic path from staying alive with no
active recording session.

Co-Authored-By: tkheyfets <timur@vellum.ai>

* Add recording generation token and gate context capture on start success

Co-Authored-By: tkheyfets <timur@vellum.ai>

* Guard stale teardown against active sessions and gate rewarm on mic auth

Co-Authored-By: tkheyfets <timur@vellum.ai>

* Move context capture to Task.detached to avoid blocking main actor

Co-Authored-By: tkheyfets <timur@vellum.ai>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: tkheyfets <timur@vellum.ai>

* [LUM-681] Fix audio tap format mismatch by resetting engine before installTap (#23766)

After audio-route changes (Bluetooth, USB mic, AirPods mode switch), the
format cached inside AVAudioInputNode diverges from the engine's actual
hardware format. Both outputFormat(forBus:) and a nil format argument to
installTap resolve to this stale value, causing:

  'Failed to create tap due to format mismatch,
   <AVAudioFormat: 2 ch, 44100 Hz, Float32, deinterleaved>'

Fix: call audioEngine.reset() before re-querying the format, then pass it
explicitly to installTap. This forces the engine to discard its cached graph
state and re-read the hardware, so the tap, node, and engine all agree.

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: tkheyfets <timur@vellum.ai>

* fix: pass transport hints through HTTP message endpoint for managed-mode conversations (#23824)

* fix: pass transport metadata through POST /v1/messages to enable host environment hints

The HTTP message handler auto-creates conversations without transport
metadata, so applyTransportMetadata() returns early and host environment
hints (hostHomeDir, hostUsername) are never injected into the LLM context.
This causes the assistant to hallucinate the user's home directory path
from their display name instead of using the actual macOS username.

Thread transport metadata from the message request body through
SendMessageDeps.getOrCreateConversation() to the daemon, and send
hostHomeDir/hostUsername from the macOS client in every message request.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: replace dynamic imports with static type imports

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ashlee@vellum.ai <ashlee@vellum.ai>
Co-authored-by: tkheyfets <timur@vellum.ai>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: reset non-version-bump files to match main

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Noa Flaherty <noa@vellum.ai>
Co-authored-by: Carson Shaar <carson.s.shaar@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ashlee@vellum.ai <ashlee@vellum.ai>
Co-authored-by: tkheyfets <timur@vellum.ai>
Co-authored-by: Tirman Sidhu <tirmansidhu@gmail.com>
Co-authored-by: David Vargas Fuertes <vargasvellum@Davids-MacBook-Pro.local>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant