Skip to content

macos(settings): make per-task override sheet editable with provider/model pickers#26136

Merged
siddseethepalli merged 3 commits into
siddseethepalli/unify-llm-callsitesfrom
run-plan/llm-callsites/pr-23
Apr 16, 2026
Merged

macos(settings): make per-task override sheet editable with provider/model pickers#26136
siddseethepalli merged 3 commits into
siddseethepalli/unify-llm-callsitesfrom
run-plan/llm-callsites/pr-23

Conversation

@siddseethepalli
Copy link
Copy Markdown
Contributor

@siddseethepalli siddseethepalli commented Apr 16, 2026

Summary

  • New CallSiteOverrideRow SwiftUI row with 'Override default' toggle, provider/model pickers, save + reset actions.
  • CallSiteOverridesSheet now lists every catalog entry (not just overridden ones), grouped by domain. Header gains 'Save All' (when there are unsaved drafts) and 'Reset All' (destructive, behind a confirmation dialog).
  • Inline validation: Save disabled when provider is set but model is empty.

Part of plan: unify-llm-callsites.md (PR 23 of 24)


Open with Devin

chatgpt-codex-connector[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

… active default provider

Codex flagged two P1s:
- syncDraftsFromStore compared drafts against the NEW persisted value to
  decide 'touched', so external store updates were treated as user edits
  and got overwritten by Save All. Track the previously-persisted value
  in lastSyncedFromStore and consider a row touched only when the draft
  differs from that baseline.
- Toggling 'Override default' on initialized provider from
  providerIds.first instead of the user's actual default provider, which
  could pin the wrong provider on save. Pass the user's default provider
  into CallSiteOverrideRow and seed from it.
@siddseethepalli
Copy link
Copy Markdown
Contributor Author

@codex review

Latest commit: 4a3f0fcfe3e50e2ffadb760d0b4df003a005b23d

@siddseethepalli
Copy link
Copy Markdown
Contributor Author

@devin review

Latest commit: 4a3f0fcfe3e50e2ffadb760d0b4df003a005b23d

…veAll/resetAll

Devin flagged that saveAll() and resetAll() were passing all-nil entries
to setCallSiteOverrides, which routed them through the field-level null
path (provider/model/profile = null). That left advanced leaves
(maxTokens, effort, temperature, contextWindow) untouched on the daemon.

Fix:
- saveAll(): filter to entries with hasOverride == true; toggled-off rows
  fall through to the entry-level null path.
- resetAll(): pass an empty list so every catalog entry hits the
  entry-level null path.
@siddseethepalli
Copy link
Copy Markdown
Contributor Author

@codex review

Latest commit: d2fa8acffb4fc708064d63f79980aa5bb8d5b6c5

@siddseethepalli
Copy link
Copy Markdown
Contributor Author

@devin review

Latest commit: d2fa8acffb4fc708064d63f79980aa5bb8d5b6c5

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d2fa8acffb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +195 to +196
.padding(.leading, VSpacing.md)
.padding(.top, VSpacing.xs)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Merge consecutive padding modifiers

This row currently stacks two .padding modifiers, which violates the explicit performance rule in clients/macos/AGENTS.md (“Never stack consecutive .padding() modifiers”). In this settings list, each extra modifier adds another layout wrapper and increases layout traversal cost, so this should be collapsed into a single .padding(EdgeInsets(...)) call to stay within the project’s required SwiftUI layout constraints.

Useful? React with 👍 / 👎.

@siddseethepalli siddseethepalli merged commit 281d2f6 into siddseethepalli/unify-llm-callsites Apr 16, 2026
6 checks passed
@siddseethepalli siddseethepalli deleted the run-plan/llm-callsites/pr-23 branch April 16, 2026 22:46
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

View 5 additional findings in Devin Review.

Open in Devin Review

Comment on lines +252 to +268
private func save(id: String) {
guard let draft = drafts[id] else { return }
if draft.hasOverride {
store.setCallSiteOverride(
id,
provider: draft.provider,
model: draft.model,
profile: draft.profile
)
} else {
store.clearCallSiteOverride(id)
}
// The draft is now the new persisted state — bump the baseline so
// any subsequent `onChange` from the store doesn't see a stale
// baseline and re-flag the row as touched.
lastSyncedFromStore[id] = drafts[id]
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Per-row Save silently retains stale daemon-side fields that the draft has nil'd

save(id:) calls store.setCallSiteOverride(id, provider:, model:, profile:) which, by its documented contract (SettingsStore.swift:3083-3087), omits nil arguments from the patch — it does not clear them. This means if the user toggles an override OFF (clearing provider/model/profile locally) then back ON (seeding provider + model, profile stays nil), the per-row Save button only patches the non-nil provider and model. Any previously-set profile (or other field) persists on the daemon.

After the optimistic update and onChangesyncDraftsFromStore(), the stale profile reappears in the draft because the optimistic cache never cleared it and the baseline matches the draft (both nil for profile), so the sync treats the row as "untouched" and overwrites the draft with the persisted value (which still has the old profile).

In contrast, saveAll() uses store.setCallSiteOverrides() which emits explicit NSNull() for nil fields (SettingsStore.swift:3197-3200), properly clearing them. This creates an inconsistency: Save All correctly clears stale fields, but per-row Save does not.

Prompt for agents
The save(id:) function at CallSiteOverridesSheet.swift:252-268 calls store.setCallSiteOverride() which only patches non-nil fields — it cannot clear fields. This means if the draft has profile=nil but the daemon previously had a profile set, the per-row Save leaves the stale profile in place. After the optimistic update syncs back, the stale value silently reappears in the draft.

The fix should ensure per-row Save clears fields that the draft has nil'd. Two approaches:

1. Change save(id:) to first call store.clearCallSiteOverride(id) and then store.setCallSiteOverride(id, ...) — this is a two-step round trip but guarantees a clean slate.

2. Add a new store method (e.g. replaceCallSiteOverride) that emits NSNull() for nil fields, similar to how setCallSiteOverrides (batch) already works at SettingsStore.swift:3197-3200. The save(id:) function would call this instead of setCallSiteOverride.

Approach 2 is cleaner because it's a single round trip and mirrors the batch save semantics.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

@siddseethepalli
Copy link
Copy Markdown
Contributor Author

Follow-up fixes shipped in #26271:

  • deepMergeOverwrite: null on scalar targets assigns null (nullable config fields); null on object targets deletes (call-site clearing)
  • Override confirmation dialog gated on actual provider ID change, not mode-only toggles
  • Per-row Save uses replaceCallSiteOverride (clear-then-set) to remove stale daemon-side leaves
  • Stacked padding merged into single EdgeInsets

siddseethepalli added a commit that referenced this pull request Apr 18, 2026
…es} (#26159)

* config(llm): add unified llm schema with call-site enum and profile refines (#26089)

* config(llm): add unified llm schema with call-site enum and profile refines

* fix(llm-schema): replace deepPartialObject helper with explicit .partial().extend()

Zod 4's readonly shape typing tripped TS2542 in the LSP for the generic walker.
Inline the one-level expansion for ContextWindowSchema and switch the superRefine
issue code to the string literal (Zod 4 deprecated ZodIssueCode).

* config(llm): add resolveCallSiteConfig resolver with deep merge (#26094)

* config(llm): add resolveCallSiteConfig resolver with deep merge

* fix(llm-resolver): deep-clone nested objects so resolved configs are isolated snapshots

Codex flagged that the merge helper aliased nested objects from llm.default
when no override touched them, so a caller mutating the returned config
would silently corrupt the source. Recurse into plain-object sources
unconditionally and add a regression test.

* config(llm): add llm field to AssistantConfigSchema (no behavior change) (#26095)

* config(llm): add llm field to AssistantConfigSchema (no behavior change)

* fix(llm-schema): add field-level defaults so partial llm configs don't trigger full config reset

Codex flagged that requiring all LLMConfigBase fields meant the loader's
leaf-deletion recovery couldn't repair partial/invalid llm blocks — falling
through to cloneDefaultConfig() and discarding the user's other valid
settings. Add .default(...) to every leaf so LLMSchema.parse({}) returns a
fully-defaulted object, matching the pattern used by sibling config schemas.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* providers: accept callSite in per-call config; resolve via resolveCallSiteConfig (#26102)

* workspace: migrate scattered LLM config keys into unified llm structure (#26101)

* workspace: migrate scattered LLM config keys into unified llm structure

* fix(migration): preserve existing llm subtree; map notification intent to both call sites

Codex flagged two issues:
- The migration assignment replaced config.llm wholesale, destroying any
  pre-existing llm.callSites/profiles when llm.default was absent. Now
  merges into existing config.llm, preserving non-conflicting entries.
- notifications.decisionModelIntent drives both notification classification
  and preference extraction, but the migration only seeded
  notificationDecision. Now seeds both call sites.

* memory: route extraction/consolidation/retrieval through call-site IDs (#26106)

* memory: route narrative/pattern/summarization/starters through call-site IDs (#26107)

* notifications: route decision and preference extraction through call-site IDs (#26109)

* calls+watcher: route guardian copy and watch handlers through call-site IDs (#26105)

* utility: route classifier and analyzer LLM calls through call-site IDs (#26111)

* macos(settings): migrate InferenceServiceCard reads/writes to llm.default.* (#26113)

* workspace+conversation: route commit message and title through call-site IDs (#26112)

* ui: route identity intro and empty-state greeting through call-site IDs (#26108)

* daemon: thread callSite through processMessage options and adapter callbacks (#26115)

* daemon: thread callSite through processMessage options and adapter callbacks

* fix(callsite-threading): complete interface contract and server.ts symmetry

Devin flagged two gaps in PR #26115:
- ProcessConversationContext interface missing callSite in its
  runAgentLoop options type (works via structural typing but contract
  was incomplete; mocks would silently drop the field).
- DaemonServer.persistAndProcessMessage didn't thread callSite to
  conversation.runAgentLoop, while DaemonServer.processMessage did.
  Aligned.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(callsite): don't default unspecified callers to 'mainAgent'

Codex flagged that defaulting to mainAgent for every turn routes them
through the new RetryProvider call-site resolver, which reads from
llm.default — but config-model.setModel still writes to services.inference
without syncing llm.default. Result: stale/incompatible model IDs after a
model switch.

Defer the cutover. agent-loop turns now keep using the legacy modelIntent
path (turnCallSite = options?.callSite, no fallback). PRs 7-11 still
explicitly pass callSite and route through the new resolver as intended.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* heartbeat: pass callSite: 'heartbeatAgent' instead of speed kwarg (#26125)

* filing: pass callSite: 'filingAgent' instead of speed kwarg (#26124)

* runtime/analyze-conversation: route through callSite: 'analyzeConversation' (#26126)

* subagent: pass callSite: 'subagentSpawn' when spawning isolated agents (#26122)

* calls: route the call agent loop through callSite: 'callAgent' (#26123)

* macos(settings): add SettingsStore APIs for per-call-site overrides (#26128)

* macos(settings): add SettingsStore APIs for per-call-site overrides

* fix(callsite-overrides): harden setCallSiteOverrides against dup-id crash and batch divergence

Devin and Codex flagged two issues:
- Dictionary(uniqueKeysWithValues:) crashes if callers pass duplicate
  CallSiteOverride.id values (external input — must be tolerant). Switch
  to Dictionary(_:uniquingKeysWith:) with last-write-wins.
- Batch updates locally cleared entries omitted from the input but only
  PATCHed entries that were present, so omitted entries appeared cleared
  in the UI but reappeared on next sync. Now the PATCH payload includes
  NSNull clears for every catalog entry not in the batch, aligning remote
  with local.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(callsite-overrides): null entire entry on clear so non-UI leaves get cleared too

Codex P2 (PR #26128 cycle 2): clearCallSiteOverride only nulled
provider/model/profile, but call-site config supports additional leaves
(maxTokens, effort, speed, thinking, contextWindow). If those were set
via manual edits, the UI would report cleared while the daemon kept
applying hidden overrides.

Switch the PATCH payload from { provider: null, model: null, profile: null }
to a single null on the entry itself. The Zod fragment treats null as
absent, so the resolver falls back to llm.default. Same fix applies to the
omitted-catalog-entry clears in setCallSiteOverrides batch.

Tests updated to assert the new shape.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* macos(settings): confirm default-provider switch when call-site overrides exist (#26133)

* macos(settings): show 'N call-site overrides' badge with read-only list sheet (#26135)

* macos(settings): show 'N call-site overrides' badge with read-only list sheet

* fix(comments): drop PR-number breadcrumbs in callsite override files

Devin flagged that comments referencing PR 22/23/24 violate clients/AGENTS.md
'Comment Quality' rule (no breadcrumbs). Replaced with timeless descriptions
of code intent.

* macos(settings): make per-task override sheet editable with provider/model pickers (#26136)

* macos(settings): make per-task override sheet editable with provider/model pickers

* fix(callsite-sheet): preserve external updates and seed override from active default provider

Codex flagged two P1s:
- syncDraftsFromStore compared drafts against the NEW persisted value to
  decide 'touched', so external store updates were treated as user edits
  and got overwritten by Save All. Track the previously-persisted value
  in lastSyncedFromStore and consider a row touched only when the draft
  differs from that baseline.
- Toggling 'Override default' on initialized provider from
  providerIds.first instead of the user's actual default provider, which
  could pin the wrong provider on save. Pass the user's default provider
  into CallSiteOverrideRow and seed from it.

* fix(callsite-sheet): use entry-level null path for cleared rows in saveAll/resetAll

Devin flagged that saveAll() and resetAll() were passing all-nil entries
to setCallSiteOverrides, which routed them through the field-level null
path (provider/model/profile = null). That left advanced leaves
(maxTokens, effort, temperature, contextWindow) untouched on the daemon.

Fix:
- saveAll(): filter to entries with hasOverride == true; toggled-off rows
  fall through to the entry-level null path.
- resetAll(): pass an empty list so every catalog entry hits the
  entry-level null path.

* config(llm): remove deprecated scattered LLM keys (#26140)

* fix(config-loader): treat JSON null as key deletion in deepMergeOverwrite (#26153)

* fix(agent-loop): default user-initiated turns to callSite: 'mainAgent' (#26154)

* fix(meet-join): migrate consent-monitor + session-manager to callSite contract (#26155)

* fix(macos): atomic provider+model save via single PATCH (#26156)

* fix(cleanup): remove dead code, refresh comments, add migration test, update docs (#26157)

* fix(r2): catalog test count, skill self-knowledge doc, AGENTS.md, loader docstring (#26158)

* fix(llm-callsite): refresh stale docstring, restore overflow budget, restore SettingsStore fallback (#26252)

* fix(llm-callsite): route provider transport and field precedence through callSite (#26254)

* fix(llm-callsite): pass CI + address subagent/thinking/temperature review comments (#26258)

* test(extension-id-guard): allow CWS URL matches; mirrors main PR #26263 (#26270)

* fix(llm-callsite): UI override state divergence, null-as-delete, migration gaps (#26271)

* Fix Chrome extension allowlist ID and clarify README dev setup (#26259)

Update the canonical allowlist to use the correct published CWS
extension ID (hphbdmpffeigpcdjkckleobjmhhokpne). Restructure the
Chrome extension README to clearly explain the allowlist merge
strategy, separate the macOS app (automatic) path from the manual
native messaging setup, and show how dev + prod extensions work
side-by-side.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(clients): enable non-contiguous glyph layout for NSTextView-backed code views (#26242)

TextKit 1 defaults NSLayoutManager.allowsNonContiguousLayout to false,
which forces full-document glyph layout from character 0 on the main
thread whenever a glyph range is queried. Attaching an NSTextView to
its scroll view (setDocumentView: -> _setSuperview: ->
setNeedsDisplayInRect: -> _glyphRangeForBoundingRect:) triggers that
query during makeNSView, producing multi-second hangs on large code
blocks.

Opt into non-contiguous layout on every TextKit 1 stack we build via
NSViewRepresentable so glyph generation is confined to the requested
bounding rect.

Also replace NSLayoutManager.ensureLayout(for:) in the code-view
sizeThatFits paths with direct lineCount * fixedLineHeight math: the
text container is unbounded horizontally (no wrapping) and paragraph
style pins minimumLineHeight == maximumLineHeight, so the geometry is
exact and avoids a second O(glyph count) main-thread path.

Fixes VELLUM-ASSISTANT-MACOS-J2.

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ashlee@vellum.ai <ashlee@vellum.ai>

* fix(contacts): show Assistant badge for assistant-type contacts (LUM-1009) (#26239)

* fix(contacts): show Assistant badge for assistant-type contacts (LUM-1009)

* Move role/contactType derivation onto Kind for valid initializer

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(llm-callsite): UI override state divergence, null-as-delete, migration gaps

- deepMergeOverwrite: null on scalar/null targets assigns null (preserves
  nullable config fields like activeHoursStart); null on object targets
  still deletes (call-site clearing). Fixes regression where PATCH with
  null for nullable fields was deleted then re-defaulted.
- InferenceServiceCard: override confirmation dialog only fires when the
  resolved provider ID actually changes, not on mode-only toggles where
  both old and new resolve to the same provider.
- CallSiteOverridesSheet: per-row Save uses replaceCallSiteOverride
  (clear-then-set) so stale daemon-side leaves are removed. The
  partial-update setCallSiteOverride would retain fields the draft nil'd.
- CallSiteOverrideRow: merge consecutive .padding modifiers into single
  EdgeInsets call per macOS AGENTS.md layout rule.
- SettingsStore: add replaceCallSiteOverride for full-entry replacement.

---------

Co-authored-by: Noa Flaherty <noa@vellum.ai>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ashlee@vellum.ai <ashlee@vellum.ai>

* fix(llm-callsite): seed latency-optimized defaults and fix guardian provider routing (#26275)

* fix(meet-bot): address review feedback — Docker build, scraper races, audio capture, storage writer (#26264)

* fix(meet): chat concurrency, dispose teardown, and wake adapter fidelity (#26265)

* fix: heartbeat dual-emit, analysis dedup, test hermiticity, credential executor discovery (#26266)

* fix: model default fallback, empty-response nudge scan (#26268)

- Update FALLBACK_DEFAULT_MODEL to claude-opus-4-7 + test
- Fix resolveModel to check Anthropic catalog (not just current default)
  so stale persisted defaults (e.g. claude-opus-4-6) don't get sent
  to non-Anthropic providers
- Fix priorAssistantHadVisibleText backward scan to check ALL prior
  assistant messages, not just the most recent one

Addresses review feedback from PRs #26247, #26164.

* fix(meet): TTS stream races, barge-in tracking, ffmpeg error classification (#26267)

* Fix extension-id-sync-guard test after canonical ID update (#26263)

The guard test asserts that canonical extension IDs appear only in the
allowlist config file. After updating the canonical ID to match the
published CWS extension, it now collides with CWS URLs in README and
browser-execution.ts. Fix by stripping CWS URLs before checking for
bare ID occurrences, and ignore .codex-worktrees (repo copies).
Also remove hardcoded CWS ID from README in favor of reading from
the canonical config.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(llm-callsite): seed latency-optimized defaults, fix guardian provider routing, clean stale comments

- Add LATENCY_OPTIMIZED_CALLSITE_DEFAULTS to schema for new installs
- Create migration 040 to seed latency-optimized call-site entries for existing workspaces
- Fix guardian-action-generators to use getConfiguredProvider() instead of bypassing call-site resolution
- Restore commitMessage maxTokens: 120 and temperature: 0.2 via call-site defaults
- Remove stale PR-reference comments from analyze-conversation.ts and voice-session-bridge.ts

Addresses consolidated review feedback from PRs #26101-#26140.

---------

Co-authored-by: Noa Flaherty <noa@vellum.ai>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(retry): stop forwarding contextWindow/provider to provider request body (#26280)

* chore(skills): regenerate catalog.json

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Noa Flaherty <noa@vellum.ai>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ashlee@vellum.ai <ashlee@vellum.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant