workspace+conversation: route commit message and title through call-site IDs#26112
Conversation
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| config: { | ||
| // `callSite` lets the provider resolve `max_tokens` and | ||
| // `temperature` from `llm.callSites.commitMessage` (populated by | ||
| // the workspace migration from the legacy | ||
| // `workspaceGit.commitMessageLLM.{maxTokens,temperature}` keys). | ||
| // Operational fields (`enabled`, `timeoutMs`, `breaker`, | ||
| // `maxFilesInPrompt`, `maxDiffBytes`, `minRemainingTurnBudgetMs`) | ||
| // remain on `workspaceGit.commitMessageLLM` and are read above. | ||
| callSite: "commitMessage", | ||
| // `fastModel` overrides the resolver's `model` because commit | ||
| // message generation enforces its own provider-specific fast | ||
| // model selection (see `PROVIDER_DEFAULT_FAST_MODELS` and | ||
| // `providerFastModelOverrides`). | ||
| model: fastModel, | ||
| max_tokens: llmConfig.maxTokens, | ||
| temperature: llmConfig.temperature, | ||
| }, |
There was a problem hiding this comment.
🔴 Commit message generation loses specialized max_tokens (120→64000) and temperature (0.2→null) for users with default settings
The PR removes the explicit max_tokens: llmConfig.maxTokens (default 120) and temperature: llmConfig.temperature (default 0.2) from the provider call config, relying instead on callSite: "commitMessage" resolution via resolveCallSiteConfig. However, the migration at assistant/src/workspace/migrations/038-unify-llm-callsite-configs.ts:172-185 only populates llm.callSites.commitMessage when maxTokens/temperature are explicitly present in the raw config.json file. Most users rely on Zod schema defaults (120 and 0.2 at assistant/src/config/schemas/workspace-git.ts:110-121), which are applied at parse time and aren't stored on disk. For these users, no llm.callSites.commitMessage entry exists, so resolveCallSiteConfig falls through to llm.default (assistant/src/config/schemas/llm.ts:222-231), which has maxTokens: 64000, temperature: null, effort: "max", and thinking: { enabled: true }. This causes a 533× increase in max_tokens, enables expensive extended thinking on Anthropic, and removes the low temperature setting — all for a call site that generates short commit messages (the system prompt says "Total output must be under 300 characters"). This violates the AGENTS.md backwards compatibility rule: "Never ship a change that silently breaks existing behavior."
Prompt for agents
The commit message generator at provider-commit-message-generator.ts:265-279 replaced explicit max_tokens and temperature with callSite resolution, but the migration (038-unify-llm-callsite-configs.ts) only copies these values when they are explicitly present in config.json. Users relying on Zod schema defaults (maxTokens=120, temperature=0.2) will silently get llm.default values (maxTokens=64000, temperature=null, effort=max, thinking=enabled).
Two possible fixes:
1. Keep the explicit max_tokens and temperature in the sendMessage config alongside callSite, so the normalizeViaCallSite function in retry.ts preserves them (it already has 'per-call explicit fields win' semantics for max_tokens and temperature):
config: { callSite: 'commitMessage', model: fastModel, max_tokens: llmConfig.maxTokens, temperature: llmConfig.temperature }
2. Update the migration to always write the schema defaults for commitMessage when the commitMessageLLM block exists but the individual keys are absent. This would seed llm.callSites.commitMessage with maxTokens=120 and temperature=0.2 for all users who have the feature enabled.
Option 1 is safer and maintains the existing behavior exactly while still opting into callSite for model/effort/thinking resolution. Option 2 is more aligned with the full callSite migration vision but risks missing edge cases.
Was this helpful? React with 👍 or 👎 to provide feedback.
| systemPrompt: buildTitleSystemPrompt(), | ||
| tools: [], | ||
| modelIntent: "quality-optimized", | ||
| callSite: "conversationTitle", |
There was a problem hiding this comment.
🚩 Conversation title generation model and thinking behavior changes
The conversation title service previously used modelIntent: "quality-optimized" which resolved to provider-specific quality models (e.g., claude-opus-4-7 for Anthropic via assistant/src/providers/model-intents.ts:16). After this PR, it uses callSite: "conversationTitle" which, absent an explicit llm.callSites.conversationTitle entry, falls through to llm.default — resolving to claude-opus-4-6 (the LLMConfigBase default at assistant/src/config/schemas/llm.ts:224). This is a minor model change.
More notably, the resolver also injects effort: "max" and thinking: { enabled: true, streamThinking: true } from llm.default. For Anthropic (which is in THINKING_AWARE_PROVIDERS at assistant/src/providers/retry.ts:36), this means extended thinking is now enabled for title generation — a 2-6 word output. The max_tokens: 1024 set by btw-sidechain is preserved (the resolver's if (nextConfig.max_tokens === undefined) check at assistant/src/providers/retry.ts:194 doesn't overwrite it), so token output is bounded, but the thinking overhead adds latency and cost for a trivial generation task. This isn't a correctness bug but is a meaningful efficiency regression worth considering.
Was this helpful? React with 👍 or 👎 to provide feedback.
| const options = callArgs[3] as { | ||
| config: { model: string; callSite: string }; | ||
| }; | ||
| expect(options.config.model).toBe("claude-haiku-4-5-20251001"); | ||
| expect(options.config.callSite).toBe("commitMessage"); |
There was a problem hiding this comment.
🚩 Test mocks bypass RetryProvider resolution, hiding the max_tokens regression
The commit message generator test at assistant/src/__tests__/provider-commit-message-generator.test.ts:77-79 mocks resolveConfiguredProvider to return a bare mock provider (not a RetryProvider). This means callSite resolution via normalizeViaCallSite never runs in the test. The test at line 243 only verifies that callSite: "commitMessage" is present in the config object, not that the resolution produces correct max_tokens/temperature values. This is why the regression from the reported bug passes tests — the test doesn't exercise the production code path where RetryProvider resolves callSite into concrete config values.
Was this helpful? React with 👍 or 👎 to provide feedback.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 951475a0c3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| max_tokens: params.maxTokens ?? 1024, | ||
| tool_choice: { type: "none" }, | ||
| modelIntent: params.modelIntent ?? "latency-optimized", | ||
| ...(params.callSite !== undefined | ||
| ? { callSite: params.callSite } |
There was a problem hiding this comment.
runBtwSidechain now uses callSite, but it still always sends max_tokens: params.maxTokens ?? 1024. In RetryProvider.normalizeViaCallSite, resolved call-site maxTokens is only applied when max_tokens is undefined, so any llm.callSites.<id>.maxTokens setting is ignored (including for conversationTitle). This makes the new call-site path unable to honor configured token budgets.
Useful? React with 👍 / 👎.
| callSite: "commitMessage", | ||
| // `fastModel` overrides the resolver's `model` because commit |
There was a problem hiding this comment.
This change removes the explicit workspaceGit.commitMessageLLM.maxTokens pass-through and relies on callSite: "commitMessage" to supply limits. Migration 038 only backfills llm.callSites.commitMessage.maxTokens when the legacy key exists in raw config.json, so workspaces that enabled this feature without explicitly setting maxTokens will now inherit llm.default.maxTokens (default 64000) instead of the previous 120-token cap, increasing latency/cost risk and weakening the intended short-output guardrail.
Useful? React with 👍 / 👎.
siddseethepalli
merged commit 8de79e1
into
siddseethepalli/unify-llm-callsites
…es} (#26159) * config(llm): add unified llm schema with call-site enum and profile refines (#26089) * config(llm): add unified llm schema with call-site enum and profile refines * fix(llm-schema): replace deepPartialObject helper with explicit .partial().extend() Zod 4's readonly shape typing tripped TS2542 in the LSP for the generic walker. Inline the one-level expansion for ContextWindowSchema and switch the superRefine issue code to the string literal (Zod 4 deprecated ZodIssueCode). * config(llm): add resolveCallSiteConfig resolver with deep merge (#26094) * config(llm): add resolveCallSiteConfig resolver with deep merge * fix(llm-resolver): deep-clone nested objects so resolved configs are isolated snapshots Codex flagged that the merge helper aliased nested objects from llm.default when no override touched them, so a caller mutating the returned config would silently corrupt the source. Recurse into plain-object sources unconditionally and add a regression test. * config(llm): add llm field to AssistantConfigSchema (no behavior change) (#26095) * config(llm): add llm field to AssistantConfigSchema (no behavior change) * fix(llm-schema): add field-level defaults so partial llm configs don't trigger full config reset Codex flagged that requiring all LLMConfigBase fields meant the loader's leaf-deletion recovery couldn't repair partial/invalid llm blocks — falling through to cloneDefaultConfig() and discarding the user's other valid settings. Add .default(...) to every leaf so LLMSchema.parse({}) returns a fully-defaulted object, matching the pattern used by sibling config schemas. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * providers: accept callSite in per-call config; resolve via resolveCallSiteConfig (#26102) * workspace: migrate scattered LLM config keys into unified llm structure (#26101) * workspace: migrate scattered LLM config keys into unified llm structure * fix(migration): preserve existing llm subtree; map notification intent to both call sites Codex flagged two issues: - The migration assignment replaced config.llm wholesale, destroying any pre-existing llm.callSites/profiles when llm.default was absent. Now merges into existing config.llm, preserving non-conflicting entries. - notifications.decisionModelIntent drives both notification classification and preference extraction, but the migration only seeded notificationDecision. Now seeds both call sites. * memory: route extraction/consolidation/retrieval through call-site IDs (#26106) * memory: route narrative/pattern/summarization/starters through call-site IDs (#26107) * notifications: route decision and preference extraction through call-site IDs (#26109) * calls+watcher: route guardian copy and watch handlers through call-site IDs (#26105) * utility: route classifier and analyzer LLM calls through call-site IDs (#26111) * macos(settings): migrate InferenceServiceCard reads/writes to llm.default.* (#26113) * workspace+conversation: route commit message and title through call-site IDs (#26112) * ui: route identity intro and empty-state greeting through call-site IDs (#26108) * daemon: thread callSite through processMessage options and adapter callbacks (#26115) * daemon: thread callSite through processMessage options and adapter callbacks * fix(callsite-threading): complete interface contract and server.ts symmetry Devin flagged two gaps in PR #26115: - ProcessConversationContext interface missing callSite in its runAgentLoop options type (works via structural typing but contract was incomplete; mocks would silently drop the field). - DaemonServer.persistAndProcessMessage didn't thread callSite to conversation.runAgentLoop, while DaemonServer.processMessage did. Aligned. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(callsite): don't default unspecified callers to 'mainAgent' Codex flagged that defaulting to mainAgent for every turn routes them through the new RetryProvider call-site resolver, which reads from llm.default — but config-model.setModel still writes to services.inference without syncing llm.default. Result: stale/incompatible model IDs after a model switch. Defer the cutover. agent-loop turns now keep using the legacy modelIntent path (turnCallSite = options?.callSite, no fallback). PRs 7-11 still explicitly pass callSite and route through the new resolver as intended. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * heartbeat: pass callSite: 'heartbeatAgent' instead of speed kwarg (#26125) * filing: pass callSite: 'filingAgent' instead of speed kwarg (#26124) * runtime/analyze-conversation: route through callSite: 'analyzeConversation' (#26126) * subagent: pass callSite: 'subagentSpawn' when spawning isolated agents (#26122) * calls: route the call agent loop through callSite: 'callAgent' (#26123) * macos(settings): add SettingsStore APIs for per-call-site overrides (#26128) * macos(settings): add SettingsStore APIs for per-call-site overrides * fix(callsite-overrides): harden setCallSiteOverrides against dup-id crash and batch divergence Devin and Codex flagged two issues: - Dictionary(uniqueKeysWithValues:) crashes if callers pass duplicate CallSiteOverride.id values (external input — must be tolerant). Switch to Dictionary(_:uniquingKeysWith:) with last-write-wins. - Batch updates locally cleared entries omitted from the input but only PATCHed entries that were present, so omitted entries appeared cleared in the UI but reappeared on next sync. Now the PATCH payload includes NSNull clears for every catalog entry not in the batch, aligning remote with local. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(callsite-overrides): null entire entry on clear so non-UI leaves get cleared too Codex P2 (PR #26128 cycle 2): clearCallSiteOverride only nulled provider/model/profile, but call-site config supports additional leaves (maxTokens, effort, speed, thinking, contextWindow). If those were set via manual edits, the UI would report cleared while the daemon kept applying hidden overrides. Switch the PATCH payload from { provider: null, model: null, profile: null } to a single null on the entry itself. The Zod fragment treats null as absent, so the resolver falls back to llm.default. Same fix applies to the omitted-catalog-entry clears in setCallSiteOverrides batch. Tests updated to assert the new shape. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * macos(settings): confirm default-provider switch when call-site overrides exist (#26133) * macos(settings): show 'N call-site overrides' badge with read-only list sheet (#26135) * macos(settings): show 'N call-site overrides' badge with read-only list sheet * fix(comments): drop PR-number breadcrumbs in callsite override files Devin flagged that comments referencing PR 22/23/24 violate clients/AGENTS.md 'Comment Quality' rule (no breadcrumbs). Replaced with timeless descriptions of code intent. * macos(settings): make per-task override sheet editable with provider/model pickers (#26136) * macos(settings): make per-task override sheet editable with provider/model pickers * fix(callsite-sheet): preserve external updates and seed override from active default provider Codex flagged two P1s: - syncDraftsFromStore compared drafts against the NEW persisted value to decide 'touched', so external store updates were treated as user edits and got overwritten by Save All. Track the previously-persisted value in lastSyncedFromStore and consider a row touched only when the draft differs from that baseline. - Toggling 'Override default' on initialized provider from providerIds.first instead of the user's actual default provider, which could pin the wrong provider on save. Pass the user's default provider into CallSiteOverrideRow and seed from it. * fix(callsite-sheet): use entry-level null path for cleared rows in saveAll/resetAll Devin flagged that saveAll() and resetAll() were passing all-nil entries to setCallSiteOverrides, which routed them through the field-level null path (provider/model/profile = null). That left advanced leaves (maxTokens, effort, temperature, contextWindow) untouched on the daemon. Fix: - saveAll(): filter to entries with hasOverride == true; toggled-off rows fall through to the entry-level null path. - resetAll(): pass an empty list so every catalog entry hits the entry-level null path. * config(llm): remove deprecated scattered LLM keys (#26140) * fix(config-loader): treat JSON null as key deletion in deepMergeOverwrite (#26153) * fix(agent-loop): default user-initiated turns to callSite: 'mainAgent' (#26154) * fix(meet-join): migrate consent-monitor + session-manager to callSite contract (#26155) * fix(macos): atomic provider+model save via single PATCH (#26156) * fix(cleanup): remove dead code, refresh comments, add migration test, update docs (#26157) * fix(r2): catalog test count, skill self-knowledge doc, AGENTS.md, loader docstring (#26158) * fix(llm-callsite): refresh stale docstring, restore overflow budget, restore SettingsStore fallback (#26252) * fix(llm-callsite): route provider transport and field precedence through callSite (#26254) * fix(llm-callsite): pass CI + address subagent/thinking/temperature review comments (#26258) * test(extension-id-guard): allow CWS URL matches; mirrors main PR #26263 (#26270) * fix(llm-callsite): UI override state divergence, null-as-delete, migration gaps (#26271) * Fix Chrome extension allowlist ID and clarify README dev setup (#26259) Update the canonical allowlist to use the correct published CWS extension ID (hphbdmpffeigpcdjkckleobjmhhokpne). Restructure the Chrome extension README to clearly explain the allowlist merge strategy, separate the macOS app (automatic) path from the manual native messaging setup, and show how dev + prod extensions work side-by-side. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(clients): enable non-contiguous glyph layout for NSTextView-backed code views (#26242) TextKit 1 defaults NSLayoutManager.allowsNonContiguousLayout to false, which forces full-document glyph layout from character 0 on the main thread whenever a glyph range is queried. Attaching an NSTextView to its scroll view (setDocumentView: -> _setSuperview: -> setNeedsDisplayInRect: -> _glyphRangeForBoundingRect:) triggers that query during makeNSView, producing multi-second hangs on large code blocks. Opt into non-contiguous layout on every TextKit 1 stack we build via NSViewRepresentable so glyph generation is confined to the requested bounding rect. Also replace NSLayoutManager.ensureLayout(for:) in the code-view sizeThatFits paths with direct lineCount * fixedLineHeight math: the text container is unbounded horizontally (no wrapping) and paragraph style pins minimumLineHeight == maximumLineHeight, so the geometry is exact and avoids a second O(glyph count) main-thread path. Fixes VELLUM-ASSISTANT-MACOS-J2. Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: ashlee@vellum.ai <ashlee@vellum.ai> * fix(contacts): show Assistant badge for assistant-type contacts (LUM-1009) (#26239) * fix(contacts): show Assistant badge for assistant-type contacts (LUM-1009) * Move role/contactType derivation onto Kind for valid initializer --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(llm-callsite): UI override state divergence, null-as-delete, migration gaps - deepMergeOverwrite: null on scalar/null targets assigns null (preserves nullable config fields like activeHoursStart); null on object targets still deletes (call-site clearing). Fixes regression where PATCH with null for nullable fields was deleted then re-defaulted. - InferenceServiceCard: override confirmation dialog only fires when the resolved provider ID actually changes, not on mode-only toggles where both old and new resolve to the same provider. - CallSiteOverridesSheet: per-row Save uses replaceCallSiteOverride (clear-then-set) so stale daemon-side leaves are removed. The partial-update setCallSiteOverride would retain fields the draft nil'd. - CallSiteOverrideRow: merge consecutive .padding modifiers into single EdgeInsets call per macOS AGENTS.md layout rule. - SettingsStore: add replaceCallSiteOverride for full-entry replacement. --------- Co-authored-by: Noa Flaherty <noa@vellum.ai> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: ashlee@vellum.ai <ashlee@vellum.ai> * fix(llm-callsite): seed latency-optimized defaults and fix guardian provider routing (#26275) * fix(meet-bot): address review feedback — Docker build, scraper races, audio capture, storage writer (#26264) * fix(meet): chat concurrency, dispose teardown, and wake adapter fidelity (#26265) * fix: heartbeat dual-emit, analysis dedup, test hermiticity, credential executor discovery (#26266) * fix: model default fallback, empty-response nudge scan (#26268) - Update FALLBACK_DEFAULT_MODEL to claude-opus-4-7 + test - Fix resolveModel to check Anthropic catalog (not just current default) so stale persisted defaults (e.g. claude-opus-4-6) don't get sent to non-Anthropic providers - Fix priorAssistantHadVisibleText backward scan to check ALL prior assistant messages, not just the most recent one Addresses review feedback from PRs #26247, #26164. * fix(meet): TTS stream races, barge-in tracking, ffmpeg error classification (#26267) * Fix extension-id-sync-guard test after canonical ID update (#26263) The guard test asserts that canonical extension IDs appear only in the allowlist config file. After updating the canonical ID to match the published CWS extension, it now collides with CWS URLs in README and browser-execution.ts. Fix by stripping CWS URLs before checking for bare ID occurrences, and ignore .codex-worktrees (repo copies). Also remove hardcoded CWS ID from README in favor of reading from the canonical config. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(llm-callsite): seed latency-optimized defaults, fix guardian provider routing, clean stale comments - Add LATENCY_OPTIMIZED_CALLSITE_DEFAULTS to schema for new installs - Create migration 040 to seed latency-optimized call-site entries for existing workspaces - Fix guardian-action-generators to use getConfiguredProvider() instead of bypassing call-site resolution - Restore commitMessage maxTokens: 120 and temperature: 0.2 via call-site defaults - Remove stale PR-reference comments from analyze-conversation.ts and voice-session-bridge.ts Addresses consolidated review feedback from PRs #26101-#26140. --------- Co-authored-by: Noa Flaherty <noa@vellum.ai> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(retry): stop forwarding contextWindow/provider to provider request body (#26280) * chore(skills): regenerate catalog.json --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Noa Flaherty <noa@vellum.ai> Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: ashlee@vellum.ai <ashlee@vellum.ai>
1 participant
Summary
memory/conversation-title-service.ts->callSite: 'conversationTitle'.workspace/provider-commit-message-generator.ts->callSite: 'commitMessage'. Operational fields (timeoutMs,maxFilesInPrompt,breaker, etc.) stay onworkspaceGit.commitMessageLLM; only LLM config (maxTokens,temperature) flows through the new resolver.runtime/btw-sidechain.tsextended with optionalcallSitepassthrough so the title service can opt into the resolver path; legacymodelIntentbranch preserved for unmigrated callers.Part of plan: unify-llm-callsites.md (PR 14 of 24)