memory: route extraction/consolidation/retrieval through call-site IDs#26106
Merged
siddseethepalli merged 1 commit intoApr 16, 2026
Merged
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e2898ad6e5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| { | ||
| config: { | ||
| modelIntent: "quality-optimized" as const, | ||
| callSite: "memoryExtraction" as const, |
There was a problem hiding this comment.
Switching this call from modelIntent: "quality-optimized" to callSite changes default behavior unless llm.callSites.memoryExtraction (and the analogous consolidation sites) are explicitly configured. resolveCallSiteConfig falls back to llm.default when a call-site entry is missing (assistant/src/config/llm-resolver.ts), and there is no migration/default seeding for these memory call sites, so users with a fast global default model now run extraction/consolidation on that fast model and get lower-fidelity graph diffs than before.
Useful? React with 👍 / 👎.
| { | ||
| config: { | ||
| modelIntent: "latency-optimized" as const, | ||
| callSite: "memoryRetrieval" as const, |
There was a problem hiding this comment.
This request now sets config.callSite, but the provider is still obtained earlier with getConfiguredProvider() (no call-site argument), so llm.callSites.memoryRetrieval.provider is ignored. In configs that set a different provider/model for this site, the call still goes through the global provider and can fail with model/provider mismatch errors; use getConfiguredProvider("memoryRetrieval") so provider routing matches the call-site config.
Useful? React with 👍 / 👎.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
🚩 getConfiguredProvider() not passed callSite — pre-existing pattern, not a PR regression
All six call sites in this PR call getConfiguredProvider() without the callSite parameter (e.g. retriever.ts:84, retriever.ts:183, retriever.ts:276, consolidation.ts:265, consolidation.ts:431, extraction.ts:847). The getConfiguredProvider(callSite?) signature at provider-send-message.ts:93 supports call-site-aware provider resolution via resolveConfiguredProvider at line 48-80, which would use resolveCallSiteConfig(callSite, config.llm).provider instead of the default config.services.inference.provider. If a user configures a different provider for a call site (e.g. llm.callSites.memoryRetrieval.provider = "openai"), the transport would still be the default provider while the model would be resolved for the configured provider — a mismatch. However, this same pattern exists across all other callers in the codebase (every getConfiguredProvider() call outside tests omits the callSite), so this is a pre-existing architectural limitation, not a regression from this PR. The test at retry-callsite.test.ts:312 demonstrates the intended future pattern.
(Refers to line 84)
Was this helpful? React with 👍 or 👎 to provide feedback.
| { | ||
| config: { | ||
| modelIntent: "latency-optimized" as const, | ||
| callSite: "memoryRetrieval" as const, |
Contributor
There was a problem hiding this comment.
🔴 dedupForTurn silently upgrades from latency-optimized (haiku) to quality model (opus)
dedupForTurn previously used modelIntent: "latency-optimized" which resolved to a fast/cheap model (e.g. claude-haiku-4-5-20251001 per assistant/src/providers/model-intents.ts:15). The new callSite: "memoryRetrieval" falls through to llm.default.model (defaults to claude-opus-4-6 per assistant/src/config/schemas/llm.ts:224) since no llm.callSites.memoryRetrieval entry exists and the workspace migration (assistant/src/workspace/migrations/038-unify-llm-callsite-configs.ts) doesn't create one for this implicit intent. This function runs on every conversation turn, and its JSDoc at line 172-175 explicitly states it "Uses a latency-optimized model". The switch from haiku to opus causes a major latency and cost regression — opus is roughly 30-60× more expensive and significantly slower, threatening the loadContextMemory "p90 < 2s" budget documented at line 361.
Prompt for agents
The root cause is that dedupForTurn, dedupCrossCategory, and rerankAndDedup all share a single callSite ID (memoryRetrieval) but previously had different model intents (latency-optimized vs quality-optimized). The callSite system resolves all three to the same default config, losing the latency/quality distinction.
Possible fixes:
1. Split into separate call-site IDs: e.g. memoryRetrievalRerank (quality) vs memoryRetrievalDedup (latency). Register them in LLMCallSiteEnum in assistant/src/config/schemas/llm.ts and use the appropriate one in each function.
2. Add a default llm.callSites.memoryRetrieval entry with a fast model/speed in the migration (038-unify-llm-callsite-configs.ts) — but this only helps existing users, not fresh installs.
3. Use a profile-based approach: define a fast profile and reference it from a memoryRetrievalDedup call site.
Option 1 is cleanest since these functions genuinely have different latency/quality requirements. The per-turn dedup functions (dedupForTurn at line 177, dedupCrossCategory at line 271) need a fast model; rerankAndDedup (line 76) can use the quality model.
Was this helpful? React with 👍 or 👎 to provide feedback.
| { | ||
| config: { | ||
| modelIntent: "latency-optimized" as const, | ||
| callSite: "memoryRetrieval" as const, |
Contributor
There was a problem hiding this comment.
🔴 dedupCrossCategory silently upgrades from latency-optimized (haiku) to quality model (opus)
Same issue as dedupForTurn: dedupCrossCategory previously used modelIntent: "latency-optimized" (resolving to haiku) but now uses callSite: "memoryRetrieval" which resolves to the global default opus model. This function runs during every context load and its design intent is lightweight dedup-only processing (see JSDoc at line 265-269). The shared callSite: "memoryRetrieval" with rerankAndDedup (which was previously quality-optimized) collapses the latency/quality distinction.
Was this helpful? React with 👍 or 👎 to provide feedback.
siddseethepalli
merged commit 5eb05dc
into
siddseethepalli/unify-llm-callsites
11 of 12 checks passed
siddseethepalli
added a commit
that referenced
this pull request
Apr 18, 2026
…es} (#26159) * config(llm): add unified llm schema with call-site enum and profile refines (#26089) * config(llm): add unified llm schema with call-site enum and profile refines * fix(llm-schema): replace deepPartialObject helper with explicit .partial().extend() Zod 4's readonly shape typing tripped TS2542 in the LSP for the generic walker. Inline the one-level expansion for ContextWindowSchema and switch the superRefine issue code to the string literal (Zod 4 deprecated ZodIssueCode). * config(llm): add resolveCallSiteConfig resolver with deep merge (#26094) * config(llm): add resolveCallSiteConfig resolver with deep merge * fix(llm-resolver): deep-clone nested objects so resolved configs are isolated snapshots Codex flagged that the merge helper aliased nested objects from llm.default when no override touched them, so a caller mutating the returned config would silently corrupt the source. Recurse into plain-object sources unconditionally and add a regression test. * config(llm): add llm field to AssistantConfigSchema (no behavior change) (#26095) * config(llm): add llm field to AssistantConfigSchema (no behavior change) * fix(llm-schema): add field-level defaults so partial llm configs don't trigger full config reset Codex flagged that requiring all LLMConfigBase fields meant the loader's leaf-deletion recovery couldn't repair partial/invalid llm blocks — falling through to cloneDefaultConfig() and discarding the user's other valid settings. Add .default(...) to every leaf so LLMSchema.parse({}) returns a fully-defaulted object, matching the pattern used by sibling config schemas. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * providers: accept callSite in per-call config; resolve via resolveCallSiteConfig (#26102) * workspace: migrate scattered LLM config keys into unified llm structure (#26101) * workspace: migrate scattered LLM config keys into unified llm structure * fix(migration): preserve existing llm subtree; map notification intent to both call sites Codex flagged two issues: - The migration assignment replaced config.llm wholesale, destroying any pre-existing llm.callSites/profiles when llm.default was absent. Now merges into existing config.llm, preserving non-conflicting entries. - notifications.decisionModelIntent drives both notification classification and preference extraction, but the migration only seeded notificationDecision. Now seeds both call sites. * memory: route extraction/consolidation/retrieval through call-site IDs (#26106) * memory: route narrative/pattern/summarization/starters through call-site IDs (#26107) * notifications: route decision and preference extraction through call-site IDs (#26109) * calls+watcher: route guardian copy and watch handlers through call-site IDs (#26105) * utility: route classifier and analyzer LLM calls through call-site IDs (#26111) * macos(settings): migrate InferenceServiceCard reads/writes to llm.default.* (#26113) * workspace+conversation: route commit message and title through call-site IDs (#26112) * ui: route identity intro and empty-state greeting through call-site IDs (#26108) * daemon: thread callSite through processMessage options and adapter callbacks (#26115) * daemon: thread callSite through processMessage options and adapter callbacks * fix(callsite-threading): complete interface contract and server.ts symmetry Devin flagged two gaps in PR #26115: - ProcessConversationContext interface missing callSite in its runAgentLoop options type (works via structural typing but contract was incomplete; mocks would silently drop the field). - DaemonServer.persistAndProcessMessage didn't thread callSite to conversation.runAgentLoop, while DaemonServer.processMessage did. Aligned. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(callsite): don't default unspecified callers to 'mainAgent' Codex flagged that defaulting to mainAgent for every turn routes them through the new RetryProvider call-site resolver, which reads from llm.default — but config-model.setModel still writes to services.inference without syncing llm.default. Result: stale/incompatible model IDs after a model switch. Defer the cutover. agent-loop turns now keep using the legacy modelIntent path (turnCallSite = options?.callSite, no fallback). PRs 7-11 still explicitly pass callSite and route through the new resolver as intended. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * heartbeat: pass callSite: 'heartbeatAgent' instead of speed kwarg (#26125) * filing: pass callSite: 'filingAgent' instead of speed kwarg (#26124) * runtime/analyze-conversation: route through callSite: 'analyzeConversation' (#26126) * subagent: pass callSite: 'subagentSpawn' when spawning isolated agents (#26122) * calls: route the call agent loop through callSite: 'callAgent' (#26123) * macos(settings): add SettingsStore APIs for per-call-site overrides (#26128) * macos(settings): add SettingsStore APIs for per-call-site overrides * fix(callsite-overrides): harden setCallSiteOverrides against dup-id crash and batch divergence Devin and Codex flagged two issues: - Dictionary(uniqueKeysWithValues:) crashes if callers pass duplicate CallSiteOverride.id values (external input — must be tolerant). Switch to Dictionary(_:uniquingKeysWith:) with last-write-wins. - Batch updates locally cleared entries omitted from the input but only PATCHed entries that were present, so omitted entries appeared cleared in the UI but reappeared on next sync. Now the PATCH payload includes NSNull clears for every catalog entry not in the batch, aligning remote with local. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(callsite-overrides): null entire entry on clear so non-UI leaves get cleared too Codex P2 (PR #26128 cycle 2): clearCallSiteOverride only nulled provider/model/profile, but call-site config supports additional leaves (maxTokens, effort, speed, thinking, contextWindow). If those were set via manual edits, the UI would report cleared while the daemon kept applying hidden overrides. Switch the PATCH payload from { provider: null, model: null, profile: null } to a single null on the entry itself. The Zod fragment treats null as absent, so the resolver falls back to llm.default. Same fix applies to the omitted-catalog-entry clears in setCallSiteOverrides batch. Tests updated to assert the new shape. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * macos(settings): confirm default-provider switch when call-site overrides exist (#26133) * macos(settings): show 'N call-site overrides' badge with read-only list sheet (#26135) * macos(settings): show 'N call-site overrides' badge with read-only list sheet * fix(comments): drop PR-number breadcrumbs in callsite override files Devin flagged that comments referencing PR 22/23/24 violate clients/AGENTS.md 'Comment Quality' rule (no breadcrumbs). Replaced with timeless descriptions of code intent. * macos(settings): make per-task override sheet editable with provider/model pickers (#26136) * macos(settings): make per-task override sheet editable with provider/model pickers * fix(callsite-sheet): preserve external updates and seed override from active default provider Codex flagged two P1s: - syncDraftsFromStore compared drafts against the NEW persisted value to decide 'touched', so external store updates were treated as user edits and got overwritten by Save All. Track the previously-persisted value in lastSyncedFromStore and consider a row touched only when the draft differs from that baseline. - Toggling 'Override default' on initialized provider from providerIds.first instead of the user's actual default provider, which could pin the wrong provider on save. Pass the user's default provider into CallSiteOverrideRow and seed from it. * fix(callsite-sheet): use entry-level null path for cleared rows in saveAll/resetAll Devin flagged that saveAll() and resetAll() were passing all-nil entries to setCallSiteOverrides, which routed them through the field-level null path (provider/model/profile = null). That left advanced leaves (maxTokens, effort, temperature, contextWindow) untouched on the daemon. Fix: - saveAll(): filter to entries with hasOverride == true; toggled-off rows fall through to the entry-level null path. - resetAll(): pass an empty list so every catalog entry hits the entry-level null path. * config(llm): remove deprecated scattered LLM keys (#26140) * fix(config-loader): treat JSON null as key deletion in deepMergeOverwrite (#26153) * fix(agent-loop): default user-initiated turns to callSite: 'mainAgent' (#26154) * fix(meet-join): migrate consent-monitor + session-manager to callSite contract (#26155) * fix(macos): atomic provider+model save via single PATCH (#26156) * fix(cleanup): remove dead code, refresh comments, add migration test, update docs (#26157) * fix(r2): catalog test count, skill self-knowledge doc, AGENTS.md, loader docstring (#26158) * fix(llm-callsite): refresh stale docstring, restore overflow budget, restore SettingsStore fallback (#26252) * fix(llm-callsite): route provider transport and field precedence through callSite (#26254) * fix(llm-callsite): pass CI + address subagent/thinking/temperature review comments (#26258) * test(extension-id-guard): allow CWS URL matches; mirrors main PR #26263 (#26270) * fix(llm-callsite): UI override state divergence, null-as-delete, migration gaps (#26271) * Fix Chrome extension allowlist ID and clarify README dev setup (#26259) Update the canonical allowlist to use the correct published CWS extension ID (hphbdmpffeigpcdjkckleobjmhhokpne). Restructure the Chrome extension README to clearly explain the allowlist merge strategy, separate the macOS app (automatic) path from the manual native messaging setup, and show how dev + prod extensions work side-by-side. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(clients): enable non-contiguous glyph layout for NSTextView-backed code views (#26242) TextKit 1 defaults NSLayoutManager.allowsNonContiguousLayout to false, which forces full-document glyph layout from character 0 on the main thread whenever a glyph range is queried. Attaching an NSTextView to its scroll view (setDocumentView: -> _setSuperview: -> setNeedsDisplayInRect: -> _glyphRangeForBoundingRect:) triggers that query during makeNSView, producing multi-second hangs on large code blocks. Opt into non-contiguous layout on every TextKit 1 stack we build via NSViewRepresentable so glyph generation is confined to the requested bounding rect. Also replace NSLayoutManager.ensureLayout(for:) in the code-view sizeThatFits paths with direct lineCount * fixedLineHeight math: the text container is unbounded horizontally (no wrapping) and paragraph style pins minimumLineHeight == maximumLineHeight, so the geometry is exact and avoids a second O(glyph count) main-thread path. Fixes VELLUM-ASSISTANT-MACOS-J2. Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: ashlee@vellum.ai <ashlee@vellum.ai> * fix(contacts): show Assistant badge for assistant-type contacts (LUM-1009) (#26239) * fix(contacts): show Assistant badge for assistant-type contacts (LUM-1009) * Move role/contactType derivation onto Kind for valid initializer --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(llm-callsite): UI override state divergence, null-as-delete, migration gaps - deepMergeOverwrite: null on scalar/null targets assigns null (preserves nullable config fields like activeHoursStart); null on object targets still deletes (call-site clearing). Fixes regression where PATCH with null for nullable fields was deleted then re-defaulted. - InferenceServiceCard: override confirmation dialog only fires when the resolved provider ID actually changes, not on mode-only toggles where both old and new resolve to the same provider. - CallSiteOverridesSheet: per-row Save uses replaceCallSiteOverride (clear-then-set) so stale daemon-side leaves are removed. The partial-update setCallSiteOverride would retain fields the draft nil'd. - CallSiteOverrideRow: merge consecutive .padding modifiers into single EdgeInsets call per macOS AGENTS.md layout rule. - SettingsStore: add replaceCallSiteOverride for full-entry replacement. --------- Co-authored-by: Noa Flaherty <noa@vellum.ai> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: ashlee@vellum.ai <ashlee@vellum.ai> * fix(llm-callsite): seed latency-optimized defaults and fix guardian provider routing (#26275) * fix(meet-bot): address review feedback — Docker build, scraper races, audio capture, storage writer (#26264) * fix(meet): chat concurrency, dispose teardown, and wake adapter fidelity (#26265) * fix: heartbeat dual-emit, analysis dedup, test hermiticity, credential executor discovery (#26266) * fix: model default fallback, empty-response nudge scan (#26268) - Update FALLBACK_DEFAULT_MODEL to claude-opus-4-7 + test - Fix resolveModel to check Anthropic catalog (not just current default) so stale persisted defaults (e.g. claude-opus-4-6) don't get sent to non-Anthropic providers - Fix priorAssistantHadVisibleText backward scan to check ALL prior assistant messages, not just the most recent one Addresses review feedback from PRs #26247, #26164. * fix(meet): TTS stream races, barge-in tracking, ffmpeg error classification (#26267) * Fix extension-id-sync-guard test after canonical ID update (#26263) The guard test asserts that canonical extension IDs appear only in the allowlist config file. After updating the canonical ID to match the published CWS extension, it now collides with CWS URLs in README and browser-execution.ts. Fix by stripping CWS URLs before checking for bare ID occurrences, and ignore .codex-worktrees (repo copies). Also remove hardcoded CWS ID from README in favor of reading from the canonical config. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(llm-callsite): seed latency-optimized defaults, fix guardian provider routing, clean stale comments - Add LATENCY_OPTIMIZED_CALLSITE_DEFAULTS to schema for new installs - Create migration 040 to seed latency-optimized call-site entries for existing workspaces - Fix guardian-action-generators to use getConfiguredProvider() instead of bypassing call-site resolution - Restore commitMessage maxTokens: 120 and temperature: 0.2 via call-site defaults - Remove stale PR-reference comments from analyze-conversation.ts and voice-session-bridge.ts Addresses consolidated review feedback from PRs #26101-#26140. --------- Co-authored-by: Noa Flaherty <noa@vellum.ai> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(retry): stop forwarding contextWindow/provider to provider request body (#26280) * chore(skills): regenerate catalog.json --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Noa Flaherty <noa@vellum.ai> Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: ashlee@vellum.ai <ashlee@vellum.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
memory/graph/extraction.ts→callSite: 'memoryExtraction'memory/graph/consolidation.ts→callSite: 'memoryConsolidation'memory/graph/retriever.ts→callSite: 'memoryRetrieval'(collapses two prior intents into one configurable site)callSite.Part of plan: unify-llm-callsites.md (PR 12 of 24)