feat(agent-core-v2): add dedicated compaction model with fallback

[arrrrny]

Related Issue

Internal feature work on a personal fork — no upstream issue to link.

Problem

Context compaction always ran against the current conversation model. There was no way to route compaction/summarization through a dedicated, potentially cheaper or specialized model — the same capability the visual model already provides for vision tasks.

What changed

• Add a [compaction_model] config section plus a compaction-model experimental flag, mirroring the existing visual-model pattern.
• When enabled, AgentFullCompactionService resolves the dedicated model and issues the compaction request against it instead of the current model.
• If the dedicated model errors or is inaccessible (e.g. an uncatalogued alias), compaction transparently falls back to the current model on the same round and still completes. Telemetry reports the model actually used, so the dedicated model is never a single point of failure.
• Add resolver unit tests (test/session/compaction/configSection.test.ts) and end-to-end fallback integration tests (test/agent/fullCompaction/compaction-model.test.ts).

Checklist

• I have read the CONTRIBUTING document.
• I have added tests that prove my feature works.
• Ran gen-changesets skill (changeset included under .changeset/).
• Ran gen-docs skill, or this PR needs no doc update.

[arrrrny]

@coderabbitai review

[coderabbitai]

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Warning

Review limit reached

@arrrrny, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 59 minutes

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 1ea9a507-3baf-416a-a92d-90fbdb334889

📥 Commits

Reviewing files that changed from the base of the PR and between 617f7a8 and de6fbf1.

📒 Files selected for processing (14)

• .changeset/compaction-model-option.md
• packages/agent-core-v2/src/agent/fullCompaction/fullCompactionService.ts
• packages/agent-core-v2/src/agent/llmRequester/llmRequester.ts
• packages/agent-core-v2/src/agent/llmRequester/llmRequesterService.ts
• packages/agent-core-v2/src/agent/profile/profile.ts
• packages/agent-core-v2/src/agent/profile/profileService.ts
• packages/agent-core-v2/src/app/kosongConfig/compactionModelOverlay.ts
• packages/agent-core-v2/src/app/kosongConfig/configSection.ts
• packages/agent-core-v2/src/app/telemetry/events.ts
• packages/agent-core-v2/src/index.ts
• packages/agent-core-v2/src/session/compaction/configSection.ts
• packages/agent-core-v2/src/session/compaction/flag.ts
• packages/agent-core-v2/test/agent/fullCompaction/compaction-model.test.ts
• packages/agent-core-v2/test/session/compaction/configSection.test.ts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[arrrrny]

Fetch URLs
2 pages

effectiveMaxOutputSize is never updated on CompactionTruncatedError retry, breaking truncation handling.
The variable is computed once before the do…while loop as const. When a CompactionTruncatedError occurs, compactionMaxOutputSize is halved and the loop retries, but effectiveMaxOutputSize still holds the original value, so the request keeps asking for the same (too-large) limit. The same issue exists after falling back to the current model: effectiveMaxOutputSize should be recomputed against the current model’s cap, not the (possibly larger or stale) dedicated model’s boundMaxOutputSize.
thinking_effort telemetry reports the current conversation model’s level even when a dedicated compaction model is used.
thinkingEffort is captured from resolvedModel (the caller’s model) before the dedicated model is resolved. When compaction succeeds on the dedicated model, compaction_finished telemetry still sends the caller’s thinkingLevel instead of the bound compaction model’s actual thinkingLevel.
diff
Copy
diff --git a/packages/agent-core-v2/src/agent/fullCompaction/fullCompactionService.ts b/packages/agent-core-v2/src/agent/fullCompaction/fullCompactionService.ts
--- a/packages/agent-core-v2/src/agent/fullCompaction/fullCompactionService.ts
+++ b/packages/agent-core-v2/src/agent/fullCompaction/fullCompactionService.ts
@@ -649,6 +649,7 @@ export class AgentFullCompactionService extends Service implements IAgentFullCom
? Math.min(maxContextTokens, DEFAULT_COMPACTION_MAX_COMPLETION_TOKENS)
: undefined;
const compactionMaxOutputSize = resolvedModel.maxOutputSize ?? defaultCompactionCap;

•  const currentModelMaxOutputSize = compactionMaxOutputSize;
  
   const binding = compactionModelBindingFor(this.configService, this.flags, {
     modelAlias: currentModelAlias,
  

@@ -662,6 +663,7 @@ export class AgentFullCompactionService extends Service implements IAgentFullCom
if (hasDedicatedModel) {
try {
boundModel = this.profile.resolveModelContextFor(dedicatedModelAlias);

•      thinkingEffort = boundModel.thinkingLevel;
     } catch (error) {
       this.log.warn(
         `compaction model "${dedicatedModelAlias}" is not configured; falling back to current model "${currentModelAlias}"`,
  

@@ -673,7 +675,7 @@ export class AgentFullCompactionService extends Service implements IAgentFullCom
const boundMaxOutputSize = boundModel.maxOutputSize ?? defaultCompactionCap;
let compactionRequestModel = hasDedicatedModel ? dedicatedModelAlias : undefined;
let effectiveModelAlias = hasDedicatedModel ? dedicatedModelAlias : currentModelAlias;

•  const effectiveMaxOutputSize = Math.min(
  

•  let effectiveMaxOutputSize = Math.min(
     compactionMaxOutputSize ?? DEFAULT_COMPACTION_MAX_COMPLETION_TOKENS,
     boundMaxOutputSize ?? DEFAULT_COMPACTION_MAX_COMPLETION_TOKENS,
   );
  

@@ -751,6 +753,11 @@ export class AgentFullCompactionService extends Service implements IAgentFullCom
if (error instanceof CompactionTruncatedError) {
compactionMaxOutputSize = Math.floor((compactionMaxOutputSize ?? DEFAULT_COMPACTION_MAX_COMPLETION_TOKENS) / 2);

•        effectiveMaxOutputSize = Math.min(
  

•          compactionMaxOutputSize ?? DEFAULT_COMPACTION_MAX_COMPLETION_TOKENS,
  

•          (usingFallbackModel ? currentModelMaxOutputSize : boundMaxOutputSize) ??
  

•            DEFAULT_COMPACTION_MAX_COMPLETION_TOKENS,
  

•        );
         retryCount = 0;
         continue;
       }
  

@@ -758,6 +765,10 @@ export class AgentFullCompactionService extends Service implements IAgentFullCom
) {
usingFallbackModel = true;
effectiveModelAlias = currentModelAlias;
compactionRequestModel = undefined;

•        effectiveMaxOutputSize = Math.min(
  

•          compactionMaxOutputSize ?? DEFAULT_COMPACTION_MAX_COMPLETION_TOKENS,
  

•          currentModelMaxOutputSize ?? DEFAULT_COMPACTION_MAX_COMPLETION_TOKENS,
  

•        );
         this.log.warn(
           `compaction model "${dedicatedModelAlias}" failed; falling back to current model "${currentModelAlias}"`,
           { cause: wrapCompactionModelError(error, dedicatedModelAlias) },
  

local review verdict: findings — 1) effectiveMaxOutputSize never recomputed on CompactionTruncatedError retry, breaking truncation handling; 2) thinking_effort telemetry reports caller model's level instead of dedicated compaction model's level — reviewed #1

[arrrrny]

Fetch URLs
2 pages

effectiveMaxOutputSize is declared const and never recomputed when compactionMaxOutputSize is halved on CompactionTruncatedError, so every retry after truncation still requests the original (too-large) limit. The same variable is also not recomputed after falling back to the current model, so the fallback request may carry the dedicated model's cap instead of the current model's cap.
thinkingEffort is captured from the caller's resolvedModel before the dedicated compaction model is resolved and is never updated. When a dedicated model is configured (especially with defaultEffort), the compaction instruction template and compaction_finished telemetry still report the caller's model thinking level instead of the dedicated model's.
diff
Copy
diff --git a/packages/agent-core-v2/src/agent/fullCompaction/fullCompactionService.ts b/packages/agent-core-v2/src/agent/fullCompaction/fullCompactionService.ts
index 014355e4f7..b5e5aa29b4 100644
--- a/packages/agent-core-v2/src/agent/fullCompaction/fullCompactionService.ts
+++ b/packages/agent-core-v2/src/agent/fullCompaction/fullCompactionService.ts
@@ -649,6 +649,7 @@ export class AgentFullCompactionService extends Service implements IAgentFullCom
? Math.min(maxContextTokens, DEFAULT_COMPACTION_MAX_COMPLETION_TOKENS)
: undefined;
const compactionMaxOutputSize = resolvedModel.maxOutputSize ?? defaultCompactionCap;

•  const currentModelMaxOutputSize = compactionMaxOutputSize;
  
   const binding = compactionModelBindingFor(this.configService, this.flags, {
     modelAlias: currentModelAlias,
  

@@ -662,6 +663,7 @@ export class AgentFullCompactionService extends Service implements IAgentFullCom
if (hasDedicatedModel) {
try {
boundModel = this.profile.resolveModelContextFor(dedicatedModelAlias);

•      thinkingEffort = binding.thinking ?? boundModel.thinkingLevel;
     } catch (error) {
       this.log.warn(
         `compaction model "${dedicatedModelAlias}" is not configured; falling back to current model "${currentModelAlias}"`,
  

@@ -673,7 +675,7 @@ export class AgentFullCompactionService extends Service implements IAgentFullCom
const boundMaxOutputSize = boundModel.maxOutputSize ?? defaultCompactionCap;
let compactionRequestModel = hasDedicatedModel ? dedicatedModelAlias : undefined;
let effectiveModelAlias = hasDedicatedModel ? dedicatedModelAlias : currentModelAlias;

•  const effectiveMaxOutputSize = Math.min(
  

•  let effectiveMaxOutputSize = Math.min(
     compactionMaxOutputSize ?? DEFAULT_COMPACTION_MAX_COMPLETION_TOKENS,
     boundMaxOutputSize ?? DEFAULT_COMPACTION_MAX_COMPLETION_TOKENS,
   );
  

@@ -751,6 +753,11 @@ export class AgentFullCompactionService extends Service implements IAgentFullCom
if (error instanceof CompactionTruncatedError) {
compactionMaxOutputSize = Math.floor((compactionMaxOutputSize ?? DEFAULT_COMPACTION_MAX_COMPLETION_TOKENS) / 2);

•        effectiveMaxOutputSize = Math.min(
  

•          compactionMaxOutputSize ?? DEFAULT_COMPACTION_MAX_COMPLETION_TOKENS,
  

•          (usingFallbackModel ? currentModelMaxOutputSize : boundMaxOutputSize) ??
  

•            DEFAULT_COMPACTION_MAX_COMPLETION_TOKENS,
  

•        );
         retryCount = 0;
         continue;
       }
  

@@ -758,6 +765,10 @@ export class AgentFullCompactionService extends Service implements IAgentFullCom
) {
usingFallbackModel = true;
effectiveModelAlias = currentModelAlias;

•        thinkingEffort = resolvedModel.thinkingLevel;
         compactionRequestModel = undefined;
  

•        effectiveMaxOutputSize = Math.min(
  

•          compactionMaxOutputSize ?? DEFAULT_COMPACTION_MAX_COMPLETION_TOKENS,
  

•          currentModelMaxOutputSize ?? DEFAULT_COMPACTION_MAX_COMPLETION_TOKENS,
  

•        );
         this.log.warn(
           `compaction model "${dedicatedModelAlias}" failed; falling back to current model "${currentModelAlias}"`,
           { cause: wrapCompactionModelError(error, dedicatedModelAlias) },
  

diff --git a/packages/agent-core-v2/test/agent/fullCompaction/compaction-model.test.ts b/packages/agent-core-v2/test/agent/fullCompaction/compaction-model.test.ts
index 05ae86d903..c8a3f2e1b4 100644
--- a/packages/agent-core-v2/test/agent/fullCompaction/compaction-model.test.ts
+++ b/packages/agent-core-v2/test/agent/fullCompaction/compaction-model.test.ts
@@ -198,4 +198,52 @@ describe('FullCompaction — dedicated compaction model', () => {
expect(finished?.properties?.['model']).toBe('kimi-code');
});

• it('uses the dedicated model thinking level when configured', async () => {
• vi.stubEnv(COMPACTION_MODEL_FLAG_ENV, 'true');
• const { ctx, records } = makeAgent({

•  initialConfig: { compactionModel: { model: 'kimi/compaction', defaultEffort: 'high' } },
  

• });
• seedHistory(ctx);
• await runManualCompaction(ctx, records);
• const finished = compactionFinished(records);
• expect(finished).toBeDefined();
• expect(finished?.properties?.['thinking_effort']).toBe('high');
• });
• it('reverts to the current model thinking level on fallback', async () => {
• vi.stubEnv(COMPACTION_MODEL_FLAG_ENV, 'true');
• let callCount = 0;
• const generate: GenerateFn = async (_chat, _systemPrompt, _tools, _history, _callbacks, options) => {

•  options?.signal?.throwIfAborted();
  

•  callCount += 1;
  

•  if (callCount === 1) {
  

•    throw new APIConnectionError('simulated connection failure');
  

•  }
  

•  const message: Message = {
  

•    role: 'assistant',
  

•    content: [{ type: 'text', text: 'Compacted summary.' }],
  

•    toolCalls: [],
  

•  };
  

•  options?.onStreamEnd?.();
  

•  return {
  

•    id: 'mock-fallback',
  

•    message,
  

•    usage: { inputOther: 0, output: 0, inputCacheRead: 0, inputCacheCreation: 0 },
  

•    finishReason: 'completed',
  

•    rawFinishReason: 'stop',
  

•    traceId: null,
  

•  };
  

• };
• const { ctx, records } = makeAgent({

•  generate,
  

•  initialConfig: { compactionModel: { model: 'kimi/compaction', defaultEffort: 'high' } },
  

• });
• seedHistory(ctx);
• await runManualCompaction(ctx, records);
• expect(callCount).toBe(2);
• const finished = compactionFinished(records);
• expect(finished).toBeDefined();
• expect(finished?.properties?.['thinking_effort']).not.toBe('high');
• });
  });
  local review verdict: findings — 1) effectiveMaxOutputSize never recomputed on CompactionTruncatedError retry or fallback, breaking truncation handling; 2) thinking_effort telemetry and prompt use caller model's level instead of dedicated compaction model's level — reviewed feat(agent-core-v2): add dedicated compaction model with fallback #1