feat: pi-parity for chat + agent LLM loops (shared stream layer, reasoning round-trip, developer role)

changelogs/v2.6.5.md

@@ -0,0 +1,13 @@
+## What's new
+
+### Fixes
+
+- **Cairn Agent no longer fails against providers that reject the top-level `system` parameter.** The agent loop sent the system prompt as an Anthropic-style top-level `system` field in the request body (`{ model, system: …, messages: … }`), which strict OpenAI-compatible providers reject outright with *unknown parameter system* — breaking the agent (and compaction) entirely on those providers, while regular chat worked because it sends the prompt as a normal `role: "system"` message. The agent loop and the context-compaction call now both send the system prompt the same way chat does, as a `role: "system"` message, so pi-agent works identically against any endpoint that chat does.
+- **A cut-off agent reply can no longer execute a half-built tool call.** The agent loop never read `finish_reason` from the stream, so a model that hit its output-token limit mid-call could still have its (possibly truncated) arguments executed — the same failure chat guarded against. The agent loop now captures `finish_reason` like chat does: any tool call from a turn that ended with `finish_reason: "length"` is refused, shown as failed, and the model is asked to re-issue with complete arguments.
+- **Agent reasoning now round-trips to the same model, pi-style, instead of being thrown away.** Previously the agent stripped chain-of-thought from every re-sent message, which breaks multi-turn reasoning continuity on models that expect their thinking back (Claude/OpenAI encrypted reasoning, DeepSeek/Qwen `reasoning_content`). The agent now records which reasoning field the provider used and sends it back under that same field — but only to the *same* model that produced it. If you switch provider/model mid-session, the old model's reasoning is converted to plain text (pi behaviour) so the new model keeps the context without risking an unknown-field rejection.
+- **Reasoning is never baked into the reply text anymore.** When a "thinking" model exhausts its output budget having emitted only chain-of-thought, the reasoning stays in its own panel instead of being echoed back as message content — matching pi. Reasoning-only turns are still preserved and round-tripped to the same model.
+- **An interrupted stream can no longer execute a half-built tool call.** A connection that drops mid-call ends the stream without any `finish_reason`. Both chat and the agent now treat that the same as an output-token cut: buffered tool calls are refused, shown as failed, and the model is asked to re-issue with complete arguments.
+- **Empty assistant turns no longer poison the next agent request.** A reasoning model that stopped mid-thought can leave behind an assistant turn with neither content nor tool calls once reasoning is stripped — replaying it makes strict providers reject the whole request with *content or tool_calls must be set*. The agent now filters those turns out of outgoing requests, matching chat.
+- **Chat and the agent now share one LLM streaming layer.** The SSE parsing (reasoning-field capture, `finish_reason` tracking, tool-call buffering), the request-body construction, the truncated-tool-call guard, and the message-preparation step (system prompt, reasoning round-trip, empty-turn filtering) were duplicated between `chat-loop` and the agent loop and had silently drifted. They now live in a single shared module (`llm-stream`) consumed by both loops, so a fix or improvement lands in both at once.
+- **Reasoning models get the `developer` system role on supported providers (pi parity).** The system prompt is now sent as `role: "developer"` instead of `role: "system"` when the selected model is a reasoning/thinking model (from the models.dev catalog) and the provider supports it — the OpenAI convention pi follows. It falls back to `system` on providers known to reject the `developer` role (DeepSeek, NVIDIA, xAI, Together, Moonshot, ZAI, Cloudflare, Ant Ling, on-device Llama, and non-`anthropic/`/`openai/` OpenRouter models), so this can't break a provider that doesn't understand it.
+- **The ask-questions form keeps the question visible while you answer.** The question text used to live as the *placeholder* inside the answer field, so it vanished the moment you started typing — and the form could also arrive out of view if you'd scrolled up. The question is now rendered as persistent text above each answer field (in both chat and the agent), and the thread auto-scrolls to the form so you always see what's being asked next to where you type.

electron/ipc/chat.ts

@@ -16,6 +16,7 @@ import { TOOLS, buildSystemPrompt, type ChatRequest } from "../lib/tools";
 import { getExternalToolDefs } from "../lib/external-tools";
 import { getCachedConfig, cacheLlmConnection } from "../lib/config-cache";
 import { runToolLoop } from "../lib/chat-loop";
+import { resolveSystemRole } from "../lib/llm-stream";
 import { resolveLlmApiKey } from "../lib/secure-store";
 import { buildAttachmentParts } from "../../shared/models/pdf-attach";
 import { resolveCreditSpec, probeCredits } from "../lib/provider-credits";
@@ -153,7 +154,10 @@ export function registerChatHandler(db: Database.Database, workspacePath: string
       : { role: "user", content: req.message };
 
     const messages: OpenAIMessage[] = [
-      { role: "system", content: buildSystemPrompt(req) },
+      {
+        role: resolveSystemRole({ isReasoningModel: req.config?.isReasoningModel, baseUrl, provider, modelId: model }),
+        content: buildSystemPrompt(req),
+      },
       ...(req.history ?? [])
         // Drop assistant turns that carry neither content nor tool_calls — a
         // thinking model that timed out or stopped mid-reasoning leaves such a

electron/ipc/pi-agent.ts

@@ -56,6 +56,7 @@ interface PiAgentPromptRequest {
     temperature?: number;
     maxTokens?: number;
     autoApprove?: boolean;
+    isReasoningModel?: boolean;
   };
 }
 
@@ -75,6 +76,7 @@ interface PiAgentApprovePlanRequest {
     temperature?: number;
     maxTokens?: number;
     autoApprove?: boolean;
+    isReasoningModel?: boolean;
   };
 }
 
@@ -235,6 +237,7 @@ export function registerPiAgentHandler(
       temperature: reqConfig?.temperature ?? 0.3,
       maxTokens:   reqConfig?.maxTokens,
       autoApprove: reqConfig?.autoApprove !== undefined ? reqConfig.autoApprove : true,
+      isReasoningModel: reqConfig?.isReasoningModel,
     };
 
     let session = sessions.get(sessionId);
@@ -330,6 +333,7 @@ export function registerPiAgentHandler(
       temperature: reqConfig?.temperature ?? 0.3,
       maxTokens:   reqConfig?.maxTokens,
       autoApprove: reqConfig?.autoApprove !== undefined ? reqConfig.autoApprove : true,
+      isReasoningModel: reqConfig?.isReasoningModel,
     };
 
     let session = sessions.get(sessionId);

electron/lib/chat-loop.ts

@@ -16,12 +16,12 @@ import type { BrowserWindow } from "electron";
 import type Database from "better-sqlite3";
 import type { OpenAIMessage } from "./llm";
 import { buildApiUrl } from "./llm";
+import { buildChatCompletionsBody, consumeAssistantStream, failToolCallsFromTruncatedMessage, interruptedStreamToolCallError } from "./llm-stream";
 import { TOOLS, type ChatRequest } from "./tools";
 import { executeTool } from "../ipc/chat-executor";
 import { executeExternalTool, isExternalToolName, externalToolLabel } from "./external-tools";
 import { extractExternalRef } from "./external-ref";
 import { externalOutputError } from "./tool-result";
-import { iterSseData } from "./sse";
 import { traceTool } from "./tool-trace";
 import { parseToolArgs } from "./parse-tool-args";
 
@@ -202,18 +202,13 @@ export async function runToolLoop(
           method: "POST",
           headers,
           signal,
-          body: JSON.stringify({
+          body: JSON.stringify(buildChatCompletionsBody({
             model,
             messages,
             tools: combinedTools,
-            tool_choice: "auto",
-            // Only send max_tokens when the user set an explicit cap — omitting
-            // it lets the model finish naturally (see maxTokens above).
-            ...(maxTokens ? { max_tokens: maxTokens } : {}),
+            maxTokens,
             temperature,
-            stream: true,
-            stream_options: { include_usage: true },
-          }),
+          })),
         });
       } catch (_err) {
         if (signal?.aborted) return { exhausted: true, content: "", reasoning: accumulatedReasoning };
@@ -228,118 +223,56 @@ export async function runToolLoop(
       const reader = response.body?.getReader();
       if (!reader) return { exhausted: true, content: "No response stream", reasoning: "" };
 
-      let contentBuffer = "";
-      const toolCallBuffers: Map<number, { id: string; name: string; args: string; thought_signature?: string }> = new Map();
-
-      for await (const jsonStr of iterSseData(reader, signal ?? undefined)) {
-        try {
-          // eslint-disable-next-line @typescript-eslint/no-explicit-any
-          const chunk = JSON.parse(jsonStr) as any;
-          if (chunk.usage && onUsage) {
-            const topCost = chunk.cost as { request_cost_usd?: unknown } | undefined;
-            onUsage(
-              chunk.usage.prompt_tokens ?? 0,
-              chunk.usage.completion_tokens ?? 0,
-              chunk.usage.completion_tokens_details?.reasoning_tokens ?? 0,
-              typeof chunk.usage.cost === "number"
-                ? chunk.usage.cost
-                : typeof topCost?.request_cost_usd === "number"
-                  ? topCost.request_cost_usd
-                  : undefined,
-            );
-          }
-          const delta = chunk.choices?.[0]?.delta;
-          // Capture the finish reason — "length" means the model hit max_tokens.
-          // A reasoning model can hit it having emitted only chain-of-thought.
-          if (chunk.choices?.[0]?.finish_reason) {
-            turnFinishReason = chunk.choices[0].finish_reason;
-          }
-          if (!delta) continue;
-
-          if (delta.content) {
-            contentBuffer += delta.content;
-            accumulatedContent += delta.content;
-            if (onToken) onToken(delta.content);
-          }
-
-          // Reasoning / thinking stream (Claude thinking_delta, OpenAI delta.reasoning,
-          // DeepSeek/Qwen-style delta.reasoning_content).
-          // Models that don't expose reasoning text simply never emit this field —
-          // the panel stays hidden. Reasoning is NOT merged into content/tool JSON.
-          const thought = delta.reasoning_content ?? delta.reasoning;
-          if (thought) {
-            accumulatedReasoning += thought;
-            if (onThought) onThought(thought);
-          }
-
-          if (delta.tool_calls) {
-            for (const tc of delta.tool_calls) {
-              const idx: number = tc.index ?? 0;
-              if (!toolCallBuffers.has(idx)) {
-                toolCallBuffers.set(idx, { id: tc.id ?? "", name: tc.function?.name ?? "", args: "" });
-              }
-              const buf = toolCallBuffers.get(idx)!;
-              if (tc.id) buf.id = tc.id;
-              if (tc.function?.name) buf.name = tc.function.name;
-              if (tc.function?.arguments) buf.args += tc.function.arguments;
-              // Gemini 3.x thought signature — opaque blob to round-trip back.
-              if (tc.thought_signature) buf.thought_signature = tc.thought_signature;
-            }
-          }
-        } catch { /* skip malformed SSE JSON line */ }
-      }
+      // Shared SSE parse (same as the agent loop) so reasoning-field capture,
+      // finish_reason tracking, and tool-call buffering can never diverge again.
+      const turn = await consumeAssistantStream(reader, {
+        signal,
+        onToken: (delta) => { if (onToken) onToken(delta); },
+        onThought: (delta) => { if (onThought) onThought(delta); },
+        onUsage: (usage) => {
+          if (!onUsage) return;
+          const topCost = usage.chunkCost as { request_cost_usd?: unknown } | undefined;
+          const raw = usage.raw as { cost?: unknown };
+          onUsage(
+            usage.promptTokens,
+            usage.completionTokens,
+            usage.reasoningTokens,
+            typeof raw.cost === "number"
+              ? raw.cost
+              : typeof topCost?.request_cost_usd === "number"
+                ? topCost.request_cost_usd
+                : undefined,
+          );
+        },
+      });
+      // Reasoning and content are accumulated here (onToken/onThought only stream).
+      accumulatedContent += turn.content;
+      accumulatedReasoning += turn.reasoning;
+      turnFinishReason = turn.finishReason;
 
       // Dev trace: per-tool assembled arguments.
-      for (const [idx, buf] of toolCallBuffers.entries()) {
+      for (let i = 0; i < turn.toolCalls.length; i++) {
+        const tc = turn.toolCalls[i];
         traceTool("sse-args", {
-          toolIndex: idx,
-          toolName: buf.name,
-          arguments: buf.args,
+          toolIndex: turn.toolCallIndexes[i],
+          toolName: tc.function.name,
+          arguments: tc.function.arguments,
         });
       }
 
       if (signal?.aborted) return { exhausted: true, content: "", reasoning: accumulatedReasoning };
 
-      const toolCalls = toolCallBuffers.size > 0
-        ? Array.from(toolCallBuffers.entries())
-            .sort(([a], [b]) => a - b)
-            .map(([, buf]) => ({
-              id: buf.id,
-              type: "function" as const,
-              function: { name: buf.name, arguments: buf.args },
-              ...(buf.thought_signature ? { thought_signature: buf.thought_signature } : {}),
-            }))
-        : undefined;
+      const toolCalls = turn.toolCalls.length > 0 ? turn.toolCalls : undefined;
 
       assistantMsg = {
         role: "assistant" as const,
-        content: contentBuffer || null,
+        content: turn.content || null,
         // Note: reasoning is intentionally NOT included here. It is
         // accumulated separately in `accumulatedReasoning` and returned
-        // to the caller for UI/persistence. Sending it back to the API
-        // would violate both OpenAI and Anthropic message schemas.
+        // to the caller for UI/persistence (the ThinkingPanel renders it).
+        // Reasoning is never baked into `content` — matching pi.
         tool_calls: toolCalls,
       };
-
-      // Reasoning-budget exhaustion safety net (cloud/streaming). By default we
-      // omit max_tokens so this shouldn't happen, but a user-set cap (or a
-      // provider's own server-side limit) can still stop a "thinking" model with
-      // finish_reason:"length" having emitted only reasoning and no content.
-      // That empty turn is useless to show AND, if persisted, poisons the next
-      // request (the provider rejects an assistant message with neither content
-      // nor tool_calls — a 400). Surface the captured reasoning so the reply is
-      // never empty.
-      if (
-        (!assistantMsg.content || !assistantMsg.content.trim()) &&
-        !assistantMsg.tool_calls?.length &&
-        turnFinishReason === "length" &&
-        accumulatedReasoning.trim()
-      ) {
-        const surfaced = `*[This model hit its output-token limit before giving a final answer. Its reasoning is shown below — raise or clear "Max output tokens" in AI settings for a complete reply.]*\n\n${accumulatedReasoning.trim()}`;
-        assistantMsg.content = surfaced;
-        accumulatedContent = surfaced;
-        if (onToken) onToken(surfaced);
-      }
     }
 
     // No tool calls — model is ready to produce its final reply
@@ -348,29 +281,26 @@ export async function runToolLoop(
       return { exhausted: false, content: accumulatedContent, reasoning: accumulatedReasoning };
     }
 
-    // Output-token-limit truncation guard. A "length" finish means the model ran
-    // out of tokens mid-turn, so every tool call in this message may carry
-    // truncated arguments. The worst case is a cut that lands on a well-formed
-    // boundary: the partial JSON parses cleanly and would be executed with
-    // silently missing fields — something no JSON parser can detect. Mirroring
-    // pi's `failToolCallsFromTruncatedMessage`, refuse to execute ANY of them:
-    // emit the chip + a structured error so the model re-issues with complete
+    // Output-token-limit / interrupted-stream truncation guard. A "length"
+    // finish (or a stream that ended without ANY finish_reason — connection cut
+    // mid-call) means the tool calls in this message may carry truncated
+    // arguments. The worst case is a cut that lands on a well-formed boundary:
+    // the partial JSON parses cleanly and would be executed with silently
+    // missing fields — something no JSON parser can detect. Mirroring pi's
+    // `failToolCallsFromTruncatedMessage`, refuse to execute ANY of them: emit
+    // the chip + a structured error so the model re-issues with complete
     // arguments (or the user raises the output-token cap).
-    if (turnFinishReason === "length") {
+    const streamInterrupted = turnFinishReason === null && (assistantMsg.tool_calls?.length ?? 0) > 0;
+    if (turnFinishReason === "length" || streamInterrupted) {
       messages.push(assistantMsg);
-      // Tailor the guidance to the config: on Auto we send NO max_tokens, so a
-      // "length" cut comes from the provider's own output cap, not our setting.
-      const capHint = maxTokens
-        ? "Try raising the Max output tokens setting."
-        : "Max output tokens is set to Auto (no cap is sent), so this is the provider's own output limit — try a manual Max output tokens value or a model with a larger output cap.";
-      const truncatedError =
-        `Tool call not executed: the model hit its output-token limit, so the arguments may be truncated. ` +
-        `Re-issue the tool call with complete arguments. ${capHint}`;
-      for (const call of assistantMsg.tool_calls) {
-        emitToolCall({ tool: call.function.name, label: externalToolLabel(call.function.name, db), args: {}, callId: call.id });
-        emitToolCallDone?.({ tool: call.function.name, callId: call.id, ok: false, error: truncatedError });
-        messages.push({ role: "tool", tool_call_id: call.id, content: JSON.stringify({ error: truncatedError }) });
-      }
+      const toolResults = failToolCallsFromTruncatedMessage(assistantMsg.tool_calls, {
+        maxTokens,
+        error: streamInterrupted ? interruptedStreamToolCallError() : undefined,
+        labelFor: (name) => externalToolLabel(name, db),
+        emitStart: (tool, label, callId, args) => emitToolCall({ tool, label, args, callId }),
+        emitEnd: (tool, label, ok, output, callId, _args) => emitToolCallDone?.({ tool, callId, ok, error: output }),
+      });
+      for (const tr of toolResults) messages.push(tr);
       continue;
     }
 

electron/lib/compaction.ts

@@ -132,8 +132,13 @@ export async function generateSummary(
     signal,
     body: JSON.stringify({
       model,
-      system: SUMMARIZATION_SYSTEM_PROMPT,
-      messages: [{ role: "user", content: SUMMARIZATION_USER_PROMPT(conversationText) }],
+      // System prompt as a `role: "system"` message (not a top-level `system:`
+      // field) so strict OpenAI-compatible providers don't reject the request
+      // with "unknown parameter system".
+      messages: [
+        { role: "system", content: SUMMARIZATION_SYSTEM_PROMPT },
+        { role: "user", content: SUMMARIZATION_USER_PROMPT(conversationText) },
+      ],
       max_tokens: SUMMARY_MAX_TOKENS,
       temperature: 0.1, // deterministic summary
       // Must stream to prevent proxy connection drop timeouts (e.g. gateway/reverse proxy limits on blocking sync calls returning 504 Gateway Time-out)