fix: treat overlong prompts as clean truncated rollouts

[mikasenghaas]

Summary

Overlong prompts (the prompt + requested completion exceeding the model's context window) are a budget limit, not a crash. Previously the model call failed, the interception server returned a 502, and the rollout was recorded as an error. This makes them produce a clean, truncated rollout instead — so a conversation that simply grows past the context window is preserved as data, not lost to an error.

Detection lives in the clients; the truncation policy lives in the interception server.

• v1/errors.py — add OverlongPromptError(ModelError).
• v1/clients/openai.py — model_error(e) maps a provider 4xx whose message matches a context-length phrase to OverlongPromptError (any other failure stays a ModelError).
• v1/clients/renderer.py — rebadge the renderers-native client-side OverlongPromptError (raised pre-flight from GET /v1/models, so it is not an OpenAIError) and the engine 4xx. Without this the renderer path's overlong would slip through uncaught.
• v1/interception/server.py — handle_chat catches OverlongPromptError, sets a context_length truncation stop, and mirrors the existing refused path: it returns the last good turn when a simulated conversation already produced one, else refuses the call to halt the harness cleanly (Harness.run treats an exit after a stop condition as expected).
• v1/trace.py — Trace.is_truncated counts context_length.

Mirrors the v0 path (@handle_openai_overlong_prompt + renderer_client's RendererOverlongPromptError rebadge).

Verification

Ran a multi-turn echo rollout (echo-multi-v1, user-simulator driven, with a long phrases override) against a local vLLM served with --max-model-len 1024, so the conversation grows until the prompt overflows the context window.

Before (the overlong turn surfaces as a hard failure):

openai.InternalServerError: Error code: 502 - "The prompt is 1025 tokens, which exceeds
the model's maximum context length of 1024 tokens" → ProgramError: harness exited 1

After — for both the openai and renderers clients, and both branches:

client	scenario	result
openai	multi-turn overflow (last is not None)	has_error=False, stop_condition=context_length, is_truncated=True, 5 turns preserved
renderers	multi-turn overflow	has_error=False, stop_condition=context_length, is_truncated=True, 5 turns preserved
openai	first-turn overflow (last is None, harness halted via 400)	has_error=False, stop_condition=context_length, is_truncated=True, 0 turns
renderers	first-turn overflow	same

Note

Treat overlong prompts as clean truncated rollouts in the interception server

• Introduces OverlongPromptError in errors.py as a ModelError subclass for context-length failures, distinct from generic model errors.
• Updates openai.py and renderer.py to raise OverlongPromptError instead of ModelError when the underlying error text indicates a context-length overflow.
• Updates the InterceptionServer.handle_chat handler in server.py to catch OverlongPromptError, stop the trace with reason "context_length", and return the last assistant turn as a normal completion (or a 400 if no prior turn exists) instead of a 502.
• Marks traces stopped due to "context_length" as truncated via Trace.is_truncated in trace.py.
• Behavioral Change: overlong-prompt failures that previously produced 502 errors now produce either a valid completion response or a 400, and the trace is flagged as truncated.

^{Macroscope summarized 36d1455.}

---

Note

Medium Risk
Changes rollout completion semantics for a common failure mode (context overflow); incorrect phrase matching could misclassify other API errors as truncation.

Overview
Context-window overflows are treated as clean truncations instead of rollout failures.

Adds OverlongPromptError and maps provider/renderer context-length failures to it via model_error() (OpenAI client) and renderer pre-flight rebadging. The interception server catches this like other budget stops: sets stop_condition to context_length, returns the last good assistant turn when one exists, or a 400 rollout stopped: context_length on the first turn. Trace.is_truncated now includes context_length.

Rollouts that grow past the model limit keep prior turns as data (has_error=False) rather than ending in a 502 and ProgramError.

^{Reviewed by Cursor Bugbot for commit 36d1455. Bugbot is set up for automated code reviews on this repo. Configure here.}

[macroscopeapp]

Approvability

Verdict: Approved

Straightforward bug fix that gracefully handles overlong prompt errors as clean truncations rather than crashes. The change is self-contained, limited to error handling paths, and the author is the primary maintainer of these files.

^{You can customize Macroscope's approvability policy. Learn more.}