Skip to content
11 changes: 10 additions & 1 deletion docs/design/full-turn-multimodal-routing.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ Missing or false `agent` capability keeps the existing Vision Bridge transcripti
- If the selected vision model is not agent-capable, transcribe through Vision Bridge and answer on the primary.
- If the selected vision model is agent-capable, keep the original image parts and set a turn-local exact model selector.
- The exact provider, model, and endpoint are reused for provider retries, tool execution, tool-result continuations, and blocking ACP Stop Hook continuations.
- Headless tool execution receives the same runtime view as the selected image model; queued notification and cron drains remain independent turns and do not inherit it.
- Configured fallback models are disabled for that turn. Failure to resolve the exact route fails closed instead of sending raw image data to the primary.
- The next independent user turn clears the selector and returns to the primary. Every model request, including side queries, receives only media modalities supported by its exact target.

Expand All @@ -39,4 +40,12 @@ LLM-based automatic chat compression remains on the primary-model path. A full-t

## Entry points

Phase 1 covers the interactive TUI and ACP. Non-interactive routing is intentionally unchanged until it has an equivalent turn-local lifecycle.
Phase 1 covers the interactive TUI, ACP, and non-interactive CLI.

Textual `@` paths are resolved to their canonical target before MIME detection, workspace checks, ignore filtering, and file reads. Both the user-supplied alias and canonical target must pass ignore filtering, so a symlink cannot disguise an ignored file or a non-image target.

## Durable visual context

Historical images are represented by deterministic `Image #<id>` references. Below the payload threshold all historical images remain attached; at or above it only the configured recent images are attached. A prompt that explicitly names one or more image IDs attaches only those images, then follows the same bridge or full-turn routing decision as a newly supplied image.

Resume rebuilds the session-scoped image store from the original JSONL messages, including images older than the latest compression checkpoint. Compaction restoration headers retain the same IDs, and `/clear` drops the store. Raw bytes stay in the attachment layer and are never inserted into text sent to a text-only model.
Loading