feat(code): warn before expensive cold-cache turns

[Mason Daugherty (mdrxy)]

Docs

When an interactive chat message reaches the front of the queue, dcode now warns before sending it to the model if its prompt cache may need re-warming, showing the estimated cold-input cost and warm-cache delta.

flowchart TD
    A["Queued message reaches front of queue"] --> B{"Qualifying message?"}
    B -- "No (e.g. using a slash command, ACP mode)" --> S["Send the message"]
    B -- "Yes - sent inside the interactive TUI and not a slash command" --> C["Load checkpointed last request time + model identity"]
    C -- "Missing or malformed:\nlog a warning and treat as cold (see Fig. 1 for specifics)" --> D
    C -- "Loaded" --> D{"Known cache policy for current model + endpoint?"}
    D -- "No (e.g. custom endpoint or other provider)" --> S
    D -- Yes --> E{"Idle past window, age unknown, or cache-affecting params changed?"}
    E -- "No: within window" --> S
    E -- Yes --> E2{"Context ≥ minimum cacheable tokens?"}
    E2 -- "No: tiny context" --> S
    E2 -- Yes --> F{"Re-warm price delta ≥ threshold?"}
    F -- "No: below threshold or unpriceable" --> S
    F -- Yes --> M["Show blocking warning modal"]
    M -- "Keep draft (Esc)" --> R["Restore draft to chat input"]
    M -- "Cannot be shown (render failure, or another prompt owns the modal slot)" --> R
    M -- "Send anyway" --> S
    M -- "Send + don't warn again this session" --> SS["Mute until restart (in memory)"] --> S
    M -- "Send + don't warn again ever" --> SP["Persist to warnings.suppress in config.toml"] --> S

Loading

Fig. 1 — the "treat as cold" fallback

The main flow compares the checkpointed identity (what last warmed the cache) against the current model (what's about to run). Policy and cost always come from the current model, which is known from live config — so a missing checkpointed identity doesn't need its own policy; it just forces the comparison to "changed":

flowchart TD
    subgraph P["Checkpointed identity (written by middleware after each completed model call)"]
        P1 -- "Request time bad, absent, or in the future" --> P2["Log, discard time:\nthe age is unknown, not zero"]
        P1 -- "Model spec bad" --> P3["Log warning, discard spec"]
    end
    subgraph Q["Current identity (from live config)"]
        Q1["Effective model spec\n(always resolvable)"] --> Q2["Resolve policy + price against this model"]
    end
    P2 -- "No age to compare against a window" --> OUT2["Continue down the cold-cache path:\nreason = age_unknown"]
    P3 -- "Discarded spec reads as the model having changed in last vs. current" --> OUT["Continue down the cold-cache path:\nidentity_changed = true"]
    Q2 --> OUT
    Q2 --> OUT2

Loading

The three causes are kept distinct because each gets its own modal copy, and reporting one as another states something untrue about the user's configuration. In particular an unknown age is never reported as a model change, which would claim a change that never happened.

The retention windows come from documented provider policies:

Provider	Condition	Retention window	Warning wording
Anthropic	any	5m	"expired"
OpenAI	prompt_cache_retention: in_memory	1h	"expired"
OpenAI	prompt_cache_retention: 24h	24h	"expired"
OpenAI	GPT-5.6 or newer, no explicit retention	30m minimum	"may be cold" (provider may retain longer)

The window is either a documented maximum or a documented minimum, and the wording follows from which. Anthropic's TTL and OpenAI's prompt_cache_retention ceilings are maximums, so once the window passes the entry is gone ("expired"). GPT-5.6+'s 30 minutes is a guaranteed minimum that the provider may exceed, so past it the cache can only be called "may be cold".

The two OpenAI knobs are independent, which is why the table is ordered as it is: prompt_cache_retention states a maximum lifetime while the GPT-5.6+ guarantee states a minimum one, so an explicitly configured retention is the later and firmer bound and takes precedence on those models too. Warning a user who asked for 24h at the 30-minute mark would contradict their own configuration.

Anthropic is always 5m regardless of a configured cache_control.ttl. AnthropicPromptCachingMiddleware runs inside ConfigurableModelMiddleware and rewrites model_settings["cache_control"] with its own TTL (5m, which this stack never overrides), so a user-supplied 1h never reaches the API. Treating it as an hour would suppress the warning for 55 minutes of a cache that died at five.

The warning combines three sources of information:

1. Provider retention policies (from the table above) — how long each provider keeps cache entries
2. Checkpointed timing — when you last sent a request to this model
3. Pricing overrides — your configured cost per token

If the estimated re-warm cost reaches your threshold, a modal appears before sending with four keyboard-driven choices. Cost is estimated by pricing two synthetic usage payloads — one billed as a full cache read, one as a cold request — through the ordinary cost-tracking path, so the pricing catalog stays the single source of truth. Both figures are upper bounds and are rounded upward so the displayed estimate never reads lower than the modelled spend.

How a miss is priced follows the model: Anthropic carries a write premium over plain input, GPT-5.6+ bills a miss as a cache write, and OpenAI before 5.6 charges no write surcharge at all, so those misses price at the plain input rate. Tagging a pre-5.6 miss as a write would apply a premium the provider never charges and overstate both the displayed cost and the threshold comparison.

The model identity check looks at the model name plus the invocation parameters that actually select or invalidate a cache entry — prompt_cache_retention, prompt_cache_key, prompt_cache_options, and cache_control. A change to any of them invalidates the prefix, so it warns regardless of how recent the last turn was. Unrelated knobs are deliberately excluded: /effort rewrites reasoning_effort wholesale, and temperature or max_tokens are just as inert for caching, so comparing whole parameter maps would open the modal asserting that the cached prefix "cannot be reused" when nothing about the prefix moved.

Anything that prevents the confirmation from being shown keeps the draft rather than sending: if the modal fails to render, or another prompt already owns the modal slot, the message is not dispatched and the text is restored to the chat input. A warning that cannot be presented must not become an implicit authorization to spend. Where a draft genuinely cannot be put back — the input already holds different text, or the active thread changed while the modal was open — the notification says the message was not sent and points at history recall rather than implying it was merely deferred.

Only interactive chat submissions are checked — prompts sent programmatically (e.g. via ACP from an editor client), slash commands, requests to custom endpoints, and requests without pricing information all skip this warning entirely. The default threshold is $0.50; adjust it with warnings.cold_cache_min_delta_usd.

The request time is kept fresh from two directions. A completed turn reads it back from the checkpoint; a turn that reached the model but ended without a readable checkpoint — an interrupt, say — stamps it locally instead. Without that second path the field would never advance past its initial value, and every send after the first would open the modal reporting no record of a turn that plainly happened.

If the checkpointed request time or model identity cannot be loaded (for example, a checkpoint written by a newer version, or a malformed value), dcode logs a warning naming the affected channel and proceeds down the cold-cache path rather than silently assuming the cache is warm: an unusable request time makes the age unknown, and a missing model identity is treated as a model change. A request time in the future — a corrected clock, or a thread synced from a skewed machine — is treated as unknown age too, since clamping it to zero would place it inside every retention window and suppress the warning outright.

Skipping the warning means sending at full cold-cache price, so the paths that skip it on failure are made visible rather than left to the log alone: if the evaluation itself fails, the session says once that cost checks are unavailable; an unparseable configured base_url warns where it is parsed, since it silently disables warnings for that provider; and a pricing catalog that produces a non-finite figure is reported as a data defect rather than reading as "no price published". A failed persistent suppression names its cause — a malformed [warnings] table needs one line of TOML, not the chmod that a generic "check file permissions" would send the user to.

Made by Open SWE

References

• Plan: https://openswe.vercel.app/agents/f30f3674-c02d-e6cd-6bf2-89df728709cb/plan

[open-swe]

Open SWE Review found 1 potential issue.

Open in Web • View Open SWE trace

[open-swe]

Open SWE Review found 1 potential issue.

Open in Web • View Open SWE trace

[open-swe]

Open SWE Review found 1 potential issue.

Open in Web • View Open SWE trace

[open-swe]

Open SWE Review found 1 potential issue.

Open in Web • View Open SWE trace

[open-swe]

Open SWE Review found 2 potential issues.

Open in Web • View Open SWE trace