perf(models-dev): load disk cache synchronously, refresh in background

[light-merlin-dark]

Problem

fetch_models_dev() blocks startup on a network GET to models.dev/api.json on every cold start, then serializes and writes a 1.7MB JSON cache to disk. This adds ~600ms to AIAgent.__init__ via ContextCompressor.__init__ → get_model_context_length() → lookup_models_dev_context() → fetch_models_dev().

Profiled breakdown:

• HTTP GET: 400ms (blocking)
• JSON serialize + atomic write: 200ms (1.7MB)

This runs on every cold process start even when a fresh disk cache exists from a recent run.

Solution

Restructure fetch_models_dev() to:

1. Load the disk cache synchronously (fast, ~25ms)
2. Kick off a background thread for the network refresh
3. Only block on the network fetch when no disk cache exists at all (first-ever run)

This follows the same pattern already used in run_agent.py for OpenRouter model metadata pre-warming.

Metrics

Measured on macOS M4 Max, Python 3.11, Hermes v0.8.0:

	Before	After
Cold start (disk cache exists)	600ms blocking	~25ms (disk load + background thread spawn)
First-ever start (no cache)	600ms blocking	600ms blocking (unchanged)

Startup impact: AIAgent.__init__ drops from ~1.2s to ~0.6s on subsequent runs.

Files changed

• agent/models_dev.py — fetch_models_dev() loads disk cache first; added _background_refresh() helper.

---

Merlin (@EnchantedRobot on X) & GLM 5.1 via OpenCode

[kshitijk4poor]

Thanks @light-merlin-dark — closing as substantially superseded on current main (verified against f64e4f4).

The primary win here (skip the network fetch when the disk cache is fresh) landed via commit 775c0e2 ("perf(models_dev): cache-first lookup, skip network when disk cache is fresh", #22808): fetch_models_dev now short-circuits on a fresh-by-mtime disk cache before any network call, with the in-memory TTL anchored to the disk file's age.

The piece main does NOT have is your stale-cache path: serving the stale disk data immediately and refreshing in a background daemon thread (main still does a synchronous network fetch when the disk cache is expired). If that cold-start-with-stale-cache case matters to you, a focused follow-up PR for just the background refresh would be considered on its own merits — with attention to the thread-spawn-in-library-code trade-off.