perf(agent): defer synchronous httpx.post out of AIAgent.__init__ (#32221)

[rodboev]

Summary

ContextCompressor.__init__ called get_model_context_length() synchronously during agent construction. Inside the resolver at step 5e (agent/model_metadata.py:1716), _query_ollama_api_show() opens a synchronous httpx.Client and POSTs to {base_url}/api/show against any base_url, with a 5-second timeout configured at agent/model_metadata.py:1086. For Ollama endpoints this resolves the GGUF context length; for every other server (OpenAI, Anthropic, CLIProxyAPI, etc.) it returns 404/405 after a full round-trip. cProfile benchmarking by the reporter showed this adds roughly 150 to 160ms per construction on local networks, and in a 10-iteration loop each construction paid the full cost: there is no amortization.

For the legacy single-agent CLI case where one agent is instantiated at startup, the penalty is invisible. For architectures that instantiate ephemeral per-turn agents or run dense multi-agent orchestration (the reporter's use case), every construction blocks on this probe, destroying the latency budget. The probe is also a security anti-pattern: burying synchronous network egress inside a constructor creates an unobservable SSRF vector if the agent is constructed in a web context with user-controlled routing, and a hung Ollama server locks the main thread during object creation.

The compression feasibility check was already deferred to first turn in agent_init.py:1616-1622 (lazy via _compression_feasibility_checked), but the ContextCompressor.__init__ context-length resolution at agent/context_compressor.py:616 still ran eagerly. This is the remaining synchronous network I/O in the constructor path.

The fix replaces context_length and its derived values (threshold_tokens, tail_token_budget, max_summary_tokens) with lazy-resolved properties. ContextCompressor.__init__ stores the resolution parameters but does not call get_model_context_length(). The first access to .context_length triggers resolution and caches the result; subsequent accesses return the cached value. A setter ensures the update_model() and switch_model paths that assign compressor.context_length = N directly still work, bypassing the resolver entirely. For agents with quiet_mode=True (gateway subagents, ephemeral agents), the property is never accessed during construction, so the blocking probe is fully deferred. The config_context_length fast path (step 0 of get_model_context_length) still short-circuits without any network I/O when the user has configured an explicit context length. Related issues: #8499, #12977, #13492.

Changes

• agent/context_compressor.py: replace eager get_model_context_length() call in __init__ with stored parameters; add _resolve_context_length() private method, context_length lazy property with getter/setter, and lazy properties for threshold_tokens, tail_token_budget, max_summary_tokens (+64 lines, -20 lines)
• tests/agent/test_context_compressor.py: update compressor fixture and TestSummaryTargetRatio to resolve context_length inside the mock's with block; add TestLazyContextResolution with 3 tests verifying init deferral, setter bypass, and config fast-path (+65 lines)

Validation

Scenario	Before	After
CLI agent construction (quiet_mode=False)	get_model_context_length called in ContextCompressor.__init__; httpx.post blocks for ~150ms	get_model_context_length called on first .context_length access (triggered by init_agent print at line 1606); same latency for CLI, just via property
Gateway/subagent construction (quiet_mode=True)	same blocking probe during construction	probe deferred until first .context_length access (first turn, not construction); construction is ~150ms faster
Ephemeral agent loop (10x construction)	10 x ~150ms = ~1.5s of mandatory network latency	construction cost is memory allocation only; probes deferred to first use
config_context_length set in config.yaml	step 0 returns immediately (unchanged)	step 0 returns immediately (unchanged); no regression
update_model(model, context_length=N)	sets self.context_length = N	goes through setter, caches N, no resolver call (unchanged behavior)
switch_model / fallback activation	assigns compressor.context_length	goes through setter (unchanged behavior)
Existing compressor tests	pass	pass (no behavior change for resolved values)
Existing feasibility tests	pass	pass (already test the deferred path)

Test plan

• pytest tests/agent/test_context_compressor.py -v --timeout=0 — 94 passed
• pytest tests/run_agent/test_compression_feasibility.py -v --timeout=0 — 16 passed

Not in scope

Making get_model_context_length itself async is deliberately left out. The function is called from many synchronous paths (CLI, config resolution, model switch, plugin init) and converting it would require async propagation across a large surface. The lazy property approach eliminates the blocking I/O from the constructor path without touching the resolver itself. A follow-up could introduce an async variant for the gateway's event-loop context, but this PR focuses on the constructor latency that the report describes.

Upstream

Closes #32221.
Reported by @twocash.

[teknium1]

Thanks for isolating the constructor-side context-resolution cost. The eager resolution remains present on current main, but the branch predates important initialization and compression changes.

Problems

• The proposed getter does not defer the work across current agent construction: agent/agent_init.py:1871 reads context_compressor.context_length immediately after constructing it for the minimum-context guard. That invokes the new getter during the same initialization path.
• Current agent/context_compressor.py:1082-1102 applies the raise-only 75% threshold floor for windows below 512K after resolving context length. The added _resolve_context_length() in this PR only caches the value, so it would omit that current invariant after salvage.

Suggested changes

• Preserve the current minimum-context validation and decide explicitly where it may resolve metadata without defeating the intended deferral.
• Apply the current effective-threshold calculation when lazy resolution first occurs, and add an AIAgent-path regression test plus a sub-512K threshold test.

Automated hermes-sweeper review.

[rodboev]

You're right, the lazy property alone still left three eager-read sites alive: validation, session metadata, and the deferred runtime snapshot.

I moved the quiet-path resolution to first-turn setup, kept the non-quiet CLI banner as the explicit eager path, and reapplied the current _effective_threshold_percent() policy when context length resolves or gets assigned directly. That keeps the sub-512K 75 percent floor, the 512K boundary behavior, and the setter-based model-switch paths intact.

I also widened the change across the remaining eager-read surfaces in agent/agent_init.py, agent/turn_context.py, and agent/agent_runtime_helpers.py, then added regressions for quiet AIAgent construction, first-turn resolution before model I/O, below-minimum first-turn failure, threshold-floor behavior, the 512K boundary, setter bypass, and primary-runtime restoration.

[kshitijk4poor]

Merged via #57229 — both your commits were cherry-picked onto current main with authorship preserved (perf(agent): defer synchronous httpx.post out of AIAgent.__init__ + test fallout fix), closing #32221. We added a follow-up so the deferral also holds on the interactive (quiet_mode=False) path, plus coherence guards on the context_length setter. Thanks for the contribution!