feat(context-engine): add preassemble() hook for per-turn rewrite

[100yenadmin]

Summary

Adds an additive, non-breaking ABC method ContextEngine.preassemble(messages, budget_tokens=None) -> list[Message] (default no-op) with a single call site in run_agent.py right after sanitization and before the retry loop. Lets pluggable context engines (LCM-style, etc.) substitute the message list every turn — not just on overflow — without breaking session lineage or rotating the SQLite session ID.

Diff: +110 / 0, 3 files. 22/22 tests pass.

Why

Today ContextEngine.compress() only fires when should_compress() returns True (overflow). Plugin engines that maintain a lossless conversation pyramid (raw + summaries DAG) need to substitute evicted raw turns with summary stubs on every prompt build to keep the prompt under budget while preserving long-range recall — not just at the overflow boundary.

The existing pre_llm_call hook can't do this: it concatenates plugin output into the current user message and never rewrites the message list (intentional, to preserve the prompt cache prefix). The new preassemble() hook gives engines explicit, structured per-turn rewrite capability while keeping pre_llm_call semantics unchanged.

This is independent of and complementary to #22929 (Wire `on_pre_compress` into the context compression pipeline): `on_pre_compress` would fire at compress-time; `preassemble` fires at every API call.

What

• `agent/context_engine.py`: add `preassemble()` method to the ABC. Default returns `messages` unchanged.
• `run_agent.py`: call `self.context_compressor.preassemble(api_messages, budget_tokens=threshold_tokens)` after surrogate sanitization and before the retry loop. Exception-safe — failures log a warning and fall back to unmodified messages. Fires once per turn at the API-call boundary (not inside the retry loop).
• `tests/agent/test_context_engine.py`: 3 new tests covering default no-op behavior (with and without `budget_tokens`) and a subclass-override case where `preassemble` substitutes a stub for middle turns while preserving the system message at index 0 and the final user message.

API design

```python
def preassemble(
self,
messages: List[Dict[str, Any]],
budget_tokens: int = None,
) -> List[Dict[str, Any]]:
"""Per-turn rewrite hook called immediately before each model API request.

Engines may return a substituted message list (e.g., replace evicted
raw turns with summary stubs while preserving the assistant/tool-call
pairing invariant). Default is a no-op: return ``messages`` unchanged.
...
\"\"\"
return messages

```

System message at index 0 (if present) is documented as the engine's responsibility to preserve.

Impact

• Additive only. Default no-op. `ContextCompressor` (the built-in engine) keeps the no-op default; zero behavior change for non-plugin users.
• Test suite: 22/22 pass (3 new + 19 existing) on `tests/agent/test_context_engine.py`.
• Blast radius (independent verification): GitNexus code-graph impact analysis on `ContextEngine` shows LOW risk; 4 direct callers (`ContextCompressor`, `hermes_cli/plugins.py`, `agent/context_compressor.py`, `plugins/context_engine/init.py`); 0 affected processes.
• Exception-safety: a misbehaving plugin engine's `preassemble` can't break the agent loop — failures are logged and the original `api_messages` is sent unchanged.
• Retry loop: hook fires once per turn, BEFORE the retry loop. Retries on transient API errors reuse the substituted messages (intended — we don't want to re-substitute on every retry, that would invalidate the prompt cache).

Context

Drafted as part of electricsheephq/lossless-hermes — a community port of Lossless Claw to Python/Hermes. Full design rationale and the ADR-documented fallback (if this isn't merged) live at docs/adr/010-always-on-assembly.md. The fallback is workable but breaks session lineage (forces `should_compress=True` every turn → SQLite session ID rotation → memory-provider lineage warnings). This patch eliminates that compromise.

The docstring at `agent/context_engine.py:5` already names LCM as a planned tenant of the plugin slot, so this lands on the surface the ABC was designed to support.

Test plan

• `pytest tests/agent/test_context_engine.py -x -q` → 22 passed
• Hermes maintainer review (this PR)
• Smoke test in CI on the upstream matrix

Reversibility

100% reversible: revert the 3-file commit. Zero schema changes, zero external contract changes, zero new dependencies.

[teknium1]

Thanks for the focused ContextEngine extension. The need is still present on current main, but this implementation needs a redesigned salvage rather than a direct transplant.

Problems

• The hunk is stale: run_agent.py:5775-5795 now only forwards to agent/conversation_loop.py; current request assembly and retry handling are at agent/conversation_loop.py:787-1161.
• The proposed call occurs after cache-control and validation. Current main applies cache control at agent/conversation_loop.py:889-894 and request sanitizers through :952; accepting an arbitrary replacement list after those passes bypasses them. The PR test in d07bd3dae1a7a0c26ef75ba61cbeb0c85e5bb8fd also constructs system → user → user, contrary to the role-alternation invariant.
• AGENTS.md:19-23 makes prompt-cache stability an invariant. A per-request rewrite needs an explicit cache contract before it becomes a host API.

Suggested changes

• Move this into a dedicated helper in agent/conversation_loop.py, define its ordering with cache control, validation, pressure checks, and retries, and fail open on invalid outputs.
• Add end-to-end host tests for persistence isolation, retry reuse, invalid replacements, role/tool pairing, and cache-control preservation.
• Coordinate the public hook shape with the open selection/observation proposal in feat(context-engine): add select_context() + on_turn_complete() hooks (RFC #36765; consolidates #41918/#24949/#47109/#50053) #51226.

This is an automated hermes-sweeper review.

[100yenadmin]

Closing in favor of #51226 / RFC #36765. The select_context() design there lands the per-turn context-rewrite capability at the right seam on current main (agent/conversation_loop.py — the review here correctly noted this PR's hunk targeted the old run_agent.py location), and #51226's scope explicitly covers the selection use-case this PR was for.

The review's other points stand too: a per-request rewrite hook needs a defined prompt-cache contract before becoming host API, and #51226 is the right venue for that conversation. Thanks @teknium1 for the thorough salvage analysis.