Release v0.51.99 (Release BW / stage-392 / 5-PR batch)

CHANGELOG.md

@@ -4,6 +4,16 @@
 ## [Unreleased]
 
 
+## [v0.51.99] — 2026-05-20 — Release BW (stage-392 — 5-PR batch — compact tool activity grouping + CLI sidebar scan cap + title-generation API key forwarding + post-compression replay dedup + clarify popup stability)
+
+### Fixed
+
+- **PR #2638** by @dobby-d-elf — Keep compact tool activity grouped under a single Activity disclosure within an assistant turn. The compact renderer was splitting Thinking and interim tool activity into multiple visible fragments inside the same turn. Composes cleanly with v0.51.96's #2620 (second-level dedup key) — the bundled `merge_session_messages_append_only()` change now only adds to `seen_message_keys` when the key is an authoritative `message_id`-prefixed key, preserving the v0.51.96 invariant that two distinct state-only same-second rows must remain visible.
+- **PR #2647** by @Michaelyklam (closes #2628) — Cap the CLI/agent session sidebar bridge to a recent-candidate window before joining message rows. Previously `show_cli_sessions` ran a `LEFT JOIN messages ... GROUP BY s.id` across the entire Hermes `state.db`, aggregating 100k+ message rows before applying the visible sidebar cap. Large installs paid that cost on every sidebar read. The capped path now selects an oversampled candidate set (8× the visible limit) ordered by `started_at DESC`, then runs the message aggregate inside that window. Uncapped callers (full scans, exports) are unchanged.
+- **PR #2650** by @starship-s — Forward the configured `auxiliary.title_generation.api_key` for config-derived title routes. Completes #2612 / v0.51.96 — the prior PR routed provider/model/base_url through but the API key was left to fall back to the chat client's key, which doesn't match the Hermes Agent task-config shape and silently failed for setups where the title model lives behind a different account. The new `caller_supplied_route` guard prevents leakage of the title-generation key to caller-supplied (active-agent fallback) routes.
+- **PR #2651** by @LumenYoung (refs #1217) — Dedupe replayed active-context tails before appending agent result deltas to the WebUI display transcript, and apply the same replay protection to persisted `context_messages`. Without this, post-compression continuation re-fed an already-present tail into the next model turn, inflating the model-facing context and bloating the visible transcript with duplicate assistant cards.
+- **PR #2643** by @arshkumarsingh (closes #2639) — Require a stable `clarify_id` and wait for the backend ack before hiding the WebUI "Clarification needed" popup. Three bugs conspired to cause stale clarifications to silently fail: (a) `_ClarifyEntry` had no unique identifier so the frontend couldn't reference a specific pending prompt; (b) the backend used FIFO resolution which silently dropped late/stale responses; (c) the frontend hid the popup before the POST returned, so users saw a successful submit while the agent fell back to its best-judgement timeout path. Server-side `clarify_id` is now generated in `_ClarifyEntry.__init__` (UUID-based), propagated through SSE/poll payloads, sent back by the browser, and matched via `resolve_clarify_by_id()`. The popup stays visible until the POST returns; a 409/`stale:true` response keeps the draft and shows a toast. The legacy `not bool(clarify_id)` quirk that always returned `ok:true` is gone.
+
 ## [v0.51.98] — 2026-05-20 — Release BV (stage-391 — 1-PR follow-on — custom_providers allowlist priority over live /v1/models)
 
 ### Fixed

api/agent_sessions.py

@@ -401,16 +401,15 @@ def read_importable_agent_session_rows(
         )
 
         where_clauses = ["s.source IS NOT NULL"]
-        params: list[str] = []
+        params: list[object] = []
         if exclude_sources:
             excluded = tuple(str(source) for source in exclude_sources if source)
             if excluded:
                 placeholders = ", ".join("?" for _ in excluded)
                 where_clauses.append(f"s.source NOT IN ({placeholders})")
                 params.extend(excluded)
 
-        cur.execute(
-            f"""
+        select_sql = f"""
             SELECT s.id, s.title, s.model, s.message_count,
                    s.started_at, s.source,
                    {session_source_expr},
@@ -428,14 +427,47 @@ def read_importable_agent_session_rows(
                    COUNT(m.id) AS actual_message_count,
                    {user_message_count_expr} AS actual_user_message_count,
                    MAX(m.timestamp) AS last_activity
-            FROM sessions s
-            LEFT JOIN messages m ON m.session_id = s.id
-            WHERE {' AND '.join(where_clauses)}
-            GROUP BY s.id
-            ORDER BY COALESCE(MAX(m.timestamp), s.started_at) DESC
-            """,
-            params,
-        )
+        """
+        if limit is not None:
+            result_limit = max(0, int(limit))
+            if result_limit == 0:
+                return []
+            # The sidebar only needs a small visible window. Bound the expensive
+            # messages join to a recent-session candidate set instead of
+            # aggregating every historical Hermes state.db session before
+            # slicing in Python. Oversampling preserves room for hidden
+            # compression segments or other rows filtered after projection.
+            candidate_limit = max(result_limit * 8, result_limit)
+            cur.execute(
+                f"""
+                WITH candidates AS (
+                    SELECT s.id
+                    FROM sessions s
+                    WHERE {' AND '.join(where_clauses)}
+                    ORDER BY s.started_at DESC
+                    LIMIT ?
+                )
+                {select_sql}
+                FROM sessions s
+                JOIN candidates c ON c.id = s.id
+                LEFT JOIN messages m ON m.session_id = s.id
+                GROUP BY s.id
+                ORDER BY COALESCE(MAX(m.timestamp), s.started_at) DESC
+                """,
+                [*params, candidate_limit],
+            )
+        else:
+            cur.execute(
+                f"""
+                {select_sql}
+                FROM sessions s
+                LEFT JOIN messages m ON m.session_id = s.id
+                WHERE {' AND '.join(where_clauses)}
+                GROUP BY s.id
+                ORDER BY COALESCE(MAX(m.timestamp), s.started_at) DESC
+                """,
+                params,
+            )
         projected = _project_agent_session_rows([dict(row) for row in cur.fetchall()])
         projected = [_with_normalized_source(row) for row in projected]
         projected = [row for row in projected if is_cli_session_row_visible(row)]

api/clarify.py

@@ -9,6 +9,7 @@
 import queue
 import threading
 import time
+import uuid
 from typing import Optional
 
 
@@ -25,12 +26,13 @@
 class _ClarifyEntry:
     """One pending clarify request inside a session."""
 
-    __slots__ = ("event", "data", "result")
+    __slots__ = ("event", "data", "result", "clarify_id")
 
     def __init__(self, data: dict):
         self.event = threading.Event()
         self.data = data
         self.result: Optional[str] = None
+        self.clarify_id: str = data.get("clarify_id", "") or uuid.uuid4().hex[:12]
 
 
 def register_gateway_notify(session_key: str, cb) -> None:
@@ -120,6 +122,11 @@ def submit_pending(session_key: str, data: dict) -> _ClarifyEntry:
                 == list(data.get("choices_offered") or [])
             ):
                 entry = last
+                # Dedup re-uses the existing entry with its original clarify_id.
+                # If a future caller pre-populates clarify_id in data, it is
+                # silently discarded here — the original entry's id wins.
+                # Today no caller sets clarify_id (it's generated by __init__),
+                # so this is a non-issue.
                 cb = _gateway_notify_cbs.get(session_key)
                 # Keep _pending aligned to the oldest unresolved entry.
                 _pending[session_key] = gw_queue[0].data
@@ -131,6 +138,8 @@ def submit_pending(session_key: str, data: dict) -> _ClarifyEntry:
                 return entry
 
         entry = _ClarifyEntry(data)
+        # Ensure clarify_id is present in the serialised data the frontend receives.
+        entry.data["clarify_id"] = entry.clarify_id
         gw_queue.append(entry)
         _pending[session_key] = gw_queue[0].data
         cb = _gateway_notify_cbs.get(session_key)
@@ -179,3 +188,28 @@ def resolve_clarify(session_key: str, response: str, resolve_all: bool = False)
         entry.event.set()
         count += 1
     return count
+
+
+def resolve_clarify_by_id(session_key: str, clarify_id: str, response: str) -> bool:
+    """Resolve a specific pending clarify request by its stable id.
+
+    Returns True if the id was found and resolved, False otherwise.
+    """
+    with _lock:
+        q = _gateway_queues.get(session_key)
+        if not q:
+            _pending.pop(session_key, None)
+            return False
+        for i, entry in enumerate(q):
+            if entry.clarify_id == clarify_id:
+                q.pop(i)
+                if q:
+                    _pending[session_key] = q[0].data
+                    _clarify_sse_notify(session_key, dict(q[0].data), len(q))
+                else:
+                    _clear_queue_locked(session_key)
+                    _clarify_sse_notify(session_key, None, 0)
+                entry.result = response
+                entry.event.set()
+                return True
+        return False

api/models.py

@@ -2578,7 +2578,8 @@ def merge_session_messages_append_only(sidecar_messages: list, state_messages: l
             and timestamp <= max_sidecar_timestamp
         ):
             continue
-        seen_message_keys.add(key)
+        if key[0] == "message_id":
+            seen_message_keys.add(key)
         merged_messages.append(msg)
     return merged_messages
 

api/routes.py

@@ -2386,6 +2386,7 @@ def submit_pending(session_key: str, approval: dict) -> None:
         submit_pending as submit_clarify_pending,
         get_pending as get_clarify_pending,
         resolve_clarify,
+        resolve_clarify_by_id,
         sse_subscribe as clarify_sse_subscribe,
         sse_unsubscribe as clarify_sse_unsubscribe,
     )
@@ -2394,6 +2395,7 @@ def submit_pending(session_key: str, approval: dict) -> None:
     get_clarify_pending = lambda *a, **k: None
     clarify_sse_subscribe = None
     resolve_clarify = lambda *a, **k: 0
+    resolve_clarify_by_id = lambda *a, **k: False
 
 
 # ── Login page locale strings ─────────────────────────────────────────────────
@@ -9080,14 +9082,16 @@ def _handle_approval_respond(handler, body):
 
 def _resolve_clarify_legacy(sid: str, clarify_id: str, response: str) -> bool:
     """Resolve clarify through the existing callback path without new state."""
-    # The legacy clarify queue is FIFO and does not yet expose stable ids to the
-    # browser, so clarify_id is accepted by the adapter contract but not used to
-    # create a parallel callback registry in the WebUI process.
+    # When a stable clarify_id is provided, match the specific entry so stale
+    # or late responses from the frontend are reliably rejected (issue #2639).
+    if clarify_id:
+        from api.clarify import resolve_clarify_by_id
+        return resolve_clarify_by_id(sid, clarify_id, response)
+    # Legacy path: resolve the oldest pending entry.  Return the REAL result
+    # instead of the old unconditional True so the frontend can detect when
+    # there is no pending prompt to resolve.
     resolved = resolve_clarify(sid, response, resolve_all=False)
-    # Preserve the historical no-id response shape for old clients/tests: a
-    # plain /api/clarify/respond call returns ok even when no pending prompt is
-    # active. Explicit stale ids remain bounded as not-active under the adapter.
-    return bool(resolved) or not bool(clarify_id)
+    return bool(resolved)
 
 
 def _handle_clarify_respond(handler, body):
@@ -9111,7 +9115,18 @@ def _handle_clarify_respond(handler, body):
         ok = adapter.respond_clarify(sid, clarify_id, response).accepted
     else:
         ok = _resolve_clarify_legacy(sid, clarify_id, response)
-    return j(handler, {"ok": ok, "response": response})
+
+    if not ok:
+        # Both the runtime adapter and legacy paths set ok=False for
+        # stale/expired/wrong-session responses.  The 409 status applies
+        # uniformly regardless of which path resolved the clarify request.
+        return j(handler, {
+            "ok": False,
+            "error": "Clarification prompt expired or not found. The agent may have already proceeded.",
+            "stale": True,
+        }, status=409)
+
+    return j(handler, {"ok": True, "response": response})
 
 
 class _ManualCompressionMemoryHandler:

api/streaming.py

@@ -1369,11 +1369,15 @@ def generate_title_raw_via_aux(
         return None, 'missing_exchange'
     qa, prompts = _title_prompts(user_text, assistant_text)
     configured = _get_aux_title_config()
+    caller_supplied_route = bool(provider or model or base_url)
     provider = provider or configured.get('provider', '') or ''
     if str(provider).strip().lower() == 'auto':
         provider = ''
     model = model or configured.get('model', '') or ''
     base_url = base_url or configured.get('base_url', '') or ''
+    api_key = ''
+    if not caller_supplied_route:
+        api_key = str(configured.get('api_key', '') or '').strip()
     base_max_tokens = _title_completion_budget(provider, model, base_url)
     reasoning_extra = {"reasoning": {"enabled": False}}
     if _is_minimax_route(provider, model, base_url):
@@ -1395,6 +1399,7 @@ def generate_title_raw_via_aux(
                         provider=provider or None,
                         model=model or None,
                         base_url=base_url or None,
+                        api_key=api_key or None,
                         messages=messages,
                         max_tokens=max_tokens,
                         temperature=0.2,
@@ -2199,6 +2204,52 @@ def _messages_have_prefix(messages, prefix):
     return True
 
 
+def _message_replay_key(msg):
+    """Return a stable comparison key for replay/overlap de-duplication."""
+    identity = _message_identity(msg)
+    if identity is not None:
+        return identity
+    if not isinstance(msg, dict):
+        return None
+    return (
+        str(msg.get('role') or ''),
+        _message_text(msg.get('content', '')),
+        str(msg.get('tool_call_id') or ''),
+        json.dumps(msg.get('tool_calls') or [], sort_keys=True, ensure_ascii=False),
+    )
+
+
+def _strip_replayed_prefix(existing_messages, candidates):
+    """Drop a candidate prefix that is already the suffix of existing_messages.
+
+    Compression/continuation can replay the active tail from state.db after the
+    previous WebUI context/display already contains it. Prefix-only merge logic
+    then treats that replayed tail as a fresh delta and duplicates a whole turn.
+    Strip the largest exact suffix/prefix overlap before appending.
+    """
+    existing_messages = list(existing_messages or [])
+    candidates = list(candidates or [])
+    max_overlap = min(len(existing_messages), len(candidates))
+    for overlap in range(max_overlap, 0, -1):
+        left = [_message_replay_key(m) for m in existing_messages[-overlap:]]
+        right = [_message_replay_key(m) for m in candidates[:overlap]]
+        if left == right:
+            return candidates[overlap:]
+    return candidates
+
+
+def _dedupe_replayed_active_context(previous_context, result_messages):
+    """Keep model context append-only without re-appending a replayed tail."""
+    previous_context = list(previous_context or [])
+    result_messages = list(result_messages or [])
+    if not previous_context or not result_messages:
+        return result_messages
+    if not _messages_have_prefix(result_messages, previous_context):
+        return result_messages
+    candidates = result_messages[len(previous_context):]
+    return previous_context + _strip_replayed_prefix(previous_context, candidates)
+
+
 def _is_context_compression_marker(msg):
     if not isinstance(msg, dict):
         return False
@@ -2443,6 +2494,8 @@ def _merge_display_messages_after_agent_result(previous_display, previous_contex
 
     if _messages_have_prefix(result_messages, previous_context):
         candidates = result_messages[len(previous_context):]
+        candidates = _strip_replayed_prefix(previous_display, candidates)
+        candidates = _strip_replayed_prefix(previous_context, candidates)
     else:
         current_user_idx = _find_current_user_turn(result_messages, msg_text)
         marker_candidates = [
@@ -4322,6 +4375,10 @@ def _periodic_checkpoint():
                     _previous_context_messages,
                     _result_messages,
                 )
+                _next_context_messages = _dedupe_replayed_active_context(
+                    _previous_context_messages,
+                    _next_context_messages,
+                )
                 s.context_messages = _next_context_messages
                 s.messages = _merge_display_messages_after_agent_result(
                     _previous_messages,
@@ -4465,6 +4522,10 @@ def _periodic_checkpoint():
                                     _previous_context_messages,
                                     _result_messages,
                                 )
+                                _next_context_messages = _dedupe_replayed_active_context(
+                                    _previous_context_messages,
+                                    _next_context_messages,
+                                )
                                 s.context_messages = _next_context_messages
                                 s.messages = _merge_display_messages_after_agent_result(
                                     _previous_messages,
@@ -5281,6 +5342,10 @@ def _periodic_checkpoint():
                                 _next_context_messages = _restore_reasoning_metadata(
                                     _previous_context_messages, _result_messages,
                                 )
+                                _next_context_messages = _dedupe_replayed_active_context(
+                                    _previous_context_messages,
+                                    _next_context_messages,
+                                )
                                 s.context_messages = _next_context_messages
                                 s.messages = _merge_display_messages_after_agent_result(
                                     _previous_messages,