fix: cap _summary_cache with LRU (max 16 entries) by franksong2702 · Pull Request #2216 · nesquena/hermes-webui

franksong2702 · 2026-05-14T00:49:05Z

Refs #2215 (Fix A).

Thinking Path

Issue #2215 flagged _summary_cache as an unbounded plain dict. The current single-user cardinality is small, but a bounded cache is the right low-risk cleanup for long-running servers and future higher-cardinality update targets.

What Changed

Kept the existing update summary cache behavior.
Changed _summary_cache to an OrderedDict with _SUMMARY_CACHE_MAX = 16.
Refresh recency on cache hits with move_to_end().
Evict the least-recently used entry with popitem(last=False) when adding a new key at capacity.
Added regression coverage for cache reuse, bounded size, hit recency refresh, and least-recently used eviction.

Why It Matters

The summary cache should not grow without bound across a long-running WebUI process. The LRU cap preserves the fast path for recent update ranges while giving the cache an explicit memory ceiling.

Verification

pytest -q tests/test_update_banner_fixes.py::TestWhatsNewSummaryToggle::test_update_summary_cache_reuses_same_update_summary tests/test_update_banner_fixes.py::TestWhatsNewSummaryToggle::test_update_summary_cache_is_bounded_lru tests/test_update_banner_fixes.py::TestWhatsNewSummaryToggle::test_update_summary_can_be_generated_per_target_and_cached_separately tests/test_updates.py
python -m py_compile api/updates.py tests/test_update_banner_fixes.py tests/test_updates.py
git diff --check

Risks / Follow-ups

Cache miss rate can increase after eviction, but update summaries are generated infrequently and the cap is intentionally conservative.
This is only Fix A from Two cleanup follow-ups from stage-351 Opus review: _summary_cache unbounded + re.MULTILINE on 'the user is asking' strip #2215; Fix B is tracked separately in fix: drop re.MULTILINE from 'the user is asking' strip pattern #2217, so this PR intentionally uses Refs rather than auto-closing Two cleanup follow-ups from stage-351 Opus review: _summary_cache unbounded + re.MULTILINE on 'the user is asking' strip #2215.

Model Used

OpenAI GPT-5.4 via Codex CLI.

Refs nesquena#2215 Fix A: replace plain dict _summary_cache with OrderedDict-based LRU capped at 16 entries to prevent unbounded memory growth from long-running update summary generations. Add regression coverage for the bounded LRU behavior: cache hits refresh recency, a new entry at capacity evicts the least-recently used key, and cache size never exceeds the cap.

fix: cap _summary_cache with LRU (max 16 entries) (franksong2702, closes nesquena#2215 Fix A — closes nesquena#2215)

… 0.51.61) (#463) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/nesquena/hermes-webui](https://github.com/nesquena/hermes-webui) | patch | `0.51.57` → `0.51.61` | --- ### Release Notes <details> <summary>nesquena/hermes-webui (ghcr.io/nesquena/hermes-webui)</summary> ### [`v0.51.61`](https://github.com/nesquena/hermes-webui/releases/tag/v0.51.61) [Compare Source](nesquena/hermes-webui@v0.51.60...v0.51.61) ##### What's Changed - stage-354: 3-PR contributor batch — [#2228](nesquena/hermes-webui#2228) profile model picker + [#2234](nesquena/hermes-webui#2234) update-banner cleanup + [#2236](nesquena/hermes-webui#2236) silent-failure detection scope fix by [@nesquena-hermes](https://github.com/nesquena-hermes) in [#2239](nesquena/hermes-webui#2239) - Add model picker to profile creation by [@franksong2702](https://github.com/franksong2702) in [#2228](nesquena/hermes-webui#2228) - fix: silent failure detection scans only new messages, not full history by [@jasonjcwu](https://github.com/jasonjcwu) in [#2236](nesquena/hermes-webui#2236) **Full Changelog**: <nesquena/hermes-webui@v0.51.60...v0.51.61> ### [`v0.51.60`](https://github.com/nesquena/hermes-webui/blob/HEAD/CHANGELOG.md#v05160--2026-05-14--Release-AJ-stage-353--3-PR-overlapping-Appearance--critical-2223-compression-rotation-data-loss-fix--Opus-SHOULD-FIX-on-parentsessionid) [Compare Source](nesquena/hermes-webui@v0.51.59...v0.51.60) ##### Fixed - **PR [#2227](nesquena/hermes-webui#2227 by [@theh4v0c](https://github.com/theh4v0c) (closes [#2223](nesquena/hermes-webui#2223) — critical) — Context compression no longer destroys session history. The previous implementation renamed `old_sid.json` → `new_sid.json` before the new compressed session had been saved, destroying the only persistent copy of the full conversation. When the summarisation LLM call also failed, the user was left with zero recoverable messages and the bug report `Summary generation was unavailable. N message(s) were removed to free context space but could not be summarized.` text with no way to scroll back. The fix removes the destructive `old_path.rename(new_path)` call: `old_sid.json` is preserved intact as an immutable pre-compression archive, `new_sid.json` is created fresh via `s.save()`, and `parent_session_id` is set on the continuation session so the frontend can traverse the lineage chain back to the original. Even when summarisation or `s.save()` fails, the original conversation file survives on disk. New 106-line regression test file covers the no-rename invariant, parent\_session\_id stamping, and marker-only-result preservation. Stage-353 Opus SHOULD-FIX applied inline: the preservation block previously cleared `s.parent_session_id` before saving the snapshot (writing `parent=None` to `old_sid.json` on disk) and used a `if not s.parent_session_id` guard when stamping the continuation. Both bugs broke fork-of-fork compression lineage — a `/branch` fork that subsequently compressed would lose its "Forked from X" badge on the snapshot AND the continuation would skip past the snapshot back to the original fork parent. Maintainer fix removes the parent clearing during preservation (preserves the fork lineage on disk) and drops the `if not` guard (always stamps continuation to `old_sid`). Two new regression tests pin both invariants. Traversal now consistently walks new → old → old.parent → ... root. - **PR [#2222](nesquena/hermes-webui#2222 by [@franksong2702](https://github.com/franksong2702) — Settings → Appearance now wraps the "Load older messages while scrolling up" checkbox in its own `<label>` AND moves it into its own `settings-field` div instead of leaving it orphaned after the session-jump description with a stray closing `</label>`. Stage-353 maintainer resolution adopted PR [#2227](nesquena/hermes-webui#2227 stronger structural variant (each preference in its own `settings-field`) over PR [#2222](nesquena/hermes-webui#2222 smaller in-place wrap. Regression test `test_session_endless_scroll.py` pins the new per-label per-settings-field contract. ##### Added - **PR [#2225](nesquena/hermes-webui#2225 by [@franksong2702](https://github.com/franksong2702) (refs [#2224](nesquena/hermes-webui#2224)) — Adds an Extra Large option to Settings → Appearance → Font size for tablet and large-desktop readability. The new `xlarge` value is accepted by the persisted settings contract, appears alongside the existing Small / Default / Large picker options, and scales the same key UI text surfaces already covered by the font-size preference: sidebar session rows, chat message bodies/headings/code/tables, the composer textarea, workspace file rows, and app-level em/rem text. The picker grid now uses `repeat(auto-fit, minmax(96px,1fr))` instead of a fixed 3-column grid so the fourth option doesn't crowd narrow viewports. ### [`v0.51.59`](https://github.com/nesquena/hermes-webui/blob/HEAD/CHANGELOG.md#v05159--2026-05-14--Release-AI-stage-352--4-PR-clean-batch--summarycache-LRU-cap--reMULTILINE-strip-fix--Compact-sidebar-lineage-hide--CONTRIBUTORSREADME-refresh) [Compare Source](nesquena/hermes-webui@v0.51.58...v0.51.59) ##### Fixed - **PR [#2217](nesquena/hermes-webui#2217 by [@franksong2702](https://github.com/franksong2702) (refs [#2215](nesquena/hermes-webui#2215) Fix B) — Drops the leftover `re.MULTILINE` flag from the "the user is asking" pre-amble strip pattern in `api/streaming.py:695`. PR [#2213](nesquena/hermes-webui#2213) removed `re.MULTILINE` from the three sibling wrapper-strip patterns (`<think>`, MiniMax, Gemma) but missed this one instance. With `re.MULTILINE`, `^` matched the start of any line in the response, so a mid-response line that legitimately started with "The user is asking us to wait" could be stripped silently. Now the pattern only matches when the entire response leads with that wrapper, consistent with the other strips. One-flag, two-character change + regression test pinning the new behavior. - **PR [#2216](nesquena/hermes-webui#2216 by [@franksong2702](https://github.com/franksong2702) (closes [#2215](nesquena/hermes-webui#2215) Fix A) — Caps the `_summary_cache` for per-target update summaries with an `OrderedDict`-backed LRU bounded at 16 entries. Pre-fix the cache was an unbounded plain dict introduced in PR [#2207](nesquena/hermes-webui#2207); cardinality is small in practice (0-2 active update ranges per server lifetime) so this is defensive future-proofing rather than a leak being hit today. Cache hits call `move_to_end()` to refresh recency; cache writes call `popitem(last=False)` to evict the oldest entry when at capacity. Overwrites of existing keys bypass eviction. Both operations run under the existing `_cache_lock` for thread safety. With Fix A and Fix B both shipped, issue [#2215](nesquena/hermes-webui#2215) is closed. - **PR [#2219](nesquena/hermes-webui#2219 by [@franksong2702](https://github.com/franksong2702) (refs [#2218](nesquena/hermes-webui#2218)) — Compact sidebar density no longer shows compressed-session prior-turn lineage badges or expandable lineage segment rows. Pre-fix, Compact density (the default) exposed `N prior turns` badges that users read as an affordance for opening earlier conversation history — but lineage segments aren't guaranteed to have complete WebUI-loadable transcripts, so clicking them could lead to `Session not available in web UI` errors. Now the sidebar still collapses compressed continuations to the latest tip, but the lineage metadata only renders in Detailed density. Avoids the lineage report fetch/merge work entirely in Compact density since the affordance is hidden. Updates the sidebar regression test to pin the Detailed-only contract. Visual before/after evidence shipped under `docs/pr-media/2218/`. ##### Docs - **PR [#2220](nesquena/hermes-webui#2220 by [@nesquena-hermes](https://github.com/nesquena-hermes) — Refreshes `CONTRIBUTORS.md` and the README contributor section to reflect the 14 releases shipped between v0.51.44 (last refresh) and v0.51.58. Total contributors: 130 → 137. Total PR credits: 568 → 646. Seven first-time contributors added across v0.51.45–v0.51.58: [@lucasrc](https://github.com/lucasrc) (auth trilogy), [@LumenYoung](https://github.com/LumenYoung) (streaming hot path), [@MrFant](https://github.com/MrFant) (reasoning\_content whitelist), [@xz-dev](https://github.com/xz-dev) (thinking-card state + session-scoped metering), [@legeantbleu](https://github.com/legeantbleu) (French locale), [@ayushere](https://github.com/ayushere) (ctl.sh macOS compat), [@plerohellec](https://github.com/plerohellec). Bucket promotions: [@dobby-d-elf](https://github.com/dobby-d-elf) (2 → 6 PRs), [@samuelgudi](https://github.com/samuelgudi) / [@vcavichini](https://github.com/vcavichini) / [@hualong1009](https://github.com/hualong1009) / [@michael-dg](https://github.com/michael-dg) promoted from single-PR to two-PR. [@Jordan-SkyLF](https://github.com/Jordan-SkyLF) added to top-contributors with a recent burst of UX polish PRs. [@lucasrc](https://github.com/lucasrc) and [@LumenYoung](https://github.com/LumenYoung) promoted into the special-thanks roll. ### [`v0.51.58`](https://github.com/nesquena/hermes-webui/blob/HEAD/CHANGELOG.md#v05158--2026-05-13--Release-AH-stage-351--6-PR-net-positive-ready-batch--perf-CLI-scan-cache--thinking-tag-leading-only--MCP-tools-pagination--per-target-update-summaries--sweep-animation-tune--cron-badge) [Compare Source](nesquena/hermes-webui@v0.51.57...v0.51.58) ##### Fixed - **PR [#2210](nesquena/hermes-webui#2210 by [@Jordan-SkyLF](https://github.com/Jordan-SkyLF) — MCP Tools list in Settings → System no longer renders an unbounded inventory that makes the settings panel scroll-trapping. Added a toolbar (result-count summary, page-size 5/10/20/50/all, search input), bounded scroll area with consistent height, paginated rendering, and focused regression coverage for the large-inventory case. Existing WebUI-only/runtime-only contract preserved (no MCP server probing, no agent-side changes). Visual before/after evidence shipped under `docs/pr-media/2210/`. - **PR [#2213](nesquena/hermes-webui#2213 by [@franksong2702](https://github.com/franksong2702) (fixes [#2152](nesquena/hermes-webui#2152)) — Literal `<think>`/`</think>` discussions in normal assistant prose are no longer stripped from saved messages and re-renders. The old server cleanup and stored-message render regexes stripped the first closed thinking-looking block anywhere in the content. PR aligns saved/static paths with the existing streaming rule: provider reasoning wrappers (`<think>...</think>`, MiniMax `<|channel>thought...<channel|>`, Gemma 4 `<|turn|>thinking...<turn|>`) are stripped only when they lead the response (i.e. the wrapper is the first non-whitespace content). - **PR [#2149](nesquena/hermes-webui#2149 by [@starship-s](https://github.com/starship-s) — `/api/session` loads no longer pay the cost of full external CLI session discovery when opening an ordinary WebUI-native chat. Caches CLI/external session scans briefly, skips CLI metadata lookup for ordinary WebUI-native session loads, and reuses a single in-memory ID snapshot during session-index pruning. Messaging, read-only, external-agent, and CLI-marked sidecars still take the CLI metadata path; CLI-only sessions still use the existing fallback. Stage-351 maintainer fix renamed the existing `_needs_cli_session_metadata()` gate to the broader `_session_requires_cli_metadata_lookup()` from this PR — strictly more inclusive (now also covers `read_only=True` sidecars, `session_source` markers, and source\_tag/raw\_source/platform metadata so legacy-imported sidecars still get the slow path when they need it). ##### Added - **PR [#2207](nesquena/hermes-webui#2207 by [@Jordan-SkyLF](https://github.com/Jordan-SkyLF) (fixes [#1579](nesquena/hermes-webui#1579)) — Update banner now shows target-aware "What's new?" links: WebUI updates link to the WebUI comparison, Agent updates link to the Agent comparison. Agent-only and WebUI-only update states no longer show a misleading cross-target comparison action. Opt-in settings toggle enables human-readable LLM-generated update summaries for each target's diff; users can still open the original diff from the summary. Cached/generated-summary button states persist across refreshes. Extended update-banner regression coverage for the diff-link and summary flows. Visual evidence: `docs/images/update-banner-whats-new-{before,after}.png` + summary on/off variants. - **PR [#2206](nesquena/hermes-webui#2206 by [@vcavichini](https://github.com/vcavichini) — Cron list now shows a 🤖 emoji badge for jobs running in agent mode (`no_agent=false`). Cron detail view shows the configured provider/model next to the Mode badge, falling back to "default" when neither is explicitly set for agent-mode crons. UI-only change. - **PR [#2212](nesquena/hermes-webui#2212 by [@dobby-d-elf](https://github.com/dobby-d-elf) — Tunes the Activity sweep animation introduced in PR [#2203](nesquena/hermes-webui#2203) (stage-350) — softer color stops, less aggressive contrast, smoother fade. CSS-only follow-up. </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://git.erwanleboucher.dev/eleboucher/homelab/pulls/463

fix: cap _summary_cache with LRU (max 16 entries) (franksong2702, closes nesquena#2215 Fix A — closes nesquena#2215)

stage-352: 4-PR clean batch — _summary_cache LRU cap nesquena#2216 + re.MULTILINE strip fix nesquena#2217 + Compact sidebar lineage hide nesquena#2219 + CONTRIBUTORS/README refresh nesquena#2220

fix: cap _summary_cache with LRU (max 16 entries) (franksong2702, closes nesquena#2215 Fix A — closes nesquena#2215)

stage-352: 4-PR clean batch — _summary_cache LRU cap nesquena#2216 + re.MULTILINE strip fix nesquena#2217 + Compact sidebar lineage hide nesquena#2219 + CONTRIBUTORS/README refresh nesquena#2220

franksong2702 mentioned this pull request May 14, 2026

fix: drop re.MULTILINE from 'the user is asking' strip pattern #2217

Merged

This comment was marked as outdated.

Sign in to view

franksong2702 marked this pull request as draft May 14, 2026 00:57

franksong2702 force-pushed the fix/2215-summary-cache-lru branch from 5299eff to 9681761 Compare May 14, 2026 01:14

franksong2702 marked this pull request as ready for review May 14, 2026 01:17

nesquena-hermes mentioned this pull request May 14, 2026

stage-352: 4-PR clean batch — _summary_cache LRU cap #2216 + re.MULTILINE strip fix #2217 + Compact sidebar lineage hide #2219 + CONTRIBUTORS/README refresh #2220 #2221

Merged

nesquena-hermes closed this pull request by merging all changes into nesquena:master in 3f4e82b May 14, 2026

franksong2702 pushed a commit to franksong2702/hermes-webui-fork that referenced this pull request May 14, 2026

Merge pull request nesquena#2216 into stage-352

549140d

fix: cap _summary_cache with LRU (max 16 entries) (franksong2702, closes nesquena#2215 Fix A — closes nesquena#2215)

Du7chManiac mentioned this pull request May 20, 2026

sync: upstream v0.51.59 (needs review) TheCouchCoder-com/hermes-webui#22

Merged

SysAdminDoc pushed a commit to SysAdminDoc/hermes-webui that referenced this pull request Jun 26, 2026

Merge pull request nesquena#2216 into stage-352

8eb7d9c

fix: cap _summary_cache with LRU (max 16 entries) (franksong2702, closes nesquena#2215 Fix A — closes nesquena#2215)

bernyforce pushed a commit to bernyforce/hermes-webui that referenced this pull request Jul 29, 2026

Merge pull request nesquena#2216 into stage-352

299b66a

fix: cap _summary_cache with LRU (max 16 entries) (franksong2702, closes nesquena#2215 Fix A — closes nesquena#2215)

Navigation Menu

Uh oh!

fix: cap _summary_cache with LRU (max 16 entries) - #2216

fix: cap _summary_cache with LRU (max 16 entries)#2216
1 commit merged into
nesquena:masterfrom
franksong2702:fix/2215-summary-cache-lru

franksong2702 commented May 14, 2026 •

edited

Loading

Uh oh!

This comment was marked as outdated.

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

franksong2702 commented May 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Thinking Path

What Changed

Why It Matters

Verification

Risks / Follow-ups

Model Used

Uh oh!

This comment was marked as outdated.

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

franksong2702 commented May 14, 2026 •

edited

Loading