feat(gateway): periodic memory trim in idle reaper for long-lived processes

[aider4ryder]

Problem

Long-lived gateway processes (days/weeks without restart) accumulate high RSS even though the memory trim infrastructure from #63708 is deployed. Root cause: trim_memory() is only called at turn completion and agent close, but the idle reaper thread (running every 5 min) never triggers it unless a session is actually reaped.

For active sessions that never hit the 6h idle TTL, gc.collect() + malloc_trim(0) are never invoked, so glibc retains freed heap pages as RSS indefinitely.

Solution

Add trim_memory(reason="idle reaper periodic trim") at the end of _reap_idle_sessions() so every reaper scan (default 300s) releases releasable pages regardless of whether any session was reaped.

This builds on the carried #63708 infrastructure:

• hermes_cli/mem_trim.py: config-driven malloc_trim(0) with cooldown and kill-switch
• Post-turn trim in _run_prompt_submit finally block
• Force trim on AIAgent.close()
• context.memory_trim config schema (enabled, cooldown_seconds)

Observability

Before (gateway running 28h, PID 115324):

RSS: 1033 MB

Expected after: periodic trim every 5 min should keep RSS bounded. The 60s cooldown in trim_memory prevents redundant trims when turn-completion and reaper fire close together.

Testing

• 8 mem_trim unit tests (config, cooldown, kill-switch, libc failure)
• New test_reap_idle_sessions_calls_periodic_trim — verifies trim fires even with zero reaped sessions
• 324 gateway tests pass, no regressions

PYTHONPATH=. pytest tests/hermes_cli/test_mem_trim.py tests/test_tui_gateway_server.py -x -q
# 10 passed, 324 passed

[teknium1]

Thanks for extending the allocator-trim work with the idle-reaper boundary. The current implementation confirms the reaper runs every 300 seconds but has no trim on current main (tui_gateway/server.py:847,875-881,953-966).

Problems

• The new periodic call reads context.memory_trim on the daemon reaper thread. In remote multi-profile mode, profile selection is a request/turn ContextVar (tui_gateway/server.py:995-1038,8985-8987), while the reaper starts at module initialization (tui_gateway/server.py:953-966). Since the helper calls load_config(), periodic enabled/cooldown values resolve to the launch profile, not a remotely served session profile. The added tests cover turn cleanup but not this periodic configuration path.
• The PR is conflicting and contains the open allocator-trim implementation plus an unrelated text-verbosity commit. The final memory commit itself is limited to six files (97c5008445fb).

Suggested changes

• Define the intended process-wide profile/config behavior and add a regression test for the periodic reaper under remote-profile mode.
• Salvage the focused memory-trim commit only.

Automated hermes-sweeper review.

[teknium1]

trim_memory() reads load_config() here, but this daemon reaper has no per-session profile ContextVar. In desktop remote-profile mode, turns bind a profile only in their own thread, so this periodic call always uses the launch profile's context.memory_trim settings. Please define the intended process-wide profile policy and cover this path with a remote-profile test.

[kshitijk4poor]

Thanks @aider4ryder for kicking off the memory-trim work here! Closing this in favor of your own #66355, which supersedes it with the config-driven version — configurable cooldown, RSS telemetry, and broader lifecycle coverage (gateway housekeeping, slash_worker, agent close) plus unit tests. This branch also picked up an unrelated text-verbosity commit, so consolidating on the clean #66355 keeps review simple. Let's continue the trim discussion over there.