fix(gateway): refresh prompt policy in long-lived sessions

[BeliefanX]

Summary

• refresh the gateway's configured prompt/personality policy before each normal turn in the long-lived runner
• include prompt-policy inputs in the gateway agent cache signature so agent construction state changes invalidate correctly
• propagate the configured prompt into background tasks and API-server-created agents
• add regression coverage for personality fallback, cache signature keys, background prompt propagation, and API prompt composition

Notes

This intentionally keeps scope small and uses the local repair diff only for gateway prompt/cache/personality behavior. It does not include user-local SOUL/profile default changes, delegate result slimming/default behavior, auto-continue resume markers, or unrelated API profile/session/job routing changes.

Related/extends #32007. Also complements the cache-signature cluster: #37880, #39022, #39284, #39454, and #28078.

API server prompt ordering is configured/global prompt first, then request instructions, matching the normal gateway "platform/config first, request-specific last" composition and covered by test.

Tests

• uv run pytest tests/gateway/test_runtime_config_env_expansion.py tests/gateway/test_background_command.py::TestRunBackgroundTask::test_successful_task_sends_result tests/gateway/test_api_server.py::TestAdapterInit -q — 24 passed
• ulimit -n 4096; uv run pytest tests/gateway/test_runtime_config_env_expansion.py tests/gateway/test_background_command.py tests/gateway/test_api_server.py — 196 passed, 1 failed: existing macOS /var vs /private/var temp path assertion in test_media_files_routed_by_type (unrelated to this prompt change)
• python -m pytest tests/gateway/test_discord_channel_prompts.py -q — 10 passed
• python -m pytest tests/gateway/test_runtime_config_env_expansion.py -q — 17 passed
• python scripts/run_tests_parallel.py --slice 4/6 (local system env) — blocked by missing optional acp, missing mcp.server.auth, local kanban/test_run_agent timeouts; CI slice was re-run after the fixture fix

[liuhao1024]

Verification: reviewed diff + tests — no issues found.

Checked:

• 3-tier precedence (env var > agent.system_prompt > display.personality) is correct and well-tested with 7 new tests
• _resolve_personality_prompt handles both string and dict personality values
• _resolve_display_personality_prompt does case-insensitive fallback for personality key lookup
• Cache-busting config keys correctly include all prompt-policy paths so stale agents are evicted on config change
• _reload_runtime_env_preserving_config_authority() moved before ephemeral prompt loading — correct ordering so manual config edits take effect on next turn
• Background tasks now receive the ephemeral prompt via _load_ephemeral_system_prompt()
• API server correctly combines config-level and request-level ephemeral prompts with "\n\n".join()

Architectural concern noted: _reload_runtime_env_preserving_config_authority() now runs on every message turn (not just on credential refresh). This is intentional per the PR description ("manual edits to config.yaml / personalities should take effect on the next turn") but adds a config reload to the hot path. Acceptable for correctness.

[teknium1]

Thanks for isolating the gateway, background, and API propagation paths. This needs re-scoping before salvage.

Problems

• gateway/run.py:14192-14193 reloads and replaces the prompt on every turn; the new signature keys at gateway/run.py:12468-12473 then rebuild the cached agent after edits. That conflicts with the byte-stable prompt/cache requirement in AGENTS.md:19-23,88-91.
• The new fallback at gateway/run.py:3431 makes display.personality active for the gateway, but current main labels it CLI-only (gateway/display_config.py:30-31) and explicitly tests that it remains inactive without agent.system_prompt (tests/tui_gateway/test_make_agent_provider.py:160-195).
• gateway/run.py:1853 only tries a lowercase key, so it is not generally case-insensitive.

Suggested changes

• Defer prompt-policy changes to a new session/restart or explicit user-confirmed invalidation; do not alter a live conversation's cached prompt.
• Remove the display-personality fallback unless the cross-surface contract is intentionally changed, then normalize all personality keys for lookup.

Automated hermes-sweeper review.

[teknium1]

This reload plus the new prompt-policy signature keys changes the prompt and reconstructs the cached agent within an existing conversation. AGENTS.md:19-23,88-91 requires the system prompt to remain byte-stable for the conversation; defer this to a new session/restart or an explicit cache-invalidating user action.

[teknium1]

This activates display.personality for gateway agents, but current main defines that display setting as CLI-only (gateway/display_config.py:30-31) and tests the same inactive-until-agent.system_prompt contract for the TUI (tests/tui_gateway/test_make_agent_provider.py:160-195). Please remove this fallback unless the cross-surface behavior is intentionally being redesigned.

[teknium1]

This is only a lowercase fallback, not a case-insensitive lookup: display.personality: MIXEDCASE will not find an agent.personalities key named MixedCase. If this resolver remains, normalize and compare all configured keys.

[BeliefanX]

Closing this version after re-review against current main. Its per-turn prompt replacement/cache-key rebuild conflicts with the byte-stable per-conversation system-prompt invariant, and display.personality is currently CLI-only. If controlled live refresh becomes a confirmed requirement, it should return as a smaller current-main change at an explicit session boundary/action. Thanks to the reviewers for clarifying the contract.