Skip to content

fix(streaming): respect auxiliary.title_generation config for session titles - #925

Closed
starship-s wants to merge 3 commits into
nesquena:masterfrom
starship-s:fix/aux-title-model-routing
Closed

starship-s wants to merge 3 commits into
nesquena:masterfrom
starship-s:fix/aux-title-model-routing

Conversation

@starship-s

@starship-s starship-s commented Apr 23, 2026

Copy link
Copy Markdown
Contributor

Thinking Path

  • Hermes WebUI currently has two valid title-generation goals that should coexist rather than fight each other:
    • by default, prefer the active session runtime so title generation inherits provider-specific transport and API-mode behavior
    • when the user explicitly configures auxiliary.title_generation, that dedicated route should actually be honored
  • Before this change, the WebUI always passed the active agent's provider / model / base_url into the auxiliary title helper.
  • In Hermes Agent, _resolve_task_provider_model() treats those explicit args as priority 1 over config.yaml, so auxiliary.title_generation could be configured in Hermes and still be silently ignored by WebUI title generation.
  • That created a user-visible mismatch: the Hermes config and model UI suggest “use this dedicated title model”, but the WebUI still uses the chat model unless the active-session route fails first.
  • The safest fix is not to remove the active-session-model path. Instead:
    • preserve current behavior when no dedicated title config exists
    • but when auxiliary.title_generation is explicitly configured, let the aux resolver actually read that config
  • While tightening this path, I also fixed two adjacent edge cases exposed by the same flow:
    • the dedicated-title path's fallback tuple did not include llm_invalid_aux, so some invalid aux outputs skipped the intended agent fallback
    • the aux title timeout was still hardcoded to 15.0s instead of respecting the configured per-task timeout

What Changed

  • Added _aux_title_configured() in api/streaming.py to detect whether auxiliary.title_generation is meaningfully configured.
    • returns true when any of:
      • provider is set and not auto
      • model is set
      • base_url is set
  • Preserved the current default routing model:
    • if no dedicated title config exists, WebUI still prefers the active session model first
    • this keeps the existing provider/runtime behavior intact for the default case
  • Kept the existing dedicated-title routing behavior:
    • when a dedicated title config exists, the aux title helper is called without leaking the active chat model into the resolver
  • Added _aux_title_timeout() so the aux title path reads auxiliary.title_generation.timeout instead of hardcoding 15.0
  • Hardened _aux_title_timeout() to reject:
    • non-numeric values
    • zero
    • negative values
      and fall back to the default with a debug log
  • Fixed the aux→agent fallback condition in _run_background_title_update():
    • agent fallback now triggers for both llm_error_aux and llm_invalid_aux
  • Added focused regression coverage in tests/test_title_aux_routing.py for:
    • provider-only config
    • base_url-only config
    • timeout override from config
    • invalid timeout values falling back safely
    • llm_invalid_aux correctly triggering agent fallback
    • successful aux title generation not triggering agent fallback

Why It Matters

  • Makes WebUI title generation consistent with Hermes configuration semantics:
    • explicit auxiliary.title_generation settings now actually mean something
  • Preserves current behavior for users who did not configure a dedicated title model
  • Avoids cases where a chat model with different language/style behavior overrides a deliberately chosen title-generation model
  • Makes slow local or proxy-backed title models usable by honoring the configured timeout
  • Prevents an invalid aux title response from skipping the intended agent fallback path
  • Adds regression tests around the exact routing and fallback guarantees this path now depends on

Verification

Targeted verification run locally:

pytest tests/test_title_aux_routing.py tests/test_title_sanitization.py -q

Result:

  • 32 passed

The new regression coverage specifically verifies:

  • provider configured with blank model is still treated as explicit title config
  • base_url configured with blank model is still treated as explicit title config
  • auxiliary.title_generation.timeout is used instead of a hardcoded 15.0
  • invalid timeout values fall back safely
  • llm_invalid_aux and llm_error_aux both trigger agent fallback
  • successful aux title generation does not trigger agent fallback

Risks / Follow-ups

  • This intentionally does not remove the active-session-model path; it only gives explicit title config precedence when present.
  • The WebUI still owns its own prompt/sanitization logic for title generation; this PR does not try to unify it with the agent's separate title helper.
  • If maintainers want even stricter parity later, a follow-up could centralize title-routing policy in one shared helper rather than duplicating the precedence logic in WebUI.

Model Used

  • GPT-5.4-mini (code archaeology, branch analysis)
  • Claude Code Sonnet + GLM-5.1 (implementation)
  • Claude Opus 4.7 + GPT-5.5 (code review, PR drafting)

- Respect auxiliary.title_generation model config for session titles,
  preventing e.g. a Chinese chat model from producing Chinese-only titles.
- Fix aux→agent fallback tuple: change ('llm_error_aux', 'llm_error',
  'llm_invalid') to ('llm_error_aux', 'llm_invalid_aux') so the agent
  route is correctly invoked when the aux helper produces an invalid title.
- Tighten _aux_title_timeout to reject zero, negative, and non-numeric
  values, falling back to default with a debug log.
- Add regression tests for fallback routing and timeout edge cases.
@nesquena-hermes

Copy link
Copy Markdown
Collaborator

Triage review — fix(streaming): respect auxiliary.title_generation config for session titles

This PR fixes a real config-vs-behavior mismatch: users who explicitly configure auxiliary.title_generation in config.yaml expected that configuration to actually control title generation, but the WebUI was silently overriding it with the active chat model. Solid bug fix with clear motivation.

What's well done:

  • The _aux_title_configured() detection correctly distinguishes "explicitly configured" vs "not configured" rather than blindly changing behavior for all users
  • Preserves current default behavior when no dedicated title config exists — no regression for existing users
  • llm_invalid_aux being added to the fallback tuple is a real correctness fix
  • Configurable timeout reads from config.yaml and validates properly (rejects zero/negative/non-numeric)
  • Targeted regression test suite (test_title_aux_routing.py) covers the specific routing guarantees being added
  • 32 tests passing

One question for maintainer: The _aux_title_configured() helper returns True when base_url is set even without a model. This handles proxy scenarios where the base URL routes to a default model. Worth confirming this is the intended semantic — some proxy setups may want base_url alone to route differently.

No blocking issues found. Ready for maintainer review and merge.

@nesquena-hermes

Copy link
Copy Markdown
Collaborator

Triage review — branch rebased onto current master as PR #931

The implementation is correct and all 23 tests pass. Two improvements addressed over #925's branch state:

Please see PR #931 for the review and merge path. This PR can be closed once #931 merges.

@nesquena-hermes

Copy link
Copy Markdown
Collaborator

Merged to master as PR #931 (v0.50.182). Thank you @starship-s!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants