Skip to content

fix(gateway): require API_SERVER_KEY to load the api_server platform - #36180

Closed
arimu1 wants to merge 2 commits into
NousResearch:mainfrom
arimu1:fix/api-server-response-store-fd-leak
Closed

fix(gateway): require API_SERVER_KEY to load the api_server platform#36180
arimu1 wants to merge 2 commits into
NousResearch:mainfrom
arimu1:fix/api-server-response-store-fd-leak

Conversation

@arimu1

@arimu1 arimu1 commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #36111. API_SERVER_ENABLED=true without a key caused the adapter to be instantiated (opening a ResponseStore/SQLite connection) even though the HTTP server would immediately refuse to start.

Fix

  • gateway/config.py — change the load condition from or to if api_server_key, so the platform only loads when a key is actually set. Setting only API_SERVER_KEY (without the explicit enabled flag) still works — the key alone is sufficient intent.
  • gateway/config.py — fix the _PLATFORM_CONNECTED_CHECKERS entry for API_SERVER, which was a no-op lambda: True; it now returns True only when the platform config has a key stored, matching actual readiness.

Scope change

An earlier version of this PR also called response_store.close() in APIServerAdapter.disconnect() to fix a related fd leak. That part is now redundant — 4b06c98 ("fix(gateway): close ResponseStore + dispose unowned adapter on reconnect failure", #37011) already covers it more thoroughly (and added its own regression tests). I've dropped that half during a rebase onto current main and kept only the require-key fix, which isn't covered elsewhere.

Test plan

  • pytest tests/gateway/test_api_server.py tests/gateway/test_platform_reconnect_fd_leak.py tests/gateway/test_platform_connected_checkers.py tests/gateway/test_api_server_bind_guard.py — 215 passed, 1 skipped
  • ruff check clean
  • Updated test_env_override_* / test_api_server_in_connected_platforms / test_checker_returns_true_when_configured tests that assumed the old no-key-required behavior

🤖 Generated with Claude Code

@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists labels Jun 1, 2026

@mxnstrexgl mxnstrexgl left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — automated review passed. No security, quality, or test coverage issues detected.

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Approved

Review

fix(gateway): close ResponseStore on disconnect and require key to load api_server platform

Excellent 3-part fix for a production FD leak. Key observations:

  • Root cause analysis: Excellent — three layered causes identified (platform loaded without key, connected checker always true, ResponseStore never closed) with clear evidence (122 open FDs after 12 hours).
  • Fix completeness: All three root causes are addressed independently:
    1. if api_server_key (not api_server_enabled or api_server_key) — prevents platform instantiation without auth
    2. lambda cfg: bool(cfg.extra.get("key")) — honest connected checker
    3. ResponseStore.close() + null in disconnect() — proper teardown
  • Backward compatibility: Setting only API_SERVER_KEY (without API_SERVER_ENABLED) continues to work, matching existing docs.
  • Tests confirmed: 32 test_api_server_toolset + 32 test_config pass.

Looks Good

  • Minimal, targeted changes (5 lines + 2 deletions)
  • Clear commit message
  • Each root cause has its own independent fix

Reviewed by Hermes Agent

@arimu1

arimu1 commented Jun 6, 2026

Copy link
Copy Markdown
Contributor Author

Friendly ping — this has two approvals and no blocking comments. Happy to rebase or make any adjustments if needed before merge.

@arimu1
arimu1 force-pushed the fix/api-server-response-store-fd-leak branch from 11ff7b4 to c349453 Compare June 28, 2026 09:52
@arimu1 arimu1 changed the title fix(gateway): close ResponseStore on disconnect and require key to load api_server platform fix(gateway): require API_SERVER_KEY to load the api_server platform Jun 28, 2026
@arimu1

arimu1 commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

Following up again — still approved by both reviewers with no blocking comments. Let me know if there's anything I can do to help get this merged, or if it needs a rebase.

@arimu1
arimu1 force-pushed the fix/api-server-response-store-fd-leak branch from c349453 to d222ea6 Compare July 11, 2026 03:09

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for preserving the narrower configuration/status fix after the ResponseStore teardown work landed. The current-main premise is still valid: gateway/config.py:1761 loads a keyless API-server platform, gateway/platforms/api_server.py:880 constructs ResponseStore, and startup later rejects the missing key at gateway/platforms/api_server.py:4785-4810.

Problems

  • gateway/config.py:632 adds os.getenv("TWILIO_ACCOUNT_SID") to a gateway connected checker. Current gateway config reads are profile-scoped in gateway/config.py:174-189; 0f154e780e71c74f8a1cdccb25c97a6abd8e5a57 added that isolation. The raw lookup can report a secondary profile connected using a default-profile SMS credential.
  • gateway/config.py:633 treats every nonempty API key as ready, but the real startup guard requires a non-placeholder key of at least 16 characters (gateway/platforms/api_server.py:4785-4810). The new sk-mykey fixture demonstrates the mismatch.

Suggested changes

  • Remove or profile-scope the SMS checker.
  • Reuse the startup key-validity criterion for the API-server checker and test missing, placeholder, short, and strong keys.

Automated hermes-sweeper review.

Comment thread gateway/config.py Outdated
Platform.API_SERVER: lambda cfg: True,
Platform.EMAIL: lambda cfg: bool(cfg.extra.get("address")),
Platform.SMS: lambda cfg: bool(os.getenv("TWILIO_ACCOUNT_SID")),
Platform.API_SERVER: lambda cfg: bool(cfg.extra.get("key")) if cfg else False,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This bypasses the active profile secret scope. Current gateway config reads use the scoped helper at gateway/config.py:174-189 so secondary profiles cannot inherit default-profile credentials; please remove this raw lookup or derive SMS readiness from scoped/configured state.

Comment thread gateway/config.py
Platform.API_SERVER: lambda cfg: True,
Platform.EMAIL: lambda cfg: bool(cfg.extra.get("address")),
Platform.SMS: lambda cfg: bool(os.getenv("TWILIO_ACCOUNT_SID")),
Platform.API_SERVER: lambda cfg: bool(cfg.extra.get("key")) if cfg else False,
Platform.WEBHOOK: lambda cfg: True,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reports any nonempty key as connected, but APIServerAdapter._api_key_passes_startup_guard() requires has_usable_secret(..., min_length=16) at gateway/platforms/api_server.py:4785-4810. Please apply the same criterion here and use a strong valid key in the checker test.

@teknium1 teknium1 added sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users labels Jul 13, 2026
@arimu1
arimu1 force-pushed the fix/api-server-response-store-fd-leak branch from d222ea6 to dc16c3e Compare July 18, 2026 22:58
@arimu1

arimu1 commented Jul 19, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @teknium1 — addressed in the latest push.

  • API server connected-check now uses has_usable_secret(..., min_length=16) to match the startup guard
  • SMS readiness uses the profile-scoped env helper instead of a raw os.getenv lookup

Happy to adjust further if anything else looks off.

nawfalaf and others added 2 commits July 23, 2026 12:30
Previously API_SERVER_ENABLED=true without a key caused the adapter to
be instantiated (opening a ResponseStore/SQLite connection) even
though the HTTP server would immediately refuse to start. Changing the
load condition from `or` to `if api_server_key` prevents the spurious
load. Setting only API_SERVER_KEY (without the explicit flag) still
works — the key alone is sufficient intent to enable the platform.

Also fixes the _PLATFORM_CONNECTED_CHECKERS entry for API_SERVER,
which was a no-op `lambda: True`; it now returns True only when the
platform config has a `key` stored, matching actual readiness.

Fixes NousResearch#36111

Note: an earlier version of this PR also closed the adapter's
ResponseStore in disconnect() to fix a related fd leak — that part is
now redundant, since 4b06c98 ("fix(gateway): close ResponseStore +
dispose unowned adapter on reconnect failure", NousResearch#37011) already covers
it more thoroughly. Dropped that half and kept only the require-key
fix, which isn't covered elsewhere.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@arimu1
arimu1 force-pushed the fix/api-server-response-store-fd-leak branch from dc16c3e to 86f4375 Compare July 23, 2026 05:34
@arimu1

arimu1 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto latest main to resolve a merge conflict in gateway/config.py (a single reformatted line in the unrelated thread_sessions_per_user YAML-loading block collided with an upstream change that added a nested gateway.thread_sessions_per_user fallback — kept upstream's version, dropped the incidental reformat). The change is unchanged in intent — still requires API_SERVER_KEY to load the api_server platform. Tests in tests/gateway/test_api_server.py, tests/gateway/test_platform_connected_checkers.py, and tests/gateway/test_config.py all pass (376/376) on the rebased branch. @mxnstrexgl @tonydwb flagging in case the force-push reset your approval.

teknium1 pushed a commit that referenced this pull request Jul 23, 2026
…platform at load time

Salvaged from PR #36180 (commits 68dfeb4 and 86f4375 by arimu1),
re-applied onto current main with the incidental black-reformat churn
stripped out (~1,700 lines -> the semantic change + tests).

Previously gateway/config.py enrolled the api_server platform on
`api_server_enabled or api_server_key`, so API_SERVER_ENABLED=true with
no key (or a weak/placeholder key) still loaded the platform: the
adapter is instantiated (ResponseStore/SQLite opened in __init__), the
reconnect watcher spins, and the startup guard refuses at connect() —
logging errors forever. Now the platform is enrolled only when
API_SERVER_KEY passes the same strength bar as the adapter's startup
guard (has_usable_secret, min_length=16), via a shared
_has_usable_api_server_key() helper.

The no-op `lambda cfg: True` connected-checker for API_SERVER is also
replaced with the same key check, so get_connected_platforms() only
reports the platform "up" when it could actually start.

Known limitation (intentionally out of scope): a YAML config with
`platforms.api_server.enabled: true` and no key still loads the
platform; this gate covers the env-override path only.

Dropped from the original PR: EMAIL/SMS checker additions (scope creep
beyond the PR title; absent on current main) and the wholesale black
reformat of gateway/config.py and tests.

Fixes #36111

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@teknium1

Copy link
Copy Markdown
Contributor

Merged via PR #70166 — both your commits' semantic changes landed with your authorship preserved in git log (9e4b898). We trimmed the incidental reformat churn and the EMAIL/SMS checker additions to keep the diff reviewable, but the load-time gate and the honest connected-checker are exactly your fix, matched to the startup guard's 16-char bar as in your post-review revision. Earliest PR in the #36111 cluster — thanks for sticking with it through the feedback rounds!

randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
…platform at load time

Salvaged from PR NousResearch#36180 (commits 68dfeb4 and 86f4375 by arimu1),
re-applied onto current main with the incidental black-reformat churn
stripped out (~1,700 lines -> the semantic change + tests).

Previously gateway/config.py enrolled the api_server platform on
`api_server_enabled or api_server_key`, so API_SERVER_ENABLED=true with
no key (or a weak/placeholder key) still loaded the platform: the
adapter is instantiated (ResponseStore/SQLite opened in __init__), the
reconnect watcher spins, and the startup guard refuses at connect() —
logging errors forever. Now the platform is enrolled only when
API_SERVER_KEY passes the same strength bar as the adapter's startup
guard (has_usable_secret, min_length=16), via a shared
_has_usable_api_server_key() helper.

The no-op `lambda cfg: True` connected-checker for API_SERVER is also
replaced with the same key check, so get_connected_platforms() only
reports the platform "up" when it could actually start.

Known limitation (intentionally out of scope): a YAML config with
`platforms.api_server.enabled: true` and no key still loads the
platform; this gate covers the env-override path only.

Dropped from the original PR: EMAIL/SMS checker additions (scope creep
beyond the PR title; absent on current main) and the wholesale black
reformat of gateway/config.py and tests.

Fixes NousResearch#36111

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

File descriptor leak in api_server platform: ResponseStore SQLite connections not closed on retry

6 participants