Skip to content

test(gateway): add webhook toolset regression coverage - #20731

Closed
dlkakbs wants to merge 1 commit into
NousResearch:mainfrom
dlkakbs:fix/webhook-toolset-regression-tests
Closed

test(gateway): add webhook toolset regression coverage#20731
dlkakbs wants to merge 1 commit into
NousResearch:mainfrom
dlkakbs:fix/webhook-toolset-regression-tests

Conversation

@dlkakbs

@dlkakbs dlkakbs commented May 6, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds explicit regression coverage for the webhook platform/toolset wiring that previously caused KeyError failures on webhook-triggered agent runs.

This PR verifies that:

  • hermes-webhook is defined and validates as a toolset
  • the webhook platform resolves to default_toolset="hermes-webhook"
  • hermes_cli.tools_config.PLATFORMS exports the same mapping
  • hermes-gateway still includes hermes-webhook

Context

This is a focused test-only follow-up to earlier webhook platform registration work.

Related history:

What landed on main from that chain was the runtime/config wiring:

  • webhook platform registration
  • hermes-webhook toolset registration
  • inclusion in hermes-gateway

What did not land was the dedicated regression coverage from #3775.

This PR adds that missing explicit test coverage only, without changing runtime behavior.

Why this is still useful

main already has broader platform/toolset consistency coverage, but this PR adds a webhook-specific regression test for the exact wiring that was previously missing and that motivated #3775.

That makes the original failure mode easier to catch directly if webhook registration regresses again.

@alt-glitch alt-glitch added type/test Test coverage or test infrastructure P3 Low — cosmetic, nice to have comp/gateway Gateway runner, session dispatch, delivery platform/webhook Webhook / API server labels May 6, 2026
@teknium1

teknium1 commented May 7, 2026

Copy link
Copy Markdown
Contributor

Thanks for the follow-up — closing as redundant.

The exact wiring this file pins is already covered generically by TestPlatformToolsetConsistency in tests/hermes_cli/test_tools_config.py (around L558):

  • test_all_platforms_have_toolset_definitions asserts every entry in tools_config.PLATFORMS (including webhook) has its default_toolset defined in TOOLSETS — would fail if hermes-webhook got unwired.
  • test_gateway_toolset_includes_all_messaging_platforms asserts every messaging platform's toolset is in hermes-gateway["includes"] — webhook is in that sweep.

So the original failure mode (webhook platform registered without its toolset, or missing from the gateway bundle) already fails the existing generic suite. Adding a webhook-named duplicate doesn't give us additional regression coverage — it just doubles the assertion surface for one platform while every other platform rides on the generic check.

Appreciate the diligence on the #3775#4363#4660 chain — the runtime wiring is landed and guarded.

@teknium1 teknium1 closed this May 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P3 Low — cosmetic, nice to have platform/webhook Webhook / API server type/test Test coverage or test infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants