Skip to content

fix(gateway): make Slack capability guidance accurate - #69094

Closed
KCAYAAI wants to merge 1 commit into
NousResearch:mainfrom
KCAYAAI:fix/slack-capability-prompt
Closed

fix(gateway): make Slack capability guidance accurate#69094
KCAYAAI wants to merge 1 commit into
NousResearch:mainfrom
KCAYAAI:fix/slack-capability-prompt

Conversation

@KCAYAAI

@KCAYAAI KCAYAAI commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes Slack session guidance that currently turns a gateway implementation detail into a false universal capability denial.

Hermes now distinguishes:

  • Gateway conversation support: Bolt/Socket Mode receives an event and the gateway delivers the model's normal reply to the invoking Slack conversation outside the model tool loop.
  • Agent-callable operational Slack access: only explicitly declared operations on an actual selected, requirement-passing tool whose strict capability check passes for the active profile.

A configured Slack token proves gateway authentication, not model-tool exposure. Conversely, the absence of a built-in Slack tool does not prove that a deployment-specific plugin exposes no bounded Slack operations.

The conservative state therefore says that no operational Slack capability has been both declared and currently verified for the conversation; it does not claim that no Slack-related tool exists anywhere. The capability-present state lists only the declaring tool and its identifier-style operations.

Related Issue

Related to #6533 and #6536.

This is an explicit metadata-based alternative to #62135, #6545, #36676, and #68627. Those PRs soften the note or infer capability from known/Slack-looking tool or MCP names. This change deliberately does not infer operational capability from tokens, executables, terminal access, environment names, tool names, or MCP server names.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • tools/registry.py
    • Add immutable platform capability descriptors attached to real registered tools.
    • Validate platform, tool, and operation identifiers as inert lowercase metadata.
    • Require a strict capability check, evaluated without availability-cache grace.
    • Partition normal requirement caches by active HERMES_HOME.
    • Scope user-directory plugin tools to the profile that loaded them for schema selection, capability lookup, and dispatch.
    • Fail closed without logging exception messages or tracebacks from capability checks.
  • hermes_cli/plugins.py
    • Expose trailing, backwards-compatible platform_capabilities and platform_capability_check_fn registration arguments.
    • Automatically bind user-directory plugin tools to their loading profile.
  • model_tools.py
    • Resolve descriptors from the normal selected/requirement-passing model schema.
    • Re-run the strict identity check uncached and guard against concurrent registry generations.
    • Reject forged cross-profile calls before coercion, middleware, hooks, or handler execution.
  • gateway/run.py
    • Resolve Slack capability snapshots in a worker thread under the routed profile scope.
    • Pin immutable descriptors to the conversation rather than an individual cached agent.
    • Preserve the snapshot across model/fallback agent rebuilds and clear it at true conversation boundaries.
    • Skip capability-worker setup for non-Slack sessions.
  • gateway/session.py
    • Render epistemically precise conservative and capability-present Slack guidance.
    • Revalidate descriptor identifiers before prompt insertion.
    • Preserve Discord behavior and avoid any ordinary-turn send_message claim.
  • Tests cover registration, positional compatibility, selected-tool gating, profile A/B selection and execution isolation, strict true-to-mismatch transitions, exception log hygiene, registry races, event-loop liveness, prompt injection, operation allowlisting, and conversation snapshot lifecycle.
  • Update Slack user and plugin developer documentation.

How to Test

  1. Run the focused implementation suite:
    scripts/run_tests.sh \
      tests/tools/test_registry.py \
      tests/hermes_cli/test_plugins.py \
      tests/test_model_tools.py \
      tests/gateway/test_session.py \
      tests/gateway/test_prompt_tail_freeze.py -q
  2. Run profile/cache/lifecycle and Slack adapter regressions listed under Screenshots / Logs.
  3. Run Ruff, Python compilation, git diff --check, and the documentation production build.
  4. Install a test plugin that declares a Slack operation with a strict identity check. Confirm that the operation appears only under the profile that loaded it, and that a mismatched identity produces the conservative prompt state.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs and explained the architectural differences above
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run the complete canonical suite; 44,200 tests passed and 12 unrelated tests failed (details below)
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on Ubuntu Linux

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • cli-config.yaml.example is N/A; no configuration key was added
  • CONTRIBUTING.md / AGENTS.md changes are N/A
  • I've considered cross-platform impact; the new runtime path uses stdlib asyncio.to_thread, pathlib, and existing profile context variables
  • Tool descriptions/schemas are N/A; this adds optional registry metadata without changing existing tool schemas

Screenshots / Logs

Focused implementation suite:

378 passed, 0 failed

Relevant broader verification after the final rebase:

Profile/cache/conversation-lifecycle suites: 125 passed, 0 failed.
Slack adapters + model-tools bridge/search suites: 382 passed, 0 failed.
Ruff check: passed.
Python compileall: passed.
git diff --check: passed.
Docusaurus production build: passed; pre-existing unrelated broken-link/anchor warnings remain.

Complete canonical suite after the final rebase:

2145 test files completed: 44,200 passed, 12 failed.
Failures were outside this feature's changed files:
- tests/agent/test_skill_utils.py: 1 mtime cache-invalidation failure
- tests/gateway/test_agent_cache.py: 1 Honcho mtime failure
- tests/honcho_plugin/test_pin_peer_name.py: 2 Honcho mtime failures
- tests/tools/test_execute_code_approval_cluster.py: 8 approval-cluster baseline failures

Immediate isolated rerun of those four files:
- 176 passed, 9 failed
- Honcho pin-transition tests passed 43/43
- gateway agent-cache passed 80/80 on retry
- the skill-utils mtime test and eight approval-cluster tests remained failing

These failures do not execute the new Slack capability metadata, profile dispatch gates, prompt rendering, or conversation snapshot path. The focused and broader feature suites above are green.

Type/format baseline review:

Ruff lint on all changed Python files: passed.
git diff --check: passed.
ty reports 166 existing diagnostics across the five changed source files, with 0 diagnostics on lines added by this PR.
The repository-wide Ruff formatter baseline would reformat the existing large files, so no broad formatting rewrite was included.

No live Slack API calls or test posts were made. No credentials, scopes, permissions, or gateway processes were changed. After installation, restart each affected gateway so plugin registration and in-memory conversation capability snapshots are rebuilt; the next turn may resume the same persisted conversation.

@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/gateway Gateway runner, session dispatch, delivery comp/cli CLI entry point, hermes_cli/, setup wizard comp/tools Tool registry, model_tools, toolsets platform/slack Slack app adapter needs-decision Awaiting maintainer decision before any implementation labels Jul 22, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.
Related to #6545, #62135, and #68627. These patches target the same Slack capability-guidance problem but make different contract choices: this PR requires explicit declared capability metadata rather than inferring from configured tools or names. Maintainer decision needed.

@teknium1

Copy link
Copy Markdown
Contributor

Closing as superseded by #70194 (merged): covered by the derivation approach landed via #70194.

Thanks for the work — it's credited in #70194's summary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery comp/tools Tool registry, model_tools, toolsets needs-decision Awaiting maintainer decision before any implementation P3 Low — cosmetic, nice to have platform/slack Slack app adapter type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants