Skip to content

fix(hermes): enable platforms.slack in sandbox config.yaml when slack channel is attached - #4714

Merged
cv merged 5 commits into
mainfrom
fix/4712-platforms-slack-enable
Jun 3, 2026
Merged

fix(hermes): enable platforms.slack in sandbox config.yaml when slack channel is attached#4714
cv merged 5 commits into
mainfrom
fix/4712-platforms-slack-enable

Conversation

@laitingsheng

@laitingsheng laitingsheng commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Summary

nemohermes <sandbox> channels add slack + rebuild left /sandbox/.hermes/config.yaml without a platforms.slack block, so the Hermes gateway booted with only api_server, slack_bolt never started, and the bot silently received no Slack events.

Related Issue

Fixes #4712

Changes

  • agents/hermes/config/hermes-config.ts:buildHermesConfig — emit platforms.slack = { enabled: true } when enabledChannels.has("slack"). Discord/Telegram stay top-level only (their adapters auto-enable from config.discord / config.telegram); Slack has no top-level block in the Hermes schema, so the platforms-map entry is the only enable signal.
  • test/generate-hermes-config.test.ts — invert the Slack assertion from absent → { enabled: true }; add a negative-path test (no slack channel → platforms.slack undefined, only api_server) and a positive-path test that enables Slack without an allow-list (still emits platforms.slack). Telegram assertion unchanged.
  • test/e2e-scenario/validation_suites/messaging/slack/00-slack-provider-state.sh — add a Hermes branch alongside the existing OpenClaw branch. Reads /sandbox/.hermes/config.yaml via openshell sandbox exec, asserts platforms.slack.enabled is true (PyYAML path with grep fallback when the module is unavailable), and tails gateway.log to verify the gateway booted with more than one platform / connected to Slack — directly reproducing the failing condition from the issue.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Verification

  • npx prek run --all-files passes
  • npm test passes
  • Tests added or updated for new or changed behavior
  • No secrets, API keys, or credentials committed
  • Docs updated for user-facing behavior changes
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Tinson Lai tinsonl@nvidia.com

Summary by CodeRabbit

  • Improvements

    • Slack platform config is now constructed and applied only when Slack is enabled in messaging channels.
  • Tests

    • Validation and E2E tests tightened to assert presence and enabled state of the Slack platform, allowed-channel handling, and gateway startup signals; dry-run validations emit expected pass messages.
  • Chores

    • Scenario and manifest inputs updated to require an additional Slack credential (SLACK_APP_TOKEN).

… channel is attached

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
@coderabbitai

coderabbitai Bot commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b1e0a171-e955-48f2-b667-a4592d26fa54

📥 Commits

Reviewing files that changed from the base of the PR and between 1d98b77 and c34b8a7.

📒 Files selected for processing (3)
  • test/e2e-scenario/manifests/hermes-nvidia-slack.yaml
  • test/e2e-scenario/scenarios/scenarios/baseline.ts
  • test/e2e-scenario/validation_suites/messaging/slack/00-slack-provider-state.sh
🚧 Files skipped from review as they are similar to previous changes (3)
  • test/e2e-scenario/scenarios/scenarios/baseline.ts
  • test/e2e-scenario/manifests/hermes-nvidia-slack.yaml
  • test/e2e-scenario/validation_suites/messaging/slack/00-slack-provider-state.sh

📝 Walkthrough

Walkthrough

Conditionally adds a platforms.slack block to generated Hermes config when Slack is included in messaging.enabledChannels, updates unit tests to cover presence/absence cases, and tightens e2e/provider-state validation to assert platforms.slack.enabled == true and verify Slack gateway startup logs.

Changes

Slack Platform Configuration Fix

Layer / File(s) Summary
Slack platform conditional config generation
agents/hermes/config/hermes-config.ts
buildHermesConfig now builds a local platforms object seeded with api_server, conditionally adds platforms.slack when Slack is in settings.messaging.enabledChannels, then assigns it to config.platforms.
Config generation unit test updates
test/generate-hermes-config.test.ts
Tests updated: assert config.platforms.slack is { enabled: true } when Slack tokens/channels are configured; added cases for omitted platforms.slack when no messaging channels and enabled platforms.slack when Slack is selected even with empty allowlist.
Hermes e2e config validation tightening
test/e2e/test-hermes-slack-e2e.sh
E2E test Python probe now requires platforms and platforms.slack to be mappings and asserts platforms.slack.enabled is True, emitting explicit errors on failure.
Slack provider-state validation (sandbox)
test/e2e-scenario/validation_suites/messaging/slack/00-slack-provider-state.sh
Adds Hermes provider-state branch that captures agent context, validates /sandbox/.hermes/config.yaml platforms.slack.enabled == true, checks SLACK_ALLOWED_CHANNELS in .env, and inspects gateway logs for Slack startup/auth/socket-mode evidence.
E2E manifest & scenario secret updates
test/e2e-scenario/manifests/hermes-nvidia-slack.yaml, test/e2e-scenario/scenarios/scenarios/baseline.ts
Adds SLACK_APP_TOKEN to the manifest credentialRefs and to scenario requiredSecrets for Slack-enabled scenario inputs.

Sequence Diagram(s)

sequenceDiagram
  participant Developer
  participant buildHermesConfig
  participant settings
  participant config
  participant e2e_tests
  Developer->>buildHermesConfig: invoke generation
  buildHermesConfig->>settings: read messaging.enabledChannels
  buildHermesConfig->>buildHermesConfig: create local platforms (api_server)
  alt Slack present
    buildHermesConfig->>buildHermesConfig: add platforms.slack.enabled = true
  end
  buildHermesConfig->>config: set config.platforms
  Developer->>e2e_tests: validate config.yaml and gateway logs
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Suggested labels

enhancement: messaging, v0.0.57

Suggested reviewers

  • prekshivyas

Poem

🐰 A config hop, a tiny cheer,
Platforms built so Slack is near,
Tests now check the YAML true,
Logs show sockets born anew,
The rabbit nods—"Platform's here!"

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title clearly and concisely summarizes the main change: enabling platforms.slack in the sandbox config.yaml when the Slack channel is attached, which directly addresses issue #4712.
Linked Issues check ✅ Passed All changes align with issue #4712 requirements: buildHermesConfig now emits platforms.slack when Slack is in enabledChannels, tests validate this behavior, and e2e validation ensures Socket Mode connectivity.
Out of Scope Changes check ✅ Passed All changes are directly scoped to fixing the missing platforms.slack block in config.yaml and validating Slack Socket Mode connectivity; no unrelated modifications detected.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/4712-platforms-slack-enable

Comment @coderabbitai help to get the list of available commands and usage tips.

@laitingsheng laitingsheng added integration: slack Slack integration or channel behavior integration: hermes Hermes integration behavior bug-fix PR fixes a bug or regression labels Jun 3, 2026
@github-actions

github-actions Bot commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: hermes-slack-e2e, ubuntu-repo-cloud-hermes-slack
Optional E2E: hermes-e2e, messaging-providers-e2e

Dispatch hint: hermes-slack-e2e

Auto-dispatched E2E: hermes-slack-e2e via nightly-e2e.yaml at c34b8a785ce450683a3251291b6c9a5063be7312nightly run

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

  • hermes-slack-e2e (high): Primary existing coverage for the changed runtime path: installs/onboards Hermes with Slack enabled and validates platforms.slack config, Slack providers, credential placeholder isolation, token leak checks, health, and Slack network policy.
  • ubuntu-repo-cloud-hermes-slack (medium): Canonical scenario covering the touched Hermes Slack manifest, baseline scenario registry secret requirements, and messaging Slack provider-state validation suite.

Optional E2E

  • hermes-e2e (high): Useful adjacent confidence for the shared Hermes config generation/refactor path without Slack-specific providers; validates baseline Hermes onboarding, sandbox health, and inference still work.
  • messaging-providers-e2e (medium): Adjacent provider/placeholder/L7 proxy coverage for messaging credentials including Slack, but less directly targeted than hermes-slack-e2e for the Hermes platforms.slack behavior.

New E2E recommendations

  • Live Hermes Slack Socket Mode scenario coverage (high): Existing hermes-slack-e2e uses fake Slack tokens, while the scenario validation now checks gateway log evidence that the Slack platform reached Socket Mode/Bolt running state. Add a dispatchable real-secret Hermes Slack scenario or nightly job that passes SLACK_BOT_TOKEN_REAL, SLACK_APP_TOKEN_REAL, and a Slack channel id into the Hermes Slack flow and exercises the new provider-state assertions without relying only on dry-run/static checks.
    • Suggested test: Add a dispatchable Hermes Slack real Socket Mode E2E job for ubuntu-repo-cloud-hermes-slack using real Slack CI secrets and the messaging-slack validation suite.

Dispatch hint

  • Workflow: .github/workflows/nightly-e2e.yaml
  • jobs input: hermes-slack-e2e

@github-actions

github-actions Bot commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

E2E Scenario Advisor Recommendation

Required scenario E2E: e2e-scenarios-all
Optional scenario E2E: None

Dispatch required scenario E2E:

  • gh workflow run e2e-scenarios-all.yaml --ref <pr-head-ref>

Workflow run

Full scenario advisor summary

E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required scenario E2E

  • e2e-scenarios-all: Scenario catalog/definition metadata changed in test/e2e-scenario/scenarios/scenarios/baseline.ts, which requires the full scenario fan-out by policy. The PR also changes the Hermes Slack scenario manifest and Slack validation script, so the all-scenarios run will cover the affected ubuntu-repo-cloud-hermes-slack and adjacent messaging surfaces.
    • Dispatch: gh workflow run e2e-scenarios-all.yaml --ref <pr-head-ref>

Optional scenario E2E

  • None.

Relevant changed files

  • agents/hermes/config/hermes-config.ts
  • test/e2e-scenario/manifests/hermes-nvidia-slack.yaml
  • test/e2e-scenario/scenarios/scenarios/baseline.ts
  • test/e2e-scenario/validation_suites/messaging/slack/00-slack-provider-state.sh

@github-actions

github-actions Bot commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor

Findings: 0 needs attention, 2 worth checking, 0 nice ideas
Since last review: 1 prior item resolved, 0 still apply, 2 new items found

Review findings

🛠️ Needs attention

  • None.

🔎 Worth checking

  • Typed Hermes Slack scenario does not select the messaging-slack suite (test/e2e-scenario/scenarios/scenarios/baseline.ts:183): This PR expands the Slack provider-state validation to assert Hermes `platforms.slack.enabled` and gateway Slack startup, but the typed canonical Hermes Slack scenario still lists only `suiteIds: ["smoke"]`. The YAML runtime resolver metadata already selects `messaging-slack`, so the active `run-scenario.sh` path appears covered, but this edited TypeScript source-of-truth can still produce a plan that skips the new assertions.
    • Recommendation: Either add `messaging-slack` to the typed Hermes Slack scenario as well, or document/remove the typed registry path if the YAML resolver is now the sole runtime source of truth.
    • Evidence: `baseline.ts` keeps `ubuntu-repo-cloud-hermes-slack` at `suiteIds: ["smoke"]`, while `test/e2e-scenario/nemoclaw_scenarios/scenarios.yaml` has `ubuntu-repo-docker__cloud-nvidia-hermes-slack` suites `smoke` and `messaging-slack`; the new Hermes checks live in `00-slack-provider-state.sh`, which runs through `messaging-slack`.
  • Issue's @-mention response acceptance is only indirectly covered (test/e2e-scenario/validation_suites/messaging/slack/00-slack-provider-state.sh:138): The linked issue's expected result includes the bot responding to @-mentions in the allowed channel. The added validation proves `platforms.slack.enabled` and checks Socket Mode/Bolt startup markers, which is strong proxy evidence for the reported root cause, but it does not directly exercise a Slack event and bot reply path.

🌱 Nice ideas

  • None.
Since last review details

Current findings:

  • Typed Hermes Slack scenario does not select the messaging-slack suite (test/e2e-scenario/scenarios/scenarios/baseline.ts:183): This PR expands the Slack provider-state validation to assert Hermes `platforms.slack.enabled` and gateway Slack startup, but the typed canonical Hermes Slack scenario still lists only `suiteIds: ["smoke"]`. The YAML runtime resolver metadata already selects `messaging-slack`, so the active `run-scenario.sh` path appears covered, but this edited TypeScript source-of-truth can still produce a plan that skips the new assertions.
    • Recommendation: Either add `messaging-slack` to the typed Hermes Slack scenario as well, or document/remove the typed registry path if the YAML resolver is now the sole runtime source of truth.
    • Evidence: `baseline.ts` keeps `ubuntu-repo-cloud-hermes-slack` at `suiteIds: ["smoke"]`, while `test/e2e-scenario/nemoclaw_scenarios/scenarios.yaml` has `ubuntu-repo-docker__cloud-nvidia-hermes-slack` suites `smoke` and `messaging-slack`; the new Hermes checks live in `00-slack-provider-state.sh`, which runs through `messaging-slack`.
  • Issue's @-mention response acceptance is only indirectly covered (test/e2e-scenario/validation_suites/messaging/slack/00-slack-provider-state.sh:138): The linked issue's expected result includes the bot responding to @-mentions in the allowed channel. The added validation proves `platforms.slack.enabled` and checks Socket Mode/Bolt startup markers, which is strong proxy evidence for the reported root cause, but it does not directly exercise a Slack event and bot reply path.

Workflow run details

This is an automated advisory review. A human maintainer must make the final merge decision.

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
…rio checks

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
@github-actions

github-actions Bot commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 26894341551
Target ref: fe399d8be63fa3202fc0eaa6e5f0fc509774b624
Workflow ref: main
Requested jobs: hermes-slack-e2e
Summary: 1 passed, 0 failed, 0 skipped

Job Result
hermes-slack-e2e ✅ success

…cross-reference

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
…mes Slack scenario

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
test/e2e-scenario/scenarios/scenarios/baseline.ts (1)

171-171: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Add SLACK_APP_TOKEN to ubuntu-repo-cloud-openclaw-slack required secrets

  • ubuntu-repo-cloud-openclaw-slack currently requires only ["NVIDIA_API_KEY", "SLACK_BOT_TOKEN"], but the Slack integration/validation requires both SLACK_BOT_TOKEN and SLACK_APP_TOKEN (Slack is disabled/rejected if SLACK_APP_TOKEN is missing).
  • Align ubuntu-repo-cloud-openclaw-slack with ubuntu-repo-cloud-hermes-slack which already includes SLACK_APP_TOKEN.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e-scenario/scenarios/scenarios/baseline.ts` at line 171, Update the
requiredSecrets for the ubuntu-repo-cloud-openclaw-slack scenario by adding
"SLACK_APP_TOKEN" to the existing array (currently
["NVIDIA_API_KEY","SLACK_BOT_TOKEN"]) so it matches
ubuntu-repo-cloud-hermes-slack; locate the requiredSecrets declaration in
baseline.ts for the scenario named ubuntu-repo-cloud-openclaw-slack and include
"SLACK_APP_TOKEN" in that array.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@test/e2e-scenario/scenarios/scenarios/baseline.ts`:
- Line 171: Update the requiredSecrets for the ubuntu-repo-cloud-openclaw-slack
scenario by adding "SLACK_APP_TOKEN" to the existing array (currently
["NVIDIA_API_KEY","SLACK_BOT_TOKEN"]) so it matches
ubuntu-repo-cloud-hermes-slack; locate the requiredSecrets declaration in
baseline.ts for the scenario named ubuntu-repo-cloud-openclaw-slack and include
"SLACK_APP_TOKEN" in that array.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 2f7e2826-72c2-4deb-8fc3-ca00ceb42fb4

📥 Commits

Reviewing files that changed from the base of the PR and between 1d98b77 and c34b8a7.

📒 Files selected for processing (3)
  • test/e2e-scenario/manifests/hermes-nvidia-slack.yaml
  • test/e2e-scenario/scenarios/scenarios/baseline.ts
  • test/e2e-scenario/validation_suites/messaging/slack/00-slack-provider-state.sh
✅ Files skipped from review due to trivial changes (1)
  • test/e2e-scenario/manifests/hermes-nvidia-slack.yaml
🚧 Files skipped from review as they are similar to previous changes (1)
  • test/e2e-scenario/validation_suites/messaging/slack/00-slack-provider-state.sh

@github-actions

github-actions Bot commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 26896833295
Target ref: c34b8a785ce450683a3251291b6c9a5063be7312
Workflow ref: main
Requested jobs: hermes-slack-e2e
Summary: 1 passed, 0 failed, 0 skipped

Job Result
hermes-slack-e2e ✅ success

@wscurran
wscurran requested a review from cv June 3, 2026 16:46
@cv cv self-assigned this Jun 3, 2026
@cv
cv merged commit 77fbcd9 into main Jun 3, 2026
33 checks passed
@cv
cv deleted the fix/4712-platforms-slack-enable branch June 3, 2026 19:23
@wscurran wscurran added the NV QA Bugs found by the NVIDIA QA Team label Jun 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug-fix PR fixes a bug or regression integration: hermes Hermes integration behavior integration: slack Slack integration or channel behavior NV QA Bugs found by the NVIDIA QA Team

Projects

None yet

3 participants