diff --git a/.agents/skills/nemoclaw-contributor-onboard-messaging-channel/SKILL.md b/.agents/skills/nemoclaw-contributor-onboard-messaging-channel/SKILL.md new file mode 100644 index 00000000000..ce5458b996f --- /dev/null +++ b/.agents/skills/nemoclaw-contributor-onboard-messaging-channel/SKILL.md @@ -0,0 +1,102 @@ +--- +name: nemoclaw-contributor-onboard-messaging-channel +description: Guide NemoClaw contributors through adding or reviewing a new messaging channel in the manifest-first messaging architecture. Use when onboarding a channel for OpenClaw, Hermes, or both; mapping upstream channel docs and source code into NemoClaw manifests; confirming credentials, plugin/package installs, reachability checks, network policy presets, docs, and tests. Trigger keywords - add messaging channel, onboard messaging channel, new channel, messaging integration, channel manifest, OpenClaw channel, Hermes channel, plugin install, reachability check. +--- + +# Onboard Messaging Channel + +Use this skill to add a messaging channel end-to-end without leaking channel-specific logic into core NemoClaw code. + +## Intake + +Gather inputs progressively. Do not ask the full intake checklist in one message. +Ask exactly one concise clarification at a time, choosing the earliest unresolved blocker: + +1. If the channel name is missing, ask for the channel name. +2. If target agents are missing, ask whether the channel should support OpenClaw, Hermes, or both. +3. If upstream references are missing, ask for the official docs link and source-of-truth implementation link or path. Include Telegram as the format example: docs `https://docs.openclaw.ai/channels/telegram`, source `https://github.com/openclaw/openclaw/tree/main/extensions/telegram`. +4. After references are available, read them and the local messaging package before asking about credentials, plugin installs, reachability, or network policy. +5. Ask follow-up questions one by one only for details that remain ambiguous after source analysis. + +Use this intake checklist internally while analyzing source: + +- Channel name and target agents: OpenClaw, Hermes, or both. Treat unsupported agents as intentionally out of scope unless the user provides source evidence. +- Official channel documentation link and source-of-truth implementation link or path. Prefer upstream extension/runtime code over README prose when they conflict. +- Required credentials and config inputs: token environment variables, bot or app IDs, user IDs, workspace/guild/group IDs, allowlists, app secrets, webhook secrets, socket-mode/app tokens, QR pairing, callback URLs, or proxy settings. +- Plugin or package install requirements: package name, install manager, version pinning, bundled versus external status, extension ID, and whether the package must be installed during image build. +- Reachability or health evidence: endpoint or command, HTTP method, auth semantics, success response, invalid-credential response, transient-network behavior, and whether tests need a skip env var for fake credentials. +- Network reachability: exact hostnames required at runtime, whether they are agent-only or bridge-only, and whether the policy should be opt-in. + +When asking a follow-up, include the source-derived fact that made the question necessary. Example: "The upstream extension enables a webhook secret, but I do not see whether NemoClaw should prompt for it. Should this be a required input?" + +## Source Analysis + +Before editing, read: + +- Root `AGENTS.md` and `CONTRIBUTING.md`. +- `src/lib/messaging/AGENTS.md`. +- The closest existing channel manifests and tests under `src/lib/messaging/channels/`. +- The upstream docs and source code supplied by the user. + +Compare the new channel to existing patterns: + +- Token plus API reachability: Telegram-style. +- Multiple credentials, socket mode, or channel-owned conflicts: Slack-style. +- Allowlists or scoped IDs: Discord-style. +- QR or pairing flow with runtime status: WeChat or WhatsApp-style. +- Agent-specific plugin install and config render: channels that require external agent extensions. + +When docs and source disagree, implement from source code and note the inference in the final handoff. + +## Implementation Workflow + +Start with the manifest. Add core code only when the manifest vocabulary cannot express a reusable concept. + +1. Add `src/lib/messaging/channels//manifest.ts` with `auth`, `inputs`, `credentials`, `policyPresets`, `render`, `runtime`, `agentPackages`, `state`, and `hooks` as needed. +2. Add `channels//template-resolver.ts` only for derived render values, such as allowlist normalization, booleans, proxy URLs, or agent-specific schema differences. +3. Add hooks under `channels//hooks/` only for enrollment, external reachability checks, QR capture, conflict checks, runtime status, or health probes that cannot be static manifest data. +4. Register the manifest in `channels/built-ins.ts`, template resolver in `channels/template-resolver.ts`, and hook handlers in `hooks/builtins.ts`. +5. Add `nemoclaw-blueprint/policies/presets/.yaml` when the manifest declares a policy preset. Keep messaging-specific egress opt-in unless the project policy says otherwise. +6. Update `agents/openclaw/manifest.yaml` and/or `agents/hermes/manifest.yaml` so supported platforms match the manifest `supportedAgents`. +7. Add agent package install metadata when the channel needs an external agent plugin. For OpenClaw plugin packages, use this shape unless source evidence says otherwise: + + ```ts + agentPackages: [ + { + id: "openclawPluginPackage", + agent: "openclaw", + manager: "openclaw-plugin", + spec: "npm:@openclaw/@{{openclaw.version}}", + pin: true, + required: true, + }, + ], + ``` + +8. Update docs for user-facing behavior, usually `docs/manage-sandboxes/messaging-channels.mdx`, command references, network policy references, and troubleshooting. + +## Quality Gates + +- Validate the runtime config schema from upstream code. Do not copy another channel's nested config shape blindly without source evidence. +- If `render` enables a plugin entry, confirm the install source exists or document why it is bundled. +- Keep Hermes unsupported when only OpenClaw source support exists, and vice versa. +- Keep channel-specific conditionals out of onboard, rebuild, compiler, applier, and generated-config entrypoints unless the change is a general manifest capability. +- Persist only non-secret state. Plans may contain placeholders, availability flags, and hashes, never raw tokens. +- Mock external APIs in tests. Unit tests must not call real messaging providers. +- Use a skip env var for live reachability hooks when fake credentials are valid for local tests. +- Make policy hostnames exact and scoped to the channel preset. + +## Verification + +Use the narrowest tests that cover the changed behavior: + +```bash +npm run build:cli +npm run typecheck:cli +npx vitest run src/lib/messaging/channels/manifests.test.ts src/lib/messaging/channels/metadata.test.ts src/lib/messaging/compiler/manifest-compiler.test.ts +npx vitest run src/lib/messaging/channels//hooks +npx vitest run test/messaging-build-applier.test.ts +``` + +Add channel-specific config render, hook, policy, and channel add/remove tests when those surfaces change. +Run `npm run docs` for documentation changes and `npx prek run --files ` before handoff. If broad hooks expose unrelated failures, report the failure with the targeted passing evidence. diff --git a/.agents/skills/nemoclaw-skills-guide/SKILL.md b/.agents/skills/nemoclaw-skills-guide/SKILL.md index ea338fe7e58..c264a58148e 100644 --- a/.agents/skills/nemoclaw-skills-guide/SKILL.md +++ b/.agents/skills/nemoclaw-skills-guide/SKILL.md @@ -27,10 +27,10 @@ Covers installation, inference configuration, network policy management, monitor For project maintainers. Covers the daily maintainer cadence (morning standup, daytime loop, evening handoff), workflow policy reference, cutting releases, drafting release notes, finding PRs to review, comparing PRs, cross-issue sweeps, triage, normalizing issue and PR title tags, performing security code reviews, and verifying whether stale bug reports still reproduce on the latest release. -### `nemoclaw-contributor-*` (2 skills) +### `nemoclaw-contributor-*` (3 skills) For contributors to the NemoClaw codebase. -Covers creating pull requests that follow the project template and drafting documentation updates from recent commits. +Covers creating pull requests that follow the project template, drafting documentation updates from recent commits, and onboarding new messaging channels. ## Skill Catalog @@ -74,6 +74,7 @@ Covers creating pull requests that follow the project template and drafting docu | Skill | Summary | |-------|---------| | `nemoclaw-contributor-create-pr` | Create GitHub pull requests that follow the NemoClaw PR template, including pre-PR checks, conventional commit titles, and DCO sign-off. | +| `nemoclaw-contributor-onboard-messaging-channel` | Add or review a new messaging channel with manifest-first implementation, upstream source analysis, plugin install confirmation, reachability checks, policies, docs, and tests. | | `nemoclaw-contributor-update-docs` | Scan recent git commits for user-facing changes, draft or update documentation pages, and refresh generated user skills during release prep. | ## Getting Started @@ -89,7 +90,7 @@ Skills are cumulative. Each role includes the skills from the roles above it: | Role | Skills included | Count | Start with | |------|----------------|-------|------------| | User | `nemoclaw-user-*` | 10 | `nemoclaw-user-get-started` | -| Contributor | `nemoclaw-user-*` + `nemoclaw-contributor-*` | 12 | `nemoclaw-user-overview` | -| Maintainer | All skills | 25 | `nemoclaw-maintainer-morning` | +| Contributor | `nemoclaw-user-*` + `nemoclaw-contributor-*` | 13 | `nemoclaw-user-overview` | +| Maintainer | All skills | 26 | `nemoclaw-maintainer-morning` | After identifying the role, present the applicable skills from the Skill Catalog above and recommend the starting skill. diff --git a/.markdownlint-cli2.yaml b/.markdownlint-cli2.yaml index 4bcf0a209cb..56e36db6c33 100644 --- a/.markdownlint-cli2.yaml +++ b/.markdownlint-cli2.yaml @@ -6,6 +6,10 @@ # # All default rules are enabled unless listed below. +# Skills require SPDX comments before YAML frontmatter. Keep the frontmatter +# parser aware of that prefix so Markdown body rules still run normally. +frontMatter: "^(?:\\n\\n\\n?)?---\\n[^]*?\\n---\\n?" + config: default: true diff --git a/AGENTS.md b/AGENTS.md index 1fc7d26750c..80254ea35d2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -31,6 +31,10 @@ This repo ships agent skills under `.agents/skills/`, organized into three audie | `docs/` | MDX/Markdown | User-facing docs (Fern MDX plus legacy MyST source during migration) | | `fern/` | YAML/CSS/SVG | Fern site configuration and shared assets | +Package-specific guides: + +- Messaging architecture and channel migration guidance: [`src/lib/messaging/AGENTS.md`](src/lib/messaging/AGENTS.md) + ## Quick Reference | Task | Command | diff --git a/src/lib/messaging/AGENTS.md b/src/lib/messaging/AGENTS.md new file mode 100644 index 00000000000..cf734c8fa95 --- /dev/null +++ b/src/lib/messaging/AGENTS.md @@ -0,0 +1,83 @@ + + + +# Agent Instructions for `src/lib/messaging` + +## Purpose + +This package owns NemoClaw's manifest-first messaging architecture. It turns channel declarations for Telegram, Discord, Slack, WeChat, and WhatsApp into a serializable `SandboxMessagingPlan`, then applies that plan during onboard, channel add/remove/start/stop, rebuild, image build, runtime setup, diagnostics, and conflict checks. + +The design goal is to keep messaging channel behavior out of core onboard/rebuild logic. Add channel-specific behavior to manifests, template resolvers, hooks, runtime assets, and policy metadata first; only change shared engines when the manifest vocabulary cannot express the required behavior. + +## Data Flow + +1. Channel manifests live in `channels//manifest.ts` and are registered by `channels/built-ins.ts`. +2. `MessagingWorkflowPlanner` selects the right workflow shape for onboard, add, remove, start, stop, or rebuild. +3. `ManifestCompiler` and `compiler/engines/*` compile manifests into a `SandboxMessagingPlan`. +4. `MessagingSetupApplier` serializes the plan through `NEMOCLAW_MESSAGING_PLAN_B64`. +5. `onboard/dockerfile-patch.ts` bakes the plan into the sandbox build. +6. `applier/build/messaging-build-applier.mts` applies agent install, render, post-agent-install build files, and writes the reduced runtime plan artifact. +7. `MessagingHostStateApplier` persists durable plan state under the sandbox registry entry. +8. Rebuild reads the persisted plan, stages a fresh build plan, and reapplies OpenClaw render/post-install hooks after `openclaw doctor` rewrites config. + +## Package Map + +| Path | Ownership | +|---|---| +| `manifest/` | Serializable manifest and plan contracts. Keep these JSON-compatible. | +| `channels/` | Built-in channel manifests, channel metadata helpers, template resolvers, runtime preload assets, and channel hook implementations. | +| `compiler/` | Manifest-to-plan compilation. It may resolve env/config inputs and run enrollment/reachability/build hooks, but should not mutate OpenShell or registry state directly. | +| `hooks/` | Hook contracts, registries, runner validation, common prompt/static-output helpers, and conflict error types. | +| `applier/` | Host/OpenShell side effects: plan env serialization, provider upsert/reuse, policy apply, agent config writes, hook phase execution, conflict detection, registry persistence, and build-time applier. | +| `persistence.ts` | Compact persisted plan shape and hydration from current manifests. | +| `plan-validation.ts` | Defensive parsing for persisted or env-provided plans. | +| `diagnostics.ts` | Manifest-derived channel diagnostics used by status/doctor paths. | +| `utils.ts` | Agent/channel availability and selection helpers. | + +## Core Invariants + +- Manifests and compiled plans are serializable data. Do not put functions, classes, live clients, or raw secret values in them. +- Secret inputs must not declare `statePath`; persisted plans may contain `credentialAvailable`, `credentialHash`, and placeholders, never tokens. +- Hook implementations are resolved by stable handler IDs through `MessagingHookRegistry`. Manifests reference handlers by string; they do not import handler code. +- Hook outputs must match manifest declarations and be JSON-serializable. Add outputs to the manifest before consuming them. +- Channel render/build-file targets must stay inside `/sandbox/.openclaw` or `/sandbox/.hermes`; rely on existing applier validation instead of bypassing it. +- Disabled channels are not active. Always filter effects through `enabledPlanChannels()` or `filterEnabledPlanEntries()` when applying providers, policies, render, hooks, runtime setup, or conflicts. +- Conflict detection has two axes: generic credential-hash overlap in `applier/conflict-detection/` and channel-owned `pre-enable` hooks such as Slack Socket Mode gateway checks. +- Keep transitional compatibility tables derived from manifests. `src/lib/sandbox/channels.ts` intentionally builds legacy CLI metadata from `listBuiltInMessagingChannelManifests()`. + +## Adding or Changing a Channel + +Start with `channels//manifest.ts`. + +1. Declare `auth`, `inputs`, `credentials`, `policyPresets`, `render`, `runtime`, `agentPackages`, `state`, and `hooks` in the manifest. +2. Add template placeholders to `channels//template-resolver.ts` when static render data needs derived values such as allowlists, booleans, proxy URLs, or Hermes/OpenClaw schema differences. +3. Add hook implementations under `channels//hooks/` only for side effects or checks that cannot be represented as static manifest data. +4. Register hook handlers in the channel `hooks/index.ts` and in `hooks/builtins.ts`. +5. Add runtime preload assets under `channels//runtime/` only when the agent runtime needs boot/connect-time shims or diagnostics. +6. Add or update `nemoclaw-blueprint/policies/presets/.yaml` when the manifest declares a channel policy preset. +7. Cover the behavior with manifest/compiler tests plus applier/onboard/channel CLI tests when host effects change. + +## Where Changes Belong + +- New prompt, token, allowlist, provider, policy, render, package install, runtime setup, state hydration, or health-check metadata belongs in a channel manifest. +- Nontrivial render derivation belongs in a channel template resolver. +- Enrollment, external reachability checks, QR capture, channel-specific conflict checks, runtime status, and health probes belong in hooks. +- Provider creation/reuse, policy application, config-file writes, plan env encoding, and registry persistence belong in `applier/`. +- Onboard and `actions/sandbox/policy-channel.ts` should orchestrate planner/applier calls, not grow channel-specific rules. +- Build-time config generation should use the compiled plan and `applier/build/messaging-build-applier.mts`; do not reintroduce channel-specific config rendering in `scripts/generate-openclaw-config.mts` or `agents/hermes/generate-config.ts`. + +## Testing Guide + +Use the narrowest test that covers the changed surface: + +- Manifest shape and plan compilation: `npx vitest run src/lib/messaging/compiler src/lib/messaging/manifest src/lib/messaging/channels` +- Hook behavior: `npx vitest run src/lib/messaging/hooks src/lib/messaging/channels//hooks` +- Host/OpenShell application: `npx vitest run src/lib/messaging/applier` +- Build-time render/install behavior: `npx vitest run test/messaging-build-applier.test.ts` +- Onboard/channel CLI integration: `npx vitest run test/onboard-messaging.test.ts test/channels-add-preset.test.ts src/lib/onboard/messaging-channel-setup.test.ts` + +Mock external messaging APIs. Do not call real Telegram, Discord, Slack, WeChat, WhatsApp, NVIDIA, or OpenShell services from unit tests. + +## Documentation + +User-facing behavior changes usually need docs under `docs/manage-sandboxes/messaging-channels.mdx` or `docs/reference/commands.mdx`. Do not edit generated user skills under `.agents/skills/nemoclaw-user-*/` for normal docs changes.