Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 28 additions & 10 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ Must run from `platform/` directory (not repo root). Env vars: `DATABASE_URL`, `

See `docs/plugins/sources.md` for the two-axis source/shape plugin model.

Additional env vars documented in `.env.example` (2026-04-13 sync — all 21 distinct `os.Getenv`/`envx.*` keys now documented): `MOLECULE_ENV`, `GITHUB_WEBHOOK_SECRET`, `MOLECULE_URL` (MCP server target; same semantic as `PLATFORM_URL`).

`molecli` reads `MOLECLI_URL` (default http://localhost:8080) to locate the platform. Logs are written to `molecli.log` in the working directory (already covered by `*.log` in `.gitignore`).

### Canvas (Next.js)
Expand Down Expand Up @@ -108,20 +110,23 @@ OPENAI_API_KEY=... bash scripts/test-team-e2e.sh # E2E: Multi-template

### Unit Tests
```bash
cd platform && go test -race ./... # 487 Go tests (handlers, registry, provisioner, CLI, delegation, org, channels, wsauth — sqlmock + miniredis)
cd canvas && npm test # 352 Vitest tests (store, components, hydration, buildTree, secrets API, org template import)
cd workspace-template && python -m pytest -v # 1078 pytest tests (adds platform_auth token store for Phase 30.1)
cd sdk/python && python -m pytest -v # 87 SDK tests (agentskills.io spec validator, CLI, AgentskillsAdaptor round-trip, workspace/org/channel validators)
cd platform && go test -race ./... # 695 Go tests (handlers, registry, provisioner, CLI, delegation, org, channels, wsauth — sqlmock + miniredis; +47 on 2026-04-13 covering extracted helpers from the a2a_proxy / delegation / discovery / activity refactor)
cd canvas && npm test # 357 Vitest tests (store, components, hydration, buildTree, secrets API, org template import, ConfirmDialog singleButton + 7 native-dialog replacements)
cd workspace-template && python -m pytest -v # 1140 pytest tests (adds platform_auth token store for Phase 30.1, memory_write activity logging)
cd sdk/python && python -m pytest -v # 132 SDK tests (agentskills.io spec validator, CLI, AgentskillsAdaptor round-trip, workspace/org/channel validators, RemoteAgentClient Phase 30 flows)
cd mcp-server && npm test # 97 Jest tests (per-domain tool modules + smoke test on tool count)
```

### Integration Tests
```bash
bash tests/e2e/test_api.sh # 62 API tests against localhost:8080
bash tests/e2e/test_api.sh # 62 API tests against localhost:8080 (Phase 30.1 bearer-token auth aware; shellcheck-clean; also runs in CI `e2e-api` job)
bash tests/e2e/test_a2a_e2e.sh # 22 A2A end-to-end tests (requires 2 online agents)
bash tests/e2e/test_activity_e2e.sh # 25 activity/task E2E tests (requires 1 online agent)
bash tests/e2e/test_comprehensive_e2e.sh # 68 checks — ALL endpoints, memory, runtime, bundles, approvals
bash tests/e2e/test_activity_e2e.sh # 25 activity/task E2E tests (requires 1 online agent; re-registers detected agent to capture bearer token)
bash tests/e2e/test_comprehensive_e2e.sh # 67 checks — ALL endpoints, memory, runtime, bundles, approvals (registers workspaces immediately after create to beat the provisioner token race)
```
`test_api.sh` requires platform running. Tests full CRUD, registry, heartbeat, discovery, peers, access control, events, degraded/recovery lifecycle, activity logging, current task tracking, bundle round-trip (export → delete → import → verify).
All five E2E scripts share `tests/e2e/_lib.sh` + `tests/e2e/_extract_token.py` helpers and are shellcheck-clean. `test_api.sh` is the quick local-verify command — use it after any platform change. Tests full CRUD, registry, heartbeat, discovery, peers, access control, events, degraded/recovery lifecycle, activity logging, current task tracking, bundle round-trip (export → delete → import → verify).

**Phase 30.1 / 30.6 auth callout (future-proofing):** `/registry/heartbeat` and `/registry/update-card` require `Authorization: Bearer <token>` once a workspace has any live token on file (Phase 30.1 — legacy workspaces grandfathered). `/registry/discover/:id` and `/registry/:id/peers` additionally require `X-Workspace-ID` + bearer token on the caller side (Phase 30.6 — fail-open on DB hiccup since hierarchy check is primary). If you change these routes, update `tests/e2e/test_api.sh` and `docs/api-protocol/platform-api.md` in the same PR.

`test_a2a_e2e.sh` requires platform + two provisioned agents (Echo Agent, SEO Agent) running with a valid `OPENROUTER_API_KEY`. Tests message/send, JSON-RPC wrapping, error handling, peer discovery, agent cards, heartbeat. Timeout configurable via `A2A_TIMEOUT` env var (default 120s).

Expand All @@ -133,14 +138,18 @@ cd mcp-server
npm install && npm run build # Build MCP server
node dist/index.js # Run (stdio transport)
```
Exposes 87 tools for managing Molecule AI from Claude Code, Cursor, Codex, or any MCP client. Includes workspace CRUD, async delegation, plugins (install/uninstall/list), global secrets, pause/resume, org import, A2A chat, approvals, memory, files, config, discovery, bundles, templates, traces, activity logs, and social channels (add/update/remove/send/test). Configured in `.mcp.json`. Env: `MOLECULE_URL` (default http://localhost:8080).
Exposes **87 tools** for managing Molecule AI from Claude Code, Cursor, Codex, or any MCP client. Includes workspace CRUD, async delegation, plugins (install/uninstall/list), global secrets, pause/resume, org import, A2A chat, approvals, memory, files, config, discovery, bundles, templates, traces, activity logs, remote agents (Phase 30), and social channels (add/update/remove/send/test). Configured in `.mcp.json`. Env: `MOLECULE_URL` (default http://localhost:8080).

**Structure (refactored 2026-04-13, PRs #2/#4/#7):** `src/index.ts` shrank from 1697 → 89 lines and now only wires `createServer()`. Per-domain tool modules live in `src/tools/`: `workspaces.ts`, `agents.ts`, `secrets.ts`, `files.ts`, `memory.ts`, `plugins.ts`, `channels.ts`, `delegation.ts`, `schedules.ts`, `approvals.ts`, `discovery.ts`, `remote_agents.ts`. Each exports its handlers and a `registerXxxTools(srv)` function. Shared HTTP layer in `src/api.ts` (`PLATFORM_URL`, `apiCall<T>`, `ApiError`, `isApiError()`, `toMcpResult()`, `toMcpText()`). When adding a tool, pick the matching domain file or create a new one and wire it in `createServer()`.

### CI Pipeline
GitHub Actions (`.github/workflows/ci.yml`) runs on push to main and PRs:
- **platform-build**: Go build, vet, `go test -race` with coverage profiling (25% baseline threshold)
- **platform-build**: Go build, vet, `go test -race` with coverage profiling (25% baseline threshold; `setup-go` uses module cache)
- **canvas-build**: npm build, `vitest run` (no `--passWithNoTests` -- tests must exist and pass)
- **mcp-server-build**: npm build
- **python-lint**: `pytest --cov=. --cov-report=term-missing` (pytest-cov enabled)
- **e2e-api** (added 2026-04-13): spins up Postgres + Redis service containers, runs platform migrations via `docker exec`, then executes `tests/e2e/test_api.sh` against a locally-built binary (62/62 must pass)
- **shellcheck** (added 2026-04-13): lints every `tests/e2e/*.sh` via the shellcheck marketplace action

### Docker Compose
```bash
Expand Down Expand Up @@ -183,6 +192,15 @@ When a workspace specifies a template that doesn't exist, the Create handler fal

The A2A proxy (`POST /workspaces/:id/a2a`) enforces this for agent-to-agent calls. Canvas requests (no `X-Workspace-ID`), self-calls, and system callers (`webhook:*`, `system:*`, `test:*` prefixes via `isSystemCaller()` in `a2a_proxy.go`) bypass the check.

### Handler Decomposition (2026-04-13)
Four oversize handler functions were split into private helpers (pure refactor, behavior unchanged — 47 new unit tests cover the helpers directly; `handlers` package coverage 56.1% → 57.6%):
- `a2a_proxy.go::proxyA2ARequest` (257→56 lines) — helpers: `resolveAgentURL`, `normalizeA2APayload`, `dispatchA2A`, `handleA2ADispatchError`, `maybeMarkContainerDead`, `logA2AFailure`, `logA2ASuccess`; sentinel `proxyDispatchBuildError`
- `delegation.go::Delegate` (127→60 lines) — helpers: `bindDelegateRequest`, `lookupIdempotentDelegation`, `insertDelegationRow`; typed `insertDelegationOutcome` enum replaces `(bool, bool)` positional return
- `discovery.go::Discover` (125→40 lines) — helpers: `discoverWorkspacePeer`, `writeExternalWorkspaceURL`, `discoverHostPeer`
- `activity.go::SessionSearch` (109→24 lines) — helpers: `parseSessionSearchParams`, `buildSessionSearchQuery`, `scanSessionSearchRows`

When modifying any of these, prefer extending the helper rather than inlining back.

### JSONB Gotcha
When inserting Go `[]byte` (from `json.Marshal`) into Postgres JSONB columns, you must:
1. Convert to `string()` first
Expand Down
29 changes: 22 additions & 7 deletions PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,20 @@ for the full code audit.

---

## Phase 31 — Quality + Infra Pass (Q2 2026) — SHIPPED 2026-04-13

Completed in PRs #1–#8 and documented in `docs/edit-history/2026-04-13.md`:

- [x] **Brand migration cleanup** — LICENSE "Agent Molecule" → "Molecule AI"; new icon assets (PR #1).
- [x] **Repo structural cleanup** — moved `examples/remote-agent/` → `sdk/python/examples/`, `docs/superpowers/plans/` → `plugins/superpowers/plans/`; deleted empty `platform/plugins/`; gitignored `.agents/`, `platform/workspace-configs-templates/`, `backups/`, `logs/`, `test-results/`; added READMEs under `tests/` and `docs/` (PR #3).
- [x] **MCP per-domain split** — `mcp-server/src/index.ts` 1697 → 89 lines; 12 per-domain modules in `src/tools/`; shared `src/api.ts`; startup log now reports 87 tools (PRs #2, #4, #7).
- [x] **Canvas dialog unification** — native `confirm()`/`alert()` replaced with `ConfirmDialog` in 7 sites; new `singleButton` prop + 5 tests (vitest 352 → 357).
- [x] **Platform handler decomposition** — 4 oversize functions (`proxyA2ARequest`, `Delegate`, `Discover`, `SessionSearch`) split into testable helpers; +47 Go tests; `handlers` coverage 56.1% → 57.6%.
- [x] **Env-var documentation** — `.env.example` gained 11 previously-undocumented vars; all 21 distinct `os.Getenv`/`envx.*` keys now documented.
- [x] **E2E hardening + CI** — Phase 30.1 bearer auth + Phase 30.6 `X-Workspace-ID` requirements baked into `test_api.sh` (62/62) and `test_comprehensive_e2e.sh` (67/67); shared `_lib.sh` + `_extract_token.py`; new CI jobs `e2e-api` and `shellcheck`; `setup-go` gains module cache (PRs #5, #7, #8).

---

## PR Workflow Rules

All PRs must follow this checklist:
Expand Down Expand Up @@ -224,13 +238,14 @@ point for "what else is out there."

| Stack | Tests | Framework |
|-------|-------|-----------|
| Go (platform) | 476 | `go test -race` |
| Python (workspace) | 1,040 | pytest |
| Canvas (frontend) | 352 | Vitest |
| SDK (python) | 87 | pytest |
| **Total** | **1,955** | |

E2E: 68/68 comprehensive checks passing, 62 API tests.
| Go (platform) | 695 | `go test -race` |
| Python (workspace) | 1,140 | pytest |
| Canvas (frontend) | 357 | Vitest |
| SDK (python) | 132 | pytest |
| MCP server | 97 | Jest |
| **Total** | **2,421** | |

E2E: 67/67 comprehensive checks passing, 62/62 API tests (also gated in CI `e2e-api` job), shellcheck-clean across all 5 E2E scripts.

---

Expand Down
6 changes: 4 additions & 2 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
全球最强大的 Agent Team 治理方案。
</p>

[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
[![License: BSL 1.1](https://img.shields.io/badge/License-BSL%201.1-orange.svg)](LICENSE)
[![Go Version](https://img.shields.io/badge/go-1.25+-00ADD8?logo=go)](https://golang.org/)
[![Python Version](https://img.shields.io/badge/python-3.11+-3776AB?logo=python)](https://www.python.org/)
[![Next.js](https://img.shields.io/badge/Next.js-15-black?logo=next.js)](https://nextjs.org/)
Expand Down Expand Up @@ -287,4 +287,6 @@ npm run dev

## License

MIT
[Business Source License 1.1](LICENSE) — 版权所有 © 2025 Molecule AI。

允许个人、内部与非商业用途。不得使用本作品提供与本产品竞争的商业服务。2029 年 1 月 1 日起转为 Apache 2.0。
18 changes: 10 additions & 8 deletions docs/api-protocol/platform-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,14 +64,16 @@ Workspace creation also assigns an `awareness_namespace` on the workspace row. T

### Registry

| Method | Path | Description |
|---|---|---|
| `POST` | `/registry/register` | Workspace registration on startup |
| `POST` | `/registry/heartbeat` | Liveness and task updates |
| `POST` | `/registry/update-card` | Push Agent Card updates after runtime/skill changes |
| `GET` | `/registry/discover/:id` | Resolve workspace URL for A2A calls |
| `GET` | `/registry/:id/peers` | List reachable peers |
| `POST` | `/registry/check-access` | Validate reachability/access |
| Method | Path | Description | Auth |
|---|---|---|---|
| `POST` | `/registry/register` | Workspace registration on startup. First register issues a per-workspace bearer token in the response body (`auth_token`); re-register is idempotent and omits the token. | — |
| `POST` | `/registry/heartbeat` | Liveness and task updates. | Phase 30.1 — `Authorization: Bearer <token>` required if the workspace has any live token on file; legacy workspaces grandfathered (fail-open). |
| `POST` | `/registry/update-card` | Push Agent Card updates after runtime/skill changes. | Phase 30.1 — same grandfather rule as `/heartbeat`. |
| `GET` | `/registry/discover/:id` | Resolve workspace URL for A2A calls. | Phase 30.6 — caller sends `X-Workspace-ID` + own bearer token; fail-open on DB hiccup (hierarchy check is primary gate). |
| `GET` | `/registry/:id/peers` | List reachable peers. | Phase 30.6 — same as `/discover/:id`. |
| `POST` | `/registry/check-access` | Validate reachability/access. | — |

**Why the auth callout matters:** remote (Phase 30) agents authenticate themselves with the bearer token returned by `POST /registry/register`. Local containers are transparent to this during the lazy-bootstrap grace window — the provisioner threads the token in as an env var on first register. See `docs/development/testing-e2e.md` for how E2E scripts handle token capture. If you change these routes, update `tests/e2e/test_api.sh` in the same PR.

### Activity and recall

Expand Down
21 changes: 14 additions & 7 deletions docs/development/local-development.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,26 +92,33 @@ Docker Compose 2.x
### Unit Tests

```bash
cd platform && go test -race ./... # Go tests with race detection (358 tests)
cd canvas && npm test # Vitest tests (188 tests)
cd workspace-template && python -m pytest -v # Workspace runtime tests (148 tests)
cd platform && go test -race ./... # Go tests with race detection (695 tests)
cd canvas && npm test # Vitest tests (357 tests)
cd workspace-template && python -m pytest -v # Workspace runtime tests (1140 tests)
cd sdk/python && python -m pytest -v # SDK tests (121 tests)
cd mcp-server && npm test # MCP server tests (97 Jest tests)
```

### Integration Tests

```bash
bash test_api.sh # 62 API tests (requires platform running)
bash test_a2a_e2e.sh # 22 A2A e2e tests (requires platform + 2 agents)
bash test_activity_e2e.sh # 25 activity/task E2E tests (requires platform + 1 agent)
bash tests/e2e/test_api.sh # 62 API tests (quick local verify; Phase 30.1 bearer-auth aware; also runs in CI)
bash tests/e2e/test_a2a_e2e.sh # 22 A2A e2e tests (requires platform + 2 agents)
bash tests/e2e/test_activity_e2e.sh # 25 activity/task E2E tests (requires platform + 1 agent)
bash tests/e2e/test_comprehensive_e2e.sh # 67 endpoint/memory/bundle/approval checks
```

All E2E scripts share `tests/e2e/_lib.sh` helpers and are shellcheck-clean (enforced in CI). See [`./testing-e2e.md`](./testing-e2e.md) for auth prerequisites and what CI runs.

### CI Pipeline

GitHub Actions runs automatically on push to `main` and on PRs (`.github/workflows/ci.yml`):
- **platform-build** — Go build, vet, `go test -race` with coverage profiling (25% baseline threshold)
- **platform-build** — Go build, vet, `go test -race` with coverage profiling (25% baseline threshold; setup-go uses module cache)
- **canvas-build** — npm build, `vitest run` (no `--passWithNoTests` -- tests must exist and pass)
- **mcp-server-build** — npm build
- **python-lint** — `pytest --cov=. --cov-report=term-missing` (pytest-cov enabled)
- **e2e-api** (added 2026-04-13) — Postgres + Redis service containers, migrations applied via `docker exec`, `tests/e2e/test_api.sh` must pass 62/62
- **shellcheck** (added 2026-04-13) — lints every `tests/e2e/*.sh`

Postgres and Redis are not exposed to the host -- use `docker compose exec postgres psql` or `docker compose exec redis redis-cli` for direct access.

Expand Down
68 changes: 68 additions & 0 deletions docs/development/testing-e2e.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# E2E Testing

End-to-end test scripts live under `tests/e2e/` and exercise the platform against a real Postgres + Redis. Every script is shellcheck-clean and shares helpers from `tests/e2e/_lib.sh` + `tests/e2e/_extract_token.py`.

## Scripts

| Script | Checks | Prerequisites |
|--------|--------|--------------|
| `test_api.sh` | 62 | platform running on :8080; no live agents required |
| `test_comprehensive_e2e.sh` | 67 | platform running; spins up its own workspaces |
| `test_a2a_e2e.sh` | 22 | platform + 2 provisioned agents (Echo + SEO) with `OPENROUTER_API_KEY` |
| `test_activity_e2e.sh` | 25 | platform + 1 online agent |
| `test_claude_code_e2e.sh` | — | platform + Claude Code runtime; exercises CLI adapter |

## Auth Prerequisites (Phase 30)

After Phase 30.1, the following routes require `Authorization: Bearer <token>` once a workspace has any live token on file (legacy workspaces are grandfathered):

- `POST /registry/heartbeat`
- `POST /registry/update-card`

After Phase 30.6, the following routes additionally require `X-Workspace-ID` on the caller side (bearer token validated, fail-open on DB hiccup):

- `GET /registry/discover/:id`
- `GET /registry/:id/peers`

The scripts handle this by:

1. Creating a workspace → platform returns no token yet.
2. Calling `POST /registry/register` — response body includes `auth_token` once per workspace.
3. Extracting the token via `_extract_token.py` (reads JSON from stdin).
4. Passing it in subsequent heartbeat / discover / peers calls.

`test_comprehensive_e2e.sh` registers each workspace **immediately after creation** so the provisioner's auto-register doesn't race the test's explicit register. `test_activity_e2e.sh` re-registers a detected-already-online agent to capture a fresh bearer token.

## Running Locally

```bash
# Quickest check after any platform change:
cd platform && go build ./cmd/server && ./server &
bash tests/e2e/test_api.sh # expect 62/62 pass

# Comprehensive sweep:
bash tests/e2e/test_comprehensive_e2e.sh # expect 67/67 pass
```

Both scripts include a pre-test cleanup that deletes workspaces from previous runs so a stale DB won't cause spurious failures.

## What CI Runs

`.github/workflows/ci.yml` (added 2026-04-13):

- **e2e-api** — spins up Postgres + Redis via service containers, applies migrations with `docker exec`, builds the platform binary, runs `tests/e2e/test_api.sh`. All 62 checks must pass.
- **shellcheck** — runs the shellcheck marketplace action against every `tests/e2e/*.sh`.

The other E2E scripts are not yet in CI because they require provisioned agents and LLM credentials; run them locally before merging runtime-touching changes.

## Adding a New E2E Check

1. Source `tests/e2e/_lib.sh` for `assert_*` helpers, bearer-token extraction, and the cleanup preamble.
2. When hitting an auth-gated route, always register the workspace first and thread the returned token through subsequent requests.
3. Keep each check idempotent — the comprehensive script is expected to be re-runnable on the same DB.
4. Run `shellcheck tests/e2e/your_script.sh` locally before pushing.

## Related Docs

- [Local Development](./local-development.md)
- [Platform API](../api-protocol/platform-api.md) — route reference incl. auth requirements
Loading
Loading