test(sdk-e2e): self-contained embedded-PG ICU + connector-sync & watcher-reaction gates

.env.example

@@ -96,7 +96,7 @@ WORKER_DISALLOWED_DOMAINS=
 PUBLIC_GATEWAY_URL=https://app.lobu.ai
 
 # Optional Lobu base MCP URL override.
-# File-first projects can enable Lobu with [memory] in lobu.toml;
+# File-first projects can enable Lobu with [memory] in lobu.config.ts;
 # when set, Lobu scopes this base URL to the org declared there.
 MEMORY_URL=https://lobu.com/mcp
 

.github/workflows/ci.yml

@@ -63,6 +63,7 @@
         run: |
           cd packages/core && bun run build && cd ../..
           cd packages/connector-sdk && bun run build && cd ../..
+          cd packages/sdk && bun run build && cd ../..
           cd packages/embeddings && bun run build && cd ../..
           cd packages/connector-worker && bun run build && cd ../..
 
@@ -100,6 +101,61 @@
           files: coverage/lcov.info
           fail_ci_if_error: false
 
+  # SDK lifecycle e2e — the hard gate that the WHOLE TypeScript-SDK path runs,
+  # not just that config maps (unit/integration cover that). Boots `lobu run`
+  # (embedded Postgres — the linux binary ships as an @embedded-postgres
+  # optional dep, the same engine prod uses), auto-applies a prune:true fixture,
+  # and drives a real agent turn through a spawned worker against a deterministic
+  # mock OpenAI-compatible provider (no provider key needed). A failure here
+  # fails CI → the PR goes red. See scripts/sdk-e2e.sh.
+  sdk-e2e:
+    runs-on: ubuntu-latest
+    timeout-minutes: 20
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: ./.github/actions/setup-submodule
+        with:
+          deploy-key: ${{ secrets.OWLETTO_WEB_DEPLOY_KEY }}
+
+      - uses: oven-sh/setup-bun@v2
+        with:
+          bun-version: 1.3.5
+
+      # Node 22 — the worker uses isolated-vm (abi127 prebuild); the runner's
+      # default Node major would segfault the sandbox runtime.
+      - uses: actions/setup-node@v4
+        with:
+          node-version: "22"
+
+      - name: Install dependencies
+        run: bun install
+
+      # The worker's exec-sandbox uses bwrap on Linux; ubuntu-latest blocks
+      # unprivileged user namespaces via AppArmor, so flip the sysctl too.
+      - name: Install bubblewrap (worker exec-sandbox)
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y bubblewrap coreutils
+          sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
+          bwrap --version
+
+      # NOTE: `lobu run`'s embedded Postgres (PG18) is NEEDED-linked against
+      # ICU 60, which ubuntu-latest no longer carries. We do NOT install a
+      # system libicu60 (the old archive .deb was fragile and didn't help local
+      # Linux devs) — instead scripts/sdk-e2e.sh runs
+      # scripts/sdk-e2e/fix-embedded-pg-icu.mjs, which creates the missing
+      # `.so.60` SONAME symlinks in the embedded-postgres native/lib dir (it
+      # already ships libicu*.so.60.2). The binaries' `$ORIGIN/../lib` rpath then
+      # resolves their bundled ICU with zero system deps. Prod is unaffected
+      # (external Postgres; embedded-postgres pruned from the app image).
+
+      - name: Build all packages (CLI + server bundle + sdk + pgvector-embedded)
+        run: make build-packages
+
+      - name: SDK lifecycle e2e (apply + prune + real worker turn)
+        run: bash scripts/sdk-e2e.sh
+
   # Frontend tests run under jsdom via vitest. owletto is a submodule;
   # forks without the deploy key get a stub package and skip these.
   frontend:
@@ -187,29 +243,29 @@
       - uses: actions/setup-node@v4
         with:
           node-version: '22'
 
       - name: Install dependencies
         run: bun install
 
       - name: Build packages server depends on
         # embeddings → connector-worker order matters: connector-worker's
         # src/embeddings.ts imports `@lobu/embeddings` and resolves it via
         # the workspace symlink → dist. connector-worker itself is required
         # because `packages/server/src/utils/connector-catalog.ts` imports
         # `@lobu/connector-worker/compile` (the shared compile pipeline),
         # which resolves via the package's `exports` field to
         # `./dist/compile/index.js`. Vitest's vite-node resolver follows
         # the `import` condition and fails to load if dist is absent —
         # which transitively breaks every integration file whose import
         # graph touches `queue-helpers` / `worker-api` / `connector-catalog`.
         #
         # pgvector-embedded must build too: the test backend
         # (src/__tests__/setup/embedded-postgres-backend.ts) imports
         # `@lobu/pgvector-embedded`, so vite resolves its `import` condition
         # (./dist/index.js) at transform time even though the embedded backend
         # is never started when DATABASE_URL points at the external Postgres.
         run: |
           cd packages/core && bun run build && cd ../..
           cd packages/connector-sdk && bun run build && cd ../..
           cd packages/embeddings && bun run build && cd ../..
           cd packages/connector-worker && bun run build && cd ../..

.gitignore

@@ -95,3 +95,4 @@ packages/*/~/
 
 # Embedded PGlite dev data (make dev without DATABASE_URL)
 .lobu-dev/
+.sdk-e2e-run/

AGENTS.md

@@ -14,7 +14,7 @@
 - **MCP:** providers from `config/providers.json`; MCP servers from per-agent settings or `SKILL.md`. Workers discover MCP tools at startup and call them via the gateway proxy (JWT). Built-ins: `AskUser`, `UploadFile`. Integration auth (OAuth/refresh) lives in Lobu MCP servers — workers never see tokens.
 - **`events` is append-only.** Never `DELETE FROM events`; tombstone via `client.knowledge.delete()` / `save_knowledge({ supersedes_event_id, ... })`. `current_event_records` masks superseded rows.
 - **Connectors:** `*.connector.ts` extending `ConnectorRuntime`. npm deps → project `package.json`, bundled by esbuild at compile time; native deps → `runtime.nix.packages` (nixpkgs refs), provisioned via `nix-shell` at run. Compile happens on the CLI (`lobu apply` → `bun install --ignore-scripts`); `@lobu/connector-sdk` is externalized (runtime-provided).
-- **Guardrails** (`packages/core/src/guardrails/`): stages `input`/`output`/`pre-tool`; built-ins `secret-scan`, `pii-scan`, `forbidden-tools` (see `gateway/guardrails/builtins.ts` + `aggregator.ts`). Configure in `[agents.<id>].guardrails` (lobu.toml); all fail open on infra errors; each trip writes a `guardrail-trip` event.
+- **Guardrails** (`packages/core/src/guardrails/`): stages `input`/`output`/`pre-tool`; built-ins `secret-scan`, `pii-scan`, `forbidden-tools` (see `gateway/guardrails/builtins.ts` + `aggregator.ts`). Configure via `defineAgent({ guardrails: [...] })` in `lobu.config.ts`; all fail open on infra errors; each trip writes a `guardrail-trip` event.
 - **Network/egress:** worker subprocesses get `HTTP_PROXY=http://localhost:8118`. Access via `WORKER_ALLOWED_DOMAINS`: empty=no net (default), `github.com`=allowlist, `*`=all (not prod), `*`+`WORKER_DISALLOWED_DOMAINS`=blocklist; domains exact or `.wildcard`. Linux prod adds kernel `IPAddressDeny=any` + allow 127.0.0.1. Risky domains can route through an LLM egress judge (Haiku, 5-min cache, fail-closed; needs `ANTHROPIC_API_KEY`; `gateway/proxy/http-proxy.ts`).
 
 ## Releasing
@@ -38,4 +38,4 @@ release-please owns it: land conventional commits on `main`, merge the generated
 - **E2E before merge (hard gate)** for bug fixes: red→fix→green reproducer in the PR body; if you can't reproduce, BAIL (post the dead-end, don't open a PR). Native-app UI exempt (compile-check + draft, say so).
 - Bot behavior → `./scripts/test-bot.sh "@me ..."` (dev `@clawdotfreebot`, prod `@lobuaibot`; clear stale history via `chat_state_lists`). Prompt/behavior → promptfoo (`bun run evals`). Auth'd UI → `docs/BROWSER_TESTING.md`.
 - **Owletto e2e (Chrome/Mac):** Chrome → `make e2e-browser` launches/reuses the stable `owletto` agent-browser harness (persistent `~/.config/lobu-dev/chrome` profile + `--extension` from this worktree); drive it with `agent-browser --session owletto <cmd>`, `RESTART=1` to pick up extension/worktree changes. The extension's fixed manifest `key` pins its ID, so chrome.storage.local (gateway URL + auth) persists — pair once, reuse across sessions/worktrees, like the installed Mac app. Mac → use the installed `Owletto.app`; it reads `~/.config/lobu/config.json` each popover so task-setup-registered worktree contexts appear in its picker. Don't reintroduce a per-worktree extension symlink.
-- `.env` is the single source of truth for secrets (restart `make dev` after changes). Worker sessions persist via `./workspaces/{agentId}/`. Skills declare network/nix needs that merge into the allowlist — review skills before installing; destructive MCP calls need in-thread approval unless pre-approved in `lobu.toml`.
+- `.env` is the single source of truth for secrets (restart `make dev` after changes). Worker sessions persist via `./workspaces/{agentId}/`. Skills declare network/nix needs that merge into the allowlist — review skills before installing; destructive MCP calls need in-thread approval unless pre-approved in `lobu.config.ts`.

Makefile

@@ -1,6 +1,6 @@
 # Development Makefile for Lobu
 
-.PHONY: help setup build test clean dev build-packages ensure-submodule clean-workers test-unit test-integration test-e2e typecheck task-setup task-clean e2e-browser bump review
+.PHONY: help setup build test clean dev build-packages ensure-submodule clean-workers test-unit test-integration test-e2e test-e2e-sdk typecheck task-setup task-clean e2e-browser bump review
 
 # Default target
 help:
@@ -34,7 +34,7 @@ typecheck:
 # Build all TypeScript packages in dependency order
 build-packages:
 	@echo "📦 Building all TypeScript packages..."
-	@for pkg in core pgvector-embedded connector-sdk agent-worker openclaw-plugin embeddings connector-worker promptfoo-provider; do \
+	@for pkg in core pgvector-embedded connector-sdk client sdk agent-worker openclaw-plugin embeddings connector-worker promptfoo-provider; do \
 		echo "   📦 Building packages/$$pkg..."; \
 		( cd packages/$$pkg && bun run build ) || exit $$?; \
 	done
@@ -141,6 +141,13 @@ test-e2e:
 	@: $${DATABASE_URL?Set DATABASE_URL=postgres://… (with pgvector) before running}
 	@./scripts/run-e2e.sh
 
+# SDK lifecycle e2e: boots `lobu run` (embedded Postgres), auto-applies a
+# prune:true fixture, and drives a real agent turn through a spawned worker
+# against a deterministic mock provider (no key needed). Self-contained. This is
+# the CI `sdk-e2e` gate; run it locally the same way.
+test-e2e-sdk:
+	@./scripts/sdk-e2e.sh
+
 # Stop any embedded worker subprocesses left over from a crashed gateway.
 # Workers are normally cleaned up when the gateway exits; this target is a
 # safety net for orphaned bun processes spawned by EmbeddedDeploymentManager.

README.md

@@ -30,7 +30,7 @@ npx @lobu/cli@latest run                      # boots the stack and applies your
 npx @lobu/cli@latest chat -c local "hello"    # talk to it
 ```
 
-`lobu run` (embedded) auto-applies your `lobu.toml`, so the scaffolded agent is usable immediately. To use an external Postgres, set `DATABASE_URL` in `.env`; to push later config changes, run `lobu apply`.
+`lobu run` (embedded) auto-applies your `lobu.config.ts`, so the scaffolded agent is usable immediately. To use an external Postgres, set `DATABASE_URL` in `.env`; to push later config changes, run `lobu apply`.
 
 ## Agent configuration
 
@@ -42,7 +42,7 @@ npx @lobu/cli@latest org set my-org
 npx @lobu/cli@latest agent list
 ```
 
-Local `lobu.toml` projects are still useful for `lobu validate` and `lobu apply` workflows.
+Local `lobu.config.ts` projects are still useful for `lobu validate` and `lobu apply` workflows.
 
 ### Deployment
 
@@ -94,7 +94,7 @@ Every Lobu agent ships with tools for autonomous execution and persistence:
 | **Full Linux toolbox** — sandboxed shell, file edit, search | `bash`, `read`, `write`, `edit`, `grep`, `find`, `ls` |
 | **Conversation context** — pull earlier thread messages | `GetChannelHistory` |
 | **File & media delivery** — share reports, charts, audio | `UploadUserFile`, `GenerateAudio` |
-| **Skills** — extend via `lobu.toml` or admin settings | `lobu.toml`, Settings UI |
+| **Skills** — extend via `lobu.config.ts` or admin settings | `lobu.config.ts`, Settings UI |
 | **Connected APIs** — GitHub, Google, etc. with Lobu-managed OAuth | MCP tools via Lobu |
 | **Managed MCP proxy** — any MCP server with secret injection | [MCP Proxy](docs/SECURITY.md#credentials) |
 | **Nix + external MCP** — browsing, headless UI, custom tools | `bash` (Nix), MCP servers |