Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -119,12 +119,14 @@ jobs:
CLOUDFLARE_EMAIL_FROM: ${{ secrets.CLOUDFLARE_EMAIL_FROM }}
SENTRY_DSN: ${{ secrets.SENTRY_DSN }}
CAPABILITY_REINDEX_SECRET: ${{ secrets.CAPABILITY_REINDEX_SECRET }}
PACKAGE_INVOCATION_TOKENS: ${{ secrets.PACKAGE_INVOCATION_TOKENS }}
run: >
node tools/ci/sync-worker-secrets.ts --env production --config
"$WRANGLER_CONFIG" --set-from-env COOKIE_SECRET --set-from-env
AI_GATEWAY_ID --set-from-env-optional CLOUDFLARE_EMAIL_FROM
--set-from-env-optional SENTRY_DSN --set-from-env-optional
CLOUDFLARE_API_TOKEN --set-from-env-optional CAPABILITY_REINDEX_SECRET
--set-from-env-optional PACKAGE_INVOCATION_TOKENS

- name: 🗄️ Apply D1 Migrations
env:
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/preview.yml
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,7 @@ jobs:
PREVIEW_URL: ${{ steps.deploy_preview.outputs.url }}
AI_GATEWAY_ID: ${{ secrets.AI_GATEWAY_ID_PREVIEW }}
SENTRY_DSN: ${{ secrets.SENTRY_DSN }}
PACKAGE_INVOCATION_TOKENS: ${{ secrets.PACKAGE_INVOCATION_TOKENS }}
run: |
set -euo pipefail
ENV_FILE="packages/worker/.env.example"
Expand All @@ -277,7 +278,7 @@ jobs:
extra_args+=(--set "APP_BASE_URL=$PREVIEW_URL")
fi

node tools/ci/sync-worker-secrets.ts --env "" --name "$APP_WORKER_NAME" --config "$WRANGLER_CONFIG" --from-dotenv "$ENV_FILE" --from-dotenv "$OVERRIDES_FILE" --set-from-env AI_GATEWAY_ID --set-from-env-optional SENTRY_DSN --generate-cookie-secret --include-empty "${extra_args[@]}"
node tools/ci/sync-worker-secrets.ts --env "" --name "$APP_WORKER_NAME" --config "$WRANGLER_CONFIG" --from-dotenv "$ENV_FILE" --from-dotenv "$OVERRIDES_FILE" --set-from-env AI_GATEWAY_ID --set-from-env-optional SENTRY_DSN --set-from-env-optional PACKAGE_INVOCATION_TOKENS --generate-cookie-secret --include-empty "${extra_args[@]}"

- name: 📤 Upload Worker source maps to Sentry
env:
Expand Down
45 changes: 45 additions & 0 deletions docs/contributing/environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,51 @@ Optional Worker secret/var (see `packages/worker/src/env-schema.ts` and
to the worker. When unset, the worker rejects home connector registration and
the internal home MCP bridge cannot route `home` capabilities.

## External package invocation API

Optional Worker secret/var (see `packages/worker/src/env-schema.ts` and
`packages/worker/src/package-invocations/http.ts`):

- `PACKAGE_INVOCATION_TOKENS` — optional Worker **secret** (JSON string) whose
value is a JSON object mapping token ids to scoped bearer token configs for
`POST /api/package-invocations/:packageIdOrKodyId/:exportName`.

Each token config must include:

- `token` — the bearer token string presented by the trusted external service
- `userId` — the Kody user who owns the target saved package
- `email`
- `displayName`
- at least one package scope via `packageIds` and/or `packageKodyIds`

Optional narrowing fields:

- `exportNames` — allowed package exports (normalized like
`./dispatch-message-created`)
- `sources` — allowed `source` metadata values from the request body

Example:

```json
{
"discord-gateway-prod": {
"token": "replace-with-a-long-random-secret",
"userId": "user_123",
"email": "me@example.com",
"displayName": "Kent",
"packageKodyIds": ["discord-gateway"],
"exportNames": ["./dispatch-message-created"],
"sources": ["discord-gateway"]
}
}
```

This API is intended for trusted first-party service-to-service callers such as
an external Discord Gateway proxy. The token scope is enforced against the
resolved saved package, export name, and optional request source metadata, and
the worker records idempotency state in D1 so duplicate deliveries can replay
the original response without re-running package code.

### Remote connector secrets (Worker)
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved

See `packages/worker/src/env-schema.ts` and
Expand Down
1 change: 1 addition & 0 deletions docs/contributing/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ style, tests, MCP capabilities, and runtime architecture.
- [Setup](./setup.md), [code style](./code-style.md),
[testing](./testing-principles.md)
- [Packages and manifests](./packages-and-manifests.md)
- [External package invocation API](./package-invocation-api.md)
- [Adding capabilities](./adding-capabilities.md)
- [Architecture](./architecture/index.md)
- [MCP server patterns](./mcp-server-patterns.md) (reference for server design)
Expand Down
219 changes: 219 additions & 0 deletions docs/contributing/package-invocation-api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,219 @@
# External package invocation API

Use this API when a trusted external service should invoke a saved package
export inside Kody without going through the interactive MCP transport.

The first intended caller is a stable Discord Gateway proxy:

1. the external proxy owns the long-lived Discord websocket
2. the proxy normalizes gateway events
3. the proxy calls Kody's package invocation API
4. Kody executes the saved package export with package context, user context,
package storage, and normal secret/capability rules

Kody remains the package runtime and storage brain. The external service owns
the socket lifecycle.

## Endpoint

`POST /api/package-invocations/:packageIdOrKodyId/:exportName`

Examples:

- `/api/package-invocations/pkg_123/dispatch-message-created`
- `/api/package-invocations/discord-gateway/dispatch-message-created`

The path accepts either:

- the saved package id
- the package `kody.id`

The export name is normalized to package export form, so
`dispatch-message-created` resolves as `./dispatch-message-created`.

## Authentication

Authentication uses a private bearer token configured with the Worker secret
`PACKAGE_INVOCATION_TOKENS`.

The secret value is a JSON object that maps a token id to:

- `token` — the actual bearer secret
- `userId`
- `email`
- `displayName`
- package scope (`packageIds` and/or `packageKodyIds`)
- optional `exportNames`
- optional `sources`

Example:

```json
{
"discord-gateway-prod": {
"token": "replace-with-a-long-random-secret",
"userId": "user_123",
"email": "me@example.com",
"displayName": "Kent",
"packageKodyIds": ["discord-gateway"],
"exportNames": ["./dispatch-message-created"],
"sources": ["discord-gateway"]
}
}
```

The token is not a global backdoor:

- package lookup is still user-owned
- package access is scoped by the token config
- export access can be restricted
- `source` metadata can be restricted
- execution still uses normal package runtime machinery

## Request body

```json
{
"params": {
"messageId": "123",
"content": "hello"
},
"idempotencyKey": "discord:message-create:123",
"source": "discord-gateway",
"topic": "discord.message.created"
}
```

Fields:

- `params` — JSON object passed to the package export as runtime params
- `idempotencyKey` — required stable key for replay protection
- `source` — optional source label for auditing and token scoping
- `topic` — optional event topic label for downstream logic and logs

Comment on lines +80 to +87

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

source is documented as optional but is effectively required by the runtime.

service.ts tokenAllowsSource() returns false when source is null/empty, which causes invokePackageExport to return 403 source_not_allowed before any package or export is even resolved. So a request omitting source always fails — it isn't "optional" in practice. Either tighten this doc to say source is required (and that it must match the token's sources_json allowlist), or update tokenAllowsSource to treat a missing source as "no source claim, skip allowlist check" if you actually want it optional.

The same friction shows up in the curl example at lines 196–205, where source is included — readers could reasonably remove it based on the "optional" description and get a confusing 403.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/contributing/package-invocation-api.md` around lines 80 - 86, The docs
claim `source` is optional but the runtime enforces it: `service.ts`'s
tokenAllowsSource() returns false for null/empty `source`, causing
invokePackageExport to fail with 403; fix by either (A) updating this docs page
and the curl example to state `source` is required and must match the token's
sources_json allowlist (clarify format and behavior), or (B) change
`tokenAllowsSource()` to treat a missing/empty `source` as "no source claim —
skip allowlist check" so omission is allowed; pick one approach and update the
docs and the implementation consistently (reference tokenAllowsSource and
invokePackageExport when making the change).

## Idempotency

Kody persists package invocation idempotency in D1.

The identity key is:

- user
- token id
- package id
- export name
- idempotency key

Behavior:

- same request + same idempotency key => stored response replayed
- same idempotency key + different payload => `409 idempotency_mismatch`
- duplicate while first invocation is still active =>
`409 invocation_in_progress`

This makes duplicate Discord gateway deliveries safe when the proxy retries.

## Response shape

Success:

```json
{
"ok": true,
"package": {
"id": "pkg_123",
"kodyId": "discord-gateway"
},
"exportName": "./dispatch-message-created",
"source": "discord-gateway",
"topic": "discord.message.created",
"idempotency": {
"key": "discord:message-create:123",
"replayed": false
},
"result": {
"reply": "handled"
},
"logs": []
}
```

Replay responses return the same stored body with `idempotency.replayed: true`.

Failures return:

```json
{
"ok": false,
"error": {
"code": "package_not_found",
"message": "Saved package \"discord-gateway\" was not found for this user."
}
}
```

Execution failures return sanitized structured errors and logs. The route does
not expose Worker secrets directly.

## Runtime behavior

The API reuses the existing published package bundle path:

- resolve the saved package for the configured user
- resolve the requested package export
- load the published `module` artifact if present
- rebuild and persist the module artifact on cache miss
- execute through `runBundledModuleWithRegistry`

Execution includes:

- package context
- repo context when a published source exists
- writable package storage bound to `package:<packageId>`
- user context from the scoped token config

## Rate limiting and auditing

The endpoint is shaped for standard Cloudflare edge rate limiting:

- path is stable and narrow
- caller metadata includes `source` and `topic`
- each request is audit-logged with hashed email/IP metadata

Prefer Cloudflare WAF/rate limiting rules in front of this path rather than
adding bespoke in-Worker rate limiting first.

## Discord gateway proxy pattern

Recommended flow:

1. keep the Discord Gateway websocket in a stable external process
2. receive `MESSAGE_CREATE`
3. derive a stable idempotency key from the gateway event
4. call the Kody endpoint
5. use Kody's structured result to decide what to post back to Discord

Example request:

```bash
curl --fail --silent \
-X POST \
-H "Authorization: Bearer $PACKAGE_INVOCATION_TOKEN" \
-H "Content-Type: application/json" \
"https://kody.example.com/api/package-invocations/discord-gateway/dispatch-message-created" \
-d '{
"params": {
"messageId": "123",
"channelId": "456",
"content": "hello"
},
"idempotencyKey": "discord:message-create:123",
"source": "discord-gateway",
"topic": "discord.message.created"
}'
```

## Related

- [Packages and manifests](./packages-and-manifests.md)
- [Environment variables](./environment-variables.md)
- [Setup manifest](./setup-manifest.md)
34 changes: 34 additions & 0 deletions docs/contributing/setup-manifest.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,14 @@ automatically:
`POST /__maintenance/reindex-capabilities` to refresh builtin capability
embeddings in Vectorize. Saved package projections refresh when packages are
saved or published.)
- `PACKAGE_INVOCATION_TOKENS` (optional Worker secret; JSON object mapping token
ids to scoped private bearer token configs for
`POST /api/package-invocations/:packageIdOrKodyId/:exportName`. Each entry
declares the raw bearer token, the owning `userId`, `email`, `displayName`,
and allowed package scopes via `packageIds` and/or `packageKodyIds`. Optional
`exportNames` and `sources` narrow the token further. Use this for trusted
first-party service-to-service callers such as an external Discord gateway
proxy.)

Tests run with `CLOUDFLARE_ENV=test` (set by Playwright) and read local secrets
from `packages/worker/.env`.
Expand All @@ -118,6 +126,9 @@ Configure these GitHub Actions secrets and variables for workflows:
paste the DSN; syncs to the Worker as a secret when set in GitHub Actions)
- `CAPABILITY_REINDEX_SECRET` (optional; triggers post-deploy Vectorize reindex
when set; synced like other optional secrets)
- `PACKAGE_INVOCATION_TOKENS` (optional; JSON worker secret for scoped private
bearer tokens used by external package invocation clients such as a Discord
gateway proxy)
- `SENTRY_AUTH_TOKEN` (optional GitHub **secret**; Sentry auth token with
`project:releases` / source map upload permissions — used only by CI to run
`npm run sentry:upload-sourcemaps` after deploy)
Expand Down Expand Up @@ -183,6 +194,29 @@ How to get/set each value:
`/__maintenance/reindex-capabilities` and `/__maintenance/reindex-skills`
with `Authorization: Bearer …` to refresh capability and user-skill
embeddings.
- `PACKAGE_INVOCATION_TOKENS` (optional)
- Store a JSON object as the repository secret `PACKAGE_INVOCATION_TOKENS`.
- Each top-level key is a token id for auditing; each value must include:
- `token` — the raw bearer token string to send in `Authorization`
- `userId` — owning Kody user id
- `email` and `displayName` — user context passed into package execution
- `packageIds` and/or `packageKodyIds` — allowed saved packages
- Optional `exportNames` limits callable exports, and optional `sources`
limits request-body `source` values.
- Example:
```json
{
"discord-gateway-prod": {
"token": "replace-with-long-random-token",
"userId": "stable-user-id",
"email": "me@example.com",
"displayName": "me",
"packageKodyIds": ["discord-gateway"],
"exportNames": ["./dispatch-message-created"],
"sources": ["discord-gateway"]
}
}
```
- `DOCKERHUB_USERNAME`
- Use your Docker Hub username or organization service account name.
- Store it as the repository secret `DOCKERHUB_USERNAME`.
Expand Down
7 changes: 7 additions & 0 deletions packages/worker/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,10 @@ CLOUDFLARE_EMAIL_FROM=reset@kody.dev
# Authorization: Bearer <this value> repopulates Vectorize after deploys. CI can set
# GitHub secret CAPABILITY_REINDEX_SECRET and sync it via deploy workflow.
# CAPABILITY_REINDEX_SECRET=

# External package invocation tokens (optional Worker secret).
# POST /api/package-invocations/:packageIdOrKodyId/:exportName uses a private
# bearer token from this JSON map for first-party service-to-service calls such as
# an external Discord Gateway proxy. Each token is scoped to one signed-in user and
# an allowlist of packages/exports/sources.
# PACKAGE_INVOCATION_TOKENS={"discord-gateway":{"token":"replace-me","userId":"user-123","email":"me@example.com","displayName":"me","packageKodyIds":["discord-gateway"],"exportNames":["./dispatch-message-created"],"sources":["discord-gateway"]}}
Loading
Loading