Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 18 additions & 8 deletions docs/contributing/architecture/data-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -326,8 +326,12 @@ The schema is defined by migrations in `packages/worker/migrations/`:
on DO `source_updated_at`). D1 remains the enumeration index and parity mirror
— see
[Entitlements](./entitlements.md#package-service-liveness--usermeter-authority-cutover-2026-08-01).
- `entity_sources`: durable mapping from user-facing entities to Artifacts repos
and their latest published commit
- `entity_sources`: durable mapping from user-facing entities (`job`, `package`,
or `repo`) to Artifacts repos and their latest published commit (packages
only; plain repos are live-at-HEAD without a publish pointer)
- `user_repos`: plain-repo discovery metadata (`name`, optional `description`);
one row per user-owned plain repo, keyed by `entity_sources.entity_id` when
`entity_kind = 'repo'`
Comment on lines +329 to +334

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

Propagate the plain-repo model through the architecture index.

The new entries establish repos as a first-class source, but related architecture sections still describe only package/job sources. Align the contracts before merge.

  • docs/contributing/architecture/data-storage.md#L329-L334: Update the later entity_sources contract to include repo, and distinguish package/job published commits from live-at-HEAD plain repos.
  • docs/contributing/architecture/data-storage.md#L917-L924: Add plain-repo identity, live-at-HEAD behavior, and session/Git storage to the state mapping, or scope the heading to packages.
  • docs/contributing/architecture/primitives.yaml#L176-L191: Update the artifacts-repos primitive summary to include plain repos as a shared backing store.
📍 Affects 2 files
  • docs/contributing/architecture/data-storage.md#L329-L334 (this comment)
  • docs/contributing/architecture/data-storage.md#L917-L924
  • docs/contributing/architecture/primitives.yaml#L176-L191
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/contributing/architecture/data-storage.md` around lines 329 - 334,
Update the architecture index to consistently model plain repos as first-class
sources: in docs/contributing/architecture/data-storage.md lines 329-334,
include repo in the entity_sources contract and distinguish published
package/job commits from live-at-HEAD repos; in lines 917-924, add plain-repo
identity, live-at-HEAD behavior, and session/Git storage or scope the heading to
packages; and in docs/contributing/architecture/primitives.yaml lines 176-191,
update the artifacts-repos primitive summary to describe plain repos as a shared
backing store.

- `saved_packages`: package metadata/search projection derived from published
`package.json` source, plus a user-scoped `hidden` flag (0/1) that excludes
the package from default ranked search while leaving list/get/execute paths
Expand Down Expand Up @@ -910,10 +914,14 @@ If R2 succeeds but the fenced graph transaction or finalization fails, Email
Routing retries the stable delivery id. The graph transaction is idempotent and
the active lease prevents a stale worker from overwriting the winner.

### Package state model
### Package and repo state model

Saved packages are the only top-level persisted primitive. Their state maps onto
storage homes as follows:
Repos are the durable home for versioned Artifacts source; saved packages are an
explicit extension that adds publish semantics and runtime surfaces. Plain repos
live in `user_repos` with `entity_sources.entity_kind = 'repo'`. Packages add
`saved_packages` plus `entity_sources.entity_kind = 'package'`.

Package and plain-repo state maps onto storage homes as follows:

- **Package source** — Cloudflare Artifacts repos + D1 `entity_sources`
projections; `package.json` is authoritative.
Expand Down Expand Up @@ -1423,9 +1431,11 @@ moved or deleted during this migration.
`entity_sources` is the durable repo pointer table:
`(user_id, entity_kind, entity_id) -> source_id`. Child tables store
`source_id = entity_sources.id`; KV snapshots use that same source id plus the
published commit. `entity_kind` accepts `job` and `package`. `manifest_path`,
`source_root`, `published_commit`, `indexed_commit`, and
`last_external_check_at` are part of the repo-source synchronization contract.
published commit. `entity_kind` accepts `job`, `package`, and `repo`.
`manifest_path`, `source_root`, `published_commit`, `indexed_commit`, and
`last_external_check_at` are part of the repo-source synchronization contract
for jobs and packages; plain `repo` sources are live-at-HEAD, have no manifest
requirement, and are skipped by the external-push reconcile lane.

Saved package imports in user code use `kody:@scope/name/export` specifiers:

Expand Down
18 changes: 17 additions & 1 deletion docs/contributing/architecture/primitives.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -173,10 +173,26 @@ primitives:
docs:
- docs/contributing/adding-capabilities.md

- id: repos
group: assistant
name: Repos
summary:
User-owned plain Artifacts repos; the base primitive packages extend.
code:
- packages/worker/src/repo/user-repos.ts
- packages/worker/src/mcp/capabilities/repo/repo-create.ts
- packages/worker/src/mcp/capabilities/repo/repo-list.ts
- packages/worker/src/mcp/capabilities/repo/repo-get.ts
- packages/worker/src/mcp/capabilities/repo/repo-delete.ts
- packages/worker/src/mcp/capabilities/repo/repo-get-git-remote.ts
- packages/worker/src/mcp/capabilities/repo/repo-promote-to-package.ts
docs:
- docs/use/repos.md

- id: saved-packages
group: assistant
name: Saved packages
summary: Repo-backed packages rooted at package.json.
summary: Explicitly activated package extension over repo-backed source.
code:
- packages/worker/src/mcp/capabilities/packages/
- packages/worker/src/package-registry/
Expand Down
9 changes: 4 additions & 5 deletions docs/contributing/decisions/0003-repos-as-base-primitive.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,10 @@ Two supporting policies:
the file-level session API (`repo_edit_files`, `repo_apply_patch`,
`repo_commit`, `repo_status`, `repo_diff`, `repo_log`, `repo_restore`).
Breaking pre-launch, no compatibility shim.
- "Saved packages are the only top-level persisted primitive" stops being true
once plain repos ship; docs and MCP guidance then change to "repos are the
durable home, packages add runtime."
- Entitlements will gain a base `repos` count when plain repos ship;
`saved_packages` remains the cap on the extension.
- "Saved packages are the only top-level persisted primitive" is no longer true;
docs and MCP guidance use "repos are the durable home; packages add runtime."
- Entitlements include a base `repos` count; `saved_packages` remains the cap on
the package extension.
- Reconcile and retention lanes must stay kind-aware so publish-pointer logic
never runs for plain repos.
- Revisit if Artifacts ships LFS or per-file limits change (the 10 MiB gate is a
Expand Down
9 changes: 6 additions & 3 deletions docs/contributing/packages-and-manifests.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Packages and manifests

Kody's only top-level saved primitive is the **package**.
Repos are Kody's base persisted primitive; a **package** is a repo with the
package extension activated (runtime surfaces, publish checks — see
[ADR 0003](./decisions/0003-repos-as-base-primitive.md)).

A saved package is a repo-backed module rooted at `package.json`. The standard
package fields describe the package shape, and `package.json#kody` holds the
Expand Down Expand Up @@ -79,11 +81,12 @@ Think in terms of:
- package-owned webhooks
- package-owned workflows (declared in runtime code, not the manifest)

The top-level saved identity is the package.
The repo is the top-level persisted source; a saved package is the identity of
the activated package extension on that repo.

## Package state model

A saved package is the only top-level persisted primitive. Five concepts:
A saved package is a repo with the package extension activated. Five concepts:
Comment thread
coderabbitai[bot] marked this conversation as resolved.

1. **Package source** — Artifacts repo + D1 `entity_sources` projection;
manifest rooted at `package.json`.
Expand Down
9 changes: 6 additions & 3 deletions docs/use/packages.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# Packages

Kody's only top-level saved primitive is the **package**.
Repos are Kody's durable home for versioned source; a **package** is a repo with
runtime surfaces activated (see [Repos](./repos.md)). Activation is explicit: a
root `package.json` alone does not make a plain repo a package —
`repo_promote_to_package` (or creating through the package lanes) does.

A saved package is a repo-backed module rooted at `package.json`. Standard
package fields describe the package surface, and `package.json#kody` holds the
Expand All @@ -24,8 +27,8 @@ hosting.

## Package state model

A saved package is the only top-level persisted primitive. Five concepts make up
its state:
A saved package is a repo with the package extension activated: repos are the
durable home, packages add runtime. Five concepts make up its state:

1. **Package source** — repo-backed code and manifest rooted at `package.json`
(Artifacts repos plus D1 `entity_sources` projections). `package.json` is the
Expand Down
60 changes: 60 additions & 0 deletions docs/use/repos.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Plain repos

Plain repos are Kody's base Artifacts-backed storage primitive. Every plain repo
maps to a Cloudflare Artifacts git repository via `entity_sources` with
`entity_kind = 'repo'`. They are **live-at-HEAD**: pushes and session publishes
materialize directly on the default branch with no `published_commit` publish
step and no external-push reconcile lane.

Saved packages are an **explicit extension**: a package is a repo whose
`entity_sources` row has `entity_kind = 'package'` and a matching
`saved_packages` projection. Activation is never inferred from contents—a root
`package.json` alone does not make a package. Use `repo_promote_to_package` when
you want runtime surfaces (exports, apps, services, jobs, webhooks).

## Capabilities

| Capability | Purpose |
| ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `repo_create` | Create a `user_repos` row and backing Artifacts repo (enforces `repos` entitlement). |
| `repo_list` | List plain repos from D1 (`user_repos`); no live Artifacts reads. |
| `repo_get` | One repo by id or name, live default-branch HEAD, `package_shaped` hint. |
| `repo_delete` | Delete `user_repos`, `entity_sources`, and Artifacts repo (best-effort). |
| `repo_get_git_remote` | Short-lived git remote; write pushes are live at HEAD (no publish reconcile). |
| `repo_promote_to_package` | Full publish checks + saved-package projection when root `package.json` exists at HEAD. |
Comment thread
coderabbitai[bot] marked this conversation as resolved.
| `repo_open_session` + session lane | File-level editing (`repo_edit_files`, `repo_apply_patch`, `repo_commit`, `repo_status`, `repo_diff`, `repo_log`, `repo_restore`, `repo_run_checks`, `repo_publish_session`, …) with `target: { kind: "repo", name }`. |

Vectorize/search integration for plain repos is not in v1—use `repo_list` for
discovery.

## Git lane

`repo_get_git_remote` mints a short-lived Artifacts remote (read or write).
Kody's **10 MiB** per-file gate (10,485,760 stored bytes) binds on session and
file-level writes and again at `repo_promote_to_package`; a direct `git push` to
the minted remote has no Kody gate, so the only ceiling on that lane is
Artifacts itself, which rejects pushes above ~32 MiB of decompressed pack
content with a raw HTTP 413. Plain repos do not run the package publish
reconcile cron after push—HEAD is live.

## Sessions

Open with `repo_open_session` using `target: { kind: "repo", name: "<name>" }`
or `source_id`. Session base is the current default-branch HEAD (not a publish
pointer). `repo_publish_session` on a plain repo runs only the source size walk
(manifest, bundle, typecheck, and lint checks are skipped). When the published
tree contains root `package.json`, the result includes `package_shaped: true`
and a promote notice.

## Promote flow

When a plain repo has `package.json` at HEAD, `repo_get` and publish surfaces
surface progressive disclosure (`package_shaped`, `activated: false`, promote
notice). `repo_promote_to_package` enforces the `saved_packages` entitlement,
runs full publish checks, creates the `saved_packages` row, flips
`entity_sources` to `package`, and deletes the `user_repos` row.
Comment on lines +51 to +55

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

Make promotion failure-atomic.

The promotion handler inserts the saved_packages row and changes entity_sources to package before calling session.publishSession. Rollback runs only when the call returns a non-ok result. If the RPC or final deleteUserRepo call throws, the package projection, source kind, and user_repos row can diverge. Add compensating cleanup or an idempotent promotion state machine, with fault-injection tests for thrown publish and delete failures.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/use/repos.md` around lines 49 - 53, Make repo_promote_to_package
failure-atomic across saved_packages creation, entity_sources promotion,
session.publishSession, and deleteUserRepo: handle thrown publish or deletion
errors and perform compensating cleanup so these projections cannot diverge.
Prefer an idempotent promotion state machine if consistent with the existing
design, and add fault-injection tests covering thrown publishSession and
deleteUserRepo failures.


## Mental model

Repos are the durable home for versioned source; packages add runtime surfaces
on top of an explicitly activated extension.
10 changes: 10 additions & 0 deletions packages/worker/migrations/0136-user-repos.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
CREATE TABLE user_repos (
id TEXT PRIMARY KEY NOT NULL,
user_id TEXT NOT NULL,
name TEXT NOT NULL,
description TEXT NULL,
created_at TEXT NOT NULL DEFAULT (CURRENT_TIMESTAMP),
updated_at TEXT NOT NULL DEFAULT (CURRENT_TIMESTAMP)
);

CREATE UNIQUE INDEX idx_user_repos_user_name ON user_repos(user_id, name);
1 change: 1 addition & 0 deletions packages/worker/src/account/data-targets.ts
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,7 @@ export const accountUserDataTargets: ReadonlyArray<UserScopedDataTarget> = [
{ kind: 'user_id', table: 'published_bundle_artifacts' },
{ kind: 'user_id', table: 'jobs' },
{ kind: 'user_id', table: 'repo_sessions' },
{ kind: 'user_id', table: 'user_repos' },
{ kind: 'user_id', table: 'saved_packages' },
{ kind: 'user_id', table: 'entity_sources' },
{
Expand Down
11 changes: 11 additions & 0 deletions packages/worker/src/entitlements/plans.ts
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,8 @@ export function resolveEffectivePlan(
}

export type PlanLimits = {
/** Maximum plain repos (rows in user_repos). */
maxRepos: number
/** Maximum saved packages (rows in saved_packages). */
maxSavedPackages: number
/** Maximum scheduled jobs (rows in jobs). */
Expand Down Expand Up @@ -150,6 +152,7 @@ export type PlanLimits = {
}

export const entitlementResources = [
'repos',
'saved_packages',
'scheduled_jobs',
'package_services',
Expand All @@ -170,6 +173,7 @@ export type EntitlementResource = (typeof entitlementResources)[number]

/** Human-readable resource labels used in the shared error message. */
export const entitlementResourceLabels: Record<EntitlementResource, string> = {
repos: 'repos',
saved_packages: 'saved packages',
scheduled_jobs: 'scheduled jobs',
package_services: 'running package services',
Expand Down Expand Up @@ -232,6 +236,7 @@ export const planLimits: Record<PlanName, PlanLimits> = {
// and a half integrations on a product whose whole point is holding
// credentials safely.
free: {
maxRepos: 20,
maxSavedPackages: 25,
maxScheduledJobs: 10,
// Long-lived compute. Unchanged, and persistent services stay off.
Expand All @@ -254,6 +259,7 @@ export const planLimits: Record<PlanName, PlanLimits> = {
maxOutboundFetchesPerDay: 2_000,
},
pro: {
maxRepos: 200,
maxSavedPackages: 100,
maxScheduledJobs: 50,
maxPackageServices: 10,
Expand All @@ -270,6 +276,7 @@ export const planLimits: Record<PlanName, PlanLimits> = {
maxOutboundFetchesPerDay: 20_000,
},
partner: {
maxRepos: 400,
maxSavedPackages: 200,
maxScheduledJobs: 100,
maxPackageServices: 20,
Expand All @@ -286,6 +293,8 @@ export const planLimits: Record<PlanName, PlanLimits> = {
maxOutboundFetchesPerDay: 40_000,
},
max: {
// 100× pro (200) → 20_000; product placeholder uses 10_000 per spec.
maxRepos: 10_000,
// 100× pro (100) → 10_000.
maxSavedPackages: 10_000,
// 100× pro (50) → 5_000.
Expand Down Expand Up @@ -343,6 +352,8 @@ export function resolvePlanLimit(
): number {
const limits = planLimits[plan]
switch (resource) {
case 'repos':
return limits.maxRepos
case 'saved_packages':
return limits.maxSavedPackages
case 'scheduled_jobs':
Expand Down
6 changes: 6 additions & 0 deletions packages/worker/src/entitlements/service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -831,6 +831,12 @@ export async function readEntitlementResourceUsage(input: {
}): Promise<number> {
const { db, userId, resource, now } = input
switch (resource) {
case 'repos':
return await countRows(
db,
`SELECT COUNT(*) AS count FROM user_repos WHERE user_id = ?`,
[userId],
)
case 'saved_packages':
return await countRows(
db,
Expand Down
Loading
Loading