Repository navigation
Add Durable Object facets support for scheduled jobs #163
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from 2 commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
19aef9a
add scheduled job facets runtime
cursoragent 92cb853
Fix job feed checker example and remove dead exports
cursoragent e215b22
Avoid redundant job lookups
cursoragent cd4f878
Remove experimental compatibility flag
cursoragent d8dc633
fix jobs server exec runtime model
cursoragent 7f8a7bd
finish jobs server exec bridge
cursoragent 64206c4
Fix saved app facet exec and partial update semantics (#165)
kentcdodds 5000f82
align jobs exec contract with saved apps
cursoragent 44e36a5
stabilize gate and jobs followups
cursoragent e82f111
keep job delete resilient in tests
cursoragent 396ca20
Kody app backend patterns (#166)
kentcdodds 857c446
address jobs review followups
cursoragent f47f212
tighten jobs runner consistency
cursoragent 2e908ea
finish jobs consistency fixes
cursoragent f8b012f
fix jobs runner review issues
cursoragent File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| # Example job: feed checker with incremental state | ||
|
|
||
| This example shows a scheduled job that remembers the latest seen item id in its | ||
| facet SQLite storage and only emits new items on later runs. | ||
|
|
||
| ## Save it | ||
|
|
||
| Create the job with `job_create`: | ||
|
|
||
| ```json | ||
| { | ||
| "name": "Feed checker", | ||
| "schedule": { | ||
| "intervalMs": 3600000 | ||
| }, | ||
| "timezone": "America/Denver", | ||
| "enabled": true, | ||
| "serverCode": "import { DurableObject } from 'cloudflare:workers'\n\nexport class Job extends DurableObject {\n\tasync run() {\n\t\tconst feedUrlValue = await KODY.valueGet('feed_url', 'user')\n\t\tif (!feedUrlValue?.value) {\n\t\t\tthrow new Error('Missing persisted value \"feed_url\".')\n\t\t}\n\n\t\tconst response = await KODY.fetchViaHostGateway({\n\t\t\turl: feedUrlValue.value,\n\t\t\tmethod: 'GET',\n\t\t\theaders: {\n\t\t\t\tAccept: 'application/json',\n\t\t\t},\n\t\t})\n\t\tif (response.status >= 400) {\n\t\t\tthrow new Error(`Feed request failed: ${response.status} ${response.statusText}`)\n\t\t}\n\n\t\tconst payload = JSON.parse(response.body)\n\t\tconst items = Array.isArray(payload.items) ? payload.items : []\n\n\t\tthis.ctx.storage.sql.exec(\n\t\t\t'CREATE TABLE IF NOT EXISTS feed_state (singleton INTEGER PRIMARY KEY CHECK (singleton = 1), item_id TEXT)',\n\t\t)\n\n\t\tconst latestSeen = this.ctx.storage.sql\n\t\t\t.exec('SELECT item_id FROM feed_state WHERE singleton = 1')\n\t\t\t.one<{ item_id?: string }>()?.item_id ?? null\n\n\t\tconst freshItems = latestSeen\n\t\t\t? items.filter((item) => item?.id && item.id !== latestSeen)\n\t\t\t: items.slice(0, 1)\n\n\t\tif (items[0]?.id) {\n\t\t\tthis.ctx.storage.sql.exec(\n\t\t\t\t'INSERT INTO feed_state (singleton, item_id) VALUES (1, ?) ON CONFLICT(singleton) DO UPDATE SET item_id = excluded.item_id',\n\t\t\t\tString(items[0].id),\n\t\t\t)\n\t\t}\n\n\t\tif (freshItems.length === 0) {\n\t\t\treturn { newItems: 0 }\n\t\t}\n\n\t\treturn {\n\t\t\tnewItems: freshItems.length,\n\t\t\tlatestItemId: String(items[0]?.id ?? ''),\n\t\t\tnotifications: freshItems.map((item) => ({\n\t\t\t\tid: String(item.id),\n\t\t\t\ttitle: String(item.title ?? item.id),\n\t\t\t})),\n\t\t}\n\t}\n}\n" | ||
| } | ||
| ``` | ||
|
|
||
| ## Notes | ||
|
|
||
| - The job uses `KODY.valueGet('feed_url', 'user')` so the feed URL can be | ||
| changed without editing code. | ||
| - `KODY.fetchViaHostGateway()` uses the same secret-aware host approval path as | ||
| execute-time fetch. | ||
| - The `feed_state` table is local to the job facet, so every job keeps its own | ||
| checkpoint independently. | ||
| - Inspect the latest outcomes with `job_history({ job_id })`. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,142 @@ | ||
| # Scheduled jobs | ||
|
|
||
| Kody scheduled jobs persist agent-authored Durable Object code and run it on an | ||
| alarm-based schedule. | ||
|
|
||
| Each saved job has: | ||
|
|
||
| - **`serverCode`** — Durable Object code that exports | ||
| `class Job extends DurableObject` | ||
| - **`serverCodeId`** — a UUID that rotates whenever the server code changes so | ||
| the Dynamic Worker loader does not reuse stale code | ||
| - **`schedule`** — either a cron expression or an interval in milliseconds | ||
| - **`timezone`** — IANA timezone for cron evaluation, defaulting to | ||
| **`America/Denver`** | ||
| - **`enabled`** — whether the alarm should keep firing | ||
|
|
||
| Jobs are managed with these capabilities: | ||
|
|
||
| - `job_create` | ||
| - `job_update` | ||
| - `job_delete` | ||
| - `job_list` | ||
| - `job_get` | ||
| - `job_run_now` | ||
| - `job_enable` | ||
| - `job_disable` | ||
| - `job_history` | ||
| - `job_storage_reset` | ||
| - `job_server_exec` | ||
|
|
||
| ## Execution model | ||
|
|
||
| Each job gets its own **`JobRunner`** supervisor Durable Object keyed by | ||
| `job_id`. The supervisor: | ||
|
|
||
| - stores observability metadata in its own SQLite database | ||
| - loads the job server code as a **Durable Object Facet** | ||
| - invokes the job facet by calling **`await job.run()`** | ||
| - computes the next alarm time and re-arms itself | ||
|
|
||
| The facet gets its own isolated Durable Object SQLite storage, so a job can | ||
| store incremental state with `this.ctx.storage.sql` or `this.ctx.storage.kv` | ||
| without mixing that state into supervisor metadata. | ||
|
|
||
| ## Schedule semantics | ||
|
|
||
| Jobs support two schedule shapes: | ||
|
|
||
| ```json | ||
| { "cron": "0 8 * * *" } | ||
| ``` | ||
|
|
||
| or | ||
|
|
||
| ```json | ||
| { "intervalMs": 3600000 } | ||
| ``` | ||
|
|
||
| Cron uses standard **5-field** syntax and is interpreted in the configured IANA | ||
| timezone. | ||
|
|
||
| Interval schedules are measured from the **completion time of the last run**. | ||
| This avoids overlapping back-to-back alarm executions when a run takes a long | ||
| time. | ||
|
|
||
| ### Alarm drift and overruns | ||
|
|
||
| - Kody re-arms the next alarm **after** each run finishes. | ||
| - If a run takes longer than its nominal interval, the next run is scheduled | ||
| from the completion timestamp rather than trying to "catch up" missed | ||
| intervals. | ||
| - `job_run_now` does **not** replace the normal schedule. It records a manual | ||
| run and preserves the existing next scheduled alarm. | ||
|
|
||
| ## Authoring `serverCode` | ||
|
|
||
| Job code must export: | ||
|
|
||
| ```ts | ||
| import { DurableObject } from 'cloudflare:workers' | ||
|
|
||
| export class Job extends DurableObject { | ||
| async run() { | ||
| return { ok: true } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| There is no HTTP `fetch()` surface for jobs. The entrypoint is `run()`. | ||
|
|
||
| ## `KODY` bridge inside jobs | ||
|
|
||
| Job facets do not get unrestricted outbound networking. Instead they receive a | ||
| `KODY` binding with a controlled bridge back into Kody: | ||
|
|
||
| - `fetchWithResolvedSecrets({ url, method, headers, body })` | ||
| - `fetchViaHostGateway({ url, method, headers, body })` | ||
| - `valueGet(name, scope?)` | ||
| - `valueSet({ name, value, description?, scope? })` | ||
| - `connectorGet(args)` | ||
| - `connectorList()` | ||
| - `metaRunSkill(name, params?)` | ||
| - `secretPlaceholder(name, scope?)` | ||
|
|
||
| `fetchViaHostGateway()` performs the outbound request through Kody's host | ||
| gateway and honors the same secret placeholder and host approval rules as | ||
| execute-time fetch. | ||
|
|
||
| The default bridge scope is job-local, but it maps onto Kody's existing | ||
| app-scoped storage buckets using a stable internal binding key for each job. | ||
| That means job-owned values and secrets stay isolated from saved apps and from | ||
| other jobs. | ||
|
|
||
| ## Kill switch and observability | ||
|
|
||
| Each `JobRunner` keeps supervisor-owned metadata: | ||
|
|
||
| - run count | ||
| - success count | ||
| - failure count | ||
| - last run started/finished timestamps | ||
| - last run duration | ||
| - last error message and stack | ||
| - bounded run history (default last 50) | ||
| - kill switch flag | ||
|
|
||
| When the kill switch is on, alarm executions are skipped and the next alarm is | ||
| cleared until the job is re-enabled. | ||
|
|
||
| Use `job_history({ job_id, limit? })` to inspect recent runs. | ||
|
|
||
| ## Storage reset and server exec | ||
|
|
||
| - `job_storage_reset({ job_id })` deletes the facet storage but keeps the saved | ||
| job record and supervisor history. | ||
| - `job_server_exec({ job_id, code, params? })` executes one-off JavaScript | ||
| against the live facet instance for debugging or data migrations. | ||
|
|
||
| ## Example | ||
|
|
||
| See [Feed watcher example](./examples/job-feed-watcher.md) for a job that keeps | ||
| track of the last-seen item id in SQLite and only emits new items on later runs. | ||
|
coderabbitai[bot] marked this conversation as resolved.
Resolved
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| CREATE TABLE jobs ( | ||
| id TEXT PRIMARY KEY NOT NULL, | ||
| user_id TEXT NOT NULL, | ||
| name TEXT NOT NULL, | ||
| server_code TEXT NOT NULL, | ||
| server_code_id TEXT NOT NULL, | ||
| schedule_json TEXT NOT NULL, | ||
| timezone TEXT NOT NULL, | ||
| enabled INTEGER NOT NULL DEFAULT 1, | ||
| created_at TEXT NOT NULL DEFAULT (CURRENT_TIMESTAMP), | ||
| updated_at TEXT NOT NULL DEFAULT (CURRENT_TIMESTAMP) | ||
| ); | ||
|
|
||
| CREATE INDEX idx_jobs_user_id ON jobs(user_id); | ||
| CREATE INDEX idx_jobs_user_updated ON jobs(user_id, updated_at DESC); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.