diff --git a/docs/developers/daemon/02-serve-runtime.md b/docs/developers/daemon/02-serve-runtime.md index 34d69450e4e..591773285bf 100644 --- a/docs/developers/daemon/02-serve-runtime.md +++ b/docs/developers/daemon/02-serve-runtime.md @@ -77,19 +77,20 @@ 12. **Build `fsFactory`**: `runQwenServe` defaults to `trusted: true`; direct `createServeApp` callers default to `trusted: false` and warn once. 13. **`createHttpAcpBridge`**, see [`03-acp-bridge.md`](./03-acp-bridge.md). 14. **`createServeApp`** assembles Express. -15. **`server.listen(port, hostname)`**, then resolve the actual `getPort()` for host allowlist. -16. **Register SIGINT / SIGTERM handlers** for graceful shutdown. +15. **Create and lifecycle-bind the HTTP(S) server before listening**, then call `server.listen(port, hostname)` and resolve the actual `getPort()` for host allowlist. Conversations ownership cannot start until this listener and the remaining host startup gates are ready. +16. **Register SIGINT / SIGTERM handlers** for graceful shutdown through the shared app lifecycle. ### Graceful shutdown -1. **Phase 1 - bridge teardown** on first signal: +1. **Seal admission and begin all drains** on the first signal: - Dispose the device-flow registry and cancel pending flows. - `bridge.shutdown()` marks each channel `isDying = true`, sends graceful close to each ACP child stdin, waits `KILL_HARD_DEADLINE_MS` (10s) per channel, then calls `channel.kill()` if needed. -2. **Phase 2 - HTTP teardown**: +2. **Close the listener while app and host drains run**: - `server.close()` stops accepting new connections and lets in-flight requests finish. - `SHUTDOWN_FORCE_CLOSE_MS` (5s) triggers `server.closeAllConnections()`. - A second 2s deadline escalates again if needed. -3. **Second signal while exiting**: +3. **Release Conversations ownership only after positive shutdown proof** from the listener, app-local work, host-owned work, Live discovery cleanup, and runtime drains. Any incomplete proof rejects shutdown instead of allowing an unsafe handoff. +4. **Second signal while exiting**: - `bridge.killAllSync()` + `process.exit(1)` to avoid orphaned children blocking daemon exit. ## State and lifecycle @@ -98,9 +99,9 @@ - `url`: resolved listen URL, after ephemeral port resolution. - `port`: actual port, including `0` resolution. -- `close({ timeoutMs? })`: programmatic shutdown for embedders and tests. +- `close()`: programmatic shutdown for embedders and tests. -Calling `createServeApp` directly returns only an `Application`; the embedder owns `listen` and shutdown. +Calling `createServeApp` directly still returns only an `Application`. An embedder that needs Live/Conversations must create the actual Node server, call `getServeAppLifecycle(app).bindServer(server)` before its first `listen()`, and await `lifecycle.close()` during shutdown. Without binding, ordinary routes remain available but Live/Conversations fail closed. Calling raw `server.close()` triggers event-driven cleanup, but the embedder must still await `lifecycle.close()` to observe drain or ownership-release failures. ## Dependencies diff --git a/docs/developers/daemon/20-quickstart-operations.md b/docs/developers/daemon/20-quickstart-operations.md index dd0598abff0..5ba5f5241e9 100644 --- a/docs/developers/daemon/20-quickstart-operations.md +++ b/docs/developers/daemon/20-quickstart-operations.md @@ -259,7 +259,9 @@ serve/server.ts createServeApp() - builds Express app (**does | `- return app | v -serve/run-qwen-serve.ts server = app.listen(port, hostname, cb) +serve/run-qwen-serve.ts server = createServer(app) / https.createServer(..., app) + | |- lifecycle.bindServer(server, { startupReady, drainHost }) + | |- server.listen(port, hostname) | |- server.maxConnections = cap | |- actualPort = server.address().port | |- write "qwen serve listening on ..." @@ -272,8 +274,8 @@ commands/serve.ts await blockForever() // block forever unti Key facts: -- **`createServeApp` only builds; it does not listen.** It returns an `express()` instance with middleware and routes mounted. The caller owns `app.listen()`. `server.test.ts` uses the factory this way across roughly 25 cases, so the factory intentionally avoids owning lifecycle. -- **`() => actualPort` is a lazy closure.** `actualPort` is assigned in the `app.listen` callback. The `hostAllowlist` middleware reads it on demand, so ephemeral ports (`--port 0`) still gate the `Host` header correctly. +- **`createServeApp` only builds; it does not listen.** It returns an `express()` instance with middleware and routes mounted. Ordinary-only embedders may continue to own `app.listen()`. Embedders that use Live/Conversations must bind the actual Node server to the exported app lifecycle before listening and await that lifecycle during shutdown. +- **`() => actualPort` is a lazy closure.** `actualPort` is assigned in the `server.listen` callback. The `hostAllowlist` middleware reads it on demand, so ephemeral ports (`--port 0`) still gate the `Host` header correctly. - **`await blockForever()` is intentional.** If `yargs.parse()` resolves, the CLI top level falls through into the interactive TUI entrypoint (`gemini.tsx`). SIGINT / SIGTERM exit through `runQwenServe`'s `onSignal` path. ## 10. HTTP route file split @@ -325,11 +327,17 @@ console.log(`Daemon at ${handle.url}`); await handle.close(); // programmatic shutdown ``` -Or get the Express app directly and listen yourself: +Or get the Express app directly and bind the listener lifecycle yourself. This form is required when the embed uses Live/Conversations: ```ts -import { createServeApp } from '@qwen-code/qwen-code/serve'; - +import { createServer } from 'node:http'; +import type { AddressInfo } from 'node:net'; +import { + createServeApp, + getServeAppLifecycle, +} from '@qwen-code/qwen-code/serve'; + +let actualPort = 0; const app = createServeApp( { port: 0, @@ -337,17 +345,28 @@ const app = createServeApp( mode: 'http-bridge', maxSessions: 20, }, - () => 0, + () => actualPort, { /* deps: bridge, fsFactory, ... */ }, ); -const server = app.listen(0, '127.0.0.1', () => { - console.log('listening on', server.address()); +const lifecycle = getServeAppLifecycle(app); +const server = createServer(app); +lifecycle.bindServer(server); +await new Promise((resolve, reject) => { + server.once('error', reject); + server.listen(0, '127.0.0.1', () => resolve()); }); +actualPort = (server.address() as AddressInfo).port; +console.log('listening on', server.address()); + +// Stop admission, drain app work, close the listener, and release ownership. +await lifecycle.close(); ``` +Calling raw `server.close()` also starts the same event-driven cleanup, but it is only best effort unless the process remains alive; always await `lifecycle.close()` to receive shutdown errors. If no server is bound, Live/Conversations requests fail closed while ordinary-only app behavior is unchanged. + Note: when calling `createServeApp` directly, the default `fsFactory.trusted = false`. Agent-side ACP `writeTextFile` is rejected as `untrusted_workspace`, and a stderr warning is printed once. Either inject `deps.fsFactory` with explicit trust, inject `deps.bridge`, or accept the trust-gated default behavior. ## 13. Debugging recipes diff --git a/docs/plans/2026-08-13-standalone-pr1-runtime-boundary.md b/docs/plans/2026-08-13-standalone-pr1-runtime-boundary.md new file mode 100644 index 00000000000..caca6414f0c --- /dev/null +++ b/docs/plans/2026-08-13-standalone-pr1-runtime-boundary.md @@ -0,0 +1,524 @@ +# 实施计划：Standalone PR1 —— Conversations runtime ownership 与隔离边界 + +日期：2026-08-15 + +上游设计：`docs/design/standalone-daemon-sessions.md` + +关联：Issue #8908、PR0 #8890 + +发布基线：`origin/main` at `9aa570446aa590442e835e8a9cf501d3fe4da3e9` + +PR0 已合入：#8890，squash merge commit `c9cb53398dcf7faa9e70a30f7f38b5946cf2def1`，最终 PR head `9d08762121df9918095d08baf2295f43415fe32a` + +## Goal + +在 PR0 的 `ConversationRuntimeManager` 与 owned-runtime publication 基础上，完成两个隐藏基础能力： + +1. 同一用户的多个 supporting daemon 中，最多一个进程持有 Conversations runtime；有效外部 owner、被篡改的 owner 状态和根目录失败均返回结构化错误，且绝不回退 primary runtime。 +2. `live-conversation` runtime继续服务owner-routed session、Live、health/capabilities、user-global config reconciliation，以及既有Live只读channel与scheduled-task管理的窄兼容面；除此之外，所有普通workspace选择器、管理路由和非全局配置的后台workspace fanout默认看不到它。 + +PR1 不增加 standalone source、公开 standalone routes、SDK/standalone UI 行为或 `standalone_sessions_v1` capability；WebShell只做两类兼容收口：既有`kind: "live"` entry的ordinary selector/presentation guard（新会话、scheduled-task、workspace voice与scratch outcome列表），以及Live Sidebar catalog的capability-gated `sourceType=default`过滤。 + +## Baseline 与开工门槛 + +- PR1 不再是 stacked PR；设计分支已直接基于包含 PR0 merge commit 的最新 `origin/main`。不得重放或 rebase 到旧 PR0 head，否则会与 squash merge 重复。 +- 实现分支已在发布前将 PR1 自身提交 rebase 到 `9aa570446aa590442e835e8a9cf501d3fe4da3e9`；不重放旧 PR0 head，避免与 squash merge 重复。 +- 发布基线已包含 PR0 后续的 telemetry、background-shell active-work、cross-worktree Git guard 以及 WebShell 更新。PR1 按该基线的 handler-resolved/pre-resolved attribution contract 验证 telemetry 隔离，并让既有 bridge/session drain（包括其后台 shell）先于 owner release 完成。 +- 当前inventory用`rg`得到49个import或访问`WorkspaceRegistry`/`WorkspaceRuntime`的production TypeScript文件：43个直接选择/registry consumer，加6个只接收已选runtime或generation guard的helper；下文均已分类。这是实现门禁，不是一次性文档。实现开始和每次同步main后都要重建，尤其复核`server.ts`、`run-qwen-serve.ts`、`routes/session.ts`、`acp-http/index.ts`、Channel/Goal/multi-agent路径。 +- 最终 PR0 的 owned publication 只有 registry add 前的 `validateBeforePublication`；它不会先发布一个 non-routable entry 再 rollback。PR1 必须在这个 pre-publication seam 内完成 candidate 与 exact-root 重验，不引入第二个 publication state machine。 + +## Invariants + +- Owner record 位于真实user-home下的稳定runtime目录，不受`QWEN_HOME`、`QWEN_RUNTIME_DIR`、project workspace或project settings影响；两个不同`QWEN_HOME`但共享同一OS home/Conversations root的daemon仍必须竞争同一record。 +- 一个进程身份是 `{ pid, instanceNonce }`；相同 PID、不同 nonce 按 PID reuse/foreign owner 处理并 fail closed。 +- 只有有效且已死亡的 foreign owner 可以被替换；替换后等待固定的短 drain grace，再允许 publish/use runtime。 +- malformed、symlink、wrong owner、wrong mode、oversize 或无法证明安全的 record 均不删除、不覆盖。 +- release 只删除仍匹配当前 `{ pid, instanceNonce }` 的 record，并且只能发生在 route/session/bridge/child drain 完成且 listener close callback 已确认之后。 +- force-exit、drain error、channel-worker retry或listener secondary deadline均不进入owner unlink。release在exact unlink前失败时本进程不删除/覆盖观测状态，匹配record若仍存在则保留；missing/foreign/invalid保持原样并报compromise。若exact unlink已成功但lock cleanup失败，record已安全移除且进程内claim必须清除，`close()`仍报错并让后继通过lock recovery而非假装完整handoff。 +- 除下述source/session identity验证过的兼容catalog与精确session操作外，普通workspace selector无论使用workspace ID、原始cwd、canonical cwd或path alias，都把internal runtime当成不存在。 +- 任何internal lookup failure都不能改选primary；owner-routed lookup要么得到已验证的internal owner，要么返回错误。session owner index若指向transitioning/draining/blocked internal entry，必须保留该index并返回明确unavailable outcome，不能跳过后扫描active primary；只有active runtime明确报告session不存在或entry真正removed时才按既有契约清除stale index。 +- ordinary request的mismatch/conflict/admission error不返回internal workspace ID/cwd，也不把internal计入workspace count；capabilities的临时`kind: "live"` entry和已授权session结果是明确兼容例外。 +- Registry 仍保存完整 runtime 集合，供 shutdown、总 session-ID admission、session owner index、Live 和观测聚合使用；隔离发生在 resolver 和每个 direct consumer 边界，不改变 registry 的底层语义。 +- `GET /capabilities` 可暂时保留 `{ kind: "live" }` 兼容 entry，但不得新增 standalone capability；普通路由即使拿到该 ID 也必须拒绝。 +- `createServeApp` direct embed只有在把实际接收请求的Node listener绑定到共享lifecycle后才能claim/publish Conversations；未绑定时ordinary routes保持可用，任何internal boot/ensure都fail closed且不执行ownership I/O。绑定后的listener close、app-local drain、host drain与owner release必须由同一个lifecycle串行证明，不能让embed和`runQwenServe`各维护一套释放状态。 + +## Ownership contract + +### Stable record + +新增 `packages/cli/src/serve/conversations/conversation-runtime-ownership.ts`，默认 record 为： + +```text +~/.qwen/conversations/runtime-owner.json +``` + +最小且exact（unknown key也拒绝）schema： + +```ts +interface ConversationRuntimeOwnerRecord { + version: 1; + pid: number; + instanceNonce: string; +} +``` + +不写 URL、token、workspace path 或可由 project 配置覆盖的值。PID必须是正safe integer，nonce沿用Live的UUID/pattern约束。POSIX敏感叶目录（owner record目录与Live locator目录）为 owner-only `0700`，record 为 link count 1的regular non-symlink owner-only `0600`；Windows只承诺regular non-reparse、single-link、canonical identity与既有平台可观测的path安全，不虚构uid/mode/ACL保证。读取有固定 byte 上限。首次创建目录时，先 canonicalize并记录nearest existing ancestor，再逐级使用non-recursive `mkdir`创建缺失组件；每一级在`mkdir`/`EEXIST`后都重验parent和child identity，拒绝symlink、非目录或竞态替换。既有祖先只要求稳定的canonical identity及POSIX same-owner，不把`0700`追溯强加给历史`~/.qwen`；敏感叶目录必须满足上述严格权限。只有本次成功创建的组件可依创建mode设置权限；既有unsafe敏感叶目录不得靠recursive `mkdir`或`chmod`静默修复。`proper-lockfile` 必须显式把 `lockfilePath` 放在已验证目录内（例如 `.runtime-owner.lock`），不能使用默认的 sibling `~/.qwen/conversations.lock`。进入 lock 前记录目录的 canonical/device/inode identity，lock 后及每次 read/rename/unlink 前重验，目录替换或 symlink一律 compromised。record读取采用 `lstat -> open(no-follow where supported) -> fstat`，并要求 path/handle device+inode一致；不得在 `lstat(path)` 后直接 `readFile(path)`。写入采用 same-directory `wx` temp file、`sync`与最终安全校验；POSIX可rename-over exact validated target，Windows在lock内重验后采用平台支持的commit顺序，不声称目标已存在时仍有不可实现的atomic overwrite。Windows删除validated dead target前必须已sync current temp；若删除后current commit失败，活进程保持owner lock并完成一次不可取消grace后才release/throw，进程崩溃则由大于grace加最大临界区的stale阈值保证后继恢复锁时已跨过grace。该异常gap路径不启动runtime。只best-effort清理当前operation持有的随机temp；crash遗留和其他未知文件均忽略且不删除。 + +lock使用显式、可测试的bounded retry window覆盖正常I/O临界区；一个仍有效的foreign lock只是暂时busy，耗尽重试映射为`conversation_runtime_unavailable`，不能误报篡改。unsafe lock shape、stale-lock recovery失败、`ECOMPROMISED`或release ownership丢失才映射为`conversation_runtime_ownership_compromised`。显式`onCompromised`只记录并唤醒当前operation；每次commit/release前检查该状态，不使用library默认的异步throw handler把进程直接crash。stale阈值必须大于handoff grace加最大正常文件临界区，update间隔满足library约束，两者均可测试注入。 + +Ownership constructor必须是无 I/O、无 timer、无 process handler的纯构造。其 `stableBaseDir` 与 Live discovery 使用同一个已解析值：production沿用`getStableLiveDiscoveryBaseDir()`语义固定为真实home下的`~/.qwen`，不得改用会跟随`QWEN_HOME`的`Storage.getGlobalQwenDir()`；`runQwenServe` 的 `liveDiscoveryStableBaseDir` test/embed override必须同时传给 ownership和locator，不能出现两套“stable”目录。`proper-lockfile` 与 legacy `live/discovery` inspection在首次 `acquire()` 内动态加载；manager只 type-import ownership contract。这样不会破坏现有 serve startup import boundary，也不会因为 Live关闭而提前加载或创建稳定目录。 + +`runQwenServe`只解析一次stable base并传给app与locator。`createServeApp`在`LiveHostCoordinator`产生nonce后，通过窄factory seam `(pid, instanceNonce, stableBaseDir) => ConversationRuntimeOwnership`构造side-effect-free实例，保证默认production ownership与tests注入的fake都拿到同一identity；再把同一实例装配到manager、Live discovery gate与`app.locals`。默认factory的构造仍无I/O，真实home下的目录/record只有在下述listener binding已经成立且internal boot实际开始后才会访问。 + +`createServeApp(): Application`保持返回类型兼容，但在app上安装一个共享、one-flight的`ServeAppLifecycle`，并从`serve/index.ts`导出类型与`getServeAppLifecycle(app)` accessor： + +```ts +interface ServeAppLifecycle { + bindServer( + server: Server, + options?: { + startupReady?: Promise; + drainHost?: () => Promise; + }, + ): void; + close(options?: { timeoutMs?: number }): Promise; +} +``` + +`bindServer`必须在第一次`server.listen()`和任何internal boot attempt前，把实际接收该app请求的尚未listening Node `Server`绑定exactly once；已listening server、重复绑定、绑定不同server或boot开始后的迟到绑定都明确拒绝。这样不会存在listener已经接收请求、lifecycle却尚未拥有cleanup proof的窗口。lifecycle监听绑定后的真实`listening`/`error`/`close`结果：direct embed在listener成功后即可打开其boot admission，且首次pre-listen error直接seal/reject；`runQwenServe`则额外传入覆盖完整host startup的`startupReady` promise，只有listener与该promise都成功才打开。production的listen retry classifier仍由`runQwenServe`拥有，transient `EADDRINUSE`只尝试同一pre-bound server的下一个port，不reject `startupReady`、不调用`server.close()`、也不被lifecycle误判为shutdown；只有所有listen尝试或后续channel/runtime startup最终失败时才reject该promise并seal。为满足exactly-once binding，HTTP路径也改为先`http.createServer(app)`，与现有HTTPS路径一样在首个listen attempt前绑定并跨port retry复用同一对象，不再让每次`app.listen()`隐式创建新server。 + +`drainHost`是唯一的外层lifecycle seam，在close开始时与app-local seal一起发起，并在owner release前等待；`runQwenServe`用它纳入channel worker、process registry及其他不属于app的drain，direct embed通常省略。`RunHandle.close()`委托同一个handle，不再维护第二个ownership release gate。绑定后的embed即使直接调用`server.close()`，`close`事件也必须同步seal并启动同一条one-flight cleanup，错误保存在handle上；公开文档仍要求调用并await `lifecycle.close()`，以便在进程退出前等待drain/release并接收错误。未调用`bindServer`时ordinary app行为保持不变，explicit Live/internal请求返回结构化unavailable，capabilities返回ordinary snapshot，绝不能退成no-op ownership或写真实home。所有会触发internal route的direct-app tests都注入无外部资源fake并绑定真实ephemeral test listener；纯assembly测试可保持unbound并断言零ownership I/O。 + +boot hook等待共享lifecycle的boot-admission barrier：server必须已绑定并成功listening；`runQwenServe`还必须已经把app纳入同一cleanup owner，且其channel/runtime startup其余可失败门禁全部通过。direct embed的pre-listen error、production最终listen failure、`startupReady` rejection或shutdown均reject/seal barrier；production可重试listen error不改变barrier。`runQwenServe`遇到最终listen或host startup failure时，必须先调用并await同一个`ServeAppLifecycle.close()`，完成可证明的listener/app/host cleanup后才reject启动promise；若`drainHost`仍持有retryable worker/service lease，则沿既有runtime-failure retry语义保持cleanup owner，不能先把失败返回给一个已失去handle的caller。该路径尚未打开boot时ownership release是无I/O no-op。dedicated Live或internal catalog请求在production channel startup期间可等待barrier但不能抢先acquire。`/capabilities`是例外：channel worker在ready前会探测该route，因此barrier未open且boot未开始时必须立即返回不含internal entry的ordinary snapshot，既不等待也不触发claim；barrier open后若boot已经开始，后续capabilities才等待同一settlement并稳定反映结果。direct embed没有额外`startupReady`时仍必须先绑定并成功启动真实listener，不能靠test-only bypass伪造close proof。 + +装配阶段不得启动ownership I/O：当前`createServeApp`末段立即触发的Live runtime boot改成显式one-flight `startConversationRuntimeBoot()`。production `runQwenServe`只有在`createServeApp`成功返回、共享lifecycle已绑定server、listener成功启动，并且channel worker等其他会让runtime startup失败的门禁已通过后，才可在eager discovery publication/readiness之前主动调用；成功监听是必要但不充分条件，也不是让Live-disabled ordinary daemon无条件claim owner的新理由。所有同步listen throw、最终`error`/port retry失败和pre-runtime-ready startup failure都发生在claim之前。Live兼容面启用时的首个兼容Live catalog或dedicated Live请求可lazy触发并等待同一hook；capabilities只有在共享barrier已open后才能触发首次attempt，否则按上段返回ordinary snapshot。首次attempt settled后，capabilities只等待当前pending或读取snapshot，不因轮询重复acquire；后续显式Live/internal请求可新开attempt，允许loser在foreign owner退出后恢复。每个attempt仍one-flight并在settled后清除pending，terminal ownership compromise则由ownership对象固定拒绝。直接app测试必须使用前述显式fake ownership与bound ephemeral listener。只有在Live兼容面启用且selector精确命中configured Conversations ID/root、并且catalog显式携带`sourceType=default`时，session route才可在ordinary resolver前触发这个preflight；任意ID/cwd、无source catalog或普通workspace请求均不能因此claim owner。capabilities在boot已开始时继续等待settlement再取snapshot，成功时稳定看到active`kind: "live"`entry；ownership失败沿既有非广告语义不伪造entry，真正请求Live/internal操作时再返回structured error。`/live/start`与`/live/new`必须改为async handler，在调用同步coordinator action前await同一boot hook；该preflight的typed ownership/root/runtime error直接由Live route serializer转成`status/code/retryable`，不能被后台eager boot的catch吞掉后先返200。非HTTP Host action仍沿既有Live state/error channel报失败，不伪造HTTP响应。这样后续route assembly、listen或channel startup失败不会留下外层拿不到引用的active owner record，也不改变无Live/无standalone需求daemon的惰性。shutdown seal必须阻止尚未开始的boot，并等待已经开始的boot/ownership acquire/publish settled后再进入release gate。 + +公开给 manager/lifecycle 的窄接口： + +```ts +interface ConversationRuntimeOwnership { + acquire(): Promise<{ reclaimed: boolean }>; + release(): Promise; +} +``` + +内部状态最小化为`unclaimed → provisional → owned → released`并带不可清除的terminal-compromised flag：commit/确认current record后先进入`provisional`，只有所有lock cleanup成功且必要grace完成后才进入`owned`。任何post-commit、acquire成功前的错误把实例置为terminal provisional，并让该次及后续调用固定返回non-retryable ownership compromise；owned后观测到missing/foreign/invalid/unsafe也同样置terminal。terminal且尚未released时，`release()`拒绝unlink，即使外部后来把record恢复成相同nonce也不能洗掉compromise。这样provisional current record留给进程死亡后的后继重新执行grace，不能因旧locator已删除而跳过handoff。Windows destructive gap若current record从未commit，则在lock内完成grace后仍保持unclaimed，可按实际I/O错误重试，不属于provisional。 + +`release()`的boolean只表达“本调用是否删除了owned current record”：从未claim或成功release后的重复调用返回`false`且无I/O；provisional或terminal pre-unlink release抛structured compromise且不碰record；owned时record缺失、invalid或nonce/PID不匹配会先置terminal，再抛错并绝不删除。exact unlink一成功就转为`released`；随后lock cleanup成功则返回`true`，cleanup失败则抛structured compromise但重复release仍为无I/O `false`，因为record已经不存在，不能伪称仍由本实例claim。 + +`acquire()`使用进程内one-flight串行化并发调用，但每个新的acquire cycle都在锁内重读和校验record，不能仅依赖cached state。若本对象已经owned，只有record仍精确匹配当前PID/nonce才可幂等成功；missing、foreign、invalid或unsafe都表示运行中ownership proof被破坏，置terminal、映射non-retryable compromise且不重建/回收。in-flight `provisional`是正常中间态，所有caller等待同一promise；若acquire promise已settled而实例仍停在`provisional`，则必须同时带terminal flag，之后不能重试成owned。下表只描述unclaimed或精确same-owner的正常决策。正常成功路径在锁内完成legacy inspection与owner commit，确认locks release成功后才在锁外等待dead-owner drain grace；唯一例外是上述Windows destructive commit gap失败，它为防无record后继提前进入而在owner lock内等待grace后报错。grace仍属于同一个pending acquire，在完成前任何同进程caller都不能提前成功；另一个进程此时看到alive current owner或busy lock并fail closed。这样正常路径不为1秒等待持有filesystem lock，也不需要靠heartbeat维持grace。结果规则： + +| 当前状态 | 结果 | +| -------------------------------- | ----------------------------------------------------------------------------- | +| 无 record | 原子写入当前 owner，`reclaimed: false` | +| 与当前 PID/nonce 相同 | 幂等成功，`reclaimed: false` | +| foreign valid record，PID alive | `503 conversation_runtime_in_use`，`retryable: true` | +| foreign valid record，PID dead | 按平台serialized commit，锁外等待1,000 ms injectable grace，`reclaimed: true` | +| PID 相同、nonce 不同 | 按 active foreign owner 处理，防 PID reuse | +| unsafe/invalid/unreadable record | `503 conversation_runtime_ownership_compromised`，`retryable: false` | + +1,000 ms grace在dead Conversations owner或dead foreign legacy Live owner handoff后执行；同一次acquire若两者都stale也只等待一次，返回的`reclaimed`在任一handoff发生时为`true`。对校验通过且PID已死的foreign Live locator，在owner→Live锁序内先commit/确认当前owner record，再nonce/PID精确删除locator；两把lock都成功释放后才等待grace。commit后的grace是不可取消、只resolve的timer，shutdown等待同一个pending acquire，不能用AbortSignal让same-owner retry跳过未完成grace。无需额外的“已等待locator”journal/cache：成功acquire已完成grace；grace前进程退出或post-commit失败则provisional current record必须保留，后继会从dead owner record再次执行grace。测试通过注入`isProcessAlive`、只resolve的`wait`与base dir保持确定性，production使用`process.kill(pid, 0)`，除`ESRCH`外均视为alive。 + +### Legacy Live compatibility + +复用`live/discovery.ts`已有的size/schema/mode/owner/PID校验，不复制第二套宽松parser；同时把其platform contract与owner record对齐：mode/uid仅在POSIX强制，Windows验证regular non-reparse/single-link与可观测identity。legacy locator目录不存在时inspection直接返回absent，不为检查而创建Live目录；目录存在时先验证regular non-reparse、canonical/device/inode和POSIX owner-only属性，再把explicit lock path放在该目录内，既有unsafe目录不靠`chmod`静默修复。后续Live publish若需首次创建目录，复用owner record的nearest-existing-ancestor、逐级non-recursive `mkdir`与identity revalidation契约，不能保留现有recursive `mkdir`/unconditional `chmod`旁路。新增一个locked handoff seam，返回owner状态并允许调用方在仍持有Live lock时对已验证dead record做exact nonce/PID removal： + +- 无stable Live record或same `{pid, nonce}`：允许继续；dead foreign record在current owner commit后精确移除并触发一次drain grace； +- active foreign Live owner：映射为 `conversation_runtime_in_use`； +- malformed/unsafe stable Live record：映射为 `conversation_runtime_ownership_compromised`。 + +Conversation owner acquisition locked-inspect一次 legacy stable Live owner，再提交/确认新 owner record并完成必要 grace。不要增加无法闭合 mixed-version竞态的多阶段 handshake：旧版本在新 standalone owner 之后启动无法被强制遵守新 record，继续保留设计文档中的 mixed-version unsupported 限制。Live启用路径在 acquire后紧接着执行既有 nonce/PID-protected discovery write，因此仍会拒绝 acquisition期间已出现的 foreign Live owner。 + +只有真实home下的stable Live locator参与cross-daemon legacy arbitration；现有`runtimeBaseDir` locator可随`QWEN_HOME`改变，不能被当作user-global owner proof。但当stable与runtime base不同时，两个locator都必须等待同一boot成功才发布，shutdown也必须在owner release前对每个曾发布target取得“exact current owner removed”或“already absent”的正向证明。 + +一次Live publication只有在全部distinct target都写入当前PID/nonce后才进入ready；若后一个target失败，立即对本次已成功target做nonce/PID-protected compensating removal并保持not-ready/retry状态。cleanup成功的target可从published set移除；cleanup失败或结果不明的target必须保留到shutdown proof，不能因publish promise已失败而遗忘。该补偿不释放Conversation owner，也不把partial locator success当成endpoint ready。 + +唯一允许的嵌套顺序是owner lock→legacy Live inspection lock；任何持有Live lock的路径都不得再获取owner lock。`acquire()`返回前两者均已释放，Live publish随后单独获取Live lock；shutdown也先完成Live cleanup并释放其lock，再进入owner release。实现与测试断言没有Live→owner反向等待。 + +`LiveHostCoordinator.daemonInstanceNonce` 与 Conversations owner 使用同一 nonce。Live discovery publish 必须等待同一个Conversation boot成功：既已`acquire()`，又已revalidate/publish出active internal runtime，缺一都不写locator；这样启用Live但owner/root/runtime失败的daemon不会广告一个无权或无能力提供的endpoint。Live disable不提前release owner，owner生命周期仍是daemon lifetime。 + +### Structured errors + +新增独立的 CLI-local `conversation-runtime-errors.ts`，只定义ownership与manager共用的typed error contract，避免manager为了错误类在startup期加载ownership实现。错误固定 `status = 503`、`code`、`retryable`，响应和用户可见日志均不暴露record/root path、nonce或foreign PID： + +- `conversation_runtime_in_use`：`retryable: true` +- `conversation_runtime_ownership_compromised`：`retryable: false` +- `conversation_root_compromised`：`retryable: false` +- `conversation_runtime_unavailable`：`retryable: true` + +Ownership typed errors原样传播。`ConversationWorkspace` identity/mode/owner/exact-root失败，Conversations exact root已被non-internal entry占用，或owned runtime违反`!primary`、`trusted`、`removable === false`、`live-conversation` provenance不变量，均映射为non-retryable `conversation_root_compromised`；pre-publication runtime construction/validation的可重试失败，以及已知internal entry处于transitioning/draining/blocked等暂时不可用状态，映射为`conversation_runtime_unavailable`。serializer不根据错误message猜类型；在抛出边界显式wrap并保留cause仅供内部日志，响应使用固定sanitized message。后续PR2/PR3直接复用该contract。 + +Live-enabled daemon的后台eager boot遇到ownership/root错误时保持现有降级边界：ordinary primary/secondary workspace服务可继续启动，但不发布internal runtime、`kind: "live"` entry或Live locator；首个真正请求Conversations/Live的操作返回上述structured error。不得把后台错误升级为整个ordinary daemon启动失败，也不得吞掉后再回退primary。 + +## Isolation contract + +### Default-deny resolver + +在 `workspace-registry.ts` 把 derived scope 固化到 `WorkspaceEntry`（例如 `internal: boolean`；replacement 不得改变该 scope），并增加两个最小 predicate： + +```ts +isConversationRuntime(runtime): runtime.provenance === 'live-conversation' +isConversationEntry(entry): entry.internal +``` + +entry-level scope 是必需的：transitioning、draining 或 blocked entry 没有 active runtime，普通 resolver 仍必须把它识别为 internal，而不是从已关闭的 `current.runtime` 重新推断 scope 或泄露成 `workspace_runtime_unavailable`。`removed` entry 按当前 registry contract 会立即从ID/cwd index与list中删除，不需要虚构publication rollback状态。不要新增第二个 registry。`workspace-route-runtime.ts` 中面向普通 workspace 的 entry/runtime/path resolvers 默认过滤 internal runtime，包括 direct ID fast path、exact cwd、canonical scan 与 lexical fallback；`sendWorkspaceMismatch` 的 `workspaceCount` 只计算普通 workspace。 + +Owner-routed session 和 Live service 不调用这些 user-workspace resolvers，而是继续通过 session owner index、exact transcript ownership 或 `ConversationRuntimeManager` 明确 opt in。没有调用者需要一个“任意 internal path selector” helper；若实现过程中出现这种需求，应先证明它是 owner-routed，而不是添加通用逃生口。 + +窄compatibility resolver只能接受已知configured internal ID或exact root，并先读取固化entry scope：active/current才返回runtime；transitioning/draining/blocked返回typed`conversation_runtime_unavailable`，removed/unknown返回not-found或mismatch；任何分支都不回退primary。普通resolver对同一inactive internal仍按隐藏workspace处理，不泄露其存在。 + +现有WebShell会从capabilities的兼容`kind: "live"` entry发起Live catalog读取，并把返回session的internal cwd传给load/resume；PR1不能通过blanket deny破坏它。`routes/session.ts`因此只有以下窄例外，且不得复用为通用workspace resolver： + +- singular/plural list GET仅在selector精确命中active internal entry、请求显式带现有projectless `sourceType=default`过滤时进入兼容catalog路径；返回结果仍按compatible Live/legacy projectless metadata过滤，不能只信query，也不能让未来`sourceType=standalone`提前穿透。pagination/filtering必须保留底层`nextCursor`/`truncated`语义，不能用过滤后的当前页长度推断catalog已完整。 +- 带精确session ID的load/resume、transcript/export、archive/unarchive/delete与organization操作，可在对应archive lock内证明该ID的location、source与internal transcript ownership后opt in；batch要求每个ID都通过且全部解析到同一runtime，任一失败、歧义或跨runtime则整个mutation在副作用前拒绝。 +- aggregate `session-info`、session-groups CRUD、无source filter的catalog list以及仅凭internal cwd/ID的操作仍按ordinary workspace拒绝。普通top-level session creation不得选择internal；已有internal owner session发起并由owner index证明的branch/fork/side-task/sub-session派生创建继续允许。 + +这保持设计文档允许的“owner-routed session/catalog operations”兼容面，同时让settings/Git/files/ACP/voice等普通workspace表面无法借`kind: live` entry寻址internal。当前实码中Live Sidebar的`WorkspaceSection`直接传`selectedSessionSource`：默认tab会发送`default`，但Channel tab会改成`channel`。下述最小WebShell兼容改动必须让Live section在daemon广告`session_source_metadata`时固定发送`sourceType=default`，不随project tab切换；旧daemon未广告该feature时仍传`undefined`并维持unfiltered legacy请求。不能为迁就client而放宽新daemon。 + +`POST /session/:id/load|resume` 保留 PR0/Live 兼容，但 internal opt-in 不能由 cwd 单独授权。resolver先按普通 workspace规则处理；若请求显式命中 internal ID/cwd，只能形成尚未授权的 candidate，不能设置 telemetry、预留session ID、materialize目录或调用bridge。进入该session ID的既有archive shared lock后，必须先满足以下任一ownership入口，再完成共同校验： + +- session owner index精确命中同一个 internal runtime；或 +- `assertSessionLoadable` 在该runtime catalog中返回实际location（`undefined`不是成功），且随后source helper证明它是compatible Live或既有projectless legacy transcript。 + +无论从哪个入口进入，bridge调用前都再次要求 transcript location存在、source兼容、runtime generation仍open；这些检查与requested-session-ID reservation和load/resume保持在同一个archive shared section内，避免校验后换档。未知session、foreign/project source、owner冲突或candidate失效统一拒绝，不触碰internal bridge并且不回退primary。owner-routed transcript/status等现有按session ID入口继续使用owner index，不新增通用internal path resolver。这里仅兼容PR0已支持的Live/legacy projectless source；PR1不接受未来显式`standalone` source，也不创建新route。 + +无selector的精确transcript/batch resolver在扫描active ordinary runtime前，还必须检查`listManaged()`中的internal persistence target：若该ID在inactive internal entry中实际存在或读取返回structured compromise，分别返回runtime-unavailable或原错误，不能因`list()`跳过inactive entry而命中primary同UUID。该检查只发生在session ID的archive lock内，不返回internal identity，也不把任意cwd变成selector。 + +### Reserved registration path + +普通 startup、persisted restore 和 `POST /workspaces` 不得把 Conversations root 本身或其子目录注册为 `existing` runtime： + +- `ConversationWorkspace.rootPath` 提供不创建目录的 configured root；root 已存在时同时比较安全 canonical identity。 +- 显式 `--workspace` 命中时启动失败并返回不含真实 canonical target 的明确 reserved-workspace error。 +- persisted registration 命中时跳过并写 sanitized warning，不启动 child。 +- 动态 registration 命中时返回 `409 conversation_workspace_reserved`，且发生在 persistence、runtime creation 和 registry mutation之前。 +- 遗留 registration store 中已存在的 reserved root/child 仍可作为 `active: false` 的持久化脏数据列出并删除，但 list/forget 不能把它绑定到 internal runtime：不返回 `restartRequired`，不修改 internal metadata，也不触发 runtime removal。 +- 更高层的父 workspace 不在 PR1 禁止范围内；阻止它会破坏既有 broad-workspace 用法。internal runtime 的精确 registry entry 仍由 default-deny selector 隐藏，文件系统的父 workspace containment policy 不在本 PR 改写。 + +Owned publication继续只接受exact validated Conversations root。若registry已有non-internal exact entry，manager固定返回non-retryable `conversation_root_compromised`，不复用、不替换、不回退primary。 + +## Direct-consumer classification + +实现时必须按下表逐项落测试；仅改 shared resolver 不算完成。 + +| Consumer | Scope | PR1 行为 | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `ConversationRuntimeManager` | internal exact owner | acquire owner 后才 revalidate；在最终 PR0 的pre-publication validator中再验candidate/exact root，通过后才publish/use；失败无primary fallback | +| `routes/session-runtime.ts`、`routes/permission.ts`、`routes/sse-events.ts`、session owner index、requested session-ID admission/persistence targets，以及除下述A2UI例外外的既有owner-routed`/session/:id/*` | owner/global identity | 保留internal用于跨runtime UUID查重和按ID路由（含prompt/status/subagent/permission/SSE/shell等）；indexed internal处于inactive state时返回unavailable且禁止scan/fallback primary；ordinary冲突响应要redact internal owner ID/cwd，不能为隐藏它而跳过查重 | +| `routes/session.ts` creation/catalog/session selectors | ordinary + narrow compatibility | ordinary top-level creation/aggregate/group拒绝internal；owner-routed派生创建保留，source-filtered list和精确session-ID操作仅在owner/locked transcript/source proof后opt in，batch先全量验证 | +| generic settings/trust/Git/files/GitHub/extensions/skills/MCP/memory/agents/tools/status/lifecycle/workspace-permissions/voice/channel-notify routes | ordinary workspace | ID和cwd均返回`workspace_mismatch`，不调用internal service/bridge/fs/worker | +| workspace-qualified channel management与observed contacts | ordinary + narrow Live compatibility | 普通runtime不变；active internal只允许既有GET read surface，所有POST/PUT/PATCH/DELETE仍拒绝。显式compatibility resolver不得变成generic selector，也不得触发任意internal boot；internal handler全程持有activity gate lease | +| workspace-qualified scheduled-task routes | ordinary + narrow Live compatibility | active internal仅允许list及对已存在Live-owned task的update/delete/manual-run；base create保持`live_session_creation_reserved`。internal handler全程持有activity gate lease；keepalive/rehydration继续排除internal，不能借兼容面创建standalone durable task | +| primary-bound Goals、A2UI action、workspace auth/models/setup-GitHub/channel-control | legacy primary surface | 保持绑定ordinary primary；不得因`:id`或user-global写入而改选internal。A2UI当前不证明session ownership，PR1不借其session ID扩大internal访问；user-global reconciliation仅走既有显式fanout | +| `acp-http/index.ts` REST mount、ACP WS、Voice WS | ordinary workspace transport | internal 不创建 secondary mount；upgrade 返回 400 mismatch，不能落到 primary mount | +| `routes/workspace-management.ts` | user management + internal publisher | patch/delete/promote/list-by-selector 排除 internal；遗留reserved registration只能按inactive store entry清理；`publishOwnedRuntime` 是唯一明确 internal admission | +| `routes/workspace-extensions.ts` | targeted workspace + user-global config | workspace-qualified route和全局`POST /extensions/install`的workspace activation均拒绝internal；user-global mutation可按既有语义reconcile internal，但不能借此返回或选择它；internal reconciliation全程持有activity gate lease且shutdown后不晚启动 | +| `channel-worker-group.ts`、channel grouping、scheduled keepalive | ordinary background workspace | 排除 internal；即使注入了伪造 group 也 fail closed | +| device-flow event fanout | daemon-global session auth | 保留 trusted internal bridge，避免 owner-routed session的 auth事件丢失；该 fanout不提供 workspace selector | +| per-runtime sub-session launcher、session-originated channel delivery与bridge callbacks | runtime-owned session capability | 保留已授权internal session的既有能力并参与shutdown；不得反向提供cwd/ID selector，普通channel grouping/keepalive仍排除internal | +| `fs/workspace-file-system.ts`、`server/fs-factory.ts`、`routes/workspace-extensions-controller.ts`、`virtual-subagent-sessions.ts`、`voice/workspace-voice-coordinator.ts`、`workspace-runtime-storage.ts` | admitted-runtime helper | 自身不选择registry entry，只接收调用方已授权runtime或generation guard；不得blanket拒绝internal而破坏owner session，也不得新增反向ID/cwd选择器。隔离在其所有调用点证明 | +| capability feature predicates | mixed compatibility/user surface | generation等owner-routed能力可继续计入internal；`multi_workspace_sessions`、workspace-qualified ACP/voice/memory与scratch registration只由ordinary runtimes驱动 | +| telemetry URL workspace selector | ordinary selector + proven owner | ID/cwd过滤internal；被拒绝/未知/非法selector不产生workspace hash，也不误记到primary。任何获准的精确internal session操作（legacy或workspace-qualified）只能在handler完成owner/locked transcript proof后设置internal attribution；source-filtered catalog可保持无attribution；非workspace route仍沿用primary attribution | +| `live/live-session-coordinator.ts`、`live/live-task-service.ts`、`live/realtime-startup-context.ts` | dedicated Live | 明确保留 internal；project selector仍拒绝 internal，projectless/owner lookup可使用 | +| `routes/health.ts`、usage dashboard | aggregate observability | 可聚合 internal counters/usage；不返回 path/provenance或internal workspace identity | +| `routes/capabilities.ts` | compatibility allowlist | 仅active/current internal entry可按固化entry scope展示`kind: "live"`；inactive internal隐藏，不能因current缺失退化成普通workspace；limits仍反映实际admission pools | +| `daemon-status.ts`、`routes/daemon-status.ts` | aggregate + presentation | process/session/resource aggregate可计入 internal；普通 `workspaces[]` 与 path-bearing issue文本不把它呈现为 user workspace | +| metrics/resource sampling、`workspace-trust-reconciler.ts`、runtime drain/removal、shutdown | process ownership/lifecycle | 保留 internal；trust reconciler继续跳过 user-policy replacement，shutdown必须 dispose它 | +| runtime-owned settings/tool persistence callbacks | internal runtime owner callback | 允许已知runtime保存自身状态；普通workspace settings/tools route仍走default-deny resolver，不能借callback seam按任意cwd选择internal；异步internal callback必须由bridge lifecycle或activity gate持有 | + +Shared ordinary resolver覆盖的route文件至少包括下列清单；其中channel read与scheduled-task既有Live操作必须使用单独、method/operation受限的compatibility seam，不能被default-deny resolver误杀，也不能把该seam复用到其他route： + +```text +channel-notify.ts +scheduled-tasks.ts +workspace-channel-management.ts +workspace-channel-observed-contacts.ts +workspace-extensions.ts +workspace-file-read.ts / workspace-file-write.ts +workspace-git.ts / workspace-git-branches.ts / workspace-git-diff.ts / workspace-git-log.ts +workspace-github-prs.ts +workspace-lifecycle.ts +workspace-mcp-control.ts +workspace-permissions.ts +workspace-settings.ts +workspace-skills.ts +workspace-status.ts +workspace-tools.ts +workspace-trust.ts +workspace-voice.ts +workspace-agents.ts +workspace-memory.ts +``` + +每次同步 main 后，任何新增的 direct registry consumer 必须加入表中并归类；无法明确 owner scope 的 consumer 默认按 ordinary workspace 处理。 + +## Shutdown ordering + +共享`ServeAppLifecycle.close()`拥有listener、app-local drain与ownership release gate；`RunHandle.close()`只向它委托common shutdown，并在绑定时用唯一的`drainHost`回调纳入channel worker、process registry等host-owned drain，不把`finish()`等同于listener已关闭，也不另建release state machine。handle的第一个同步阶段先设置daemon-wide admission seal，让已装配的HTTP/upgrade入口拒绝新工作；若listener已成功启动，则立即发起唯一一次`server.close()`并保存其callback结果，不要等bridge/child drain完成才停止接收新请求。若embed先直接调用了`server.close()`，绑定时安装的`close` listener同步执行相同seal并启动同一个cleanup promise；之后调用`ServeAppLifecycle.close()`只await/retry该状态，不创建第二条清理链。从未成功listen的startup-failure分支不对non-listening server发起新close，仍只接受已有listener close event/callback的无错proof；该分支在设计上也不应已claim owner。callback可以先于其他drain完成，但只记录正向proof，绝不提前release： + +1. seal daemon-wide route/upgrade admission、workspace management、Live coordinator和session maintenance，并同步保存各component的drain promise；不得在这里先等待某个activity归零； +2. 立即停止会产生新工作的trust monitor/maintenance/event producers，调用绑定时提供的`drainHost`，并向SSE、ACP/voice transports、channel workers和所有runtime bridge发起cooperative drain/abort；`drainHost`必须在调用时同步发起host seal/stop并返回可等待promise，不能等app-local drain结束后才停止host producer。各component drain先封住自身admission，再等待或取消其owned lease，最后dispose child。所有允许internal的入口必须映射到一个明确drain owner：manager boot/acquire归boot hook，dedicated Live归Live coordinator，transcript/export/archive/organization与load-resume validation归`SessionArchiveCoordinator`，bridge/session/SSE操作归bridge或subscriber drain，source-filtered internal catalog、Live channel/scheduled-task兼容面、user-global extension reconciliation及其他非bridge异步callback归一个只在internal proof后进入的窄`ConversationRuntimeActivityGate`。该gate只提供`run(task)`与`sealAndWait()`，不解析ID/cwd、不成为第二个policy framework。`runSharedMany`与`runExclusiveMany`都必须在seal后拒绝新工作、计入同一个maintenance drain；activity gate也必须在seal后拒绝晚启动。不能只追踪mutation而漏掉已断开client后仍运行的shared filesystem或已返回202的background reconciliation。先发出能让长连接/等待中handler退出的信号，再联合等待这些component promise、`drainHost`与shared process registry，避免SSE或bridge请求与shutdown互相等待。普通generic route无法选择internal，因此无需侵入Express实现一个不可靠的全局async-handler tracker；若新增internal seam却无法归入上述drain owner，必须先补lifecycle ownership。关键stop/dispose helper必须返回或聚合错误，不能只warn后让release gate通过； +3. 等待开始阶段已发起的`server.close()`；只有callback无error且步骤2的internal component drain均有正向proof（不是仅socket被force-close）才设置`listenerCloseConfirmed = true`； +4. seal discovery toggle、停止retry，并等待所有已开始的publish/toggle/retry promise settled后，才移除当前进程在stable与runtime base下曾发布的全部Live discovery records；不能先观察absent再让迟到publish写回。每个target都要把“exact owner removed”“已不存在”“foreign/malformed”“I/O failure”分开，前两者可确认无本进程locator，后两者进入lifecycle error，不能继续用boolean/吞错后假装成功； +5. 仅当步骤 1-4 均确认成功且没有management/Live/session/trust/bridge/channel/process drain error时，调用 nonce-checked `ownership.release()`； +6. 最后完成 telemetry/logger cleanup 和 close promise settlement；这里的失败属于post-release lifecycle error，可记录/返回但不能倒推出owner record仍存在、重做release或把它混入步骤5的前置proof。 + +所有无法证明drain完成的seal/stop/dispose promise都必须显式归并到本次`close()`的lifecycle error accumulator；不能依赖`.finally()`后丢失rejection，也不能catch-log后仍通过release gate。跨重试保存的是各阶段的正向proof state，而不是永久累加所有历史transient error：首次secondary deadline/channel retry仍让该次`close()`拒绝，但迟到listener success或后续worker/service exit可更新proof并允许下一次调用通过；callback error、foreign cleanup、bridge/process dispose等非暂态失败没有正向重试证明时持续阻断。已经settled的Live boot/ensure业务失败本身不是“仍在运行”，可在seal确认没有in-flight work后继续释放其已claim owner。secondary deadline只负责让`close()`有界返回，必须记录listener-unconfirmed error，不能设置`listenerCloseConfirmed`或release。`server.listening === false`的startup-failure分支只有在现有`runtimeFailureListenerClose`保存了无error callback结果时才可release；“从未listen且从未claim”则由无I/O release no-op覆盖。 + +retryable channel/service drain、locator I/O proof缺失与listener secondary deadline必须在该次`close()`拒绝后清除settled close promise、保留全局seal与所有正向proof，从而只重开`close()`重试门，不宣称listener/bridge已恢复服务，也不重新接纳请求。`server.close`迟到callback即使首个close已settled也要记录其success/error；embed可在proof更新后再次调用共享handle的`close()`，复用已完成的drain/locator状态并完成owner release。第二次调用只有在worker/service lease真正退出、`drainHost`取得正向proof、所有曾发布Live locator均有清理正向证明且listener曾确认关闭后才release；callback永不到达则继续fail closed。若pre-unlink ownership、任一foreign/malformed Live cleanup或其他无法取得新正向proof的非暂态drain本身失败，`close()`拒绝且不修改观测到的owner/locator状态；当前匹配record仍存在时保留，missing/foreign/invalid则保持原样。exact unlink后的lock cleanup失败按前述post-unlink状态拒绝但record已安全移除。signal-owned CLI对非retryable错误随后以非零退出，使仍存在的record可在PID死亡后reclaim；retryable rejection后下一次signal可发起新close cycle，而同一cycle尚未settled时的第二次signal仍force-exit。embed caller不得把rejected handle当成已安全handoff；绑定后直接关闭server但不await handle的caller只能获得event-triggered best-effort cleanup，公开契约不保证其进程在异步release完成前保持存活。force-exit、uncaught fatal path和in-flight第二次signal均不尝试异步release。 + +Ownership只记录上述四态与terminal compromise：若foreign/compromised的是Conversations owner record且本次从未commit/确认当前nonce，仍为unclaimed，release是无I/O no-op；fresh/same-owner/dead-handoff commit后为provisional，只有完整acquire成功才owned。release与pending acquire串行；pending失败若留在provisional则拒绝unlink，owned遇到missing/foreign/malformed也绝不按“清理best effort”强删，exact unlink后的lock cleanup failure按上述post-unlink released状态处理。 + +## Implementation tasks + +### Task 0：确认 merged baseline 与 consumer inventory + +**Files:** 本计划、PR0 changed files、所有 `WorkspaceRegistry` direct consumers。 + +- [ ] 实现开始前 fetch 最新 main，确认 `c9cb53398dcf7faa9e70a30f7f38b5946cf2def1` 仍是实现基线的 ancestor；若main前进，只 rebase PR1 自身提交。 +- [ ] 记录 `git diff --stat origin/main...HEAD` 与 PR1 production line budget，确认 PR1 没有越过 core-refactor gate；不把 squash 前 PR0 head 计入 PR1 diff。 +- [ ] 以upstream design的300-550 production lines为review budget：超过550先去掉重复guard/抽象并重新审计；若安全contract客观无法在该预算内实现，先更新design并向maintainer说明，不靠隐藏的大重构硬塞。不得引入通用policy framework、第二registry或可配置lease系统。 +- [ ] 实现期行数审计：集成工作树当前约3,071行production新增、651行production删除，明显越过review budget。发布前必须先完成去重/简化审计，再把可独立验证的ownership+lifecycle、default-deny registry/transport、narrow compatibility+WebShell切成review slices；若依赖关系证明无法安全拆分，则在创建PR前由maintainer明确接受该规模。集成测试继续在完整工作树运行，不能用拆分掩盖跨slice回归。 +- [ ] 使用 `rg` 重建 shared resolver 与 direct registry consumer 清单，逐项填入 allow/deny classification。 +- [ ] 运行 PR0 focused tests，确认 baseline 不是从红灯开始。 + +### Task 1：先写 ownership RED tests，再实现 stable owner + +**Files:** + +- Create: `packages/cli/src/serve/conversations/conversation-runtime-ownership.ts` +- Create: `packages/cli/src/serve/conversations/conversation-runtime-ownership.test.ts` +- Create: `packages/cli/src/serve/conversations/conversation-runtime-errors.ts` +- Modify: `packages/cli/src/serve/live/discovery.ts` +- Modify: `packages/cli/src/serve/live/discovery.test.ts` + +- [ ] 覆盖fresh acquire、same-owner idempotency、concurrent one-flight、unclaimed active foreign owner、dead reclaim + exactly-once grace、PID reuse、owned后reacquire遇到missing/foreign/dead/invalid record一律terminal compromise且不重建、篡改后恢复exact record仍不可洗掉terminal、未claim/重复release、owned-record missing/nonce mismatch release、unlink成功后lock cleanup失败与acquire/release竞态。 +- [ ] 对owner commit之后的legacy exact removal与Live/owner lock cleanup注入错误，断言首次即返回non-retryable ownership compromise、状态停在terminal provisional、同进程不能重试成功、release不unlink current record；另以child process在只resolve grace中退出，证明后继把dead provisional record当stale owner重新等待grace。 +- [ ] 覆盖首次acquire逐级创建缺失stable tree，以及file/dir/intermediate/lock symlink、hard-link record、`mkdir`/`EEXIST`与`lstat/open`竞态、parent/directory identity replacement、wrong mode、wrong uid（平台支持时）、non-file、empty/oversize/malformed/unknown-key/unknown-version record与compromised lock；断言unsafe既有组件不被`chmod`修复且无overwrite/unlink。 +- [ ] 两个不同`QWEN_HOME`/`QWEN_RUNTIME_DIR`但相同real HOME的实例必须解析到同一个default owner/Live stable base；只有显式test/embed `liveDiscoveryStableBaseDir`能改写，且同时作用于两者。 +- [ ] 覆盖legacy Live inspection在directory absent时不创建、首次publish安全逐级创建、unsafe existing directory fail closed且不修复、active/dead/same-owner/malformed record、dead locator只在current owner commit后exact removal、commit/remove失败路径、exactly-once grace，以及Live discovery write在acquire后遇到新foreign owner时仍拒绝。 +- [ ] 覆盖lock正常busy的bounded retry与耗尽后的retryable unavailable、stale/unsafe/compromised lock的non-retryable compromise、正常commit后先release lock再等待不可取消grace、Windows destructive gap失败在lock内等待grace、shutdown与acquire并发，以及custom `onCompromised`不产生uncaught exception。 +- [ ] Live discovery removal区分exact removed、already absent、foreign/malformed和I/O failure；stable与runtime base不同时逐target记录proof，全部写入后才ready。第二target写失败时补偿移除本次已写target；补偿失败仍保留published proof requirement。shutdown只在全部曾发布target都得到前两种结果后视为本进程locator已清理。 +- [ ] 使用真实 child processes 做 contention：测试动态写一个 `.mjs` worker，通过 `node --import tsx` import TS module；A acquire 并保持存活，B 得到 `conversation_runtime_in_use`；A 被终止且不 release 后，C reclaim 并执行 grace。不能用同进程 `Promise.all` 冒充 two-process coverage。 + +### Task 2：把 ownership 接到 manager、Live discovery 和 structured errors + +**Files:** + +- Modify: `packages/cli/src/serve/conversations/conversation-runtime-manager.ts` +- Modify: `packages/cli/src/serve/conversations/conversation-runtime-manager.test.ts` +- Modify: `packages/cli/src/serve/server.ts` +- Modify: `packages/cli/src/serve/server.test.ts` +- Modify: `packages/cli/src/serve/routes/live.ts` +- Modify: `packages/cli/src/serve/routes/live.test.ts` +- Modify: `packages/cli/src/serve/index.ts` +- Modify: `packages/cli/src/serve/run-qwen-serve.ts` +- Modify: `packages/cli/src/serve/run-qwen-serve.test.ts` + +- [ ] `runQwenServe`解析一次stable base；在`LiveHostCoordinator`创建后，通过identity-bearing factory seam用同一PID/nonce/base构造side-effect-free ownership object，放入app lifecycle locals并把同一实例传给manager/discovery gate。`createServeApp`默认factory只构造、不执行I/O；tests注入无外部资源的fake。未绑定listener时internal ensure fail closed且绝不写真实home，绑定并listening后才允许boot。 +- [ ] 在`server.ts`实现唯一的`ServeAppLifecycle`并从`serve/index.ts`导出类型与`getServeAppLifecycle(app)`；保持`createServeApp(): Application`返回类型不变。`bindServer`只接受一个尚未listening的真实Node server，在首次listen前绑定并观察后续`listening`/`error`/`close`状态，把可选`startupReady`和`drainHost`纳入同一boot/release gate。`runQwenServe`必须绑定并委托该handle，不能保留平行的owner release逻辑；HTTP与HTTPS都先显式create/bind一个server并跨port retry复用，transient listen error不close/seal，最终startup failure才reject host readiness。direct embed绑定后的raw `server.close()`也启动同一cleanup，awaitable shutdown走`ServeAppLifecycle.close()`。 +- [ ] manager `ensure()` 先 acquire，再 root revalidate/publish；concurrent ensure仍只 publish一次，owner/root/runtime errors按contract映射；wrong provenance/primary/trusted/removable候选均为non-retryable root compromise。 +- [ ] Live discovery enable/publish 等待同一个boot同时证明 acquire和active internal publication；contention/root/runtime失败时不写 locator、不启动/复用错误 runtime、不 fallback primary。 +- [ ] `createServeApp` assembly不启动owner I/O；所有eager/lazy internal caller先共享lifecycle boot-admission barrier。production仅在server已绑定、listener成功、app已被cleanup owner捕获、channel/runtime startup其余可失败门禁通过且现有Live eager-boot条件成立后，在discovery publication/readiness前调用one-flight hook；direct-app Live-enabled capabilities/Live catalog/dedicated Live request必须使用显式fake ownership、pre-listen bound ephemeral listener并在listener ready后lazy触发。production channel startup期间的capabilities探测必须200返回ordinary snapshot且不等待/claim，防止worker-ready↔barrier死锁；barrier open且boot开始后capabilities才等待settlement，settled failure后轮询不反复acquire，显式Live/internal请求仍可重试。Live catalog preflight只对精确configured internal target + `sourceType=default`生效；任意ordinary selector和无source catalog不触发claim。Live-disabled ordinary daemon不claim；ownership失败不伪造entry；特别覆盖unbound direct app零I/O、already-listening/重复/异server/late binding拒绝、direct pre-listen error seal、production transient port retry不seal/不换server、最终listen failure与channel startup failure在启动promise reject前走共享close、retryable host drain保留cleanup owner、channel worker在ready前真实fetch capabilities、loser在winner退出后由显式请求成功retry、Live请求与channel startup并发时不提前acquire，以及assembly throw、boot-before-close、close-before-boot均无泄漏/无晚启动。 +- [ ] Live disable不 release；ownership已成功后发生的root/runtime初始化失败可在operator修复后由同一daemon显式retry（`retryable: false`仍禁止client自动重试unsafe root），foreign daemon仍被owner挡住；post-commit ownership compromise保持terminal provisional，不能在同进程“修复”后跳过grace。 +- [ ] 为`/live/start`与`/live/new`增加awaitable runtime-ready preflight；后台eager boot失败不影响ordinary routes，但这两个真实Live请求必须重用同一one-flight并在coordinator action前失败，不得先返200。添加route-level structured error serializer tests，断言status/code/retryable且response/用户可见log无base dir、canonical root、nonce、foreign PID；既有`LiveUnavailableError`响应保持兼容。 + +### Task 3：实现 lifecycle-safe release + +**Files:** + +- Create: `packages/cli/src/serve/conversations/conversation-runtime-activity.ts` +- Create: `packages/cli/src/serve/conversations/conversation-runtime-activity.test.ts` +- Modify: `packages/cli/src/serve/server.ts` +- Modify: `packages/cli/src/serve/server.test.ts` +- Modify: `packages/cli/src/serve/server/session-archive.ts` +- Modify: `packages/cli/src/serve/server/session-archive.test.ts` +- Modify: `packages/cli/src/serve/run-qwen-serve.ts` +- Modify: `packages/cli/src/serve/run-qwen-serve.test.ts` + +- [ ] 注入fake ownership，分别经`RunHandle.close()`与direct embed共享handle逐个卡住management/live/session/trust/activity drain、`drainHost`、bridge child、process registry、Live discovery publish/toggle/retry、stable或runtime-base cleanup和`server.close` callback，证明release只发生在全部完成后，且seal后没有迟到locator write；每个rejection都进入lifecycle accumulator而非被`.finally()`/catch-log吞掉。 +- [ ] 实现最小`ConversationRuntimeActivityGate`，只计数已通过internal proof的非bridge异步操作；`sealAndWait()`同步拒绝晚启动并等待已有task finally释放，不读取selector、不捕获普通route。断言`close()`同步封住daemon-wide HTTP/upgrade admission并只发起一次`server.close()`；先向SSE与各component发出cooperative drain/abort，再联合等待internal drain owners退出，不能先等activity而饿死其退出信号，也不能把force-close后的listener callback误当成handler已settled。`SessionArchiveCoordinator`在seal后同时拒绝/等待shared与exclusive操作；逐项证明每个internal opt-in归属manager boot、Live、archive coordinator、activity gate或bridge/subscriber drain。在internal export/shared filesystem操作、已返回202的extension reconciliation、Live channel/scheduled-task兼容操作、SSE、bridge或worker drain被卡住时不release，后到请求不能进入runtime。listener callback早于drain完成也不release，而drain完成但callback未到也不release。 +- [ ] 覆盖正常close、每类drain error、close callback error、bridge error、channel retry后第二次close、force-close后callback成功、secondary deadline时拒绝且不release、迟到success callback后embed第二次close完成release、direct embed调用共享`close()`、direct embed先raw `server.close()`再await共享handle、未await event cleanup的明确best-effort边界、pre-eager-hook daemon startup failure仍unclaimed、Conversation boot失败的unclaimed/provisional/owned状态、telemetry/logger post-release cleanup失败不重做release、重复close与第二次signal force exit。 +- [ ] 断言drain/listener proof不完整时不调用unlink且匹配record保持；release校验遇到missing/foreign/invalid时不修改观测状态；exact unlink后的lock cleanup失败则`close()`拒绝但record已不存在、claim已清除；完整成功路径release恰好一次且位于Live discovery removal之后。 + +### Task 4：把普通 workspace resolver 改成 default deny + +**Files:** + +- Modify: `packages/cli/src/serve/workspace-registry.ts` +- Modify: `packages/cli/src/serve/workspace-registry.test.ts` +- Modify: `packages/cli/src/serve/workspace-route-runtime.ts` +- Modify: `packages/cli/src/serve/workspace-route-runtime.test.ts` +- Modify: `packages/cli/src/serve/routes/session-runtime.ts` +- Modify: `packages/cli/src/serve/routes/session-runtime.test.ts` +- Modify: `packages/cli/src/serve/routes/session.ts` +- Modify: `packages/cli/src/serve/multi-workspace-sessions.test.ts` +- Modify: `packages/cli/src/serve/live/live-task-service.ts` +- Modify: `packages/cli/src/serve/live/live-task-service.test.ts` + +- [ ] 对 entry、active runtime、managed runtime 的 ID/cwd/canonical/lexical selector 写 RED matrix，internal一律 mismatch，普通 primary/secondary行为不变。 +- [ ] `activateReplacement`拒绝 user/internal scope变化；transitioning、draining和blocked entry仍按固化scope过滤，removed entry按registry现有删除契约不可再选择。 +- [ ] 扩展session owner resolution为显式unavailable outcome：internal entry进入transitioning/draining/blocked时不按ordinary replacement逻辑清空其owner index；indexed internal处于这些状态时保留index并禁止scan到primary，active owner明确session-not-found或entry removed才清除stale index。无index的精确transcript/batch lookup也先在archive lock内检查managed internal persistence target，再扫描active ordinary runtime。逐一更新`routes/session-runtime.ts`、`routes/session.ts`、permission/SSE消费者与`live/live-task-service.ts`，返回sanitized runtime-unavailable；分别用indexed与cold-persisted internal + primary同UUID夹具证明无fallback。 +- [ ] ordinary top-level session creation不能选internal，restore不能由cwd单独授权internal；未知session + internal cwd也不能fallback primary。owner-routed branch/fork/side-task/sub-session派生创建保持可用且沿用internal runtime/private-directory规则。 +- [ ] singular/plural catalog按窄例外分类：无source list、session-info、groups CRUD拒绝internal；显式`sourceType=default`的Live list在输出metadata过滤后兼容，并在internal proof后、任何catalog I/O前持有activity gate lease；精确session和batch操作在locked per-ID proof后兼容，batch先验证全部且要求同一runtime，跨runtime/歧义整批拒绝后才允许产生副作用。 +- [ ] active owner-routed Live session的全部既有session-ID操作（含prompt/status/subagent/permission/SSE/shell等）与精确transcript操作，以及cold compatible Live/legacy transcript的load/resume/transcript/export/archive路径继续按上述owner/locked proof opt in；A2UI仍按表中primary-bound例外处理，UUID admission继续跨internal查重。用当前WebShell list/load请求形状做fixture，避免方案自洽但实际UI回归。 +- [ ] 精确configured internal target + `sourceType=default`的catalog在ordinary resolver前等待boot，并把boot typed error原样序列化；精确internal load/resume candidate可等待同一boot，但boot成功仍不等于session授权，必须再完成locked location/source/owner proof才能调bridge。无source、任意ID/cwd和ordinary selector断言不触发boot。 +- [ ] internal restore candidate在source/location验证前不设置telemetry、不reserve ID、不materialize、不调用bridge；`readCreationMetadata()`的空对象不能让不存在的session通过。owner冲突、project source和generation变化均fail closed。 +- [ ] mismatch、ambiguous-owner、workspace-conflict与requested-ID admission响应均不泄露internal ID/cwd/count；查重和内部日志关联仍保留sanitized/hash identity。 + +### Task 5：封住 HTTP、WebSocket 与 workspace-management 旁路 + +**Files:** + +- Modify: `packages/cli/src/serve/acp-http/index.ts` +- Modify: `packages/cli/src/serve/acp-http/workspace-qualified-acp.test.ts` +- Modify: `packages/cli/src/serve/routes/workspace-qualified-voice.test.ts` +- Modify: `packages/cli/src/serve/routes/workspace-management.ts` +- Modify: `packages/cli/src/serve/routes/workspace-management.test.ts` +- Modify: `packages/cli/src/serve/run-qwen-serve.ts` +- Modify: `packages/cli/src/serve/run-qwen-serve.test.ts` + +- [ ] ACP REST、ACP WS、Voice WS分别用 internal ID 和 encoded cwd测试；断言 400、无 mount/upgrade/bridge调用、无 primary fallback。 +- [ ] secondary mount factory自身再做 internal guard，防调用者漏过滤。 +- [ ] patch/delete/persist/promote/select internal均不可达；owned publication仍可发布唯一 exact internal root。 +- [ ] 增加不创建root的reserved-path classifier，覆盖configured/canonical root、child、alias与path-boundary；随后覆盖显式startup reserved root、persisted root/child skip、dynamic root/child `409 conversation_workspace_reserved`，以及父workspace在internal已发布和publication in-flight两种状态都保持兼容。 +- [ ] legacy store若含reserved root/child，registration GET仅把它作为inactive persisted entry呈现；DELETE只移除store记录，不绑定/修改/移除internal runtime，也不返回`restartRequired`。 + +### Task 6：参数化覆盖所有 generic route family 与后台 consumer + +**Files:** + +- Modify: `packages/cli/src/serve/routes/workspace-extensions.ts` +- Modify: `packages/cli/src/serve/routes/workspace-qualified-extensions.test.ts` +- Modify: `packages/cli/src/serve/routes/workspace-channel-management.ts` +- Modify: `packages/cli/src/serve/routes/workspace-channel-management.test.ts` +- Modify: `packages/cli/src/serve/routes/workspace-channel-observed-contacts.ts` +- Modify: `packages/cli/src/serve/routes/workspace-channel-observed-contacts.test.ts` +- Modify: `packages/cli/src/serve/routes/scheduled-tasks.ts` +- Modify: `packages/cli/src/serve/routes/scheduled-tasks.test.ts` +- Modify: `packages/cli/src/serve/routes/channel-notify.test.ts` +- Modify: `packages/cli/src/serve/routes/workspace-trust.test.ts` +- Modify: `packages/cli/src/serve/routes/capabilities.ts` +- Modify: `packages/cli/src/serve/routes/health.ts` +- Modify: `packages/cli/src/serve/server.ts` +- Modify: `packages/cli/src/serve/server/telemetry.ts` +- Modify: `packages/cli/src/serve/server/telemetry.test.ts` +- Modify: `packages/cli/src/serve/daemon-status.ts` +- Modify: `packages/cli/src/serve/daemon-status.test.ts` +- Modify: `packages/cli/src/serve/workspace-trust-reconciler.ts` +- Modify: `packages/cli/src/serve/workspace-trust-reconciler.test.ts` + +- [ ] 建立一个 internal runtime route harness，按 Direct-consumer classification 对每个 generic route family至少测试 ID/cwd一种选择，并对高风险 mutation同时测两种。 +- [ ] 每个断言不仅检查 response，还检查 internal bridge/workspace service/fs/extension manager/channel worker没有调用。 +- [ ] 复核primary-bound `goals.ts`、`a2ui-action.ts`、`workspace-auth.ts`、`workspace-models.ts`、`workspace-setup-github.ts`与`workspace-channel-control.ts`：不新增internal选择/fanout；全部既有owner-routed session-ID路径（含permission/SSE/shell）仍通过owner index命中internal，legacy unqualified permission继续只走primary。 +- [ ] extension targeted routes和全局install接口的workspace activation均排除internal；global extension reconciliation继续覆盖internal且不暴露selector，并在每个internal target的异步刷新外持有activity gate lease，gate sealed后不晚启动。device-flow fanout、per-runtime sub-session launcher、session-originated channel delivery和bridge callbacks继续覆盖trusted internal session。channel worker grouping和scheduled keepalive排除internal；runtime-owned settings/tool persistence callback继续可保存internal自身状态但没有任意cwd入口，非bridge异步callback同样持有activity gate lease。 +- [ ] 保留上游设计的两类Live兼容例外：qualified channel management/observed contacts对active internal只开放GET read surface；qualified scheduled tasks对active internal允许list与既有task的PATCH/DELETE/manual-run，POST base create仍拒绝。internal handler在proof后、任何service/fs调用前取得activity gate lease并在finally释放。按每个HTTP method测试，断言兼容resolver不被其他generic route调用、不因任意ID/cwd触发boot、不启动channel worker或scheduled keepalive，也不能创建新的internal task/session；shutdown seal后返回daemon-draining且无调用。 +- [ ] telemetry resolver改为可返回“无workspace attribution”：internal、unknown、malformed workspace selector不产生workspace hash且不记到primary，非workspace route和有效普通workspace的既有attribution不变；telemetry失败仍不影响请求处理。 +- [ ] 逐项审计因PR1而新增internal owner routing的session telemetry route：当前legacy`GET /session/:id/export`与`PATCH /session/:id/organization`是pre-resolved primary attribution，workspace-qualified transcript/export/batch routes也在handler proof前pre-resolve。凡按Task 4通过owner/locked transcript proof支持internal的精确或batch操作，都必须改为handler-resolved并只在proof成功后设置最终owner cwd；source-filtered internal catalog可保持无attribution。A2UI与unqualified permission保持明确primary-bound。测试同时覆盖legacy与workspace-qualified获准internal操作得到internal hash、proof失败/未知owner不产生hash，以及任何internal candidate都不先污染primary attribution。 +- [ ] capabilities feature predicates逐项分类：owner-routed generation与process/per-runtime admission limits保留internal；internal alone不触发`multi_workspace_sessions`、workspace-qualified ACP/voice/memory或scratch registration。每个变化都对应实际普通selector/registration表面，不能blanket-filter。 +- [ ] health aggregate保持可用；capabilities仅把active/current internal按固化scope展示为兼容`kind: "live"`，transitioning/blocked/draining internal不退化成普通entry，removed internal按registry契约不再展示，limits仍反映实际runtime；ordinary selector features无internal/standalone误广告。daemon status不在ordinary`workspaces[]`/path issue中暴露internal。 +- [ ] trust reconciliation、Live task/projectless路径、realtime startup和 shutdown aggregate保留明确 internal行为，增加回归测试防止过度过滤。 + +### Task 7：收紧WebShell compatibility boundary + +**Files:** + +- Modify: `packages/web-shell/client/App.tsx` +- Modify: `packages/web-shell/client/App.test.tsx` +- Modify: `packages/web-shell/client/components/sidebar/WebShellSidebar.tsx` +- Modify: `packages/web-shell/client/components/sidebar/WebShellSidebar.workspace-removal.test.tsx` +- Modify: `packages/web-shell/client/voice/voice-workspace-target.ts` +- Modify: `packages/web-shell/client/voice/voice-workspace-target.test.ts` + +- [ ] 从capabilities全量`workspaces`派生`ordinaryWorkspaces = kind !== "live"`；全量集合继续支持Live sidebar/catalog与已授权session identity，Composer、新session防御性校验、scheduled-task target及scratch outcome workspace展示必须使用ordinary集合。 +- [ ] Live `WorkspaceSection`改为`sourceType={sourceMetadataEnabled ? 'default' : undefined}`，不再沿用`selectedSessionSource`；feature缺失的旧daemon保持unfiltered legacy请求。fixture同时在default/channel tab断言Live active query固定为default，并覆盖现有archived catalog的分页/query shape。 +- [ ] voice target resolver对`kind: "live"`返回不可用，不能生成workspace-qualified ID/cwd URL；普通primary/secondary voice保持不变。 +- [ ] 回归证明Live section及source-filtered catalog仍显示/可load，Live entry不再出现在新会话、scheduled-task或voice workspace selector；不增加Standalone文案、控件或capability。 + +### Task 8：验证、E2E 计划与审计 + +**Files:** + +- Create: `.qwen/e2e-tests/standalone-pr1-runtime-boundary.md`（实现工作产物，不提交） +- Modify: `docs/developers/daemon/02-serve-runtime.md` +- Modify: `docs/developers/daemon/20-quickstart-operations.md` +- Update design doc仅当实现发现 contract必须修订；不为重复计划内容做无意义改写。 + +- [ ] 先跑所有 changed-file focused tests；从 `packages/cli` 目录执行 Vitest。 +- [ ] 按仓库要求先用global`qwen` dry-run记录baseline：在隔离temp HOME/USERPROFILE中观察现有Live catalog/load请求形状、internal普通route当前可达性与正常shutdown行为，先断言所有root都落在temp tree；若global版本不含PR0 seam，明确标为不可比而不是伪造before。 +- [ ] 运行 `npm run format`并重新审diff，再执行`npm run build && npm run typecheck`，随后`npm run lint`。 +- [ ] build/bundle后执行two-daemon E2E：两个真实child daemon共享同一个隔离的temp HOME/USERPROFILE与stable base、各用不同primary workspace/port；覆盖owner contention时loser的ordinary workspace仍可用但无internal entry/locator且Live操作返回503、kill -9 stale reclaim（平台支持时）、Live locator compatibility、generic REST/ACP WS/Voice WS拒绝、正常shutdown handoff、无primary fallback。先断言record解析到temp tree，绝不触碰操作者真实`~/.qwen`。 +- [ ] WebShell行为验证记录Before/After evidence：Before的`kind: "live"`会出现在Composer/voice或scheduled-task ordinary selector，After这些selector不再展示/生成其目标，同时Live sidebar section、source-filtered list与精确session load仍可用。 +- [ ] 更新公开embed文档：`createServeApp`返回值保持不变；需要Live/Conversations的direct embed用`http.createServer(app)`绑定实际listener，调用`getServeAppLifecycle(app).bindServer(server)`，并以`await lifecycle.close()`完成shutdown。说明未绑定时internal能力fail closed、raw `server.close()`只触发event-driven best-effort cleanup且仍应await lifecycle，以及ordinary-only embed不受影响；给出从现有`app.listen()`示例迁移后的完整代码。 +- [ ] 在macOS/Linux可用环境验证mode/uid/PID与rename-over replacement；Windows把POSIX mode/uid标为N/A，验证regular non-reparse/single-link、平台commit顺序，以及delete→commit失败由held-lock grace、crash gap由stale阈值覆盖后继handoff，再验证PID、nonce和path semantics；不写“atomic overwrite”伪保证。 +- [ ] 检查 `git diff --check`、production/test line count和 PR template证据；PR1仍不广告 capability。 +- [ ] 按仓库规则做开放式自审；发现问题即修订并重跑验证，直到连续两轮 clean pass。 + +## Focused verification commands + +```bash +cd packages/cli +npx vitest run src/serve/conversations/conversation-runtime-ownership.test.ts +npx vitest run src/serve/conversations/conversation-runtime-activity.test.ts +npx vitest run src/serve/conversations/conversation-runtime-manager.test.ts +npx vitest run src/serve/live/discovery.test.ts +npx vitest run src/serve/live/live-task-service.test.ts +npx vitest run src/serve/live/realtime-startup-context.test.ts +npx vitest run src/serve/live/run-qwen-serve-live.test.ts +npx vitest run src/serve/routes/live.test.ts +npx vitest run src/serve/workspace-registry.test.ts +npx vitest run src/serve/workspace-route-runtime.test.ts +npx vitest run src/serve/acp-http/workspace-qualified-acp.test.ts +npx vitest run src/serve/routes/workspace-qualified-voice.test.ts +npx vitest run src/serve/routes/workspace-qualified-extensions.test.ts +npx vitest run src/serve/routes/workspace-management.test.ts +npx vitest run src/serve/multi-workspace-sessions.test.ts +npx vitest run src/serve/routes/channel-notify.test.ts +npx vitest run src/serve/routes/workspace-channel-management.test.ts +npx vitest run src/serve/routes/workspace-channel-observed-contacts.test.ts +npx vitest run src/serve/routes/scheduled-tasks.test.ts +npx vitest run src/serve/routes/session-runtime.test.ts +npx vitest run src/serve/routes/workspace-trust.test.ts +npx vitest run src/serve/server/telemetry.test.ts +npx vitest run src/serve/server/session-archive.test.ts +npx vitest run src/serve/daemon-status.test.ts +npx vitest run src/serve/serve-app-lifecycle.test.ts +npx vitest run src/serve/server.test.ts +npx vitest run src/serve/run-qwen-serve.test.ts +npx vitest run src/serve/workspace-trust-reconciler.test.ts + +cd ../web-shell +npx vitest run --config vitest.config.ts App.test.tsx +npx vitest run --config vitest.config.ts components/sidebar/WebShellSidebar.workspace-removal.test.tsx +npx vitest run --config vitest.config.ts voice/voice-workspace-target.test.ts + +cd ../.. +npm run format +npm run build +npm run bundle +npm run typecheck +npm run lint +git diff --check +``` + +本地迭代可临时加`-t`；上面的交付命令必须运行完整test file，避免regex遗漏新增用例。 + +## Explicit non-goals + +- 不创建 `StandaloneSessionService`，不新增/迁移 transcript source。 +- 不添加 standalone REST、SDK、WebUI/WebShell feature或 capability；仅做上述既有`kind: "live"` entry的ordinary-selector过滤与Live catalog source-filter兼容，不改变Live catalog UX。 +- 不承诺 old daemon在 new owner之后启动时的 mixed-version互斥。 +- 不引入 daemon-to-daemon proxy、multi-master lease、heartbeat、TTL或网络协调。 +- 不改变`createServeApp`返回类型，不新增与`RunHandle`平行的第二套ownership lifecycle；只导出一个由direct embed和`runQwenServe`共同使用的listener-bound handle/accessor。 +- 不把 Conversations 变成严格 OS sandbox；保留现有 user/global/root config语义。 +- 不重构整个 registry；一个 predicate、default-deny resolver和逐 consumer guard已足够。 +- 不改变 broad parent workspace的文件 containment模型，不在 PR1扩大到通用 filesystem policy。 + +## Exit criteria + +- 两个新版本 daemon并发时，只有一个能 publish/use Conversations runtime；active/PID-reused/compromised owner均 fail closed。 +- dead owner可在固定 grace后恢复；成功 shutdown在完整 drain和 listener确认后安全 handoff，drain/listener proof不完整时不进入owner unlink；exact unlink后的lock cleanup失败按明确post-unlink状态处理。 +- `createServeApp` direct embed可通过公开共享lifecycle安全使用Live/Conversations：未绑定listener时零ownership I/O并fail closed，绑定后无论由handle还是外部server close发起shutdown都进入同一cleanup状态机，且公开await路径能证明drain与release结果。 +- 所有 ordinary workspace HTTP、ACP WS、Voice WS、management和后台 consumer都无法通过 internal ID/cwd寻址该 runtime。 +- owner-routed Live/session行为、health/capabilities兼容、总局 UUID admission、metrics与 shutdown保持工作。 +- 没有任何 failure path回退到 primary runtime，且 `standalone_sessions_v1`仍未出现。 diff --git a/packages/cli/src/serve/acp-http/index.ts b/packages/cli/src/serve/acp-http/index.ts index 3a76a01263c..c25f7c9e892 100644 --- a/packages/cli/src/serve/acp-http/index.ts +++ b/packages/cli/src/serve/acp-http/index.ts @@ -27,6 +27,7 @@ import type { WorkspaceRegistry, WorkspaceRuntime, } from '../workspace-registry.js'; +import { isInternalWorkspaceRuntime } from '../workspace-runtime-visibility.js'; import { isPortableAbsolutePath, resolveManagedWorkspaceRuntimeByPathSelector, @@ -1378,7 +1379,9 @@ export function mountAcpHttp( const getOrCreateSecondaryMount = ( rt: WorkspaceRuntime, ): RuntimeAcpMount | undefined => { - if (rt.primary || !rt.trusted) return undefined; + if (rt.primary || !rt.trusted || isInternalWorkspaceRuntime(rt)) { + return undefined; + } const existing = secondaryMounts.get(rt.workspaceId); if (existing) return existing; const mount = createSecondaryAcpMount(rt); @@ -1678,6 +1681,12 @@ export function mountAcpHttp( socket.destroy(); return; } + if (isInternalWorkspaceRuntime(runtime)) { + logReject(`workspace-mismatch ${logSafe(selector)}`); + socket.write('HTTP/1.1 400 Bad Request\r\n\r\n'); + socket.destroy(); + return; + } if (!runtime.trusted) { logReject(`untrusted-workspace ${runtime.workspaceId}`); socket.write('HTTP/1.1 403 Forbidden\r\n\r\n'); @@ -1730,6 +1739,12 @@ export function mountAcpHttp( socket.destroy(); return; } + if (isInternalWorkspaceRuntime(rt)) { + logReject(`workspace-mismatch ${logSafe(selector)}`); + socket.write('HTTP/1.1 400 Bad Request\r\n\r\n'); + socket.destroy(); + return; + } const entry = wsRegistry.getEntryByWorkspaceId(rt.workspaceId); if ( !entry || diff --git a/packages/cli/src/serve/acp-http/workspace-qualified-acp.test.ts b/packages/cli/src/serve/acp-http/workspace-qualified-acp.test.ts index 87d75ac8670..5a991a6197e 100644 --- a/packages/cli/src/serve/acp-http/workspace-qualified-acp.test.ts +++ b/packages/cli/src/serve/acp-http/workspace-qualified-acp.test.ts @@ -950,7 +950,7 @@ describe('workspace-qualified ACP (/workspaces/:workspace/acp)', () => { expect(res.status).toBe(200); }); - it('reserves Live creation and relocates compatible ACP restores before exposing them', async () => { + it('rejects workspace-qualified ACP access to the Live runtime', async () => { const liveBridge = makeBridge(); const restoreSession = async (req: { sessionId: string }) => ({ sessionId: req.sessionId, @@ -993,246 +993,53 @@ describe('workspace-qualified ACP (/workspaces/:workspace/acp)', () => { }), ); - const rejectedNew = await sendWsRequest('/workspaces/live-id/acp', { - jsonrpc: '2.0', - id: 40, - method: 'session/new', - params: {}, - }); - expect(rejectedNew['error']).toMatchObject({ - code: -32602, - data: { errorKind: 'live_session_creation_reserved' }, - }); - expect(liveBridge.spawnOrAttach).not.toHaveBeenCalled(); - - const rejectedFork = await sendWsRequest('/workspaces/live-id/acp', { - jsonrpc: '2.0', - id: 46, - method: 'session/fork', - params: { sessionId: 'live-session' }, - }); - expect(rejectedFork['error']).toMatchObject({ - code: -32602, - data: { errorKind: 'live_session_creation_reserved' }, - }); - expect(liveBridge.branchSession).not.toHaveBeenCalled(); - - await writeStoredSession('generic-session', '/live-root'); - const restoredProjectless = await sendWsRequest('/workspaces/live-id/acp', { - jsonrpc: '2.0', - id: 41, - method: 'session/load', - params: { sessionId: 'generic-session' }, - }); - expect(restoredProjectless['result']).toMatchObject({}); - expect(materializeLiveConversationDirectory).toHaveBeenCalledWith( - 'generic-session', - ); - expect(loadSession).toHaveBeenCalledWith( - expect.objectContaining({ - sessionId: 'generic-session', - workspaceCwd: '/live-root', - }), - ); - expect(changeSessionCwd).toHaveBeenCalledWith('generic-session', { - path: '/live-root/conversation-generic-session', - allowedRoots: ['/live-root'], - managedRelocation: 'live-conversation', - }); - materializeLiveConversationDirectory.mockClear(); - loadSession.mockClear(); - changeSessionCwd.mockClear(); - - const foreignWorkspace = await fsp.mkdtemp( - path.join(os.tmpdir(), 'qwen-live-foreign-workspace-'), - ); - try { - await writeStoredSession('foreign-live-session', foreignWorkspace, { - sourceType: 'default', - sourceId: 'realtime_voice:p1:h1:a1:foreign-call', + for (const route of [ + '/workspaces/live-id/acp', + '/workspaces/%2Flive-root/acp', + ]) { + const response = await postInitialize(route); + expect(response.status).toBe(400); + await expect(response.json()).resolves.toMatchObject({ + code: 'workspace_mismatch', }); - const rejectedForeignLoad = await sendWsRequest( - '/workspaces/live-id/acp', - { - jsonrpc: '2.0', - id: 47, - method: 'session/load', - params: { - sessionId: 'foreign-live-session', - cwd: foreignWorkspace, - }, - }, - ); - expect(rejectedForeignLoad['error']).toMatchObject({ code: -32602 }); - expect(materializeLiveConversationDirectory).not.toHaveBeenCalled(); - expect(loadSession).not.toHaveBeenCalled(); - } finally { - await fsp.rm(foreignWorkspace, { recursive: true, force: true }); } - await writeStoredSession('live-session', '/live-root', { - sourceType: 'default', - sourceId: 'realtime_voice:p1:h1:a1:call-1', - }); - const restored = await sendWsRequest('/workspaces/live-id/acp', { - jsonrpc: '2.0', - id: 42, - method: 'session/load', - params: { sessionId: 'live-session' }, - }); - expect(restored['result']).toMatchObject({}); - expect(materializeLiveConversationDirectory).toHaveBeenCalledWith( - 'live-session', - ); - expect(loadSession).toHaveBeenCalledWith( - expect.objectContaining({ - sessionId: 'live-session', - workspaceCwd: '/live-root', - sourceType: 'default', - sourceId: 'realtime_voice:p1:h1:a1:call-1', + await expect( + sendWsRequest('/workspaces/live-id/acp', { + jsonrpc: '2.0', + id: 40, + method: 'session/load', + params: { sessionId: 'live-session' }, }), - ); - expect(changeSessionCwd).toHaveBeenCalledWith('live-session', { - path: '/live-root/conversation-live-session', - allowedRoots: ['/live-root'], - managedRelocation: 'live-conversation', - }); - expect( - materializeLiveConversationDirectory.mock.invocationCallOrder[0], - ).toBeLessThan(loadSession.mock.invocationCallOrder[0]!); - expect(loadSession.mock.invocationCallOrder[0]).toBeLessThan( - changeSessionCwd.mock.invocationCallOrder[0]!, - ); - - activeLiveSessionIds.add('live-session'); - const blockedClose = await new Promise>( - (resolve, reject) => { - const ws = new WebSocket( - `ws://127.0.0.1:${port}/workspaces/live-id/acp`, - { handshakeTimeout: 2000 }, - ); - ws.on('open', () => ws.send(INITIALIZE)); - ws.on('message', (data: WebSocket.RawData) => { - const message = JSON.parse(data.toString()) as Record< - string, - unknown - >; - if (message['id'] === 1) { - ws.send( - JSON.stringify({ - jsonrpc: '2.0', - id: 2, - method: 'session/load', - params: { sessionId: 'live-session' }, - }), - ); - } else if (message['id'] === 2) { - ws.send( - JSON.stringify({ - jsonrpc: '2.0', - id: 3, - method: 'session/close', - params: { sessionId: 'live-session' }, - }), - ); - } else if (message['id'] === 3) { - ws.close(); - resolve(message); - } - }); - ws.on('error', reject); - }, - ); - expect(blockedClose['error']).toMatchObject({ - code: -32600, - data: { - errorKind: 'live_session_active', - httpStatus: 409, - sessionId: 'live-session', - }, - }); - for (const [id, method] of [ - [48, '_qwen/sessions/archive'], - [49, '_qwen/sessions/delete'], - ] as const) { - const blocked = await sendWsRequest('/workspaces/live-id/acp', { + ).rejects.toThrow('Unexpected server response: 400'); + await expect( + sendWsRequest('/workspaces/%2Flive-root/acp', { jsonrpc: '2.0', - id, - method, - params: { sessionIds: ['live-session'] }, - }); - expect(blocked['error']).toMatchObject({ - code: -32600, - data: { - errorKind: 'live_session_active', - httpStatus: 409, - sessionId: 'live-session', - }, - }); - } - expect(liveBridge.closeSession).not.toHaveBeenCalled(); - activeLiveSessionIds.delete('live-session'); - - await writeStoredSession('live-resume', '/live-root', { - sourceType: 'default', - sourceId: 'realtime_voice:p1:h1:a1:call-2', - }); - const resumed = await sendWsRequest('/workspaces/live-id/acp', { - jsonrpc: '2.0', - id: 43, - method: 'session/resume', - params: { sessionId: 'live-resume' }, - }); - expect(resumed['result']).toMatchObject({}); - expect(resumeSession).toHaveBeenCalledWith( - expect.objectContaining({ - sessionId: 'live-resume', - workspaceCwd: '/live-root', - sourceType: 'default', - sourceId: 'realtime_voice:p1:h1:a1:call-2', + id: 41, + method: 'session/load', + params: { sessionId: 'live-session' }, }), - ); - expect(changeSessionCwd).toHaveBeenCalledWith('live-resume', { - path: '/live-root/conversation-live-resume', - allowedRoots: ['/live-root'], - managedRelocation: 'live-conversation', - }); - - await writeStoredSession('live-active', '/live-root', { - sourceType: 'default', - sourceId: 'realtime_voice:p1:h1:a1:call-active', - }); - const active = await sendWsRequest('/workspaces/live-id/acp', { - jsonrpc: '2.0', - id: 44, - method: 'session/load', - params: { sessionId: 'live-active' }, - }); - expect(active['result']).toMatchObject({}); - expect(changeSessionCwd).not.toHaveBeenCalledWith( - 'live-active', - expect.anything(), - ); - - await writeStoredSession('live-active-root', '/live-root', { - sourceType: 'default', - sourceId: 'realtime_voice:p1:h1:a1:call-active-root', - }); - const activeAtRoot = await sendWsRequest('/workspaces/live-id/acp', { - jsonrpc: '2.0', - id: 45, - method: 'session/load', - params: { sessionId: 'live-active-root' }, - }); - expect(activeAtRoot['error']).toMatchObject({ code: -32603 }); - expect(liveBridge.detachClient).toHaveBeenCalledWith( - 'live-active-root', - 'live-client', - ); - expect(liveBridge.killSession).not.toHaveBeenCalledWith( - 'live-active-root', - expect.anything(), - ); + ).rejects.toThrow('Unexpected server response: 400'); + for (const route of [ + '/workspaces/live-id/voice/stream', + '/workspaces/%2Flive-root/voice/stream', + ]) { + await expect( + new Promise((resolve, reject) => { + const ws = new WebSocket(`ws://127.0.0.1:${port}${route}`, { + handshakeTimeout: 2_000, + }); + ws.on('open', () => { + ws.close(); + resolve(); + }); + ws.on('error', reject); + }), + ).rejects.toThrow('Unexpected server response: 400'); + } + expect(liveBridge.spawnOrAttach).not.toHaveBeenCalled(); + expect(loadSession).not.toHaveBeenCalled(); + expect(resumeSession).not.toHaveBeenCalled(); }); it('forwards unexpected legacy POST failures to Express', async () => { diff --git a/packages/cli/src/serve/conversations/conversation-runtime-activity.test.ts b/packages/cli/src/serve/conversations/conversation-runtime-activity.test.ts new file mode 100644 index 00000000000..f5f6e5fd778 --- /dev/null +++ b/packages/cli/src/serve/conversations/conversation-runtime-activity.test.ts @@ -0,0 +1,48 @@ +/** + * @license + * Copyright 2026 Qwen Team + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, expect, it } from 'vitest'; +import { ConversationRuntimeActivityGate } from './conversation-runtime-activity.js'; + +describe('ConversationRuntimeActivityGate', () => { + it('waits for admitted work and rejects work after sealing', async () => { + const gate = new ConversationRuntimeActivityGate(); + let finish!: () => void; + const task = gate.run( + () => + new Promise((resolve) => { + finish = resolve; + }), + ); + const drain = gate.sealAndWait(); + + await expect(gate.run(async () => undefined)).rejects.toMatchObject({ + code: 'daemon_draining', + }); + let drained = false; + void drain.then(() => { + drained = true; + }); + await Promise.resolve(); + expect(drained).toBe(false); + + finish(); + await task; + await drain; + expect(drained).toBe(true); + }); + + it('releases activity when a task fails', async () => { + const gate = new ConversationRuntimeActivityGate(); + + await expect( + gate.run(async () => { + throw new Error('failed'); + }), + ).rejects.toThrow('failed'); + await expect(gate.sealAndWait()).resolves.toBeUndefined(); + }); +}); diff --git a/packages/cli/src/serve/conversations/conversation-runtime-activity.ts b/packages/cli/src/serve/conversations/conversation-runtime-activity.ts new file mode 100644 index 00000000000..b241ed6642d --- /dev/null +++ b/packages/cli/src/serve/conversations/conversation-runtime-activity.ts @@ -0,0 +1,43 @@ +/** + * @license + * Copyright 2026 Qwen Team + * SPDX-License-Identifier: Apache-2.0 + */ + +export class ConversationRuntimeActivityGate { + private sealed = false; + private active = 0; + private drain?: { promise: Promise; resolve: () => void }; + + async run(task: () => Promise): Promise { + if (this.sealed) { + throw Object.assign( + new Error('The daemon is draining and no longer accepts work.'), + { code: 'daemon_draining' }, + ); + } + this.active++; + try { + return await task(); + } finally { + this.active--; + if (this.active === 0) { + this.drain?.resolve(); + this.drain = undefined; + } + } + } + + sealAndWait(): Promise { + this.sealed = true; + if (this.active === 0) return Promise.resolve(); + if (!this.drain) { + let resolve!: () => void; + const promise = new Promise((done) => { + resolve = done; + }); + this.drain = { promise, resolve }; + } + return this.drain.promise; + } +} diff --git a/packages/cli/src/serve/conversations/conversation-runtime-errors.ts b/packages/cli/src/serve/conversations/conversation-runtime-errors.ts new file mode 100644 index 00000000000..eddb17aee93 --- /dev/null +++ b/packages/cli/src/serve/conversations/conversation-runtime-errors.ts @@ -0,0 +1,73 @@ +/** + * @license + * Copyright 2026 Qwen Team + * SPDX-License-Identifier: Apache-2.0 + */ + +export type ConversationRuntimeOwnershipErrorCode = + | 'conversation_runtime_in_use' + | 'conversation_runtime_unavailable' + | 'conversation_root_compromised' + | 'conversation_runtime_ownership_compromised'; + +const DEFAULT_MESSAGES: Record = + { + conversation_runtime_in_use: + 'The Conversations runtime is owned by another daemon.', + conversation_runtime_unavailable: + 'The Conversations runtime is temporarily unavailable.', + conversation_root_compromised: + 'The Conversations root could not be verified.', + conversation_runtime_ownership_compromised: + 'The Conversations runtime ownership state could not be verified.', + }; + +export class ConversationRuntimeOwnershipError extends Error { + readonly status = 503; + + constructor( + readonly code: ConversationRuntimeOwnershipErrorCode, + readonly retryable: boolean, + options?: { cause?: unknown }, + ) { + super(DEFAULT_MESSAGES[code], options); + this.name = 'ConversationRuntimeOwnershipError'; + } +} + +export function conversationRuntimeInUseError(): ConversationRuntimeOwnershipError { + return new ConversationRuntimeOwnershipError( + 'conversation_runtime_in_use', + true, + ); +} + +export function conversationRuntimeUnavailableError( + cause?: unknown, +): ConversationRuntimeOwnershipError { + return new ConversationRuntimeOwnershipError( + 'conversation_runtime_unavailable', + true, + { cause }, + ); +} + +export function conversationRuntimeOwnershipCompromisedError( + cause?: unknown, +): ConversationRuntimeOwnershipError { + return new ConversationRuntimeOwnershipError( + 'conversation_runtime_ownership_compromised', + false, + { cause }, + ); +} + +export function conversationRootCompromisedError( + cause?: unknown, +): ConversationRuntimeOwnershipError { + return new ConversationRuntimeOwnershipError( + 'conversation_root_compromised', + false, + { cause }, + ); +} diff --git a/packages/cli/src/serve/conversations/conversation-runtime-manager.test.ts b/packages/cli/src/serve/conversations/conversation-runtime-manager.test.ts index 2bc70e2f3e2..add5a781495 100644 --- a/packages/cli/src/serve/conversations/conversation-runtime-manager.test.ts +++ b/packages/cli/src/serve/conversations/conversation-runtime-manager.test.ts @@ -14,7 +14,11 @@ import { type WorkspaceRuntime, } from '../workspace-registry.js'; import type { ConversationWorkspace } from './conversation-workspace.js'; -import { ConversationRuntimeManager } from './conversation-runtime-manager.js'; +import type { ConversationRuntimeOwnership } from './conversation-runtime-ownership.js'; +import { + ConversationRuntimeManager, + type ConversationRuntimeManagerOptions, +} from './conversation-runtime-manager.js'; const root = { configuredRoot: '/work/conversations', @@ -96,7 +100,46 @@ function createOwnedRuntime(bridge = createBridge()): WorkspaceRuntime { }); } +function createManager( + options: Omit & { + ownership?: ConversationRuntimeOwnership; + }, +): ConversationRuntimeManager { + return new ConversationRuntimeManager({ + ...options, + ownership: options.ownership ?? { + acquire: vi.fn(async () => ({ reclaimed: false })), + release: vi.fn(async () => false), + }, + }); +} + describe('ConversationRuntimeManager', () => { + it('acquires ownership before touching the root or registry', async () => { + const workspace = createWorkspace(); + const registry = createRegistry(); + const publishRuntime = vi.fn(); + const ownershipError = new Error('owner unavailable'); + const ownership = { + acquire: vi.fn(async () => Promise.reject(ownershipError)), + release: vi.fn(async () => false), + }; + const manager = createManager({ + workspace, + registry, + publishRuntime, + ownership, + }); + + await expect(manager.ensure()).rejects.toBe(ownershipError); + expect(workspace.revalidate).not.toHaveBeenCalled(); + expect(workspace.assertExactRoot).not.toHaveBeenCalled(); + expect(publishRuntime).not.toHaveBeenCalled(); + expect( + registry.getManagedEntryByWorkspaceCwd(root.canonicalRoot), + ).toBeUndefined(); + }); + it('one-flights publication and revalidates cached reuse without preheating or binding Live', async () => { const workspace = createWorkspace(); const registry = createRegistry(); @@ -112,7 +155,7 @@ describe('ConversationRuntimeManager', () => { registry.add(candidate); return candidate; }); - const manager = new ConversationRuntimeManager({ + const manager = createManager({ workspace, registry, publishRuntime, @@ -140,7 +183,7 @@ describe('ConversationRuntimeManager', () => { const registry = createRegistry(candidate); const workspace = createWorkspace(); const publishRuntime = vi.fn(); - const manager = new ConversationRuntimeManager({ + const manager = createManager({ workspace, registry, publishRuntime, @@ -163,13 +206,16 @@ describe('ConversationRuntimeManager', () => { const registry = createRegistry(); registry.add(candidate); const publishRuntime = vi.fn(); - const manager = new ConversationRuntimeManager({ + const manager = createManager({ workspace: createWorkspace(), registry, publishRuntime, }); - await expect(manager.ensure()).rejects.toThrow(/without Live provenance/); + await expect(manager.ensure()).rejects.toMatchObject({ + code: 'conversation_root_compromised', + retryable: false, + }); expect(publishRuntime).not.toHaveBeenCalled(); }); @@ -182,13 +228,16 @@ describe('ConversationRuntimeManager', () => { return root; }); const publishRuntime = vi.fn(); - const manager = new ConversationRuntimeManager({ + const manager = createManager({ workspace, registry, publishRuntime, }); - await expect(manager.ensure()).rejects.toThrow(/no longer an active/); + await expect(manager.ensure()).rejects.toMatchObject({ + code: 'conversation_runtime_unavailable', + retryable: true, + }); expect(publishRuntime).not.toHaveBeenCalled(); }); @@ -248,13 +297,16 @@ describe('ConversationRuntimeManager', () => { }, ])('rejects an existing runtime with invalid $name', async ({ runtime }) => { const publishRuntime = vi.fn(); - const manager = new ConversationRuntimeManager({ + const manager = createManager({ workspace: createWorkspace(), registry: createRegistry(runtime), publishRuntime, }); - await expect(manager.ensure()).rejects.toThrow(/without Live provenance/); + await expect(manager.ensure()).rejects.toMatchObject({ + code: 'conversation_root_compromised', + retryable: false, + }); expect(publishRuntime).not.toHaveBeenCalled(); }); @@ -263,7 +315,7 @@ describe('ConversationRuntimeManager', () => { async (state) => { const candidate = createOwnedRuntime(); const registry = createRegistry(candidate); - const entry = registry.getEntryByWorkspaceCwd(root.canonicalRoot)!; + const entry = registry.getManagedEntryByWorkspaceCwd(root.canonicalRoot)!; if (state === 'draining') { registry.beginDrain(candidate); } else { @@ -271,13 +323,16 @@ describe('ConversationRuntimeManager', () => { registry.blockReplacement(entry, 'blocked'); } const publishRuntime = vi.fn(); - const manager = new ConversationRuntimeManager({ + const manager = createManager({ workspace: createWorkspace(), registry, publishRuntime, }); - await expect(manager.ensure()).rejects.toThrow(/no longer an active/); + await expect(manager.ensure()).rejects.toMatchObject({ + code: 'conversation_runtime_unavailable', + retryable: true, + }); expect(publishRuntime).not.toHaveBeenCalled(); }, ); @@ -286,7 +341,7 @@ describe('ConversationRuntimeManager', () => { const candidate = createOwnedRuntime(); const registry = createRegistry(candidate); const publishRuntime = vi.fn(); - const manager = new ConversationRuntimeManager({ + const manager = createManager({ workspace: createWorkspace(), registry, publishRuntime, @@ -295,7 +350,10 @@ describe('ConversationRuntimeManager', () => { registry.beginDrain(candidate); registry.completeDrain(candidate); - await expect(manager.ensure()).rejects.toThrow(/no longer an active/); + await expect(manager.ensure()).rejects.toMatchObject({ + code: 'conversation_runtime_unavailable', + retryable: true, + }); expect(publishRuntime).not.toHaveBeenCalled(); }); @@ -307,7 +365,7 @@ describe('ConversationRuntimeManager', () => { registry.add(candidate); return candidate; }); - const manager = new ConversationRuntimeManager({ + const manager = createManager({ workspace: createWorkspace(), registry, publishRuntime, @@ -316,7 +374,10 @@ describe('ConversationRuntimeManager', () => { registry.beginDrain(candidate); registry.completeDrain(candidate); - await expect(manager.ensure()).rejects.toThrow(/no longer an active/); + await expect(manager.ensure()).rejects.toMatchObject({ + code: 'conversation_runtime_unavailable', + retryable: true, + }); expect(publishRuntime).toHaveBeenCalledOnce(); }); @@ -325,17 +386,20 @@ describe('ConversationRuntimeManager', () => { const replacement = createOwnedRuntime(); const registry = createRegistry(candidate); const publishRuntime = vi.fn(); - const manager = new ConversationRuntimeManager({ + const manager = createManager({ workspace: createWorkspace(), registry, publishRuntime, }); await manager.ensure(); - const entry = registry.getEntryByWorkspaceCwd(root.canonicalRoot)!; + const entry = registry.getManagedEntryByWorkspaceCwd(root.canonicalRoot)!; registry.beginReplacement(entry, 'next'); registry.activateReplacement(entry, replacement, 'next'); - await expect(manager.ensure()).rejects.toThrow(/no longer an active/); + await expect(manager.ensure()).rejects.toMatchObject({ + code: 'conversation_runtime_unavailable', + retryable: true, + }); expect(publishRuntime).not.toHaveBeenCalled(); }); @@ -408,7 +472,7 @@ describe('ConversationRuntimeManager', () => { 'rejects a publication candidate with invalid $name', async ({ runtime }) => { const registry = createRegistry(); - const manager = new ConversationRuntimeManager({ + const manager = createManager({ workspace: createWorkspace(), registry, publishRuntime: async (_cwd, validate) => { @@ -417,7 +481,10 @@ describe('ConversationRuntimeManager', () => { }, }); - await expect(manager.ensure()).rejects.toThrow(/ownership gate/); + await expect(manager.ensure()).rejects.toMatchObject({ + code: 'conversation_root_compromised', + retryable: false, + }); expect(registry.getByWorkspaceCwd(root.canonicalRoot)).toBeUndefined(); }, ); @@ -445,13 +512,16 @@ describe('ConversationRuntimeManager', () => { registry.add(candidate); return candidate; }); - const manager = new ConversationRuntimeManager({ + const manager = createManager({ workspace, registry, publishRuntime, }); - await expect(manager.ensure()).rejects.toThrow(/exact/); + await expect(manager.ensure()).rejects.toMatchObject({ + code: 'conversation_root_compromised', + retryable: false, + }); await expect(manager.ensure()).resolves.toBe(candidate); expect(registry.getByWorkspaceCwd('/work/wrong')).toBeUndefined(); expect(publishRuntime).toHaveBeenCalledTimes(2); @@ -482,7 +552,7 @@ describe('ConversationRuntimeManager', () => { registry.add(candidate); return candidate; }); - const manager = new ConversationRuntimeManager({ + const manager = createManager({ workspace, registry, publishRuntime, @@ -512,13 +582,16 @@ describe('ConversationRuntimeManager', () => { registry.add(second); return second; }); - const manager = new ConversationRuntimeManager({ + const manager = createManager({ workspace: createWorkspace(), registry, publishRuntime, }); - await expect(manager.ensure()).rejects.toThrow(/no longer an active/); + await expect(manager.ensure()).rejects.toMatchObject({ + code: 'conversation_runtime_unavailable', + retryable: true, + }); registry.completeDrain(first); await expect(manager.ensure()).resolves.toBe(second); expect(publishRuntime).toHaveBeenCalledTimes(2); @@ -539,14 +612,20 @@ describe('ConversationRuntimeManager', () => { registry.add(candidate); return candidate; }); - const manager = new ConversationRuntimeManager({ + const manager = createManager({ workspace, registry, publishRuntime, }); - await expect(manager.ensure()).rejects.toThrow('root unavailable'); - await expect(manager.ensure()).rejects.toThrow('publication unavailable'); + await expect(manager.ensure()).rejects.toMatchObject({ + code: 'conversation_root_compromised', + retryable: false, + }); + await expect(manager.ensure()).rejects.toMatchObject({ + code: 'conversation_runtime_unavailable', + retryable: true, + }); await expect(manager.ensure()).resolves.toBe(candidate); expect(publishRuntime).toHaveBeenCalledTimes(2); }); diff --git a/packages/cli/src/serve/conversations/conversation-runtime-manager.ts b/packages/cli/src/serve/conversations/conversation-runtime-manager.ts index ea832b5fae3..35b91787400 100644 --- a/packages/cli/src/serve/conversations/conversation-runtime-manager.ts +++ b/packages/cli/src/serve/conversations/conversation-runtime-manager.ts @@ -5,12 +5,19 @@ */ import type { ConversationWorkspace } from './conversation-workspace.js'; +import type { ConversationRuntimeOwnership } from './conversation-runtime-ownership.js'; +import { + ConversationRuntimeOwnershipError, + conversationRootCompromisedError, + conversationRuntimeUnavailableError, +} from './conversation-runtime-errors.js'; import type { WorkspaceRegistry, WorkspaceRuntime, } from '../workspace-registry.js'; export interface ConversationRuntimeManagerOptions { + ownership: ConversationRuntimeOwnership; workspace: Pick; registry: WorkspaceRegistry; publishRuntime: ( @@ -35,80 +42,87 @@ export class ConversationRuntimeManager { } private async ensureOnce(): Promise { - const root = await this.options.workspace.revalidate(); + await this.options.ownership.acquire(); + const root = await this.revalidateRoot(); if (this.runtime) { - await this.options.workspace.assertExactRoot(this.runtime.workspaceCwd); - this.assertActiveRuntime( - root.canonicalRoot, - this.runtime, - 'Live conversation runtime is no longer an active owned runtime.', - ); + await this.assertExactRoot(this.runtime.workspaceCwd); + this.assertActiveRuntime(root.canonicalRoot, this.runtime); return this.runtime; } - const entry = this.options.registry.getEntryByWorkspaceCwd( + const entry = this.options.registry.getManagedEntryByWorkspaceCwd( root.canonicalRoot, ); if (entry) { const existing = entry.current?.runtime; if (entry.state !== 'active' || !existing) { - throw new Error( - 'Live conversation runtime is no longer an active owned runtime.', - ); + throw conversationRuntimeUnavailableError(); } - this.assertOwnedRuntime( - existing, - 'Live conversation root is already registered without Live provenance.', - ); - await this.options.workspace.assertExactRoot(existing.workspaceCwd); - this.assertActiveRuntime( - root.canonicalRoot, - existing, - 'Live conversation runtime is no longer an active owned runtime.', - ); + this.assertOwnedRuntime(existing); + await this.assertExactRoot(existing.workspaceCwd); + this.assertActiveRuntime(root.canonicalRoot, existing); this.runtime = existing; return existing; } - const created = await this.options.publishRuntime( - root.canonicalRoot, - async (candidate) => { - await this.options.workspace.assertExactRoot(candidate.workspaceCwd); - this.assertOwnedRuntime( - candidate, - 'Live conversation runtime failed its ownership gate.', - ); - }, - ); - this.assertActiveRuntime( - root.canonicalRoot, - created, - 'Live conversation runtime is no longer an active owned runtime.', - ); + let created: WorkspaceRuntime; + try { + created = await this.options.publishRuntime( + root.canonicalRoot, + async (candidate) => { + await this.assertExactRoot(candidate.workspaceCwd); + this.assertOwnedRuntime(candidate); + }, + ); + } catch (error) { + if (error instanceof ConversationRuntimeOwnershipError) throw error; + throw conversationRuntimeUnavailableError(error); + } + this.assertActiveRuntime(root.canonicalRoot, created); this.runtime = created; return created; } + private async revalidateRoot(): Promise< + Awaited< + ReturnType + > + > { + try { + return await this.options.workspace.revalidate(); + } catch (error) { + throw conversationRootCompromisedError(error); + } + } + + private async assertExactRoot(candidate: string): Promise { + try { + await this.options.workspace.assertExactRoot(candidate); + } catch (error) { + throw conversationRootCompromisedError(error); + } + } + private assertActiveRuntime( canonicalRoot: string, runtime: WorkspaceRuntime, - message: string, ): void { - this.assertOwnedRuntime(runtime, message); - const entry = this.options.registry.getEntryByWorkspaceCwd(canonicalRoot); + this.assertOwnedRuntime(runtime); + const entry = + this.options.registry.getManagedEntryByWorkspaceCwd(canonicalRoot); if (entry?.state !== 'active' || entry.current?.runtime !== runtime) { - throw new Error(message); + throw conversationRuntimeUnavailableError(); } } - private assertOwnedRuntime(runtime: WorkspaceRuntime, message: string): void { + private assertOwnedRuntime(runtime: WorkspaceRuntime): void { if ( runtime.primary || runtime.provenance !== 'live-conversation' || !runtime.trusted || runtime.removable !== false ) { - throw new Error(message); + throw conversationRootCompromisedError(); } } } diff --git a/packages/cli/src/serve/conversations/conversation-runtime-ownership.test.ts b/packages/cli/src/serve/conversations/conversation-runtime-ownership.test.ts new file mode 100644 index 00000000000..27ed79e631e --- /dev/null +++ b/packages/cli/src/serve/conversations/conversation-runtime-ownership.test.ts @@ -0,0 +1,663 @@ +/** + * @license + * Copyright 2026 Qwen Team + * SPDX-License-Identifier: Apache-2.0 + */ + +import * as fs from 'node:fs/promises'; +import * as os from 'node:os'; +import * as path from 'node:path'; +import { fork, type ChildProcess } from 'node:child_process'; +import { pathToFileURL } from 'node:url'; +import { afterEach, describe, expect, it, vi } from 'vitest'; +import { + createConversationRuntimeOwnership, + getConversationRuntimeOwnerPath, +} from './conversation-runtime-ownership.js'; +import { ConversationRuntimeOwnershipError } from './conversation-runtime-errors.js'; +import { + getLiveDiscoveryPath, + writeLiveDiscoveryFile, +} from '../live/discovery.js'; +import { LIVE_HOST_PROTOCOL_VERSION } from '../live/types.js'; + +const temporaryDirectories: string[] = []; +const childProcesses = new Set(); +const currentNonce = 'conversation_owner_nonce_current_01'; + +async function temporaryStableBase(): Promise { + const root = await fs.mkdtemp( + path.join(os.tmpdir(), 'qwen-conversation-owner-'), + ); + temporaryDirectories.push(root); + return path.join(root, '.qwen'); +} + +async function readRecord(stableBaseDir: string) { + return JSON.parse( + await fs.readFile(getConversationRuntimeOwnerPath(stableBaseDir), 'utf8'), + ) as { version: number; pid: number; instanceNonce: string }; +} + +afterEach(async () => { + for (const child of childProcesses) { + child.kill('SIGKILL'); + } + childProcesses.clear(); + await Promise.all( + temporaryDirectories + .splice(0) + .map((directory) => fs.rm(directory, { recursive: true, force: true })), + ); +}); + +interface OwnershipWorkerMessage { + ok: boolean; + result?: { reclaimed: boolean; elapsedMs: number }; + error?: { code?: string; message: string }; +} + +function waitForWorkerMessage( + child: ChildProcess, +): Promise { + return new Promise((resolve, reject) => { + const cleanup = () => { + child.off('error', onError); + child.off('exit', onExit); + child.off('message', onMessage); + }; + const onError = (error: Error) => { + cleanup(); + reject(error); + }; + const onExit = (code: number | null, signal: NodeJS.Signals | null) => { + cleanup(); + reject( + new Error( + `Ownership worker exited before reporting (code=${String(code)}, signal=${String(signal)}).`, + ), + ); + }; + const onMessage = (message: unknown) => { + cleanup(); + resolve(message as OwnershipWorkerMessage); + }; + child.once('error', onError); + child.once('exit', onExit); + child.once('message', onMessage); + }); +} + +describe('Conversation runtime ownership', () => { + it('arbitrates real child processes and reclaims a crashed owner after grace', async () => { + const stableBaseDir = await temporaryStableBase(); + const workerPath = path.join( + path.dirname(stableBaseDir), + 'owner-worker.mjs', + ); + const moduleUrl = pathToFileURL( + path.resolve('src/serve/conversations/conversation-runtime-ownership.ts'), + ).href; + await fs.writeFile( + workerPath, + `import { createConversationRuntimeOwnership } from ${JSON.stringify(moduleUrl)}; +const [stableBaseDir, instanceNonce, mode] = process.argv.slice(2); +const ownership = createConversationRuntimeOwnership({ + stableBaseDir, + pid: process.pid, + instanceNonce, + handoffGraceMs: 120, +}); +const startedAt = Date.now(); +try { + const acquired = await ownership.acquire(); + process.send?.({ ok: true, result: { ...acquired, elapsedMs: Date.now() - startedAt } }); + if (mode === 'hold') { + process.on('message', async (message) => { + if (message === 'release') { + await ownership.release(); + process.exit(0); + } + }); + } else { + await ownership.release(); + } +} catch (error) { + process.send?.({ + ok: false, + error: { + code: typeof error === 'object' && error !== null ? error.code : undefined, + message: error instanceof Error ? error.message : String(error), + }, + }); +} +`, + { mode: 0o600 }, + ); + const spawnWorker = (nonce: string, mode: 'hold' | 'once') => { + const child = fork(workerPath, [stableBaseDir, nonce, mode], { + execArgv: ['--import', 'tsx'], + stdio: ['ignore', 'ignore', 'inherit', 'ipc'], + }); + childProcesses.add(child); + return child; + }; + + const holder = spawnWorker('conversation_owner_child_holder_01', 'hold'); + await expect(waitForWorkerMessage(holder)).resolves.toMatchObject({ + ok: true, + result: { reclaimed: false }, + }); + + const contender = spawnWorker( + 'conversation_owner_child_contender_01', + 'once', + ); + await expect(waitForWorkerMessage(contender)).resolves.toEqual({ + ok: false, + error: { + code: 'conversation_runtime_in_use', + message: 'The Conversations runtime is owned by another daemon.', + }, + }); + + const holderExited = new Promise((resolve) => { + holder.once('exit', () => resolve()); + }); + holder.kill('SIGKILL'); + await holderExited; + childProcesses.delete(holder); + + const successor = spawnWorker( + 'conversation_owner_child_successor_01', + 'once', + ); + const successorMessage = await waitForWorkerMessage(successor); + expect(successorMessage).toMatchObject({ + ok: true, + result: { reclaimed: true }, + }); + expect(successorMessage.result?.elapsedMs).toBeGreaterThanOrEqual(100); + }); + + it('creates one exact private record and makes acquire/release idempotent', async () => { + const stableBaseDir = await temporaryStableBase(); + const ownership = createConversationRuntimeOwnership({ + stableBaseDir, + pid: process.pid, + instanceNonce: currentNonce, + }); + + await expect(ownership.acquire()).resolves.toEqual({ reclaimed: false }); + await expect(ownership.acquire()).resolves.toEqual({ reclaimed: false }); + expect(await readRecord(stableBaseDir)).toEqual({ + version: 1, + pid: process.pid, + instanceNonce: currentNonce, + }); + + if (process.platform !== 'win32') { + const directory = path.dirname( + getConversationRuntimeOwnerPath(stableBaseDir), + ); + expect((await fs.stat(directory)).mode & 0o777).toBe(0o700); + expect( + (await fs.stat(getConversationRuntimeOwnerPath(stableBaseDir))).mode & + 0o777, + ).toBe(0o600); + } + + await expect(ownership.release()).resolves.toBe(true); + await expect(ownership.release()).resolves.toBe(false); + await expect( + fs.stat(getConversationRuntimeOwnerPath(stableBaseDir)), + ).rejects.toMatchObject({ code: 'ENOENT' }); + }); + + it('one-flights concurrent acquisition', async () => { + const stableBaseDir = await temporaryStableBase(); + const wait = vi.fn(async () => undefined); + const ownership = createConversationRuntimeOwnership({ + stableBaseDir, + pid: process.pid, + instanceNonce: currentNonce, + wait, + }); + + const first = ownership.acquire(); + const second = ownership.acquire(); + expect(second).toBe(first); + await expect(first).resolves.toEqual({ reclaimed: false }); + expect(wait).not.toHaveBeenCalled(); + }); + + it('fails closed for an active foreign owner and permits a later retry', async () => { + const stableBaseDir = await temporaryStableBase(); + const previous = createConversationRuntimeOwnership({ + stableBaseDir, + pid: 999_998, + instanceNonce: 'conversation_owner_nonce_previous_01', + }); + await previous.acquire(); + const isProcessAlive = vi.fn(() => true); + const ownership = createConversationRuntimeOwnership({ + stableBaseDir, + pid: process.pid, + instanceNonce: currentNonce, + isProcessAlive, + }); + + const first = ownership.acquire(); + await expect(first).rejects.toMatchObject({ + code: 'conversation_runtime_in_use', + retryable: true, + }); + expect(isProcessAlive).toHaveBeenCalledWith(999_998); + expect((await readRecord(stableBaseDir)).instanceNonce).toBe( + 'conversation_owner_nonce_previous_01', + ); + + isProcessAlive.mockReturnValue(false); + await expect(ownership.acquire()).resolves.toEqual({ reclaimed: true }); + expect((await readRecord(stableBaseDir)).instanceNonce).toBe(currentNonce); + }); + + it('treats the same pid with a different nonce as active PID reuse', async () => { + const stableBaseDir = await temporaryStableBase(); + const previous = createConversationRuntimeOwnership({ + stableBaseDir, + pid: process.pid, + instanceNonce: 'conversation_owner_nonce_previous_03', + }); + await previous.acquire(); + const isProcessAlive = vi.fn(() => false); + const ownership = createConversationRuntimeOwnership({ + stableBaseDir, + pid: process.pid, + instanceNonce: currentNonce, + isProcessAlive, + }); + + await expect(ownership.acquire()).rejects.toMatchObject({ + code: 'conversation_runtime_in_use', + }); + expect(isProcessAlive).not.toHaveBeenCalled(); + expect((await readRecord(stableBaseDir)).instanceNonce).toBe( + 'conversation_owner_nonce_previous_03', + ); + }); + + it('reclaims a dead foreign owner only after exactly one drain grace', async () => { + const stableBaseDir = await temporaryStableBase(); + const previous = createConversationRuntimeOwnership({ + stableBaseDir, + pid: 999_999, + instanceNonce: 'conversation_owner_nonce_previous_02', + isProcessAlive: () => false, + }); + await previous.acquire(); + const wait = vi.fn(async () => undefined); + const ownership = createConversationRuntimeOwnership({ + stableBaseDir, + pid: process.pid, + instanceNonce: currentNonce, + isProcessAlive: () => false, + wait, + handoffGraceMs: 37, + }); + + await expect(ownership.acquire()).resolves.toEqual({ reclaimed: true }); + expect(wait).toHaveBeenCalledOnce(); + expect(wait).toHaveBeenCalledWith(37); + await ownership.acquire(); + expect(wait).toHaveBeenCalledOnce(); + }); + + it('rejects an active foreign legacy Live owner before writing its record', async () => { + const stableBaseDir = await temporaryStableBase(); + await fs.mkdir(stableBaseDir, { recursive: true, mode: 0o700 }); + await writeLiveDiscoveryFile(stableBaseDir, { + url: 'http://127.0.0.1:3210', + protocolVersion: LIVE_HOST_PROTOCOL_VERSION, + pid: 999_997, + instanceNonce: 'legacy_live_owner_nonce_active_01', + }); + const ownership = createConversationRuntimeOwnership({ + stableBaseDir, + pid: process.pid, + instanceNonce: currentNonce, + isProcessAlive: () => true, + }); + + await expect(ownership.acquire()).rejects.toMatchObject({ + code: 'conversation_runtime_in_use', + }); + await expect( + fs.stat(getConversationRuntimeOwnerPath(stableBaseDir)), + ).rejects.toMatchObject({ code: 'ENOENT' }); + }); + + it('commits before removing a dead legacy Live owner and waits one grace', async () => { + const stableBaseDir = await temporaryStableBase(); + await fs.mkdir(stableBaseDir, { recursive: true, mode: 0o700 }); + await writeLiveDiscoveryFile(stableBaseDir, { + url: 'http://127.0.0.1:3210', + protocolVersion: LIVE_HOST_PROTOCOL_VERSION, + pid: 999_996, + instanceNonce: 'legacy_live_owner_nonce_dead_0001', + }); + const wait = vi.fn(async () => undefined); + const ownership = createConversationRuntimeOwnership({ + stableBaseDir, + pid: process.pid, + instanceNonce: currentNonce, + isProcessAlive: () => false, + wait, + handoffGraceMs: 41, + }); + + await expect(ownership.acquire()).resolves.toEqual({ reclaimed: true }); + expect(await readRecord(stableBaseDir)).toMatchObject({ + pid: process.pid, + instanceNonce: currentNonce, + }); + await expect( + fs.stat(getLiveDiscoveryPath(stableBaseDir)), + ).rejects.toMatchObject({ code: 'ENOENT' }); + expect(wait).toHaveBeenCalledOnce(); + expect(wait).toHaveBeenCalledWith(41); + }); + + it('waits one grace after reclaiming both ownership records', async () => { + const stableBaseDir = await temporaryStableBase(); + const previous = createConversationRuntimeOwnership({ + stableBaseDir, + pid: 999_995, + instanceNonce: 'conversation_owner_nonce_previous_04', + isProcessAlive: () => false, + }); + await previous.acquire(); + await writeLiveDiscoveryFile(stableBaseDir, { + url: 'http://127.0.0.1:3210', + protocolVersion: LIVE_HOST_PROTOCOL_VERSION, + pid: 999_994, + instanceNonce: 'legacy_live_owner_nonce_dead_0002', + }); + const wait = vi.fn(async () => undefined); + const ownership = createConversationRuntimeOwnership({ + stableBaseDir, + pid: process.pid, + instanceNonce: currentNonce, + isProcessAlive: () => false, + wait, + handoffGraceMs: 43, + }); + + await expect(ownership.acquire()).resolves.toEqual({ reclaimed: true }); + expect(wait).toHaveBeenCalledOnce(); + expect(wait).toHaveBeenCalledWith(43); + expect(await readRecord(stableBaseDir)).toMatchObject({ + pid: process.pid, + instanceNonce: currentNonce, + }); + await expect( + fs.stat(getLiveDiscoveryPath(stableBaseDir)), + ).rejects.toMatchObject({ code: 'ENOENT' }); + }); + + it('never overwrites malformed or unknown-version state', async () => { + const stableBaseDir = await temporaryStableBase(); + const ownerPath = getConversationRuntimeOwnerPath(stableBaseDir); + await fs.mkdir(path.dirname(ownerPath), { recursive: true, mode: 0o700 }); + await fs.writeFile(ownerPath, '{"version":2}', { mode: 0o600 }); + const ownership = createConversationRuntimeOwnership({ + stableBaseDir, + pid: process.pid, + instanceNonce: currentNonce, + }); + + await expect(ownership.acquire()).rejects.toMatchObject({ + code: 'conversation_runtime_ownership_compromised', + retryable: false, + }); + await expect(ownership.acquire()).rejects.toBeInstanceOf( + ConversationRuntimeOwnershipError, + ); + await expect(fs.readFile(ownerPath, 'utf8')).resolves.toBe('{"version":2}'); + await expect(ownership.release()).resolves.toBe(false); + }); + + it.each(['symlink', 'hardlink', 'oversized', 'mode'] as const)( + 'rejects an unsafe %s owner file without replacing it', + async (kind) => { + if ( + process.platform === 'win32' && + (kind === 'symlink' || kind === 'mode') + ) { + return; + } + const stableBaseDir = await temporaryStableBase(); + const ownerPath = getConversationRuntimeOwnerPath(stableBaseDir); + const ownerDirectory = path.dirname(ownerPath); + const targetPath = path.join( + path.dirname(stableBaseDir), + `owner-${kind}`, + ); + await fs.mkdir(ownerDirectory, { recursive: true, mode: 0o700 }); + const validRecord = `${JSON.stringify({ + version: 1, + pid: 999_995, + instanceNonce: 'conversation_owner_nonce_unsafe_01', + })}\n`; + if (kind === 'symlink') { + await fs.writeFile(targetPath, validRecord, { mode: 0o600 }); + await fs.symlink(targetPath, ownerPath); + } else if (kind === 'hardlink') { + await fs.writeFile(targetPath, validRecord, { mode: 0o600 }); + await fs.link(targetPath, ownerPath); + } else if (kind === 'oversized') { + await fs.writeFile(ownerPath, 'x'.repeat(4 * 1024 + 1), { + mode: 0o600, + }); + } else { + await fs.writeFile(ownerPath, validRecord, { mode: 0o644 }); + await fs.chmod(ownerPath, 0o644); + } + const ownership = createConversationRuntimeOwnership({ + stableBaseDir, + pid: process.pid, + instanceNonce: currentNonce, + }); + + await expect(ownership.acquire()).rejects.toMatchObject({ + code: 'conversation_runtime_ownership_compromised', + retryable: false, + }); + const stat = await fs.lstat(ownerPath); + if (kind === 'symlink') expect(stat.isSymbolicLink()).toBe(true); + if (kind === 'hardlink') expect(stat.nlink).toBe(2); + if (kind === 'oversized') expect(stat.size).toBe(4 * 1024 + 1); + if (kind === 'mode') expect(stat.mode & 0o777).toBe(0o644); + }, + ); + + it('turns missing or foreign state after acquire into terminal compromise', async () => { + const stableBaseDir = await temporaryStableBase(); + const ownerPath = getConversationRuntimeOwnerPath(stableBaseDir); + const ownership = createConversationRuntimeOwnership({ + stableBaseDir, + pid: process.pid, + instanceNonce: currentNonce, + }); + await ownership.acquire(); + await fs.unlink(ownerPath); + + await expect(ownership.acquire()).rejects.toMatchObject({ + code: 'conversation_runtime_ownership_compromised', + retryable: false, + }); + await expect(ownership.release()).rejects.toMatchObject({ + code: 'conversation_runtime_ownership_compromised', + }); + await expect(fs.stat(ownerPath)).rejects.toMatchObject({ code: 'ENOENT' }); + }); + + it('does not recreate a missing ownership directory after acquire', async () => { + const stableBaseDir = await temporaryStableBase(); + const ownerDirectory = path.dirname( + getConversationRuntimeOwnerPath(stableBaseDir), + ); + const ownership = createConversationRuntimeOwnership({ + stableBaseDir, + pid: process.pid, + instanceNonce: currentNonce, + }); + await ownership.acquire(); + await fs.rm(ownerDirectory, { recursive: true }); + + await expect(ownership.acquire()).rejects.toMatchObject({ + code: 'conversation_runtime_ownership_compromised', + retryable: false, + }); + await expect(fs.stat(ownerDirectory)).rejects.toMatchObject({ + code: 'ENOENT', + }); + }); + + it('does not unlink a record that changes before release', async () => { + const stableBaseDir = await temporaryStableBase(); + const ownerPath = getConversationRuntimeOwnerPath(stableBaseDir); + const ownership = createConversationRuntimeOwnership({ + stableBaseDir, + pid: process.pid, + instanceNonce: currentNonce, + }); + await ownership.acquire(); + const foreign = { + version: 1, + pid: process.pid, + instanceNonce: 'conversation_owner_nonce_foreign_01', + }; + await fs.writeFile(ownerPath, `${JSON.stringify(foreign)}\n`, { + mode: 0o600, + }); + + await expect(ownership.release()).rejects.toMatchObject({ + code: 'conversation_runtime_ownership_compromised', + }); + expect(await readRecord(stableBaseDir)).toEqual(foreign); + }); + + it('rejects an unsafe existing ownership directory without repairing it', async () => { + if (process.platform === 'win32') return; + const stableBaseDir = await temporaryStableBase(); + const ownerDirectory = path.dirname( + getConversationRuntimeOwnerPath(stableBaseDir), + ); + await fs.mkdir(ownerDirectory, { recursive: true, mode: 0o755 }); + await fs.chmod(ownerDirectory, 0o755); + const ownership = createConversationRuntimeOwnership({ + stableBaseDir, + pid: process.pid, + instanceNonce: currentNonce, + }); + + await expect(ownership.acquire()).rejects.toMatchObject({ + code: 'conversation_runtime_ownership_compromised', + }); + expect((await fs.stat(ownerDirectory)).mode & 0o777).toBe(0o755); + }); + + it.skipIf(process.platform === 'win32' || process.getuid?.() === 0)( + 'treats transient directory inspection failures as retryable', + async () => { + const stableBaseDir = await temporaryStableBase(); + const parent = path.dirname(stableBaseDir); + const ownership = createConversationRuntimeOwnership({ + stableBaseDir, + pid: process.pid, + instanceNonce: currentNonce, + }); + + await fs.chmod(parent, 0o000); + try { + await expect(ownership.acquire()).rejects.toMatchObject({ + code: 'conversation_runtime_unavailable', + retryable: true, + }); + } finally { + await fs.chmod(parent, 0o700); + } + + await expect(ownership.acquire()).resolves.toEqual({ + reclaimed: false, + }); + }, + ); + + it('rejects an unsafe ownership lock shape without replacing it', async () => { + const stableBaseDir = await temporaryStableBase(); + const ownerDirectory = path.dirname( + getConversationRuntimeOwnerPath(stableBaseDir), + ); + const lockPath = path.join(ownerDirectory, '.runtime-owner.lock'); + await fs.mkdir(ownerDirectory, { recursive: true, mode: 0o700 }); + await fs.writeFile(lockPath, 'unsafe', { mode: 0o600 }); + const ownership = createConversationRuntimeOwnership({ + stableBaseDir, + pid: process.pid, + instanceNonce: currentNonce, + }); + + await expect(ownership.acquire()).rejects.toMatchObject({ + code: 'conversation_runtime_ownership_compromised', + retryable: false, + }); + await expect(fs.readFile(lockPath, 'utf8')).resolves.toBe('unsafe'); + }); + + it('rejects a symlinked ownership directory', async () => { + if (process.platform === 'win32') return; + const stableBaseDir = await temporaryStableBase(); + const ownerDirectory = path.dirname( + getConversationRuntimeOwnerPath(stableBaseDir), + ); + const target = path.join(path.dirname(stableBaseDir), 'target'); + await fs.mkdir(target, { mode: 0o700 }); + await fs.mkdir(stableBaseDir, { mode: 0o700 }); + await fs.symlink(target, ownerDirectory); + const ownership = createConversationRuntimeOwnership({ + stableBaseDir, + pid: process.pid, + instanceNonce: currentNonce, + }); + + await expect(ownership.acquire()).rejects.toMatchObject({ + code: 'conversation_runtime_ownership_compromised', + }); + await expect(fs.readdir(target)).resolves.toEqual([]); + }); + + it('rejects an ownership directory below a symlinked stable base', async () => { + if (process.platform === 'win32') return; + const stableBaseDir = await temporaryStableBase(); + const target = path.join(path.dirname(stableBaseDir), 'stable-target'); + await fs.mkdir(path.join(target, 'conversations'), { + recursive: true, + mode: 0o700, + }); + await fs.symlink(target, stableBaseDir); + const ownership = createConversationRuntimeOwnership({ + stableBaseDir, + pid: process.pid, + instanceNonce: currentNonce, + }); + + await expect(ownership.acquire()).rejects.toMatchObject({ + code: 'conversation_runtime_ownership_compromised', + }); + await expect( + fs.readdir(path.join(target, 'conversations')), + ).resolves.toEqual([]); + }); +}); diff --git a/packages/cli/src/serve/conversations/conversation-runtime-ownership.ts b/packages/cli/src/serve/conversations/conversation-runtime-ownership.ts new file mode 100644 index 00000000000..66b6a3cb091 --- /dev/null +++ b/packages/cli/src/serve/conversations/conversation-runtime-ownership.ts @@ -0,0 +1,600 @@ +/** + * @license + * Copyright 2026 Qwen Team + * SPDX-License-Identifier: Apache-2.0 + */ + +import { randomUUID } from 'node:crypto'; +import { constants as fsConstants } from 'node:fs'; +import * as fs from 'node:fs/promises'; +import * as path from 'node:path'; +import type { LockOptions } from 'proper-lockfile'; +import { + ConversationRuntimeOwnershipError, + conversationRuntimeInUseError, + conversationRuntimeOwnershipCompromisedError, + conversationRuntimeUnavailableError, +} from './conversation-runtime-errors.js'; + +const OWNER_DIRECTORY = 'conversations'; +const OWNER_FILE = 'runtime-owner.json'; +const OWNER_LOCK = '.runtime-owner.lock'; +const MAX_OWNER_BYTES = 4 * 1024; +const NONCE_PATTERN = /^[A-Za-z0-9_-]{16,256}$/; +const DEFAULT_HANDOFF_GRACE_MS = 1_000; + +interface ConversationRuntimeOwnerRecord { + version: 1; + pid: number; + instanceNonce: string; +} + +interface DirectoryIdentity { + dev: number | bigint; + ino: number | bigint; +} + +class UnsafeOwnershipStateError extends Error {} + +export interface ConversationRuntimeOwnership { + acquire(): Promise<{ reclaimed: boolean }>; + release(): Promise; +} + +export interface ConversationRuntimeOwnershipOptions { + stableBaseDir: string; + pid: number; + instanceNonce: string; + isProcessAlive?: (pid: number) => boolean; + wait?: (milliseconds: number) => Promise; + handoffGraceMs?: number; + lockOptions?: Pick; +} + +export function getConversationRuntimeOwnerPath(stableBaseDir: string): string { + return path.join(stableBaseDir, OWNER_DIRECTORY, OWNER_FILE); +} + +function processIsAlive(pid: number): boolean { + try { + process.kill(pid, 0); + return true; + } catch (error) { + return (error as NodeJS.ErrnoException).code !== 'ESRCH'; + } +} + +function delay(milliseconds: number): Promise { + return new Promise((resolve) => { + setTimeout(resolve, milliseconds); + }); +} + +function isMissing(error: unknown): boolean { + return (error as NodeJS.ErrnoException).code === 'ENOENT'; +} + +function assertOwnerRecord(value: unknown): ConversationRuntimeOwnerRecord { + if (typeof value !== 'object' || value === null || Array.isArray(value)) { + throw new UnsafeOwnershipStateError(); + } + const keys = Object.keys(value).sort(); + if ( + keys.length !== 3 || + keys[0] !== 'instanceNonce' || + keys[1] !== 'pid' || + keys[2] !== 'version' + ) { + throw new UnsafeOwnershipStateError(); + } + const record = value as Record; + if ( + record['version'] !== 1 || + !Number.isSafeInteger(record['pid']) || + (record['pid'] as number) <= 0 || + typeof record['instanceNonce'] !== 'string' || + !NONCE_PATTERN.test(record['instanceNonce']) + ) { + throw new UnsafeOwnershipStateError(); + } + return { + version: 1, + pid: record['pid'] as number, + instanceNonce: record['instanceNonce'], + }; +} + +function sameRecord( + left: ConversationRuntimeOwnerRecord, + right: ConversationRuntimeOwnerRecord, +): boolean { + return ( + left.version === right.version && + left.pid === right.pid && + left.instanceNonce === right.instanceNonce + ); +} + +async function inspectDirectory( + directory: string, + requirePrivateMode: boolean, +): Promise { + const stat = await fs.lstat(directory); + if (!stat.isDirectory() || stat.isSymbolicLink()) { + throw new UnsafeOwnershipStateError(); + } + if ( + process.platform !== 'win32' && + ((typeof process.getuid === 'function' && stat.uid !== process.getuid()) || + (requirePrivateMode && (stat.mode & 0o777) !== 0o700)) + ) { + throw new UnsafeOwnershipStateError(); + } + return { dev: stat.dev, ino: stat.ino }; +} + +async function ensureDirectory( + directory: string, + requirePrivateMode: boolean, +): Promise { + try { + return await inspectDirectory(directory, requirePrivateMode); + } catch (error) { + if (error instanceof UnsafeOwnershipStateError || !isMissing(error)) { + throw error; + } + } + + const parent = path.dirname(directory); + if (parent === directory) throw new UnsafeOwnershipStateError(); + const parentIdentity = await ensureDirectory(parent, false); + await assertDirectoryIdentity(parent, parentIdentity, false); + let created = false; + try { + await fs.mkdir(directory, { mode: 0o700 }); + created = true; + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== 'EEXIST') throw error; + } + if (created && process.platform !== 'win32') { + await fs.chmod(directory, 0o700); + } + await assertDirectoryIdentity(parent, parentIdentity, false); + return inspectDirectory(directory, requirePrivateMode); +} + +async function assertDirectoryIdentity( + directory: string, + expected: DirectoryIdentity, + requirePrivateMode = true, +): Promise { + const current = await inspectDirectory(directory, requirePrivateMode); + if (current.dev !== expected.dev || current.ino !== expected.ino) { + throw new UnsafeOwnershipStateError(); + } +} + +async function assertLockShapeIfPresent(lockPath: string): Promise { + let stat: Awaited>; + try { + stat = await fs.lstat(lockPath); + } catch (error) { + if (isMissing(error)) return; + throw error; + } + if ( + !stat.isDirectory() || + stat.isSymbolicLink() || + (process.platform !== 'win32' && + typeof process.getuid === 'function' && + stat.uid !== process.getuid()) + ) { + throw new UnsafeOwnershipStateError(); + } +} + +async function readOwnerRecord( + ownerPath: string, +): Promise { + let pathStat: Awaited>; + try { + pathStat = await fs.lstat(ownerPath); + } catch (error) { + if (isMissing(error)) return undefined; + throw error; + } + if ( + !pathStat.isFile() || + pathStat.isSymbolicLink() || + pathStat.nlink !== 1 || + pathStat.size <= 0 || + pathStat.size > MAX_OWNER_BYTES || + (process.platform !== 'win32' && + ((pathStat.mode & 0o777) !== 0o600 || + (typeof process.getuid === 'function' && + pathStat.uid !== process.getuid()))) + ) { + throw new UnsafeOwnershipStateError(); + } + + let handle: fs.FileHandle; + try { + handle = await fs.open( + ownerPath, + process.platform === 'win32' + ? fsConstants.O_RDONLY + : fsConstants.O_RDONLY | fsConstants.O_NOFOLLOW, + ); + } catch (error) { + throw new UnsafeOwnershipStateError(undefined, { cause: error }); + } + try { + const handleStat = await handle.stat(); + if ( + !handleStat.isFile() || + handleStat.nlink !== 1 || + handleStat.dev !== pathStat.dev || + handleStat.ino !== pathStat.ino || + handleStat.size !== pathStat.size || + (process.platform !== 'win32' && + ((handleStat.mode & 0o777) !== 0o600 || + (typeof process.getuid === 'function' && + handleStat.uid !== process.getuid()))) + ) { + throw new UnsafeOwnershipStateError(); + } + let parsed: unknown; + try { + parsed = JSON.parse(await handle.readFile('utf8')); + } catch (error) { + throw new UnsafeOwnershipStateError(undefined, { cause: error }); + } + return assertOwnerRecord(parsed); + } finally { + await handle.close(); + } +} + +async function syncDirectory(directory: string): Promise { + if (process.platform === 'win32') return; + const handle = await fs.open(directory, fsConstants.O_RDONLY); + try { + await handle.sync(); + } finally { + await handle.close(); + } +} + +class FileConversationRuntimeOwnership implements ConversationRuntimeOwnership { + private readonly stableBaseDir: string; + private readonly ownerDirectory: string; + private readonly ownerPath: string; + private readonly lockPath: string; + private readonly current: ConversationRuntimeOwnerRecord; + private readonly isProcessAlive: (pid: number) => boolean; + private readonly wait: (milliseconds: number) => Promise; + private readonly handoffGraceMs: number; + private readonly lockOptions: Pick< + LockOptions, + 'stale' | 'update' | 'retries' + >; + private state: 'unclaimed' | 'provisional' | 'owned' | 'released' = + 'unclaimed'; + private terminalError?: ConversationRuntimeOwnershipError; + private acquirePending?: Promise<{ reclaimed: boolean }>; + private releasePending?: Promise; + + constructor(options: ConversationRuntimeOwnershipOptions) { + this.stableBaseDir = options.stableBaseDir; + this.ownerDirectory = path.dirname( + getConversationRuntimeOwnerPath(options.stableBaseDir), + ); + this.ownerPath = getConversationRuntimeOwnerPath(options.stableBaseDir); + this.lockPath = path.join(this.ownerDirectory, OWNER_LOCK); + this.current = { + version: 1, + pid: options.pid, + instanceNonce: options.instanceNonce, + }; + this.isProcessAlive = options.isProcessAlive ?? processIsAlive; + this.wait = options.wait ?? delay; + this.handoffGraceMs = options.handoffGraceMs ?? DEFAULT_HANDOFF_GRACE_MS; + this.lockOptions = options.lockOptions ?? { + stale: 5_000, + update: 1_000, + retries: { + retries: 40, + minTimeout: 10, + maxTimeout: 50, + factor: 1.2, + randomize: true, + }, + }; + } + + acquire(): Promise<{ reclaimed: boolean }> { + if (this.acquirePending) return this.acquirePending; + const pending = this.acquireOnce().finally(() => { + if (this.acquirePending === pending) this.acquirePending = undefined; + }); + this.acquirePending = pending; + return pending; + } + + release(): Promise { + if (this.releasePending) return this.releasePending; + const pending = this.releaseOnce().finally(() => { + if (this.releasePending === pending) this.releasePending = undefined; + }); + this.releasePending = pending; + return pending; + } + + private async acquireOnce(): Promise<{ reclaimed: boolean }> { + if (this.terminalError) throw this.terminalError; + if (this.state === 'released') { + throw this.compromise(); + } + if (this.releasePending) { + await this.releasePending.catch(() => undefined); + throw this.compromise(); + } + + const identity = await this.prepareOwnerDirectory(); + let releaseLock: (() => Promise) | undefined; + let operationError: unknown; + let reclaimed = false; + let lockCompromise: unknown; + try { + await assertLockShapeIfPresent(this.lockPath); + const lockfile = (await import('proper-lockfile')).default; + releaseLock = await lockfile.lock(this.ownerDirectory, { + realpath: false, + lockfilePath: this.lockPath, + ...this.lockOptions, + onCompromised: (error) => { + lockCompromise = error; + }, + }); + await assertLockShapeIfPresent(this.lockPath); + await assertDirectoryIdentity(this.ownerDirectory, identity); + if (lockCompromise) throw new UnsafeOwnershipStateError(); + const existing = await readOwnerRecord(this.ownerPath); + const commitOwner = async (): Promise => { + if (this.state === 'owned') { + if (!existing || !sameRecord(existing, this.current)) { + throw new UnsafeOwnershipStateError(); + } + } else if (existing && sameRecord(existing, this.current)) { + this.state = 'provisional'; + } else if (existing) { + if ( + existing.pid === this.current.pid || + this.isProcessAlive(existing.pid) + ) { + throw conversationRuntimeInUseError(); + } + reclaimed = true; + await this.commitRecord(identity, existing); + } else { + await this.commitRecord(identity, undefined); + } + }; + const liveDiscovery = await import('../live/discovery.js'); + const handoff = await liveDiscovery.handoffLiveDiscoveryOwner( + this.stableBaseDir, + this.current, + commitOwner, + { + isProcessAlive: this.isProcessAlive, + waitForHandoffGrace: false, + }, + ); + reclaimed ||= handoff.reclaimed; + } catch (error) { + operationError = error; + } + + let releaseError: unknown; + if (releaseLock) { + try { + await releaseLock(); + } catch (error) { + releaseError = error; + } + } + if (lockCompromise || releaseError) { + throw this.compromise(lockCompromise ?? releaseError); + } + if (operationError) { + throw this.mapAcquireError(operationError); + } + + if (reclaimed) { + try { + await this.wait(this.handoffGraceMs); + } catch (error) { + throw this.compromise(error); + } + } + if (this.state === 'provisional') this.state = 'owned'; + return { reclaimed }; + } + + private async releaseOnce(): Promise { + if (this.acquirePending) { + await this.acquirePending.catch(() => undefined); + } + if (this.state === 'unclaimed' || this.state === 'released') return false; + if (this.state === 'provisional' || this.terminalError) { + throw this.terminalError ?? this.compromise(); + } + + const identity = await this.prepareOwnerDirectory(); + let releaseLock: (() => Promise) | undefined; + let operationError: unknown; + let lockCompromise: unknown; + try { + await assertLockShapeIfPresent(this.lockPath); + const lockfile = (await import('proper-lockfile')).default; + releaseLock = await lockfile.lock(this.ownerDirectory, { + realpath: false, + lockfilePath: this.lockPath, + ...this.lockOptions, + onCompromised: (error) => { + lockCompromise = error; + }, + }); + await assertLockShapeIfPresent(this.lockPath); + await assertDirectoryIdentity(this.ownerDirectory, identity); + const existing = await readOwnerRecord(this.ownerPath); + if (lockCompromise || !existing || !sameRecord(existing, this.current)) { + throw new UnsafeOwnershipStateError(); + } + await assertDirectoryIdentity(this.ownerDirectory, identity); + await fs.unlink(this.ownerPath); + this.state = 'released'; + await syncDirectory(this.ownerDirectory); + } catch (error) { + operationError = error; + } + + let releaseError: unknown; + if (releaseLock) { + try { + await releaseLock(); + } catch (error) { + releaseError = error; + } + } + if (lockCompromise || releaseError) { + if (this.state !== 'released') this.compromise(lockCompromise); + throw conversationRuntimeOwnershipCompromisedError( + lockCompromise ?? releaseError, + ); + } + if (operationError) { + if (this.state === 'released') { + throw conversationRuntimeOwnershipCompromisedError(operationError); + } + if (operationError instanceof UnsafeOwnershipStateError) { + throw this.compromise(operationError); + } + throw conversationRuntimeUnavailableError(operationError); + } + return true; + } + + private async prepareOwnerDirectory(): Promise { + try { + if (this.state === 'unclaimed') { + await ensureDirectory(this.stableBaseDir, false); + return await ensureDirectory(this.ownerDirectory, true); + } + await inspectDirectory(this.stableBaseDir, false); + return await inspectDirectory(this.ownerDirectory, true); + } catch (error) { + if ( + error instanceof UnsafeOwnershipStateError || + (this.state !== 'unclaimed' && isMissing(error)) + ) { + throw this.compromise(error); + } + throw conversationRuntimeUnavailableError(error); + } + } + + private async commitRecord( + identity: DirectoryIdentity, + expected: ConversationRuntimeOwnerRecord | undefined, + ): Promise { + const temporaryPath = path.join( + this.ownerDirectory, + `.runtime-owner.${this.current.pid}.${randomUUID()}.tmp`, + ); + let handle: fs.FileHandle | undefined; + let renamed = false; + try { + handle = await fs.open(temporaryPath, 'wx', 0o600); + await handle.writeFile(`${JSON.stringify(this.current)}\n`, 'utf8'); + await handle.sync(); + if (process.platform !== 'win32') await handle.chmod(0o600); + await handle.close(); + handle = undefined; + + await assertDirectoryIdentity(this.ownerDirectory, identity); + const observed = await readOwnerRecord(this.ownerPath); + if ( + (expected === undefined && observed !== undefined) || + (expected !== undefined && + (!observed || !sameRecord(observed, expected))) + ) { + throw new UnsafeOwnershipStateError(); + } + await assertDirectoryIdentity(this.ownerDirectory, identity); + + if (process.platform === 'win32' && observed) { + await fs.unlink(this.ownerPath); + try { + await fs.rename(temporaryPath, this.ownerPath); + } catch (error) { + await this.wait(this.handoffGraceMs); + throw error; + } + } else { + await fs.rename(temporaryPath, this.ownerPath); + } + renamed = true; + this.state = 'provisional'; + await syncDirectory(this.ownerDirectory); + } finally { + await handle?.close().catch(() => undefined); + if (!renamed) { + await fs.unlink(temporaryPath).catch(() => undefined); + } + } + } + + private mapAcquireError(error: unknown): ConversationRuntimeOwnershipError { + if (error instanceof ConversationRuntimeOwnershipError) return error; + if ( + error instanceof Error && + error.name === 'LiveDiscoveryOwnerActiveError' + ) { + return conversationRuntimeInUseError(); + } + if (error instanceof UnsafeOwnershipStateError) { + return this.compromise(error); + } + if ( + this.state === 'provisional' || + (error instanceof Error && error.name === 'LiveDiscoveryStateError') + ) { + return this.compromise(error); + } + const code = (error as NodeJS.ErrnoException).code; + if (code === 'ECOMPROMISED') return this.compromise(error); + return conversationRuntimeUnavailableError(error); + } + + private compromise(cause?: unknown): ConversationRuntimeOwnershipError { + this.terminalError ??= conversationRuntimeOwnershipCompromisedError(cause); + return this.terminalError; + } +} + +export function createConversationRuntimeOwnership( + options: ConversationRuntimeOwnershipOptions, +): ConversationRuntimeOwnership { + if ( + !path.isAbsolute(options.stableBaseDir) || + !Number.isSafeInteger(options.pid) || + options.pid <= 0 || + !NONCE_PATTERN.test(options.instanceNonce) || + !Number.isSafeInteger(options.handoffGraceMs ?? DEFAULT_HANDOFF_GRACE_MS) || + (options.handoffGraceMs ?? DEFAULT_HANDOFF_GRACE_MS) < 0 + ) { + throw new TypeError('Invalid Conversations runtime ownership options.'); + } + return new FileConversationRuntimeOwnership(options); +} diff --git a/packages/cli/src/serve/daemon-status.test.ts b/packages/cli/src/serve/daemon-status.test.ts index 0b84677d4c8..0bd614c4ec0 100644 --- a/packages/cli/src/serve/daemon-status.test.ts +++ b/packages/cli/src/serve/daemon-status.test.ts @@ -990,6 +990,88 @@ describe('buildDaemonStatusResponse', () => { ]); }); + it('aggregates an internal runtime without exposing its workspace path', async () => { + const internalCwd = '/private/conversations-runtime'; + const primaryBridge = { + getDaemonStatusSnapshot: () => BASE_BRIDGE_SNAPSHOT, + lastActivityAt: null, + pendingPromptTotal: 0, + activePromptCount: 1, + } as unknown as AcpSessionBridge; + const internalBridge = { + getDaemonStatusSnapshot: () => ({ + ...BASE_BRIDGE_SNAPSHOT, + limits: { ...BASE_BRIDGE_SNAPSHOT.limits, maxSessions: 10 }, + sessionCount: 8, + channelLive: false, + sessions: [ + { + sessionId: 'internal-session', + workspaceCwd: internalCwd, + createdAt: '2026-08-14T00:00:00.000Z', + clientCount: 1, + subscriberCount: 0, + attachCount: 1, + pendingPromptCount: 0, + pendingPermissionCount: 0, + hasActivePrompt: false, + lastEventId: 0, + maxJournalEvents: 10_000, + maxJournalBytes: 8 * 1024 * 1024, + }, + ], + }), + lastActivityAt: null, + pendingPromptTotal: 2, + activePromptCount: 3, + } as unknown as AcpSessionBridge; + const primary = { + workspaceId: 'primary', + workspaceCwd: BASE_WORKSPACE, + primary: true, + trusted: true, + bridge: primaryBridge, + }; + const internal = { + workspaceId: 'internal', + workspaceCwd: internalCwd, + primary: false, + trusted: true, + provenance: 'live-conversation' as const, + bridge: internalBridge, + }; + const options = makeOptions(); + options.bridge = primaryBridge; + options.workspaceRegistry = { + primary, + list: () => [primary], + listAll: () => [primary, internal], + } as unknown as BuildDaemonStatusOptions['workspaceRegistry']; + + const response = await buildDaemonStatusResponse('summary', options); + + expect(response.runtime.sessions.active).toBe(8); + expect(response.runtime.activity.activePrompts).toBe(4); + expect(response.runtime.activity.queuedPrompts).toBe(2); + expect(response.workspaces).toBeUndefined(); + expect(JSON.stringify(response.issues)).not.toContain(internalCwd); + expect(response.issues).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + code: 'session_capacity_high', + message: expect.stringContaining('internal runtime'), + }), + expect.objectContaining({ + code: 'acp_channel_down', + message: expect.stringContaining('internal runtime'), + }), + ]), + ); + + const full = await buildDaemonStatusResponse('full', options); + expect(JSON.stringify(full.full?.sessions)).not.toContain(internalCwd); + }); + it('reports every runtime issue code from daemon counters', async () => { const response = await buildDaemonStatusResponse( 'summary', diff --git a/packages/cli/src/serve/daemon-status.ts b/packages/cli/src/serve/daemon-status.ts index 63917fb95e4..b2ea34fe32a 100644 --- a/packages/cli/src/serve/daemon-status.ts +++ b/packages/cli/src/serve/daemon-status.ts @@ -43,6 +43,7 @@ import type { } from './workspace-service/index.js'; import type { TotalSessionAdmissionSnapshot } from './total-session-admission.js'; import type { WorkspaceRegistry } from './workspace-registry.js'; +import { isInternalWorkspaceRuntime } from './workspace-runtime-visibility.js'; // Re-export so downstream consumers (server.ts, routes, the SDK type mirror) // import the bucket shape from the status module alongside the rest of the @@ -159,6 +160,7 @@ interface FullDaemonStatus { interface WorkspaceBridgeStatusSnapshot { workspaceCwd: string; + internal?: boolean; snapshot: BridgeDaemonStatusSnapshot; lastActivity: number | null; } @@ -560,9 +562,12 @@ export async function buildDaemonStatusResponse( const bridgeSnapshot = input.bridge.getDaemonStatusSnapshot(); const lastActivity = input.bridge.lastActivityAt ?? null; const workspaceRuntimes = input.workspaceRegistry?.list(); + const aggregateRuntimes = + input.workspaceRegistry?.listAll?.() ?? workspaceRuntimes; const workspaceSnapshots: WorkspaceBridgeStatusSnapshot[] = - workspaceRuntimes?.map((runtime) => ({ + aggregateRuntimes?.map((runtime) => ({ workspaceCwd: runtime.workspaceCwd, + internal: isInternalWorkspaceRuntime(runtime), snapshot: runtime.bridge === input.bridge ? bridgeSnapshot @@ -606,7 +611,10 @@ export async function buildDaemonStatusResponse( .length : workspaceSnapshots.filter((item) => item.snapshot.channelLive).length; const registeredWorkspaceCount = input.workspaceRegistry - ? input.workspaceRegistry.listEntries().length + ? ( + input.workspaceRegistry.listAllEntries?.() ?? + input.workspaceRegistry.listEntries() + ).length : workspaceSnapshots.length; // Summed in the SAME synchronous pass that produced `activeAcpChildCount` // above, over the same array. Keep it that way: an `await` slipped between @@ -718,7 +726,7 @@ export async function buildDaemonStatusResponse( derivedQueuedPromptsByWorkspace[index] = derivedQueuedPromptsForWorkspace; } const queuedPrompts = - workspaceRuntimes?.reduce( + aggregateRuntimes?.reduce( (sum, runtime, index) => sum + (runtime.bridge.pendingPromptTotal ?? @@ -803,7 +811,9 @@ export async function buildDaemonStatusResponse( full = await buildFullStatus( input, acpAggregate, - workspaceSnapshots.flatMap((item) => item.snapshot.sessions), + workspaceSnapshots + .filter((item) => item.internal !== true) + .flatMap((item) => item.snapshot.sessions), ); pushFullIssues(issues, full); } @@ -936,7 +946,7 @@ export async function buildDaemonStatusResponse( : {}), activity: { activePrompts: - workspaceRuntimes?.reduce( + aggregateRuntimes?.reduce( (sum, runtime) => sum + (runtime.bridge.activePromptCount ?? 0), 0, ) ?? @@ -1096,7 +1106,7 @@ function pushRuntimeIssues( totalAdmissionSnapshot: TotalSessionAdmissionSnapshot | undefined, workspaceSnapshots: readonly WorkspaceBridgeStatusSnapshot[], ): void { - for (const { workspaceCwd, snapshot } of workspaceSnapshots) { + for (const { workspaceCwd, internal, snapshot } of workspaceSnapshots) { if ( snapshot.limits.maxSessions !== null && snapshot.limits.maxSessions > 0 && @@ -1108,7 +1118,9 @@ function pushRuntimeIssues( severity: 'warning', message: workspaceSnapshots.length > 1 - ? `Workspace ${workspaceCwd} active sessions are at ${snapshot.sessionCount}/${snapshot.limits.maxSessions}.` + ? internal + ? `An internal runtime's active sessions are at ${snapshot.sessionCount}/${snapshot.limits.maxSessions}.` + : `Workspace ${workspaceCwd} active sessions are at ${snapshot.sessionCount}/${snapshot.limits.maxSessions}.` : `Active sessions are at ${snapshot.sessionCount}/${snapshot.limits.maxSessions}.`, }); } @@ -1174,7 +1186,9 @@ function pushRuntimeIssues( severity: 'error', message: downWorkspaces.length === 1 - ? `Active sessions exist but the ACP channel is not live for ${downWorkspaces[0]!.workspaceCwd}.` + ? downWorkspaces[0]!.internal + ? 'Active sessions exist but the ACP channel is not live for an internal runtime.' + : `Active sessions exist but the ACP channel is not live for ${downWorkspaces[0]!.workspaceCwd}.` : `Active sessions exist but the ACP channel is not live for ${downWorkspaces.length} workspace(s).`, }); } diff --git a/packages/cli/src/serve/index.ts b/packages/cli/src/serve/index.ts index 1fbfa90edbf..e82940db01e 100644 --- a/packages/cli/src/serve/index.ts +++ b/packages/cli/src/serve/index.ts @@ -5,6 +5,11 @@ */ export { createServeApp, type ServeAppDeps } from './server.js'; +export { + getServeAppLifecycle, + type ServeAppLifecycle, + type ServeAppLifecycleBindingOptions, +} from './serve-app-lifecycle.js'; export { runQwenServe, type RunHandle, diff --git a/packages/cli/src/serve/live/discovery.test.ts b/packages/cli/src/serve/live/discovery.test.ts index 2b78f621441..4634c993526 100644 --- a/packages/cli/src/serve/live/discovery.test.ts +++ b/packages/cli/src/serve/live/discovery.test.ts @@ -10,7 +10,9 @@ import * as path from 'node:path'; import { afterEach, describe, expect, it } from 'vitest'; import { getLiveDiscoveryPath, + handoffLiveDiscoveryOwner, LiveDiscoveryOwnerActiveError, + LiveDiscoveryStateError, removeLiveDiscoveryFile, writeLiveDiscoveryFile, type LiveDiscoveryRecord, @@ -62,23 +64,79 @@ describe('Live discovery file', () => { ).toEqual([]); }); + it('safely creates a missing nested runtime directory tree', async () => { + const parent = await temporaryRuntime(); + const runtime = path.join(parent, 'nested', 'runtime', 'base'); + const expected = record('daemon_instance_nonce_nested_01'); + + await expect(writeLiveDiscoveryFile(runtime, expected)).resolves.toBe( + getLiveDiscoveryPath(runtime), + ); + await expect( + fs.readFile(getLiveDiscoveryPath(runtime), 'utf8'), + ).resolves.toContain(expected.instanceNonce); + if (process.platform !== 'win32') { + expect( + (await fs.stat(path.dirname(getLiveDiscoveryPath(runtime)))).mode & + 0o777, + ).toBe(0o700); + } + }); + + it('rejects a symlinked runtime base without publishing through it', async () => { + if (process.platform === 'win32') return; + const parent = await temporaryRuntime(); + const target = path.join(parent, 'target'); + const runtimeBaseDir = path.join(parent, 'runtime-link'); + await fs.mkdir(target, { mode: 0o700 }); + await fs.symlink(target, runtimeBaseDir); + + await expect( + writeLiveDiscoveryFile( + runtimeBaseDir, + record('daemon_instance_nonce_symlink_01'), + ), + ).rejects.toBeInstanceOf(LiveDiscoveryStateError); + await expect(fs.readdir(target)).resolves.toEqual([]); + }); + + it('rejects an unsafe discovery lock shape without replacing it', async () => { + const runtime = await temporaryRuntime(); + const discoveryDirectory = path.dirname(getLiveDiscoveryPath(runtime)); + const lockPath = path.join(discoveryDirectory, '.daemon.lock'); + await fs.mkdir(discoveryDirectory, { mode: 0o700 }); + await fs.writeFile(lockPath, 'unsafe', { mode: 0o600 }); + + await expect( + writeLiveDiscoveryFile( + runtime, + record('daemon_instance_nonce_unsafe_lock_01'), + ), + ).rejects.toBeInstanceOf(LiveDiscoveryStateError); + + await expect(fs.readFile(lockPath, 'utf8')).resolves.toBe('unsafe'); + await expect(fs.stat(getLiveDiscoveryPath(runtime))).rejects.toMatchObject({ + code: 'ENOENT', + }); + }); + it('only removes a record owned by the matching daemon pid and nonce', async () => { const runtime = await temporaryRuntime(); const current = record('daemon_instance_nonce_0002'); await writeLiveDiscoveryFile(runtime, current); - expect( - await removeLiveDiscoveryFile(runtime, { + await expect( + removeLiveDiscoveryFile(runtime, { pid: current.pid, instanceNonce: 'daemon_instance_nonce_old0', }), - ).toBe(false); - expect( - await removeLiveDiscoveryFile(runtime, { + ).rejects.toBeInstanceOf(LiveDiscoveryStateError); + await expect( + removeLiveDiscoveryFile(runtime, { pid: current.pid + 1, instanceNonce: current.instanceNonce, }), - ).toBe(false); + ).rejects.toBeInstanceOf(LiveDiscoveryStateError); await expect( fs.readFile(getLiveDiscoveryPath(runtime), 'utf8'), ).resolves.toContain(current.instanceNonce); @@ -115,7 +173,7 @@ describe('Live discovery file', () => { ); }); - it('reclaims only a valid record whose owner process is stale', async () => { + it('does not publish over a stale foreign owner without a handoff', async () => { const runtime = await temporaryRuntime(); const previous = record('daemon_instance_nonce_0005', 999_999); const replacement = record('daemon_instance_nonce_0006'); @@ -128,13 +186,13 @@ describe('Live discovery file', () => { return false; }, }), - ).resolves.toBe(getLiveDiscoveryPath(runtime)); + ).rejects.toBeInstanceOf(LiveDiscoveryStateError); expect( JSON.parse(await fs.readFile(getLiveDiscoveryPath(runtime), 'utf8')), - ).toEqual(replacement); + ).toEqual(previous); }); - it('reclaims a valid legacy-protocol record whose owner is stale', async () => { + it('does not publish over a stale legacy-protocol owner without a handoff', async () => { const runtime = await temporaryRuntime(); const discoveryPath = getLiveDiscoveryPath(runtime); const previous = { @@ -157,13 +215,62 @@ describe('Live discovery file', () => { return false; }, }), - ).resolves.toBe(discoveryPath); + ).rejects.toBeInstanceOf(LiveDiscoveryStateError); expect(JSON.parse(await fs.readFile(discoveryPath, 'utf8'))).toEqual( - replacement, + previous, ); expect((await fs.stat(discoveryPath)).mode & 0o777).toBe(0o600); }); + it.each([ + ['current', LIVE_HOST_PROTOCOL_VERSION, 999_996], + ['legacy', 2, 999_995], + ])( + 'reclaims a valid stale %s-protocol owner during handoff', + async (_label, protocolVersion, pid) => { + const runtime = await temporaryRuntime(); + const discoveryPath = getLiveDiscoveryPath(runtime); + const previous = { + ...record('daemon_instance_nonce_handoff_old', pid), + protocolVersion, + }; + const replacement = record('daemon_instance_nonce_handoff_new'); + await fs.mkdir(path.dirname(discoveryPath), { + recursive: true, + mode: 0o700, + }); + await fs.writeFile(discoveryPath, `${JSON.stringify(previous)}\n`, { + mode: 0o600, + }); + let committed = false; + const wait = vi.fn(async () => undefined); + + await expect( + handoffLiveDiscoveryOwner( + runtime, + replacement, + async () => { + committed = true; + }, + { + wait, + handoffGraceMs: 37, + isProcessAlive: (ownerPid) => { + expect(ownerPid).toBe(previous.pid); + return false; + }, + }, + ), + ).resolves.toEqual({ reclaimed: true }); + expect(committed).toBe(true); + expect(wait).toHaveBeenCalledOnce(); + expect(wait).toHaveBeenCalledWith(37); + await expect(fs.stat(discoveryPath)).rejects.toMatchObject({ + code: 'ENOENT', + }); + }, + ); + it('does not replace a live owner only because its protocol is legacy', async () => { const runtime = await temporaryRuntime(); const discoveryPath = getLiveDiscoveryPath(runtime); diff --git a/packages/cli/src/serve/live/discovery.ts b/packages/cli/src/serve/live/discovery.ts index 3c0357da67a..08ed0252199 100644 --- a/packages/cli/src/serve/live/discovery.ts +++ b/packages/cli/src/serve/live/discovery.ts @@ -5,17 +5,19 @@ */ import { randomUUID } from 'node:crypto'; +import { constants as fsConstants } from 'node:fs'; import * as fs from 'node:fs/promises'; import { isIP } from 'node:net'; import { homedir } from 'node:os'; import * as path from 'node:path'; -import lockfile from 'proper-lockfile'; +import type { LockOptions } from 'proper-lockfile'; import { LIVE_HOST_PROTOCOL_VERSION } from './types.js'; export const LIVE_DISCOVERY_RELATIVE_PATH = path.join('live', 'daemon.json'); const MAX_DISCOVERY_BYTES = 16 * 1024; const NONCE_PATTERN = /^[A-Za-z0-9_-]{16,256}$/; -const LOCK_OPTIONS: lockfile.LockOptions = { +const DEFAULT_HANDOFF_GRACE_MS = 1_000; +const LOCK_OPTIONS: LockOptions = { realpath: false, stale: 10_000, update: 2_000, @@ -49,6 +51,12 @@ interface ExistingLiveDiscoveryRecord { instanceNonce: string; } +interface LiveDiscoveryDirectory { + directory: string; + dev: number | bigint; + ino: number | bigint; +} + export class LiveDiscoveryOwnerActiveError extends Error { constructor(readonly ownerPid: number) { super(`Live discovery is owned by active daemon pid ${ownerPid}.`); @@ -56,6 +64,24 @@ export class LiveDiscoveryOwnerActiveError extends Error { } } +export class LiveDiscoveryStateError extends Error { + constructor(cause?: unknown) { + super('Existing Live discovery record is invalid.', { cause }); + this.name = 'LiveDiscoveryStateError'; + } +} + +export class LiveDiscoveryPublicationError extends Error { + readonly published = true; + + constructor(cause: unknown) { + super('Live discovery publication completed with cleanup errors.', { + cause, + }); + this.name = 'LiveDiscoveryPublicationError'; + } +} + export function getStableLiveDiscoveryBaseDir( homeDirectory = homedir(), ): string { @@ -125,6 +151,12 @@ function processIsAlive(pid: number): boolean { } } +function delay(milliseconds: number): Promise { + return new Promise((resolve) => { + setTimeout(resolve, milliseconds); + }); +} + async function readExistingRecord( filePath: string, ): Promise { @@ -138,40 +170,326 @@ async function readExistingRecord( if ( !stat.isFile() || stat.isSymbolicLink() || - (stat.mode & 0o777) !== 0o600 || - (typeof process.getuid === 'function' && stat.uid !== process.getuid()) || + stat.nlink !== 1 || + (process.platform !== 'win32' && + ((stat.mode & 0o777) !== 0o600 || + (typeof process.getuid === 'function' && + stat.uid !== process.getuid()))) || stat.size <= 0 || stat.size > MAX_DISCOVERY_BYTES ) { - throw new Error('Existing Live discovery record is invalid.'); + throw new LiveDiscoveryStateError(); } - let parsed: unknown; + let handle: fs.FileHandle; try { - parsed = JSON.parse(await fs.readFile(filePath, 'utf8')); + handle = await fs.open( + filePath, + process.platform === 'win32' + ? fsConstants.O_RDONLY + : fsConstants.O_RDONLY | fsConstants.O_NOFOLLOW, + ); } catch { - throw new Error('Existing Live discovery record is invalid.'); + throw new LiveDiscoveryStateError(); } - if (typeof parsed !== 'object' || parsed === null || Array.isArray(parsed)) { - throw new Error('Existing Live discovery record is invalid.'); + try { + const handleStat = await handle.stat(); + if ( + !handleStat.isFile() || + handleStat.nlink !== 1 || + handleStat.dev !== stat.dev || + handleStat.ino !== stat.ino || + handleStat.size !== stat.size || + (process.platform !== 'win32' && + ((handleStat.mode & 0o777) !== 0o600 || + (typeof process.getuid === 'function' && + handleStat.uid !== process.getuid()))) + ) { + throw new LiveDiscoveryStateError(); + } + let parsed: unknown; + try { + parsed = JSON.parse(await handle.readFile('utf8')); + } catch { + throw new LiveDiscoveryStateError(); + } + if ( + typeof parsed !== 'object' || + parsed === null || + Array.isArray(parsed) + ) { + throw new LiveDiscoveryStateError(); + } + const record = parsed as ExistingLiveDiscoveryRecord; + try { + assertSafeRecord(record); + } catch { + throw new LiveDiscoveryStateError(); + } + return record; + } finally { + await handle.close(); } - const record = parsed as ExistingLiveDiscoveryRecord; +} + +async function inspectDirectory( + directory: string, + requirePrivateMode = true, +): Promise { + const stat = await fs.lstat(directory); + if ( + !stat.isDirectory() || + stat.isSymbolicLink() || + (process.platform !== 'win32' && + ((requirePrivateMode && (stat.mode & 0o777) !== 0o700) || + (typeof process.getuid === 'function' && + stat.uid !== process.getuid()))) + ) { + throw new LiveDiscoveryStateError(); + } + return { directory, dev: stat.dev, ino: stat.ino }; +} + +async function assertDirectoryIdentity( + expected: LiveDiscoveryDirectory, + requirePrivateMode = true, +): Promise { + const current = await inspectDirectory( + expected.directory, + requirePrivateMode, + ); + if (current.dev !== expected.dev || current.ino !== expected.ino) { + throw new LiveDiscoveryStateError(); + } +} + +async function ensureDirectory( + directory: string, + requirePrivateMode: boolean, +): Promise { try { - assertSafeRecord(record); - } catch { - throw new Error('Existing Live discovery record is invalid.'); + return await inspectDirectory(directory, requirePrivateMode); + } catch (error) { + try { + await fs.lstat(directory); + throw error; + } catch (statError) { + if ((statError as NodeJS.ErrnoException).code !== 'ENOENT') throw error; + } + } + + const parent = path.dirname(directory); + if (parent === directory) throw new LiveDiscoveryStateError(); + const parentIdentity = await ensureDirectory(parent, false); + await assertDirectoryIdentity(parentIdentity, false); + let created = false; + try { + await fs.mkdir(directory, { mode: 0o700 }); + created = true; + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== 'EEXIST') throw error; } - return record; + if (created && process.platform !== 'win32') { + await fs.chmod(directory, 0o700); + } + await assertDirectoryIdentity(parentIdentity, false); + return inspectDirectory(directory, requirePrivateMode); } -async function prepareDirectory(runtimeBaseDir: string): Promise { +async function assertLockShapeIfPresent(lockPath: string): Promise { + let stat: Awaited>; + try { + stat = await fs.lstat(lockPath); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === 'ENOENT') return; + throw error; + } + if ( + !stat.isDirectory() || + stat.isSymbolicLink() || + (process.platform !== 'win32' && + typeof process.getuid === 'function' && + stat.uid !== process.getuid()) + ) { + throw new LiveDiscoveryStateError(); + } +} + +async function prepareDirectory( + runtimeBaseDir: string, + create = true, +): Promise { const directory = path.dirname(getLiveDiscoveryPath(runtimeBaseDir)); - await fs.mkdir(directory, { recursive: true, mode: 0o700 }); - const stat = await fs.lstat(directory); - if (!stat.isDirectory() || stat.isSymbolicLink()) { - throw new Error('Live discovery directory is not a regular directory.'); + let runtimeBaseIdentity: LiveDiscoveryDirectory; + try { + runtimeBaseIdentity = await inspectDirectory(runtimeBaseDir, false); + } catch (error) { + try { + await fs.lstat(runtimeBaseDir); + throw error; + } catch (statError) { + if ((statError as NodeJS.ErrnoException).code !== 'ENOENT') throw error; + } + if (!create) return undefined; + runtimeBaseIdentity = await ensureDirectory(runtimeBaseDir, false); + } + let created = false; + try { + const identity = await inspectDirectory(directory); + await assertDirectoryIdentity(runtimeBaseIdentity, false); + return identity; + } catch (error) { + try { + await fs.lstat(directory); + throw error; + } catch (statError) { + if ((statError as NodeJS.ErrnoException).code !== 'ENOENT') throw error; + } + } + if (!create) return undefined; + await assertDirectoryIdentity(runtimeBaseIdentity, false); + try { + await fs.mkdir(directory, { mode: 0o700 }); + created = true; + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== 'EEXIST') throw error; + } + if (created && process.platform !== 'win32') { + await fs.chmod(directory, 0o700); + } + await assertDirectoryIdentity(runtimeBaseIdentity, false); + return inspectDirectory(directory); +} + +async function lockDirectory(target: LiveDiscoveryDirectory): Promise<{ + release: () => Promise; + assertHealthy: () => void; +}> { + const lockPath = path.join(target.directory, '.daemon.lock'); + await assertLockShapeIfPresent(lockPath); + const lockfile = (await import('proper-lockfile')).default; + let compromised = false; + const release = await lockfile.lock(target.directory, { + ...LOCK_OPTIONS, + lockfilePath: lockPath, + onCompromised: () => { + compromised = true; + }, + }); + try { + await assertLockShapeIfPresent(lockPath); + } catch (error) { + let releaseError: unknown; + await release().catch((candidate: unknown) => { + releaseError = candidate; + }); + throw new LiveDiscoveryStateError( + releaseError + ? new AggregateError( + [error, releaseError], + 'Live discovery lock validation cleanup failed.', + ) + : error, + ); + } + return { + release, + assertHealthy: () => { + if (compromised) throw new LiveDiscoveryStateError(); + }, + }; +} + +async function syncDirectory(directory: string): Promise { + if (process.platform === 'win32') return; + const handle = await fs.open(directory, 'r'); + try { + await handle.sync(); + } finally { + await handle.close(); + } +} + +export async function handoffLiveDiscoveryOwner( + runtimeBaseDir: string, + owner: LiveDiscoveryOwner, + commitOwner: () => Promise, + options: { + isProcessAlive?: (pid: number) => boolean; + wait?: (milliseconds: number) => Promise; + handoffGraceMs?: number; + waitForHandoffGrace?: boolean; + } = {}, +): Promise<{ reclaimed: boolean }> { + const target = await prepareDirectory(runtimeBaseDir, false); + if (!target) { + await commitOwner(); + return { reclaimed: false }; } - await fs.chmod(directory, 0o700); - return directory; + const lock = await lockDirectory(target); + let result: { reclaimed: boolean } | undefined; + let operationError: unknown; + try { + lock.assertHealthy(); + await assertDirectoryIdentity(target); + const filePath = getLiveDiscoveryPath(runtimeBaseDir); + const current = await readExistingRecord(filePath); + if ( + current && + (current.pid !== owner.pid || + current.instanceNonce !== owner.instanceNonce) + ) { + if ( + current.pid === owner.pid || + (options.isProcessAlive ?? processIsAlive)(current.pid) + ) { + throw new LiveDiscoveryOwnerActiveError(current.pid); + } + await commitOwner(); + lock.assertHealthy(); + await assertDirectoryIdentity(target); + const confirmed = await readExistingRecord(filePath); + if ( + !confirmed || + confirmed.pid !== current.pid || + confirmed.instanceNonce !== current.instanceNonce + ) { + throw new LiveDiscoveryStateError(); + } + lock.assertHealthy(); + await assertDirectoryIdentity(target); + await fs.unlink(filePath); + await syncDirectory(target.directory); + lock.assertHealthy(); + result = { reclaimed: true }; + } else { + await commitOwner(); + lock.assertHealthy(); + result = { reclaimed: false }; + } + } catch (error) { + operationError = error; + } + let releaseError: unknown; + await lock.release().catch((error: unknown) => { + releaseError = error; + }); + if (releaseError) { + throw new LiveDiscoveryStateError( + operationError + ? new AggregateError( + [operationError, releaseError], + 'Live discovery handoff cleanup failed.', + ) + : releaseError, + ); + } + if (operationError) throw operationError; + if (result?.reclaimed && options.waitForHandoffGrace !== false) { + await (options.wait ?? delay)( + options.handoffGraceMs ?? DEFAULT_HANDOFF_GRACE_MS, + ); + } + return result!; } export async function writeLiveDiscoveryFile( @@ -180,23 +498,32 @@ export async function writeLiveDiscoveryFile( options: { isProcessAlive?: (pid: number) => boolean } = {}, ): Promise { assertRecord(record); - const directory = await prepareDirectory(runtimeBaseDir); + const target = await prepareDirectory(runtimeBaseDir); + if (!target) throw new LiveDiscoveryStateError(); + const { directory } = target; const filePath = getLiveDiscoveryPath(runtimeBaseDir); const temporaryPath = path.join( directory, `.daemon.${process.pid}.${randomUUID()}.tmp`, ); - const release = await lockfile.lock(directory, LOCK_OPTIONS); + const lock = await lockDirectory(target); let handle: fs.FileHandle | undefined; + let operationError: unknown; + let operationSucceeded = false; + let published = false; try { + lock.assertHealthy(); + await assertDirectoryIdentity(target); const current = await readExistingRecord(filePath); if ( current && (current.pid !== record.pid || - current.instanceNonce !== record.instanceNonce) && - (options.isProcessAlive ?? processIsAlive)(current.pid) + current.instanceNonce !== record.instanceNonce) ) { - throw new LiveDiscoveryOwnerActiveError(current.pid); + if ((options.isProcessAlive ?? processIsAlive)(current.pid)) { + throw new LiveDiscoveryOwnerActiveError(current.pid); + } + throw new LiveDiscoveryStateError(); } handle = await fs.open(temporaryPath, 'wx', 0o600); const serialized = `${JSON.stringify(record)}\n`; @@ -204,18 +531,49 @@ export async function writeLiveDiscoveryFile( throw new Error('Live discovery record is too large.'); } await handle.writeFile(serialized, 'utf8'); - await handle.sync(); await handle.chmod(0o600); + await handle.sync(); await handle.close(); handle = undefined; + lock.assertHealthy(); + await assertDirectoryIdentity(target); await fs.rename(temporaryPath, filePath); - await fs.chmod(filePath, 0o600); - return filePath; - } finally { - await handle?.close().catch(() => {}); - await fs.unlink(temporaryPath).catch(() => {}); - await release().catch(() => {}); + published = true; + await syncDirectory(directory); + lock.assertHealthy(); + operationSucceeded = true; + } catch (error) { + operationError = error; + } + const cleanupErrors: unknown[] = []; + await handle?.close().catch((error: unknown) => cleanupErrors.push(error)); + await fs.unlink(temporaryPath).catch((error: NodeJS.ErrnoException) => { + if (error.code !== 'ENOENT') cleanupErrors.push(error); + }); + await lock.release().catch((error: unknown) => cleanupErrors.push(error)); + if (!operationSucceeded) { + const cause = + cleanupErrors.length > 0 + ? new AggregateError( + [operationError, ...cleanupErrors], + 'Live discovery publication cleanup failed.', + ) + : operationError; + if (published) throw new LiveDiscoveryPublicationError(cause); + if (cleanupErrors.length > 0) { + throw cause; + } + throw operationError; } + if (cleanupErrors.length > 0) { + throw new LiveDiscoveryPublicationError( + new AggregateError( + cleanupErrors, + 'Live discovery publication cleanup failed.', + ), + ); + } + return filePath; } export async function removeLiveDiscoveryFile( @@ -227,46 +585,30 @@ export async function removeLiveDiscoveryFile( !Number.isSafeInteger(owner.pid) || owner.pid <= 0 ) { - return false; + throw new LiveDiscoveryStateError(); } - const directory = await prepareDirectory(runtimeBaseDir); + const target = await prepareDirectory(runtimeBaseDir, false); + if (!target) return false; const filePath = getLiveDiscoveryPath(runtimeBaseDir); - const release = await lockfile.lock(directory, LOCK_OPTIONS); + const lock = await lockDirectory(target); try { - let stat: Awaited>; - try { - stat = await fs.lstat(filePath); - } catch (error) { - if ((error as NodeJS.ErrnoException).code === 'ENOENT') return false; - throw error; - } - if ( - !stat.isFile() || - stat.isSymbolicLink() || - stat.size <= 0 || - stat.size > MAX_DISCOVERY_BYTES - ) { - return false; - } - let current: unknown; - try { - current = JSON.parse(await fs.readFile(filePath, 'utf8')); - } catch { - return false; - } + lock.assertHealthy(); + await assertDirectoryIdentity(target); + const current = await readExistingRecord(filePath); + if (!current) return false; if ( - typeof current !== 'object' || - current === null || - Array.isArray(current) || - (current as { instanceNonce?: unknown }).instanceNonce !== - owner.instanceNonce || - (current as { pid?: unknown }).pid !== owner.pid + current.instanceNonce !== owner.instanceNonce || + current.pid !== owner.pid ) { - return false; + throw new LiveDiscoveryStateError(); } + lock.assertHealthy(); + await assertDirectoryIdentity(target); await fs.unlink(filePath); + await syncDirectory(target.directory); + lock.assertHealthy(); return true; } finally { - await release().catch(() => {}); + await lock.release(); } } diff --git a/packages/cli/src/serve/live/live-task-service.test.ts b/packages/cli/src/serve/live/live-task-service.test.ts index b2d871b1ded..65668667df0 100644 --- a/packages/cli/src/serve/live/live-task-service.test.ts +++ b/packages/cli/src/serve/live/live-task-service.test.ts @@ -262,7 +262,8 @@ function makeHarness() { bridge: projectBridge, } as WorkspaceRuntime; const registry = { - list: () => [runtime, projectRuntime], + list: () => [projectRuntime], + listAll: () => [runtime, projectRuntime], getByWorkspaceId: (workspaceId: string) => workspaceId === projectRuntime.workspaceId ? projectRuntime : undefined, resolveLiveSessionOwner: (sessionId: string) => @@ -298,6 +299,7 @@ function makeHarness() { bridge, projectBridge, runtime, + registry, summaries, resident, sendPrompt, @@ -321,6 +323,24 @@ beforeEach(() => { }); describe('LiveTaskService', () => { + it('preserves the structured unavailable error for an inactive internal owner', async () => { + const harness = makeHarness(); + vi.spyOn(harness.registry, 'resolveLiveSessionOwner').mockReturnValue({ + kind: 'unavailable', + }); + + await expect( + harness.service.handle({ + callerSessionId: 'live-root', + name: 'read_thread', + arguments: { threadId: 'inactive-task' }, + }), + ).rejects.toMatchObject({ + code: 'conversation_runtime_unavailable', + retryable: true, + }); + }); + it('lists existing tasks in the current Codex wire shape without creating one', async () => { const harness = makeHarness(); listWorkspaceSessionsForResponse diff --git a/packages/cli/src/serve/live/live-task-service.ts b/packages/cli/src/serve/live/live-task-service.ts index 3c1ce7e27eb..0b15214d628 100644 --- a/packages/cli/src/serve/live/live-task-service.ts +++ b/packages/cli/src/serve/live/live-task-service.ts @@ -38,6 +38,7 @@ import { isCompatibleLiveSessionSource, readLoadableLiveConversationMetadata, } from '../conversations/session-source.js'; +import { conversationRuntimeUnavailableError } from '../conversations/conversation-runtime-errors.js'; const DEFAULT_LIST_LIMIT = 20; const DEFAULT_READ_TURN_LIMIT = 3; @@ -504,7 +505,7 @@ export class LiveTaskService { const all = ( await Promise.all( this.options.workspaceRegistry - .list() + .listAll() .map((runtime) => this.listRuntimeThreads(runtime, limit)), ) ) @@ -1126,12 +1127,18 @@ export class LiveTaskService { if (live.kind === 'ambiguous') { throw new Error(`Task id is ambiguous: ${threadId}`); } + if (live.kind === 'unavailable') { + throw conversationRuntimeUnavailableError(); + } const runtimes = live.kind === 'found' ? [live.runtime] : ( await Promise.all( - this.options.workspaceRegistry.list().map(async (runtime) => ({ + ( + this.options.workspaceRegistry.listAll?.() ?? + this.options.workspaceRegistry.list() + ).map(async (runtime) => ({ runtime, exists: await createWorkspaceRuntimeSessionService( diff --git a/packages/cli/src/serve/live/realtime-startup-context.test.ts b/packages/cli/src/serve/live/realtime-startup-context.test.ts index f1629c728be..ec6ce412c58 100644 --- a/packages/cli/src/serve/live/realtime-startup-context.test.ts +++ b/packages/cli/src/serve/live/realtime-startup-context.test.ts @@ -20,6 +20,9 @@ import { const sessionData = vi.hoisted(() => new Map()); const sessionLists = vi.hoisted(() => new Map()); +const sessionServiceConstructions = vi.hoisted( + () => [] as Array<{ cwd: string; runtimeBaseDir?: string }>, +); vi.mock('@qwen-code/qwen-code-core', async (importOriginal) => { const actual = @@ -27,7 +30,17 @@ vi.mock('@qwen-code/qwen-code-core', async (importOriginal) => { return { ...actual, SessionService: class { - constructor(private readonly cwd: string) {} + constructor( + private readonly cwd: string, + options?: { runtimeBaseDir?: string }, + ) { + sessionServiceConstructions.push({ + cwd, + ...(options?.runtimeBaseDir + ? { runtimeBaseDir: options.runtimeBaseDir } + : {}), + }); + } loadSession(sessionId: string) { return Promise.resolve(sessionData.get(sessionId)); @@ -75,6 +88,7 @@ function record( afterEach(() => { sessionData.clear(); sessionLists.clear(); + sessionServiceConstructions.length = 0; for (const directory of temporaryDirectories.splice(0)) { rmSync(directory, { recursive: true, force: true }); } @@ -115,6 +129,7 @@ describe('buildRealtimeStartupContext', () => { const runtime = { workspaceCwd: repo } as WorkspaceRuntime; const workspaceRegistry = { list: () => [runtime], + listAll: () => [runtime], } as unknown as WorkspaceRegistry; const context = await buildRealtimeStartupContext({ @@ -149,6 +164,7 @@ describe('buildRealtimeStartupContext', () => { const runtime = { workspaceCwd: root } as WorkspaceRuntime; const workspaceRegistry = { list: () => [runtime], + listAll: () => [runtime], } as unknown as WorkspaceRegistry; await expect( @@ -173,6 +189,7 @@ describe('buildRealtimeStartupContext', () => { const runtime = { workspaceCwd: root } as WorkspaceRuntime; const workspaceRegistry = { list: () => [runtime], + listAll: () => [runtime], } as unknown as WorkspaceRegistry; const context = await buildRealtimeStartupContext({ @@ -187,6 +204,48 @@ describe('buildRealtimeStartupContext', () => { expect(context).not.toContain('## Recent Work'); }); + it('reads the current and recent catalogs from each runtime base', async () => { + const root = temporaryDirectory(); + const currentCwd = join(root, 'current'); + const home = join(root, 'home'); + mkdirSync(currentCwd); + mkdirSync(home); + writeFileSync(join(currentCwd, 'README.md'), 'available workspace'); + const primary = { + workspaceCwd: join(root, 'primary'), + sessionRuntimeBaseDir: join(root, 'primary-runtime'), + } as WorkspaceRuntime; + const internal = { + workspaceCwd: join(root, 'conversations'), + sessionRuntimeBaseDir: join(root, 'conversation-runtime'), + provenance: 'live-conversation', + } as WorkspaceRuntime; + const workspaceRegistry = { + listAll: () => [primary, internal], + } as unknown as WorkspaceRegistry; + + await buildRealtimeStartupContext({ + runtime: internal, + workspaceRegistry, + sessionId: 'missing', + currentCwd, + userRoot: home, + }); + + expect(sessionServiceConstructions).toEqual( + expect.arrayContaining([ + { + cwd: internal.workspaceCwd, + runtimeBaseDir: internal.sessionRuntimeBaseDir, + }, + { + cwd: primary.workspaceCwd, + runtimeBaseDir: primary.sessionRuntimeBaseDir, + }, + ]), + ); + }); + it('preserves the start and end of an over-budget turn', () => { const text = `turn-start ${'middle '.repeat(1_000)} turn-end`; const truncated = truncateRealtimeTextToTokenBudget(text, 100); diff --git a/packages/cli/src/serve/live/realtime-startup-context.ts b/packages/cli/src/serve/live/realtime-startup-context.ts index 9caffa9147f..81c57ee941e 100644 --- a/packages/cli/src/serve/live/realtime-startup-context.ts +++ b/packages/cli/src/serve/live/realtime-startup-context.ts @@ -11,7 +11,6 @@ import { createDebugLogger, findGitRoot, partToString, - SessionService, type ChatRecord, type SessionListItem, } from '@qwen-code/qwen-code-core'; @@ -19,6 +18,7 @@ import type { WorkspaceRegistry, WorkspaceRuntime, } from '../workspace-registry.js'; +import { createWorkspaceRuntimeSessionService } from '../workspace-runtime-storage.js'; const STARTUP_CONTEXT_HEADER = 'Startup context from Qwen Code.\nThis is background context about recent work and machine/workspace layout. It may be incomplete or stale. Use it to inform responses, and do not repeat it back unless relevant.'; @@ -208,9 +208,9 @@ async function loadRecentThreads( ): Promise { try { const pages = await Promise.all( - workspaceRegistry.list().map(async (runtime) => { - const page = await new SessionService( - runtime.workspaceCwd, + workspaceRegistry.listAll().map(async (runtime) => { + const page = await createWorkspaceRuntimeSessionService( + runtime, ).listSessions({ size: MAX_RECENT_THREADS, archiveState: 'active', @@ -382,7 +382,7 @@ function formatSection( export async function buildRealtimeStartupContext( options: RealtimeStartupContextOptions, ): Promise { - const current = await new SessionService(options.runtime.workspaceCwd) + const current = await createWorkspaceRuntimeSessionService(options.runtime) .loadSession(options.sessionId) .catch(() => undefined); const currentThread = buildCurrentThreadSection( diff --git a/packages/cli/src/serve/live/run-qwen-serve-live.test.ts b/packages/cli/src/serve/live/run-qwen-serve-live.test.ts index e97561bce80..8846b644524 100644 --- a/packages/cli/src/serve/live/run-qwen-serve-live.test.ts +++ b/packages/cli/src/serve/live/run-qwen-serve-live.test.ts @@ -162,6 +162,21 @@ describe('qwen serve Live Host discovery', () => { process.env['QWEN_HOME'] = qwenHome; process.env['QWEN_RUNTIME_DIR'] = runtime; resetHomeEnvBootstrapForTesting(); + const runtimeDiscoveryPath = getLiveDiscoveryPath(runtime); + await fs.mkdir(path.dirname(runtimeDiscoveryPath), { + recursive: true, + mode: 0o700, + }); + await fs.writeFile( + runtimeDiscoveryPath, + `${JSON.stringify({ + url: 'http://127.0.0.1:1', + protocolVersion: LIVE_HOST_PROTOCOL_VERSION, + pid: 999_999, + instanceNonce: 'stale_runtime_owner_nonce_0001', + })}\n`, + { mode: 0o600 }, + ); let handle: Awaited> | undefined; try { handle = await runQwenServe( diff --git a/packages/cli/src/serve/multi-workspace-sessions.test.ts b/packages/cli/src/serve/multi-workspace-sessions.test.ts index 896ce17ae85..5b292bd4a46 100644 --- a/packages/cli/src/serve/multi-workspace-sessions.test.ts +++ b/packages/cli/src/serve/multi-workspace-sessions.test.ts @@ -7,7 +7,7 @@ import * as path from 'node:path'; import { promises as fsp } from 'node:fs'; import * as os from 'node:os'; -import { describe, expect, it, vi } from 'vitest'; +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import request from 'supertest'; import { SESSION_TRANSCRIPT_MAX_INDEX_BYTES, @@ -52,6 +52,15 @@ const SECONDARY_CWD = path.resolve(path.sep, 'work', 'secondary'); const UNKNOWN_CWD = path.resolve(path.sep, 'work', 'unknown'); const TEST_TOKEN = 'test-token'; const TEST_AUTHORIZATION = `Bearer ${TEST_TOKEN}`; +const LIVE_COLD_LOAD_ID = '550e8400-e29b-41d4-a716-446655440101'; +const LIVE_COLD_RESUME_ID = '550e8400-e29b-41d4-a716-446655440102'; +const LIVE_PROJECTLESS_TASK_ID = '550e8400-e29b-41d4-a716-446655440103'; +const LIVE_ACTIVE_LOAD_ID = '550e8400-e29b-41d4-a716-446655440104'; +const LIVE_ACTIVE_RESUME_ID = '550e8400-e29b-41d4-a716-446655440105'; +const LIVE_COLD_REJECTED_ID = '550e8400-e29b-41d4-a716-446655440106'; +const LIVE_COLD_ATTACHED_ID = '550e8400-e29b-41d4-a716-446655440107'; +const LIVE_COLD_REAP_REJECTED_ID = '550e8400-e29b-41d4-a716-446655440108'; +const LIVE_INVALID_CHILD_ID = '550e8400-e29b-41d4-a716-446655440109'; const baseOpts: ServeOptions = { hostname: '127.0.0.1', @@ -288,9 +297,10 @@ async function withRuntimeDir(fn: () => Promise): Promise { async function withStoredLiveCoordinators( sessionIds: readonly string[], - fn: () => Promise, + fn: (runtimeDir: string) => Promise, ): Promise { return withRuntimeDir(async () => { + const runtimeDir = Storage.getRuntimeBaseDir(); for (const sessionId of sessionIds) { await writeStoredSession({ sessionId, @@ -302,15 +312,16 @@ async function withStoredLiveCoordinators( sourceId: `${LIVE_SESSION_SOURCE_PREFIX}${sessionId}`, }); } - return fn(); + return fn(runtimeDir); }); } async function withStoredProjectlessLiveTasks( sessionIds: readonly string[], - fn: () => Promise, + fn: (runtimeDir: string) => Promise, ): Promise { return withRuntimeDir(async () => { + const runtimeDir = Storage.getRuntimeBaseDir(); for (const sessionId of sessionIds) { await writeStoredSession({ sessionId, @@ -321,17 +332,20 @@ async function withStoredProjectlessLiveTasks( sourceType: 'default', }); } - return fn(); + return fn(runtimeDir); }); } async function withStoredLiveWorkers( sessionIds: readonly string[], - fn: () => Promise, + fn: (runtimeDir: string) => Promise, ): Promise { return withRuntimeDir(async () => { - for (const sessionId of sessionIds) { - const parentSessionId = `${sessionId}-coordinator`; + const runtimeDir = Storage.getRuntimeBaseDir(); + for (const [index, sessionId] of sessionIds.entries()) { + const parentSessionId = `00000000-0000-4000-8000-${String( + index + 1, + ).padStart(12, '0')}`; await writeStoredSession({ sessionId: parentSessionId, cwd: SECONDARY_CWD, @@ -350,7 +364,7 @@ async function withStoredLiveWorkers( parentSessionId, }); } - return fn(); + return fn(runtimeDir); }); } @@ -797,15 +811,31 @@ function makeBridge( }, async branchSession(sessionId: string) { primaryOnlyMutationCalls.push({ route: 'branch', sessionId }); - throw new Error('Unexpected branchSession call'); + return { + sessionId: `${sessionId}-branch`, + workspaceCwd, + attached: false, + clientId: 'branch-client', + state: {}, + displayName: 'Branch', + forkedFrom: { sessionId, displayName: 'Source' }, + }; }, async createSideTaskSession(sessionId: string) { primaryOnlyMutationCalls.push({ route: 'side-task', sessionId }); - throw new Error('Unexpected createSideTaskSession call'); + return { + sessionId: `${sessionId}-side-task`, + workspaceCwd, + attached: false, + clientId: 'side-task-client', + state: {}, + displayName: 'Side task', + parentSessionId: sessionId, + }; }, - async launchSessionForkAgent(sessionId: string) { + async launchSessionForkAgent(sessionId: string, directive: string) { primaryOnlyMutationCalls.push({ route: 'fork', sessionId }); - throw new Error('Unexpected launchSessionForkAgent call'); + return { sessionId, description: directive, launched: true }; }, async changeSessionCwd( sessionId: string, @@ -1043,6 +1073,26 @@ function host() { } describe('multi-workspace session dispatch', () => { + let previousRuntimeDir: string | undefined; + let testRuntimeDir: string; + + beforeEach(async () => { + previousRuntimeDir = process.env['QWEN_RUNTIME_DIR']; + testRuntimeDir = await fsp.mkdtemp( + path.join(os.tmpdir(), 'qwen-multi-workspace-test-'), + ); + process.env['QWEN_RUNTIME_DIR'] = testRuntimeDir; + }); + + afterEach(async () => { + if (previousRuntimeDir === undefined) { + delete process.env['QWEN_RUNTIME_DIR']; + } else { + process.env['QWEN_RUNTIME_DIR'] = previousRuntimeDir; + } + await fsp.rm(testRuntimeDir, { recursive: true, force: true }); + }); + it('advertises workspaces and multi_workspace_sessions only when multiple runtimes are registered', async () => { const { app } = makeHarness(); const res = await request(app).get('/capabilities').set('Host', host()); @@ -1813,8 +1863,8 @@ describe('multi-workspace session dispatch', () => { it('restores cold Live load and resume sessions into their server-derived conversation directories', async () => { await withStoredLiveCoordinators( - ['live-cold-load', 'live-cold-resume'], - async () => { + [LIVE_COLD_LOAD_ID, LIVE_COLD_RESUME_ID], + async (runtimeDir) => { const operationLog: string[] = []; const materializeConversationDirectory = vi.fn( async (sessionId: string) => { @@ -1834,11 +1884,15 @@ describe('multi-workspace session dispatch', () => { liveConversationWorkspace: { materializeConversationDirectory, } as unknown as ConversationWorkspace, + secondaryRuntimeBaseDir: runtimeDir, }); - for (const action of ['load', 'resume'] as const) { + for (const { action, sessionId } of [ + { action: 'load', sessionId: LIVE_COLD_LOAD_ID }, + { action: 'resume', sessionId: LIVE_COLD_RESUME_ID }, + ] as const) { const response = await request(app) - .post(`/session/live-cold-${action}/${action}`) + .post(`/session/${sessionId}/${action}`) .set('Host', host()) .send({ cwd: SECONDARY_CWD }); @@ -1847,26 +1901,32 @@ describe('multi-workspace session dispatch', () => { } expect(operationLog).toEqual([ - 'materialize:live-cold-load', - 'load:live-cold-load', - 'change:live-cold-load', - 'materialize:live-cold-resume', - 'resume:live-cold-resume', - 'change:live-cold-resume', + `materialize:${LIVE_COLD_LOAD_ID}`, + `load:${LIVE_COLD_LOAD_ID}`, + `change:${LIVE_COLD_LOAD_ID}`, + `materialize:${LIVE_COLD_RESUME_ID}`, + `resume:${LIVE_COLD_RESUME_ID}`, + `change:${LIVE_COLD_RESUME_ID}`, ]); expect(secondaryBridge.cwdChangeCalls).toEqual([ { - sessionId: 'live-cold-load', + sessionId: LIVE_COLD_LOAD_ID, request: { - path: path.join(SECONDARY_CWD, 'conversation-live-cold-load'), + path: path.join( + SECONDARY_CWD, + `conversation-${LIVE_COLD_LOAD_ID}`, + ), allowedRoots: [SECONDARY_CWD], managedRelocation: 'live-conversation', }, }, { - sessionId: 'live-cold-resume', + sessionId: LIVE_COLD_RESUME_ID, request: { - path: path.join(SECONDARY_CWD, 'conversation-live-cold-resume'), + path: path.join( + SECONDARY_CWD, + `conversation-${LIVE_COLD_RESUME_ID}`, + ), allowedRoots: [SECONDARY_CWD], managedRelocation: 'live-conversation', }, @@ -1879,8 +1939,8 @@ describe('multi-workspace session dispatch', () => { it('loads a projectless task created from Live in the Conversations runtime', async () => { await withStoredProjectlessLiveTasks( - ['live-projectless-task'], - async () => { + [LIVE_PROJECTLESS_TASK_ID], + async (runtimeDir) => { const materializeConversationDirectory = vi.fn( async (sessionId: string) => path.join(SECONDARY_CWD, `conversation-${sessionId}`), @@ -1896,10 +1956,11 @@ describe('multi-workspace session dispatch', () => { liveConversationWorkspace: { materializeConversationDirectory, } as unknown as ConversationWorkspace, + secondaryRuntimeBaseDir: runtimeDir, }); const response = await request(app) - .post('/session/live-projectless-task/load') + .post(`/session/${LIVE_PROJECTLESS_TASK_ID}/load`) .set('Host', host()) .send({ cwd: SECONDARY_CWD }); @@ -1908,29 +1969,77 @@ describe('multi-workspace session dispatch', () => { { action: 'load', req: expect.objectContaining({ - sessionId: 'live-projectless-task', + sessionId: LIVE_PROJECTLESS_TASK_ID, workspaceCwd: SECONDARY_CWD, sourceType: 'default', }), }, ]); expect(materializeConversationDirectory).toHaveBeenCalledWith( - 'live-projectless-task', + LIVE_PROJECTLESS_TASK_ID, ); }, ); }); + it('rejects an unqualified batch mutation when a Live session id collides with an ordinary transcript', async () => { + await withRuntimeDir(async () => { + const sessionId = '550e8400-e29b-41d4-a716-446655440110'; + const runtimeDir = Storage.getRuntimeBaseDir(); + const storedAt = new Date('2026-07-08T00:00:00.000Z'); + await writeStoredSession({ + sessionId, + cwd: PRIMARY_CWD, + timestamp: storedAt.toISOString(), + prompt: 'ordinary collision', + mtime: storedAt, + }); + await writeStoredSession({ + sessionId, + cwd: SECONDARY_CWD, + timestamp: storedAt.toISOString(), + prompt: 'Live collision', + mtime: storedAt, + sourceType: 'default', + sourceId: `${LIVE_SESSION_SOURCE_PREFIX}${sessionId}`, + }); + const { app } = makeHarness({ + secondaryProvenance: 'live-conversation', + secondaryRuntimeBaseDir: runtimeDir, + }); + + const response = await request(app) + .post('/sessions/archive') + .set('Host', host()) + .send({ sessionIds: [sessionId] }); + + expect(response.status).toBe(500); + expect(response.body).toMatchObject({ + code: 'ambiguous_session_owner', + sessionId, + }); + expect(response.body).not.toHaveProperty('workspaceIds'); + await expect( + new SessionService(PRIMARY_CWD).getSessionLocation(sessionId), + ).resolves.toBe('active'); + await expect( + new SessionService(SECONDARY_CWD).getSessionLocation(sessionId), + ).resolves.toBe('active'); + }); + }); + it('opens attached active Live workers without waiting for cwd relocation', async () => { await withStoredLiveWorkers( - ['live-active-load', 'live-active-resume'], - async () => { + [LIVE_ACTIVE_LOAD_ID, LIVE_ACTIVE_RESUME_ID], + async (runtimeDir) => { const materializeConversationDirectory = vi.fn( async (sessionId: string) => path.join(SECONDARY_CWD, `conversation-${sessionId}`), ); - for (const action of ['load', 'resume'] as const) { - const sessionId = `live-active-${action}`; + for (const { action, sessionId } of [ + { action: 'load', sessionId: LIVE_ACTIVE_LOAD_ID }, + { action: 'resume', sessionId: LIVE_ACTIVE_RESUME_ID }, + ] as const) { const { app, secondaryBridge } = makeHarness({ secondaryProvenance: 'live-conversation', secondaryRestoreAttached: true, @@ -1942,6 +2051,7 @@ describe('multi-workspace session dispatch', () => { liveConversationWorkspace: { materializeConversationDirectory, } as unknown as ConversationWorkspace, + secondaryRuntimeBaseDir: runtimeDir, }); const response = await request(app) @@ -1961,129 +2071,145 @@ describe('multi-workspace session dispatch', () => { }); it('fails closed and rolls back a cold Live restore when conversation relocation is rejected', async () => { - await withStoredLiveCoordinators(['live-cold-rejected'], async () => { - const operationLog: string[] = []; - const { app, secondaryBridge } = makeHarness({ - secondaryProvenance: 'live-conversation', - secondaryOperationLog: operationLog, - secondaryChangeSessionCwdImpl: async (sessionId, req) => ({ - sessionId, - previousCwd: SECONDARY_CWD, - newCwd: `${req.path}-rejected`, - warnings: [], - }), - liveConversationWorkspace: { - materializeConversationDirectory: async (sessionId: string) => { - operationLog.push(`materialize:${sessionId}`); - return path.join(SECONDARY_CWD, `conversation-${sessionId}`); - }, - } as unknown as ConversationWorkspace, - }); + await withStoredLiveCoordinators( + [LIVE_COLD_REJECTED_ID], + async (runtimeDir) => { + const operationLog: string[] = []; + const { app, secondaryBridge } = makeHarness({ + secondaryProvenance: 'live-conversation', + secondaryOperationLog: operationLog, + secondaryChangeSessionCwdImpl: async (sessionId, req) => ({ + sessionId, + previousCwd: SECONDARY_CWD, + newCwd: `${req.path}-rejected`, + warnings: [], + }), + liveConversationWorkspace: { + materializeConversationDirectory: async (sessionId: string) => { + operationLog.push(`materialize:${sessionId}`); + return path.join(SECONDARY_CWD, `conversation-${sessionId}`); + }, + } as unknown as ConversationWorkspace, + secondaryRuntimeBaseDir: runtimeDir, + }); - const response = await request(app) - .post('/session/live-cold-rejected/resume') - .set('Host', host()) - .send({ cwd: SECONDARY_CWD }); + const response = await request(app) + .post(`/session/${LIVE_COLD_REJECTED_ID}/resume`) + .set('Host', host()) + .send({ cwd: SECONDARY_CWD }); - expect(response.status).toBe(500); - expect(secondaryBridge.killCalls).toEqual(['live-cold-rejected']); - expect(operationLog).toEqual([ - 'materialize:live-cold-rejected', - 'resume:live-cold-rejected', - 'change:live-cold-rejected', - 'kill:live-cold-rejected', - ]); - expect(secondaryBridge.closeCalls).toEqual([]); - expect(secondaryBridge.detachCalls).toEqual([]); - }); + expect(response.status).toBe(500); + expect(secondaryBridge.killCalls).toEqual([LIVE_COLD_REJECTED_ID]); + expect(operationLog).toEqual([ + `materialize:${LIVE_COLD_REJECTED_ID}`, + `resume:${LIVE_COLD_REJECTED_ID}`, + `change:${LIVE_COLD_REJECTED_ID}`, + `kill:${LIVE_COLD_REJECTED_ID}`, + ]); + expect(secondaryBridge.closeCalls).toEqual([]); + expect(secondaryBridge.detachCalls).toEqual([]); + }, + ); }); it('only detaches its lease when an attached cold Live restore relocation is rejected', async () => { - await withStoredLiveCoordinators(['live-cold-attached'], async () => { - const { app, secondaryBridge } = makeHarness({ - secondaryProvenance: 'live-conversation', - secondaryRestoreAttached: true, - secondaryChangeSessionCwdImpl: async (sessionId, req) => ({ - sessionId, - previousCwd: SECONDARY_CWD, - newCwd: `${req.path}-rejected`, - warnings: [], - }), - liveConversationWorkspace: { - materializeConversationDirectory: async (sessionId: string) => - path.join(SECONDARY_CWD, `conversation-${sessionId}`), - } as unknown as ConversationWorkspace, - }); + await withStoredLiveCoordinators( + [LIVE_COLD_ATTACHED_ID], + async (runtimeDir) => { + const { app, secondaryBridge } = makeHarness({ + secondaryProvenance: 'live-conversation', + secondaryRestoreAttached: true, + secondaryChangeSessionCwdImpl: async (sessionId, req) => ({ + sessionId, + previousCwd: SECONDARY_CWD, + newCwd: `${req.path}-rejected`, + warnings: [], + }), + liveConversationWorkspace: { + materializeConversationDirectory: async (sessionId: string) => + path.join(SECONDARY_CWD, `conversation-${sessionId}`), + } as unknown as ConversationWorkspace, + secondaryRuntimeBaseDir: runtimeDir, + }); - const response = await request(app) - .post('/session/live-cold-attached/resume') - .set('Host', host()) - .send({ cwd: SECONDARY_CWD }); + const response = await request(app) + .post(`/session/${LIVE_COLD_ATTACHED_ID}/resume`) + .set('Host', host()) + .send({ cwd: SECONDARY_CWD }); - expect(response.status).toBe(500); - expect(secondaryBridge.detachCalls).toEqual(['live-cold-attached']); - expect(secondaryBridge.killCalls).toEqual([]); - expect(secondaryBridge.closeCalls).toEqual([]); - }); + expect(response.status).toBe(500); + expect(secondaryBridge.detachCalls).toEqual([LIVE_COLD_ATTACHED_ID]); + expect(secondaryBridge.killCalls).toEqual([]); + expect(secondaryBridge.closeCalls).toEqual([]); + }, + ); }); it('does not force-close a cold Live restore when zero-attach reap is rejected', async () => { - await withStoredLiveCoordinators(['live-cold-reap-rejected'], async () => { - const { app, secondaryBridge } = makeHarness({ - secondaryProvenance: 'live-conversation', - secondaryKillSessionResult: false, - secondaryChangeSessionCwdImpl: async (sessionId, req) => ({ - sessionId, - previousCwd: SECONDARY_CWD, - newCwd: `${req.path}-rejected`, - warnings: [], - }), - liveConversationWorkspace: { - materializeConversationDirectory: async (sessionId: string) => - path.join(SECONDARY_CWD, `conversation-${sessionId}`), - } as unknown as ConversationWorkspace, - }); + await withStoredLiveCoordinators( + [LIVE_COLD_REAP_REJECTED_ID], + async (runtimeDir) => { + const { app, secondaryBridge } = makeHarness({ + secondaryProvenance: 'live-conversation', + secondaryKillSessionResult: false, + secondaryChangeSessionCwdImpl: async (sessionId, req) => ({ + sessionId, + previousCwd: SECONDARY_CWD, + newCwd: `${req.path}-rejected`, + warnings: [], + }), + liveConversationWorkspace: { + materializeConversationDirectory: async (sessionId: string) => + path.join(SECONDARY_CWD, `conversation-${sessionId}`), + } as unknown as ConversationWorkspace, + secondaryRuntimeBaseDir: runtimeDir, + }); - const response = await request(app) - .post('/session/live-cold-reap-rejected/resume') - .set('Host', host()) - .send({ cwd: SECONDARY_CWD }); + const response = await request(app) + .post(`/session/${LIVE_COLD_REAP_REJECTED_ID}/resume`) + .set('Host', host()) + .send({ cwd: SECONDARY_CWD }); - expect(response.status).toBe(500); - expect(secondaryBridge.killCalls).toEqual(['live-cold-reap-rejected']); - expect(secondaryBridge.closeCalls).toEqual([]); - }); + expect(response.status).toBe(500); + expect(secondaryBridge.killCalls).toEqual([LIVE_COLD_REAP_REJECTED_ID]); + expect(secondaryBridge.closeCalls).toEqual([]); + }, + ); }); it('does not restore a cold Live session when conversation workspace validation fails', async () => { - await withStoredLiveCoordinators(['live-invalid-child'], async () => { - const { app, secondaryBridge } = makeHarness({ - secondaryProvenance: 'live-conversation', - secondaryChangeSessionCwdImpl: async (sessionId, req) => ({ - sessionId, - previousCwd: SECONDARY_CWD, - newCwd: req.path, - warnings: [], - }), - liveConversationWorkspace: { - materializeConversationDirectory: async () => { - throw new Error( - 'Live conversation child was replaced by a symlink.', - ); - }, - } as unknown as ConversationWorkspace, - }); + await withStoredLiveCoordinators( + [LIVE_INVALID_CHILD_ID], + async (runtimeDir) => { + const { app, secondaryBridge } = makeHarness({ + secondaryProvenance: 'live-conversation', + secondaryChangeSessionCwdImpl: async (sessionId, req) => ({ + sessionId, + previousCwd: SECONDARY_CWD, + newCwd: req.path, + warnings: [], + }), + liveConversationWorkspace: { + materializeConversationDirectory: async () => { + throw new Error( + 'Live conversation child was replaced by a symlink.', + ); + }, + } as unknown as ConversationWorkspace, + secondaryRuntimeBaseDir: runtimeDir, + }); - const response = await request(app) - .post('/session/live-invalid-child/load') - .set('Host', host()) - .send({ cwd: SECONDARY_CWD }); + const response = await request(app) + .post(`/session/${LIVE_INVALID_CHILD_ID}/load`) + .set('Host', host()) + .send({ cwd: SECONDARY_CWD }); - expect(response.status).toBe(500); - expect(secondaryBridge.restoreCalls).toEqual([]); - expect(secondaryBridge.cwdChangeCalls).toEqual([]); - expect(secondaryBridge.killCalls).toEqual([]); - }); + expect(response.status).toBe(500); + expect(secondaryBridge.restoreCalls).toEqual([]); + expect(secondaryBridge.cwdChangeCalls).toEqual([]); + expect(secondaryBridge.killCalls).toEqual([]); + }, + ); }); it('rejects unknown and untrusted restore cwd before touching a bridge', async () => { @@ -2175,6 +2301,65 @@ describe('multi-workspace session dispatch', () => { }, ); + it('does not expose the Conversations runtime identity in an unsupported session route response', async () => { + const { app } = makeHarness({ + secondaryProvenance: 'live-conversation', + }); + + const response = await request(app) + .post('/session/secondary-session/cd') + .set('Host', host()) + .send({ path: path.resolve(path.sep, 'work', 'next') }); + + expect(response.status).toBe(400); + expect(response.body).toEqual({ + error: + 'Route "POST /session/:id/cd" is only available for primary workspace sessions.', + code: 'non_primary_session_route_not_supported', + sessionId: 'secondary-session', + route: 'POST /session/:id/cd', + }); + }); + + it.each([ + { + suffix: 'branch', + body: { name: 'next' }, + expectedStatus: 201, + mutation: 'branch', + }, + { + suffix: 'side-task', + body: { name: 'research' }, + expectedStatus: 201, + mutation: 'side-task', + }, + { + suffix: 'fork', + body: { directive: 'review this' }, + expectedStatus: 202, + mutation: 'fork', + }, + ] as const)( + 'routes an internal owner session $suffix operation to its bridge', + async ({ suffix, body, expectedStatus, mutation }) => { + const { app, primaryBridge, secondaryBridge } = makeHarness({ + secondaryProvenance: 'live-conversation', + }); + + const response = await request(app) + .post(`/session/secondary-session/${suffix}`) + .set('Host', host()) + .send(body); + + expect(response.status).toBe(expectedStatus); + expect(primaryBridge.primaryOnlyMutationCalls).toEqual([]); + expect(secondaryBridge.primaryOnlyMutationCalls).toEqual([ + { route: mutation, sessionId: 'secondary-session' }, + ]); + }, + ); + it('routes POST /session/:id/model to the owning non-primary workspace bridge', async () => { const { app, primaryBridge, secondaryBridge } = makeHarness(); @@ -4321,7 +4506,7 @@ describe('multi-workspace session dispatch', () => { }); }); - it('reports archive and delete conflicts while a workspace export is in flight', async () => { + it('preserves ordinary batch conflict results while an internal runtime is active', async () => { await withRuntimeDir(async () => { const sessionId = '550e8400-e29b-41d4-a716-446655440283'; await writeStoredSession({ @@ -4350,7 +4535,18 @@ describe('multi-workspace session dispatch', () => { } return result; }); - const { app } = makeHarness({ secondarySummaries: [] }); + const { app, registry } = makeHarness({ secondarySummaries: [] }); + const conversationCwd = path.join(SECONDARY_CWD, '.conversations'); + registry.add( + makeRuntime({ + workspaceId: 'conversations-id', + workspaceCwd: conversationCwd, + primary: false, + trusted: true, + provenance: 'live-conversation', + bridge: makeBridge(conversationCwd, []), + }), + ); const exportPromise = request(app) .get(`/workspaces/secondary-id/session/${sessionId}/export`) .set('Host', host()) diff --git a/packages/cli/src/serve/routes/capabilities.ts b/packages/cli/src/serve/routes/capabilities.ts index 229f90fc98b..90116bf7153 100644 --- a/packages/cli/src/serve/routes/capabilities.ts +++ b/packages/cli/src/serve/routes/capabilities.ts @@ -39,11 +39,17 @@ export function registerCapabilitiesRoutes( deps: RegisterCapabilitiesRoutesDeps, ): void { app.get('/capabilities', (_req, res) => { - const entries = deps.workspaceRegistry.listEntries(); + const entries = deps.workspaceRegistry + .listAllEntries() + .filter( + (entry) => + !entry.internal || + (entry.state === 'active' && entry.current !== undefined), + ); const activePrimary = entries.find( (entry) => entry.primary && entry.state === 'active', )?.current?.runtime; - const multiWorkspace = entries.length > 1; + const multipleAdmissionPools = entries.length > 1; const features = deps.currentServeFeatures(); const runtimeRemoval = features.includes('workspace_runtime_removal'); const envelope: CapabilitiesEnvelope = { @@ -74,7 +80,7 @@ export function registerCapabilitiesRoutes( ...(features.includes('workspace_file_upload') ? { maxWorkspaceFileUploadBytes: MAX_UPLOAD_BYTES } : {}), - ...(multiWorkspace + ...(multipleAdmissionPools ? { maxSessionsPerWorkspace: advertisedMaxSessions( deps.maxSessionsPerWorkspace, diff --git a/packages/cli/src/serve/routes/channel-notify.test.ts b/packages/cli/src/serve/routes/channel-notify.test.ts index 8989003759a..f8554d7b853 100644 --- a/packages/cli/src/serve/routes/channel-notify.test.ts +++ b/packages/cli/src/serve/routes/channel-notify.test.ts @@ -101,7 +101,7 @@ describe('channel notify routes', () => { ); }); - it('rejects qualified notifications for the Conversations runtime', async () => { + it('hides the Conversations runtime from qualified notifications', async () => { const live = runtime( 'conversations', '/work/Conversations', @@ -117,7 +117,7 @@ describe('channel notify routes', () => { .send(body); expect(response.status).toBe(400); - expect(response.body.code).toBe('live_channel_management_reserved'); + expect(response.body.code).toBe('workspace_mismatch'); expect(deliver).not.toHaveBeenCalled(); }); diff --git a/packages/cli/src/serve/routes/health.ts b/packages/cli/src/serve/routes/health.ts index 1db099b04ef..c5d75fc1dc3 100644 --- a/packages/cli/src/serve/routes/health.ts +++ b/packages/cli/src/serve/routes/health.ts @@ -56,7 +56,7 @@ export function createHealthRoutes(deps: CreateHealthRoutesDeps): HealthRoutes { try { if ( workspaceRegistry - .listEntries() + .listAllEntries() .some((entry) => entry.state === 'blocked') ) { res.status(503).json({ diff --git a/packages/cli/src/serve/routes/live.test.ts b/packages/cli/src/serve/routes/live.test.ts index 7b049e9f469..37a665ed6cb 100644 --- a/packages/cli/src/serve/routes/live.test.ts +++ b/packages/cli/src/serve/routes/live.test.ts @@ -15,6 +15,7 @@ import { LIVE_HOST_PROTOCOL_VERSION, } from '../live/types.js'; import { registerLiveRoutes } from './live.js'; +import { ConversationRuntimeOwnershipError } from '../conversations/conversation-runtime-errors.js'; class FakeSocket extends EventEmitter { readyState: number = WebSocket.OPEN; @@ -116,6 +117,43 @@ afterEach(() => { }); describe('Live routes', () => { + it.each(['/live/start', '/live/new'])( + 'serializes runtime ownership failures from %s without leaking details', + async (route) => { + const { coordinator } = harness(); + connectReady(coordinator); + const app = express(); + app.use(express.json()); + registerLiveRoutes(app, { + coordinator, + mutate: () => ((_req, _res, next) => next()) as RequestHandler, + ensureRuntimeReady: async () => { + throw new ConversationRuntimeOwnershipError( + 'conversation_runtime_in_use', + true, + { + cause: new Error( + '/private/conversations owner=1234 nonce=secret', + ), + }, + ); + }, + }); + + const response = await request(app).post(route).send({}); + + expect(response.status).toBe(503); + expect(response.body).toEqual({ + error: 'The Conversations runtime is owned by another daemon.', + code: 'conversation_runtime_in_use', + retryable: true, + }); + expect(JSON.stringify(response.body)).not.toContain('/private'); + expect(JSON.stringify(response.body)).not.toContain('1234'); + expect(JSON.stringify(response.body)).not.toContain('secret'); + }, + ); + it('returns non-secret readiness and a structured unavailable response', async () => { const { app } = harness(false); diff --git a/packages/cli/src/serve/routes/live.ts b/packages/cli/src/serve/routes/live.ts index fbc4b257a66..b6cd5a7ee22 100644 --- a/packages/cli/src/serve/routes/live.ts +++ b/packages/cli/src/serve/routes/live.ts @@ -8,20 +8,28 @@ import type { Application, RequestHandler } from 'express'; import { safeBody } from '../server/request-helpers.js'; import { LiveUnavailableError } from '../live/live-host-coordinator.js'; import type { LiveHostCoordinator } from '../live/live-host-coordinator.js'; +import { ConversationRuntimeOwnershipError } from '../conversations/conversation-runtime-errors.js'; export interface RegisterLiveRoutesDeps { coordinator: LiveHostCoordinator; + ensureRuntimeReady?: () => Promise; mutate: (options?: { strict?: boolean }) => RequestHandler; persistShortcut?: (shortcut: string) => Promise; } function sendUnavailable(res: Parameters[1], error: unknown) { + if (error instanceof ConversationRuntimeOwnershipError) { + res.status(error.status).json({ + error: error.message, + code: error.code, + retryable: error.retryable, + }); + return true; + } if (!(error instanceof LiveUnavailableError)) return false; - res.status(503).json({ - error: error.message, - code: error.code, - status: error.status, - }); + res + .status(503) + .json({ error: error.message, code: error.code, status: error.status }); return true; } @@ -33,8 +41,9 @@ export function registerLiveRoutes( res.status(200).json(deps.coordinator.getStatus()); }); - app.post('/live/start', deps.mutate(), (_req, res) => { + app.post('/live/start', deps.mutate(), async (_req, res) => { try { + await deps.ensureRuntimeReady?.(); res.status(200).json(deps.coordinator.start('resume').status); } catch (error) { if (sendUnavailable(res, error)) return; @@ -42,8 +51,9 @@ export function registerLiveRoutes( } }); - app.post('/live/new', deps.mutate(), (_req, res) => { + app.post('/live/new', deps.mutate(), async (_req, res) => { try { + await deps.ensureRuntimeReady?.(); res.status(200).json(deps.coordinator.start('new').status); } catch (error) { if (sendUnavailable(res, error)) return; diff --git a/packages/cli/src/serve/routes/scheduled-tasks.test.ts b/packages/cli/src/serve/routes/scheduled-tasks.test.ts index 80c755e45d0..05bf623ff12 100644 --- a/packages/cli/src/serve/routes/scheduled-tasks.test.ts +++ b/packages/cli/src/serve/routes/scheduled-tasks.test.ts @@ -27,6 +27,7 @@ import type { WorkspaceRuntime, } from '../workspace-registry.js'; import { ChannelDeliveryAuthorizationStore } from '../channel-delivery-authorization.js'; +import { ConversationRuntimeActivityGate } from '../conversations/conversation-runtime-activity.js'; function safeBody(req: Request): Record { return req.body && typeof req.body === 'object' @@ -1033,6 +1034,40 @@ describe('scheduled-tasks routes', () => { expect(list.body.tasks).toEqual([]); }); + it('revokes channel delivery when a manual run consumes a one-shot task', async () => { + const delivery = { + kind: 'channel', + target: { + channelName: 'dingtalk', + type: 'user' as const, + id: 'user-1', + }, + }; + const created = await create({ + cron: '0 9 1 1 *', + prompt: 'p', + recurring: false, + delivery, + }); + + const res = await request(h.app).post( + `/scheduled-tasks/${created.body.id}/run`, + ); + + expect(res.status).toBe(200); + const firedAt = res.body.lastFiredAt + 60_000; + expect( + h.channelDeliveryAuthorizations.consume(h.workspace, { + sessionId: created.body.sessionId, + deliveryId: `${created.body.id}:${firedAt}`, + source: 'scheduled', + taskId: created.body.id, + firedAt, + target: delivery.target, + }), + ).toBe(false); + }); + it('keeps a RECURRING task on manual run (only stamps lastFiredAt)', async () => { await seedTask({ id: 'rec-run', @@ -1805,6 +1840,7 @@ interface QualifiedHarness { primary: QualifiedRuntime; secondary: QualifiedRuntime; untrusted: QualifiedRuntime; + activity: ConversationRuntimeActivityGate; } /** A registry stub exposing only what the qualified route resolver touches: @@ -1816,6 +1852,9 @@ function makeStubRegistry(runtimes: QualifiedRuntime[]): WorkspaceRegistry { workspaceCwd: runtime.workspaceCwd, primary: index === 0, removable: index !== 0, + get internal() { + return runtime.provenance === 'live-conversation'; + }, registrationIds: [], lastGenerationId: 1, state: 'active' as const, @@ -1833,18 +1872,43 @@ function makeStubRegistry(runtimes: QualifiedRuntime[]): WorkspaceRegistry { appliedRevision: 'test', })); return { - list: () => runtimes.map(asRuntime), - listEntries: () => entries, + list: () => + runtimes + .filter((runtime) => runtime.provenance !== 'live-conversation') + .map(asRuntime), + listEntries: () => entries.filter((entry) => !entry.internal), + listAll: () => runtimes.map(asRuntime), + listAllEntries: () => entries, getEntryByWorkspaceId: (id: string) => entries.find((entry) => entry.workspaceId === id), getEntryByWorkspaceCwd: (cwd: string) => entries.find((entry) => entry.workspaceCwd === cwd), + getManagedEntryByWorkspaceId: (id: string) => + entries.find((entry) => entry.workspaceId === id), + getManagedEntryByWorkspaceCwd: (cwd: string) => + entries.find((entry) => entry.workspaceCwd === cwd), getByWorkspaceId: (id: string) => { const found = runtimes.find((r) => r.workspaceId === id); - return found ? asRuntime(found) : undefined; + return found?.provenance === 'live-conversation' + ? undefined + : found + ? asRuntime(found) + : undefined; }, getByWorkspaceCwd: (cwd: string) => { const found = runtimes.find((r) => r.workspaceCwd === cwd); + return found?.provenance === 'live-conversation' + ? undefined + : found + ? asRuntime(found) + : undefined; + }, + getManagedByWorkspaceId: (id: string) => { + const found = runtimes.find((runtime) => runtime.workspaceId === id); + return found ? asRuntime(found) : undefined; + }, + getManagedByWorkspaceCwd: (cwd: string) => { + const found = runtimes.find((runtime) => runtime.workspaceCwd === cwd); return found ? asRuntime(found) : undefined; }, } as unknown as WorkspaceRegistry; @@ -1873,6 +1937,7 @@ async function makeQualifiedHarness(): Promise { const secondary = await mkRuntime('secondary', true); const untrusted = await mkRuntime('untrusted', false); const runtimes = [primary, secondary, untrusted]; + const activity = new ConversationRuntimeActivityGate(); const app = express(); app.use(express.json()); @@ -1891,8 +1956,9 @@ async function makeQualifiedHarness(): Promise { mutate: () => (_req, _res, next) => next(), safeBody, manageScheduledTaskSessions: true, + conversationRuntimeActivity: activity, }); - return { app, scratch, primary, secondary, untrusted }; + return { app, scratch, primary, secondary, untrusted, activity }; } describe('workspace-qualified scheduled-tasks routes', () => { @@ -2011,4 +2077,66 @@ describe('workspace-qualified scheduled-tasks routes', () => { fsp.readFile(getCronFilePath(h.secondary.workspaceCwd), 'utf-8'), ).rejects.toThrow(); }); + + it('fails closed before internal task reads when the activity gate is absent', async () => { + h.secondary.provenance = 'live-conversation'; + const app = express(); + app.use(express.json()); + registerWorkspaceQualifiedScheduledTasksRoutes(app, { + workspaceRegistry: makeStubRegistry([ + h.primary, + h.secondary, + h.untrusted, + ]), + mutate: () => (_req, _res, next) => next(), + safeBody, + manageScheduledTaskSessions: true, + }); + + const response = await request(app).get(qualified(h.secondary.workspaceId)); + + expect(response.status).toBe(503); + expect(response.body.code).toBe('conversation_runtime_unavailable'); + }); + + it('holds the Conversations activity lease for the whole delete handler', async () => { + const created = await request(h.app) + .post(qualified(h.secondary.workspaceId)) + .send({ cron: '0 9 * * *', prompt: 'p' }); + const taskId = created.body.id as string; + h.secondary.provenance = 'live-conversation'; + let finishClose: (() => void) | undefined; + const closePending = new Promise((resolve) => { + finishClose = resolve; + }); + const originalClose = h.secondary.bridge.closeSession.bind( + h.secondary.bridge, + ); + vi.spyOn(h.secondary.bridge, 'closeSession').mockImplementation( + async (sessionId) => { + await originalClose(sessionId); + await closePending; + }, + ); + + const deletion = request(h.app) + .delete(`${qualified(h.secondary.workspaceId)}/${taskId}`) + .then((response) => response); + await vi.waitFor(() => expect(h.secondary.bridge.closed).toHaveLength(1)); + let drained = false; + const drain = h.activity.sealAndWait().then(() => { + drained = true; + }); + await Promise.resolve(); + expect(drained).toBe(false); + + finishClose?.(); + expect((await deletion).status).toBe(200); + await drain; + expect(drained).toBe(true); + + const late = await request(h.app).get(qualified(h.secondary.workspaceId)); + expect(late.status).toBe(503); + expect(late.body.code).toBe('daemon_draining'); + }); }); diff --git a/packages/cli/src/serve/routes/scheduled-tasks.ts b/packages/cli/src/serve/routes/scheduled-tasks.ts index 82af1e5cef8..2e98a558891 100644 --- a/packages/cli/src/serve/routes/scheduled-tasks.ts +++ b/packages/cli/src/serve/routes/scheduled-tasks.ts @@ -62,9 +62,11 @@ import type { } from '../workspace-registry.js'; import { requireTrustedWorkspaceRuntime, - resolveWorkspaceRuntimeFromParam, + resolveWorkspaceRuntimeWithLiveCompatibilityFromParam, + sendConversationRuntimeUnavailable, sendGenerationClosedError, } from '../workspace-route-runtime.js'; +import type { ConversationRuntimeActivityGate } from '../conversations/conversation-runtime-activity.js'; // The per-file create cap, shared with the scheduler's MAX_JOBS. The scheduler // caps DURABLE loads against a durable-only budget of MAX_JOBS (independent of @@ -141,6 +143,7 @@ interface ScheduledTaskTarget { bridge?: ScheduledTasksSessionBridge; cleanupSession?: (sessionId: string) => Promise; assertGenerationOpen?: () => void; + activity?: ConversationRuntimeActivityGate; } function requireOpenGeneration( @@ -246,16 +249,33 @@ interface RegisterWorkspaceQualifiedScheduledTasksRoutesDeps { runtime: WorkspaceRuntime, sessionId: string, ) => Promise; + conversationRuntimeActivity?: ConversationRuntimeActivityGate; } -function runWithScheduledTaskTarget( +async function runWithScheduledTaskTarget( target: ScheduledTaskTarget, - fn: () => T, -): T { - if (target.runtimeBaseDir === undefined) { - return fn(); + fn: () => T | Promise, +): Promise> { + const result = + target.runtimeBaseDir === undefined + ? fn() + : Storage.runWithResolvedRuntimeBaseDir(target.runtimeBaseDir, fn); + return (await result) as Awaited; +} + +function sendActivityGateError(res: Response, error: unknown): boolean { + if ( + !error || + typeof error !== 'object' || + (error as { code?: unknown }).code !== 'daemon_draining' + ) { + return false; } - return Storage.runWithResolvedRuntimeBaseDir(target.runtimeBaseDir, fn); + res.status(503).json({ + error: 'The daemon is draining and no longer accepts work.', + code: 'daemon_draining', + }); + return true; } /** On-the-wire task shape — normalizes the optional on-disk fields so the @@ -373,320 +393,79 @@ function registerScheduledTaskCrudRoutes( } = deps; const base = `${prefix}/scheduled-tasks`; - // ── List ────────────────────────────────────────────────────────── - app.get(base, async (req, res) => { - const target = resolveTarget(req, res); - if (!target) return; - if (!requireOpenGeneration(target, res)) return; - try { - const tasks = await runWithScheduledTaskTarget(target, () => - readCronTasks(target.workspaceCwd), - ); - if (!requireOpenGeneration(target, res)) return; - res.status(200).json({ v: 1, tasks: tasks.map(toView) }); - } catch (err) { - // A malformed/corrupt file throws (fix-or-delete contract) rather than - // reading as empty — surface it instead of hiding the user's tasks - // behind a silent []. - writeStderrLine( - `qwen serve: GET ${base} failed: ${err instanceof Error ? err.message : String(err)}`, - ); - res.status(500).json({ - error: 'Failed to read scheduled tasks (the tasks file may be corrupt)', - code: 'scheduled_tasks_read_failed', - }); - } - }); - - // ── Create ──────────────────────────────────────────────────────── - app.post(base, mutate(), async (req, res) => { - const target = resolveTarget(req, res); - if (!target) return; - if (!requireOpenGeneration(target, res)) return; - const { workspaceCwd, bridge } = target; - const body = safeBody(req); - - const cron = typeof body['cron'] === 'string' ? body['cron'].trim() : ''; - if (cron.length === 0) { - res.status(400).json({ - error: '`cron` is required and must be a non-empty string', - code: 'invalid_cron', - }); - return; - } - if (cron.length > MAX_CRON_LENGTH) { - res.status(400).json({ - error: `\`cron\` exceeds ${MAX_CRON_LENGTH}-character limit`, - code: 'invalid_cron', - }); - return; - } - const cronError = validateCron(cron); - if (cronError) { - res.status(400).json({ error: cronError, code: 'invalid_cron' }); - return; - } - - const prompt = - typeof body['prompt'] === 'string' ? body['prompt'].trim() : ''; - if (prompt.length === 0) { - res.status(400).json({ - error: '`prompt` is required and must be a non-empty string', - code: 'invalid_prompt', - }); - return; - } - if (prompt.length > MAX_PROMPT_LENGTH) { - res.status(400).json({ - error: `\`prompt\` exceeds ${MAX_PROMPT_LENGTH}-character limit`, - code: 'invalid_prompt', - }); - return; - } - - const nameResult = parseNameField(body['name']); - if (nameResult.error) { - res.status(400).json({ error: nameResult.error, code: 'invalid_name' }); - return; - } - - if ( - body['recurring'] !== undefined && - typeof body['recurring'] !== 'boolean' - ) { - res.status(400).json({ - error: '`recurring` must be a boolean', - code: 'invalid_recurring', - }); - return; - } - if (body['enabled'] !== undefined && typeof body['enabled'] !== 'boolean') { - res.status(400).json({ - error: '`enabled` must be a boolean', - code: 'invalid_enabled', - }); - return; - } - let delivery: PublicChannelDelivery | undefined; - if (body['delivery'] !== undefined) { - try { - delivery = parseChannelDelivery(body['delivery']); - } catch (err) { - if (!isChannelDeliveryError(err)) throw err; - res.status(400).json({ error: err.message, code: err.code }); - return; - } - } - const removedField = findRemovedTaskField(body); - if (removedField) { - res.status(400).json(removedFieldError(removedField)); - return; - } - const recurring = body['recurring'] !== false; - const enabled = body['enabled'] !== false; - const taskId = generateCronTaskId(); - - // Mint the task's dedicated session up front. The task is BOUND to it and - // fires only inside it — its transcript becomes the task's run history, and - // archiving/deleting the session stops the task. Done before the write so a - // task never lands on disk without its session; if the bridge is absent - // (minimal embedding) the task is created unbound (shared-owner firing). - // - // `sessionScope: 'thread'` is REQUIRED: the daemon's default scope is - // 'single', which would attach to (and reuse) the shared workspace session - // instead of minting a fresh one. Two tasks — or a task and an open chat — - // would then bind to the same session: the task renames it, scheduled runs - // land in the wrong transcript, and deleting one task closes the shared - // session. Forcing 'thread' guarantees each task gets an isolated session. - let boundSessionId: string | undefined; - if (bridge) { - // Pre-check the cap BEFORE spawning: an over-cap create must not spawn a - // session it will immediately tear down, because closeSession removes the - // live bridge entry but can leave the just-spawned+named session listed as - // an orphan with no owning task. Best-effort — the write-lock cap check - // below stays authoritative for the concurrent-create race. + const withTarget = + ( + handler: ( + req: Request, + res: Response, + target: ScheduledTaskTarget, + ) => Promise, + ): RequestHandler => + async (req, res) => { + const target = resolveTarget(req, res); + if (!target) return; + const operation = async () => { + if (!requireOpenGeneration(target, res)) return; + await handler(req, res, target); + }; try { - if ( - ( - await runWithScheduledTaskTarget(target, () => - readCronTasks(workspaceCwd), - ) - ).length >= MAX_SCHEDULED_TASKS - ) { - res.status(409).json({ - error: `Maximum number of scheduled tasks (${MAX_SCHEDULED_TASKS}) reached`, - code: 'max_tasks_reached', - }); - return; + if (target.activity) { + await target.activity.run(operation); + } else { + await operation(); } - } catch { - // Read failure → skip the pre-check; the write below is authoritative. + } catch (error) { + if (sendActivityGateError(res, error)) return; + throw error; } - if (!requireOpenGeneration(target, res)) return; + }; + + // ── List ────────────────────────────────────────────────────────── + app.get( + base, + withTarget(async (_req, res, target) => { try { - const session = await bridge.spawnOrAttach({ - workspaceCwd, - sessionScope: 'thread', - sourceType: 'scheduled_task', - sourceId: taskId, - }); - boundSessionId = session.sessionId; - if (!requireOpenGeneration(target, res)) { - await teardownBoundSession(target, boundSessionId); - return; - } - // Name the session after the task so it's recognizable in the session - // list. Best-effort — a nameless session still fires correctly. - try { - bridge.updateSessionMetadata(boundSessionId, { - displayName: scheduledTaskSessionName(nameResult.value ?? prompt), - }); - } catch { - // metadata update is non-critical - } + const tasks = await runWithScheduledTaskTarget(target, () => + readCronTasks(target.workspaceCwd), + ); + if (!requireOpenGeneration(target, res)) return; + res.status(200).json({ v: 1, tasks: tasks.map(toView) }); } catch (err) { - if (sendGenerationClosedError(res, err)) return; + if (sendActivityGateError(res, err)) return; + // A malformed/corrupt file throws (fix-or-delete contract) rather than + // reading as empty — surface it instead of hiding the user's tasks + // behind a silent []. writeStderrLine( - `qwen serve: POST ${base} failed to create the task's session: ${err instanceof Error ? err.message : String(err)}`, + `qwen serve: GET ${base} failed: ${err instanceof Error ? err.message : String(err)}`, ); res.status(500).json({ - error: "Failed to create the task's session", - code: 'scheduled_tasks_session_failed', + error: + 'Failed to read scheduled tasks (the tasks file may be corrupt)', + code: 'scheduled_tasks_read_failed', }); - return; - } - } - - const now = Date.now(); - const task: DurableCronTask = { - id: taskId, - cron, - prompt, - recurring, - createdAt: now, - // Pin to the creation minute so the scheduler can't fire during the - // minute the task was created — same guard cronScheduler.create uses. - lastFiredAt: now - (now % 60_000), - enabled, - ...(delivery !== undefined ? { delivery } : {}), - ...(boundSessionId !== undefined ? { sessionId: boundSessionId } : {}), - ...(nameResult.value !== undefined ? { name: nameResult.value } : {}), - }; - - // Best-effort teardown of the just-minted session when the create can't be - // committed. closeSession only tears down the live child; removeSession also - // deletes the persisted transcript/title record — both are needed, or a - // rejected create (the loser of a concurrent create at the cap boundary, - // which passes the pre-check but loses the authoritative write) would leave - // a named "⏰ …" session in the list with no owning task. - const rollbackSession = async () => { - if (boundSessionId !== undefined) { - await teardownBoundSession(target, boundSessionId); - } - }; - - let overCap = false; - let rollbackBefore: DurableCronTask[] | undefined; - let rollbackAfter: DurableCronTask[] | undefined; - try { - await runWithScheduledTaskTarget(target, () => - updateCronTasks( - workspaceCwd, - (tasks) => { - // Cap check under the write lock so two concurrent creates can't both - // slip past a stale count. Returning the input unchanged is a no-op - // (no write), which the flag below turns into a 409. - if (tasks.length >= MAX_SCHEDULED_TASKS) { - overCap = true; - return tasks; - } - rollbackBefore = tasks; - rollbackAfter = [...tasks, task]; - return rollbackAfter; - }, - { assertCanCommit: target.assertGenerationOpen }, - ), - ); - } catch (err) { - await rollbackSession(); - if (sendGenerationClosedError(res, err)) return; - writeStderrLine( - `qwen serve: POST ${base} failed: ${err instanceof Error ? err.message : String(err)}`, - ); - res.status(500).json({ - error: 'Failed to create scheduled task', - code: 'scheduled_tasks_write_failed', - }); - return; - } - if (rollbackBefore && rollbackAfter) { - try { - target.assertGenerationOpen?.(); - } catch (error) { - await rollbackCronMutation( - target, - rollbackBefore, - rollbackAfter, - `POST ${base}`, - ); - await rollbackSession(); - if (sendGenerationClosedError(res, error)) return; - throw error; } - } - if (overCap) { - await rollbackSession(); - res.status(409).json({ - error: `Maximum number of scheduled tasks (${MAX_SCHEDULED_TASKS}) reached`, - code: 'max_tasks_reached', - }); - return; - } - if (task.delivery && task.sessionId) { - channelDeliveryAuthorizations?.registerScheduledTask(workspaceCwd, { - sessionId: task.sessionId, - taskId: task.id, - target: task.delivery.target, - recurring: task.recurring, - lastFiredAt: task.lastFiredAt ?? undefined, - }); - } - res.status(201).json(toView(task)); - }); + }), + ); - // ── Update (name / enabled / cron / prompt / recurring / delivery) ── - app.patch(`${base}/:id`, mutate(), async (req, res) => { - const target = resolveTarget(req, res); - if (!target) return; - if (!requireOpenGeneration(target, res)) return; - const { workspaceCwd, bridge } = target; - const id = typeof req.params['id'] === 'string' ? req.params['id'] : ''; - if (id.length === 0) { - res - .status(400) - .json({ error: 'Task id is required', code: 'invalid_id' }); - return; - } - const body = safeBody(req); - - // Pre-validate every provided field OUTSIDE the write lock — cron parsing - // and type checks don't need it, and validating inside the mutate callback - // would mean holding the lock to reject a bad request. - const patch: Partial = {}; - let clearName = false; - let clearDelivery = false; - - const removedPatchField = findRemovedTaskField(body); - if (removedPatchField) { - res.status(400).json(removedFieldError(removedPatchField)); - return; - } + // ── Create ──────────────────────────────────────────────────────── + app.post( + base, + mutate(), + withTarget(async (req, res, target) => { + const { workspaceCwd, bridge } = target; + const body = safeBody(req); - if ('cron' in body) { const cron = typeof body['cron'] === 'string' ? body['cron'].trim() : ''; - if (cron.length === 0 || cron.length > MAX_CRON_LENGTH) { + if (cron.length === 0) { res.status(400).json({ - error: '`cron` must be a non-empty string within the length limit', + error: '`cron` is required and must be a non-empty string', + code: 'invalid_cron', + }); + return; + } + if (cron.length > MAX_CRON_LENGTH) { + res.status(400).json({ + error: `\`cron\` exceeds ${MAX_CRON_LENGTH}-character limit`, code: 'invalid_cron', }); return; @@ -696,332 +475,637 @@ function registerScheduledTaskCrudRoutes( res.status(400).json({ error: cronError, code: 'invalid_cron' }); return; } - patch.cron = cron; - } - if ('prompt' in body) { + const prompt = typeof body['prompt'] === 'string' ? body['prompt'].trim() : ''; - if (prompt.length === 0 || prompt.length > MAX_PROMPT_LENGTH) { + if (prompt.length === 0) { + res.status(400).json({ + error: '`prompt` is required and must be a non-empty string', + code: 'invalid_prompt', + }); + return; + } + if (prompt.length > MAX_PROMPT_LENGTH) { res.status(400).json({ - error: '`prompt` must be a non-empty string within the length limit', + error: `\`prompt\` exceeds ${MAX_PROMPT_LENGTH}-character limit`, code: 'invalid_prompt', }); return; } - patch.prompt = prompt; - } - if ('name' in body) { + const nameResult = parseNameField(body['name']); if (nameResult.error) { res.status(400).json({ error: nameResult.error, code: 'invalid_name' }); return; } - if (nameResult.value === undefined) { - clearName = true; - } else { - patch.name = nameResult.value; - } - } - if ('recurring' in body) { - if (typeof body['recurring'] !== 'boolean') { + + if ( + body['recurring'] !== undefined && + typeof body['recurring'] !== 'boolean' + ) { res.status(400).json({ error: '`recurring` must be a boolean', code: 'invalid_recurring', }); return; } - patch.recurring = body['recurring']; - } - if ('enabled' in body) { - if (typeof body['enabled'] !== 'boolean') { + if ( + body['enabled'] !== undefined && + typeof body['enabled'] !== 'boolean' + ) { res.status(400).json({ error: '`enabled` must be a boolean', code: 'invalid_enabled', }); return; } - patch.enabled = body['enabled']; - } - if ('delivery' in body) { - if (body['delivery'] === null) { - clearDelivery = true; - } else { + let delivery: PublicChannelDelivery | undefined; + if (body['delivery'] !== undefined) { try { - patch.delivery = parseChannelDelivery(body['delivery']); + delivery = parseChannelDelivery(body['delivery']); } catch (err) { if (!isChannelDeliveryError(err)) throw err; res.status(400).json({ error: err.message, code: err.code }); return; } } - } - if (Object.keys(patch).length === 0 && !clearName && !clearDelivery) { - res.status(400).json({ - error: 'No updatable fields provided', - code: 'empty_patch', - }); - return; - } + const removedField = findRemovedTaskField(body); + if (removedField) { + res.status(400).json(removedFieldError(removedField)); + return; + } + const recurring = body['recurring'] !== false; + const enabled = body['enabled'] !== false; + const taskId = generateCronTaskId(); - let found = false; - let updated: DurableCronTask | undefined; - let blockedByArchive = false; - let blockedLegacy = false; - let rollbackBefore: DurableCronTask[] | undefined; - let rollbackAfter: DurableCronTask[] | undefined; - try { - await runWithScheduledTaskTarget(target, () => - updateCronTasks( - workspaceCwd, - (tasks) => { - const idx = tasks.findIndex((t) => t.id === id); - if (idx === -1) return tasks; // not found → no write - found = true; - const current = tasks[idx]!; - // A legacy guarded task (isolated + precondition, both removed) can't be - // enabled: `toView` reports it disabled, so the only PATCH the Web Shell - // sends for it is the Enable toggle — which would 200 here and then read - // back disabled again, an Enable control that can never succeed with no - // error explaining why. Reject the enable with the recreate remediation - // instead of acknowledging an update that changes nothing runnable. - if (patch.enabled === true && taskHasLegacyCondition(current)) { - blockedLegacy = true; - return tasks; // no write - } - // A task disabled BY archiving its session (`disabledByArchive`) can't - // be re-enabled through this generic PATCH: its bound session is still - // archived and can't fire, so flipping `enabled: true` here would show - // an enabled task with a countdown that never runs. The task/session - // lifecycle must stay coupled — the caller has to unarchive the session - // (which clears the marker and reloads it). Reject and leave the file - // untouched. - if (patch.enabled === true && current.disabledByArchive === true) { - blockedByArchive = true; - return tasks; // no write - } - const next: DurableCronTask = { ...current, ...patch }; - // `name: null/""` clears the field rather than storing an empty name, - // so toView reports it as unnamed and isValidTask never sees a "". - if (clearName) delete next.name; - if (clearDelivery) delete next.delivery; - // Re-seat the task's schedule anchor to "now" whenever an edit would - // otherwise let the scheduler retroactively fire an already-past slot. - const justReEnabled = - current.enabled === false && patch.enabled === true; - // Compare the EFFECTIVE schedule, not the raw string: a cosmetic edit - // (`0 9 * * *` → `00 9 * * *`, whitespace) must not re-seat the anchor - // and drop a legitimately-pending catch-up fire. - const cronChanged = - patch.cron !== undefined && - canonicalCron(patch.cron) !== canonicalCron(current.cron); - const becameRecurring = - patch.recurring === true && current.recurring !== true; - const becameOneShot = - patch.recurring === false && current.recurring !== false; - // Re-seated REGARDLESS of enabled: a schedule edit made while the task - // is paused must not leave a stale anchor that fires retroactively when - // it's later re-enabled in a SEPARATE request (the re-enable patch has no - // schedule change of its own to trigger the re-seat). Re-seating a paused - // task's anchor is harmless — it doesn't fire until enabled. - { - const now = Date.now(); - const minute = now - (now % 60_000); + // Mint the task's dedicated session up front. The task is BOUND to it and + // fires only inside it — its transcript becomes the task's run history, and + // archiving/deleting the session stops the task. Done before the write so a + // task never lands on disk without its session; if the bridge is absent + // (minimal embedding) the task is created unbound (shared-owner firing). + // + // `sessionScope: 'thread'` is REQUIRED: the daemon's default scope is + // 'single', which would attach to (and reuse) the shared workspace session + // instead of minting a fresh one. Two tasks — or a task and an open chat — + // would then bind to the same session: the task renames it, scheduled runs + // land in the wrong transcript, and deleting one task closes the shared + // session. Forcing 'thread' guarantees each task gets an isolated session. + let boundSessionId: string | undefined; + if (bridge) { + // Pre-check the cap BEFORE spawning: an over-cap create must not spawn a + // session it will immediately tear down, because closeSession removes the + // live bridge entry but can leave the just-spawned+named session listed as + // an orphan with no owning task. Best-effort — the write-lock cap check + // below stays authoritative for the concurrent-create race. + try { + if ( + ( + await runWithScheduledTaskTarget(target, () => + readCronTasks(workspaceCwd), + ) + ).length >= MAX_SCHEDULED_TASKS + ) { + res.status(409).json({ + error: `Maximum number of scheduled tasks (${MAX_SCHEDULED_TASKS}) reached`, + code: 'max_tasks_reached', + }); + return; + } + } catch { + // Read failure → skip the pre-check; the write below is authoritative. + } + if (!requireOpenGeneration(target, res)) return; + try { + const session = await runWithScheduledTaskTarget(target, () => + bridge.spawnOrAttach({ + workspaceCwd, + sessionScope: 'thread', + sourceType: 'scheduled_task', + sourceId: taskId, + }), + ); + boundSessionId = session.sessionId; + if (!requireOpenGeneration(target, res)) { + await teardownBoundSession(target, boundSessionId); + return; + } + // Name the session after the task so it's recognizable in the session + // list. Best-effort — a nameless session still fires correctly. + try { + await runWithScheduledTaskTarget(target, async () => + bridge.updateSessionMetadata(boundSessionId!, { + displayName: scheduledTaskSessionName( + nameResult.value ?? prompt, + ), + }), + ); + } catch { + // metadata update is non-critical + } + } catch (err) { + if (sendActivityGateError(res, err)) return; + if (sendGenerationClosedError(res, err)) return; + writeStderrLine( + `qwen serve: POST ${base} failed to create the task's session: ${err instanceof Error ? err.message : String(err)}`, + ); + res.status(500).json({ + error: "Failed to create the task's session", + code: 'scheduled_tasks_session_failed', + }); + return; + } + } + + const now = Date.now(); + const task: DurableCronTask = { + id: taskId, + cron, + prompt, + recurring, + createdAt: now, + // Pin to the creation minute so the scheduler can't fire during the + // minute the task was created — same guard cronScheduler.create uses. + lastFiredAt: now - (now % 60_000), + enabled, + ...(delivery !== undefined ? { delivery } : {}), + ...(boundSessionId !== undefined ? { sessionId: boundSessionId } : {}), + ...(nameResult.value !== undefined ? { name: nameResult.value } : {}), + }; + + // Best-effort teardown of the just-minted session when the create can't be + // committed. closeSession only tears down the live child; removeSession also + // deletes the persisted transcript/title record — both are needed, or a + // rejected create (the loser of a concurrent create at the cap boundary, + // which passes the pre-check but loses the authoritative write) would leave + // a named "⏰ …" session in the list with no owning task. + const rollbackSession = async () => { + if (boundSessionId !== undefined) { + await teardownBoundSession(target, boundSessionId); + } + }; + + let overCap = false; + let rollbackBefore: DurableCronTask[] | undefined; + let rollbackAfter: DurableCronTask[] | undefined; + try { + await runWithScheduledTaskTarget(target, () => + updateCronTasks( + workspaceCwd, + (tasks) => { + // Cap check under the write lock so two concurrent creates can't both + // slip past a stale count. Returning the input unchanged is a no-op + // (no write), which the flag below turns into a 409. + if (tasks.length >= MAX_SCHEDULED_TASKS) { + overCap = true; + return tasks; + } + rollbackBefore = tasks; + rollbackAfter = [...tasks, task]; + return rollbackAfter; + }, + { assertCanCommit: target.assertGenerationOpen }, + ), + ); + } catch (err) { + await rollbackSession(); + if (sendActivityGateError(res, err)) return; + if (sendGenerationClosedError(res, err)) return; + writeStderrLine( + `qwen serve: POST ${base} failed: ${err instanceof Error ? err.message : String(err)}`, + ); + res.status(500).json({ + error: 'Failed to create scheduled task', + code: 'scheduled_tasks_write_failed', + }); + return; + } + if (rollbackBefore && rollbackAfter) { + try { + target.assertGenerationOpen?.(); + } catch (error) { + await rollbackCronMutation( + target, + rollbackBefore, + rollbackAfter, + `POST ${base}`, + ); + await rollbackSession(); + if (sendGenerationClosedError(res, error)) return; + throw error; + } + } + if (overCap) { + await rollbackSession(); + res.status(409).json({ + error: `Maximum number of scheduled tasks (${MAX_SCHEDULED_TASKS}) reached`, + code: 'max_tasks_reached', + }); + return; + } + if (task.delivery && task.sessionId) { + channelDeliveryAuthorizations?.registerScheduledTask(workspaceCwd, { + sessionId: task.sessionId, + taskId: task.id, + target: task.delivery.target, + recurring: task.recurring, + lastFiredAt: task.lastFiredAt ?? undefined, + }); + } + res.status(201).json(toView(task)); + }), + ); + + // ── Update (name / enabled / cron / prompt / recurring / delivery) ── + app.patch( + `${base}/:id`, + mutate(), + withTarget(async (req, res, target) => { + const { workspaceCwd, bridge } = target; + const id = typeof req.params['id'] === 'string' ? req.params['id'] : ''; + if (id.length === 0) { + res + .status(400) + .json({ error: 'Task id is required', code: 'invalid_id' }); + return; + } + const body = safeBody(req); + + // Pre-validate every provided field OUTSIDE the write lock — cron parsing + // and type checks don't need it, and validating inside the mutate callback + // would mean holding the lock to reject a bad request. + const patch: Partial = {}; + let clearName = false; + let clearDelivery = false; + + const removedPatchField = findRemovedTaskField(body); + if (removedPatchField) { + res.status(400).json(removedFieldError(removedPatchField)); + return; + } + + if ('cron' in body) { + const cron = + typeof body['cron'] === 'string' ? body['cron'].trim() : ''; + if (cron.length === 0 || cron.length > MAX_CRON_LENGTH) { + res.status(400).json({ + error: '`cron` must be a non-empty string within the length limit', + code: 'invalid_cron', + }); + return; + } + const cronError = validateCron(cron); + if (cronError) { + res.status(400).json({ error: cronError, code: 'invalid_cron' }); + return; + } + patch.cron = cron; + } + if ('prompt' in body) { + const prompt = + typeof body['prompt'] === 'string' ? body['prompt'].trim() : ''; + if (prompt.length === 0 || prompt.length > MAX_PROMPT_LENGTH) { + res.status(400).json({ + error: + '`prompt` must be a non-empty string within the length limit', + code: 'invalid_prompt', + }); + return; + } + patch.prompt = prompt; + } + if ('name' in body) { + const nameResult = parseNameField(body['name']); + if (nameResult.error) { + res + .status(400) + .json({ error: nameResult.error, code: 'invalid_name' }); + return; + } + if (nameResult.value === undefined) { + clearName = true; + } else { + patch.name = nameResult.value; + } + } + if ('recurring' in body) { + if (typeof body['recurring'] !== 'boolean') { + res.status(400).json({ + error: '`recurring` must be a boolean', + code: 'invalid_recurring', + }); + return; + } + patch.recurring = body['recurring']; + } + if ('enabled' in body) { + if (typeof body['enabled'] !== 'boolean') { + res.status(400).json({ + error: '`enabled` must be a boolean', + code: 'invalid_enabled', + }); + return; + } + patch.enabled = body['enabled']; + } + if ('delivery' in body) { + if (body['delivery'] === null) { + clearDelivery = true; + } else { + try { + patch.delivery = parseChannelDelivery(body['delivery']); + } catch (err) { + if (!isChannelDeliveryError(err)) throw err; + res.status(400).json({ error: err.message, code: err.code }); + return; + } + } + } + if (Object.keys(patch).length === 0 && !clearName && !clearDelivery) { + res.status(400).json({ + error: 'No updatable fields provided', + code: 'empty_patch', + }); + return; + } + + let found = false; + let updated: DurableCronTask | undefined; + let blockedByArchive = false; + let blockedLegacy = false; + let rollbackBefore: DurableCronTask[] | undefined; + let rollbackAfter: DurableCronTask[] | undefined; + try { + await runWithScheduledTaskTarget(target, () => + updateCronTasks( + workspaceCwd, + (tasks) => { + const idx = tasks.findIndex((t) => t.id === id); + if (idx === -1) return tasks; // not found → no write + found = true; + const current = tasks[idx]!; + // A legacy guarded task (isolated + precondition, both removed) can't be + // enabled: `toView` reports it disabled, so the only PATCH the Web Shell + // sends for it is the Enable toggle — which would 200 here and then read + // back disabled again, an Enable control that can never succeed with no + // error explaining why. Reject the enable with the recreate remediation + // instead of acknowledging an update that changes nothing runnable. + if (patch.enabled === true && taskHasLegacyCondition(current)) { + blockedLegacy = true; + return tasks; // no write + } + // A task disabled BY archiving its session (`disabledByArchive`) can't + // be re-enabled through this generic PATCH: its bound session is still + // archived and can't fire, so flipping `enabled: true` here would show + // an enabled task with a countdown that never runs. The task/session + // lifecycle must stay coupled — the caller has to unarchive the session + // (which clears the marker and reloads it). Reject and leave the file + // untouched. if ( - next.recurring && - (justReEnabled || cronChanged || becameRecurring) - ) { - // A recurring task's anchor is lastFiredAt: resume from now so a - // re-enable / cron edit / one-shot→recurring flip doesn't retroactively - // fire a past slot (matters most for a bound task, whose catch-up runs - // on every file-watch reload). - next.lastFiredAt = minute; - } else if ( - !next.recurring && - (justReEnabled || cronChanged || becameOneShot) + patch.enabled === true && + current.disabledByArchive === true ) { - // A one-shot's anchor is createdAt. Re-seat it on a schedule change - // (cron edit, or recurring→one-shot) OR a re-enable so the task fires - // at its NEXT occurrence — otherwise the scheduler reads its original - // long-past slot as a MISSED one-shot and fires + permanently deletes - // it. A one-shot disabled past its slot then re-enabled would - // otherwise be silently destroyed on the next reload. - next.createdAt = now; - next.lastFiredAt = minute; + blockedByArchive = true; + return tasks; // no write } - } - updated = next; - rollbackBefore = tasks; - rollbackAfter = tasks.map((t, i) => (i === idx ? next : t)); - return rollbackAfter; - }, - { assertCanCommit: target.assertGenerationOpen }, - ), - ); - } catch (err) { - if (sendGenerationClosedError(res, err)) return; - writeStderrLine( - `qwen serve: PATCH ${base}/${id} failed: ${err instanceof Error ? err.message : String(err)}`, - ); - res.status(500).json({ - error: 'Failed to update scheduled task', - code: 'scheduled_tasks_write_failed', - }); - return; - } - if (rollbackBefore && rollbackAfter) { - try { - target.assertGenerationOpen?.(); - } catch (error) { - await rollbackCronMutation( - target, - rollbackBefore, - rollbackAfter, - `PATCH ${base}/${id}`, + const next: DurableCronTask = { ...current, ...patch }; + // `name: null/""` clears the field rather than storing an empty name, + // so toView reports it as unnamed and isValidTask never sees a "". + if (clearName) delete next.name; + if (clearDelivery) delete next.delivery; + // Re-seat the task's schedule anchor to "now" whenever an edit would + // otherwise let the scheduler retroactively fire an already-past slot. + const justReEnabled = + current.enabled === false && patch.enabled === true; + // Compare the EFFECTIVE schedule, not the raw string: a cosmetic edit + // (`0 9 * * *` → `00 9 * * *`, whitespace) must not re-seat the anchor + // and drop a legitimately-pending catch-up fire. + const cronChanged = + patch.cron !== undefined && + canonicalCron(patch.cron) !== canonicalCron(current.cron); + const becameRecurring = + patch.recurring === true && current.recurring !== true; + const becameOneShot = + patch.recurring === false && current.recurring !== false; + // Re-seated REGARDLESS of enabled: a schedule edit made while the task + // is paused must not leave a stale anchor that fires retroactively when + // it's later re-enabled in a SEPARATE request (the re-enable patch has no + // schedule change of its own to trigger the re-seat). Re-seating a paused + // task's anchor is harmless — it doesn't fire until enabled. + { + const now = Date.now(); + const minute = now - (now % 60_000); + if ( + next.recurring && + (justReEnabled || cronChanged || becameRecurring) + ) { + // A recurring task's anchor is lastFiredAt: resume from now so a + // re-enable / cron edit / one-shot→recurring flip doesn't retroactively + // fire a past slot (matters most for a bound task, whose catch-up runs + // on every file-watch reload). + next.lastFiredAt = minute; + } else if ( + !next.recurring && + (justReEnabled || cronChanged || becameOneShot) + ) { + // A one-shot's anchor is createdAt. Re-seat it on a schedule change + // (cron edit, or recurring→one-shot) OR a re-enable so the task fires + // at its NEXT occurrence — otherwise the scheduler reads its original + // long-past slot as a MISSED one-shot and fires + permanently deletes + // it. A one-shot disabled past its slot then re-enabled would + // otherwise be silently destroyed on the next reload. + next.createdAt = now; + next.lastFiredAt = minute; + } + } + updated = next; + rollbackBefore = tasks; + rollbackAfter = tasks.map((t, i) => (i === idx ? next : t)); + return rollbackAfter; + }, + { assertCanCommit: target.assertGenerationOpen }, + ), ); - if (sendGenerationClosedError(res, error)) return; - throw error; + } catch (err) { + if (sendActivityGateError(res, err)) return; + if (sendGenerationClosedError(res, err)) return; + writeStderrLine( + `qwen serve: PATCH ${base}/${id} failed: ${err instanceof Error ? err.message : String(err)}`, + ); + res.status(500).json({ + error: 'Failed to update scheduled task', + code: 'scheduled_tasks_write_failed', + }); + return; } - } - if (blockedLegacy) { - res.status(409).json({ - error: - 'This task uses the removed isolated run mode with a precondition and can no longer be enabled or run. Recreate it (and call the `create_sub_session` tool from the prompt if you need per-run isolation).', - code: 'task_legacy_unsupported', - }); - return; - } - if (blockedByArchive) { - res.status(409).json({ - error: - 'This task was disabled by archiving its session; unarchive the session to re-enable it.', - code: 'task_session_archived', - }); - return; - } - if (!found || !updated) { - res.status(404).json({ error: 'Task not found', code: 'task_not_found' }); - return; - } - // Keep the bound session's display name in sync with the task's effective - // label (its name, or its prompt when unnamed) — the session was named - // after the task at create, so a rename (or a prompt edit while unnamed) - // should follow. Only when the effective label actually changed, so a bare - // cron/enabled edit doesn't touch the session. Best-effort: a metadata - // failure must not fail the PATCH the schedule already committed. - const effectiveLabelChanged = - patch.name !== undefined || - clearName || - (patch.prompt !== undefined && updated.name === undefined); - if (bridge && updated.sessionId && effectiveLabelChanged) { - try { - bridge.updateSessionMetadata(updated.sessionId, { - displayName: scheduledTaskSessionName(updated.name ?? updated.prompt), + if (rollbackBefore && rollbackAfter) { + try { + target.assertGenerationOpen?.(); + } catch (error) { + await rollbackCronMutation( + target, + rollbackBefore, + rollbackAfter, + `PATCH ${base}/${id}`, + ); + if (sendGenerationClosedError(res, error)) return; + throw error; + } + } + if (blockedLegacy) { + res.status(409).json({ + error: + 'This task uses the removed isolated run mode with a precondition and can no longer be enabled or run. Recreate it (and call the `create_sub_session` tool from the prompt if you need per-run isolation).', + code: 'task_legacy_unsupported', }); - } catch { - // non-critical — the schedule change already persisted + return; } - } - if (updated.delivery && updated.sessionId) { - channelDeliveryAuthorizations?.registerScheduledTask(workspaceCwd, { - sessionId: updated.sessionId, - taskId: updated.id, - target: updated.delivery.target, - recurring: updated.recurring, - lastFiredAt: updated.lastFiredAt ?? undefined, - }); - } - if (clearDelivery && updated.sessionId) { - channelDeliveryAuthorizations?.revokeScheduledTask( - workspaceCwd, - updated.sessionId, - updated.id, - ); - } - res.status(200).json(toView(updated)); - }); + if (blockedByArchive) { + res.status(409).json({ + error: + 'This task was disabled by archiving its session; unarchive the session to re-enable it.', + code: 'task_session_archived', + }); + return; + } + if (!found || !updated) { + res + .status(404) + .json({ error: 'Task not found', code: 'task_not_found' }); + return; + } + // Keep the bound session's display name in sync with the task's effective + // label (its name, or its prompt when unnamed) — the session was named + // after the task at create, so a rename (or a prompt edit while unnamed) + // should follow. Only when the effective label actually changed, so a bare + // cron/enabled edit doesn't touch the session. Best-effort: a metadata + // failure must not fail the PATCH the schedule already committed. + const effectiveLabelChanged = + patch.name !== undefined || + clearName || + (patch.prompt !== undefined && updated.name === undefined); + if (bridge && updated.sessionId && effectiveLabelChanged) { + try { + bridge.updateSessionMetadata(updated.sessionId, { + displayName: scheduledTaskSessionName( + updated.name ?? updated.prompt, + ), + }); + } catch { + // non-critical — the schedule change already persisted + } + } + if (updated.delivery && updated.sessionId) { + channelDeliveryAuthorizations?.registerScheduledTask(workspaceCwd, { + sessionId: updated.sessionId, + taskId: updated.id, + target: updated.delivery.target, + recurring: updated.recurring, + lastFiredAt: updated.lastFiredAt ?? undefined, + }); + } + if (clearDelivery && updated.sessionId) { + channelDeliveryAuthorizations?.revokeScheduledTask( + workspaceCwd, + updated.sessionId, + updated.id, + ); + } + res.status(200).json(toView(updated)); + }), + ); // ── Delete ──────────────────────────────────────────────────────── - app.delete(`${base}/:id`, mutate(), async (req, res) => { - const target = resolveTarget(req, res); - if (!target) return; - if (!requireOpenGeneration(target, res)) return; - const { workspaceCwd, bridge } = target; - const id = typeof req.params['id'] === 'string' ? req.params['id'] : ''; - if (id.length === 0) { - res - .status(400) - .json({ error: 'Task id is required', code: 'invalid_id' }); - return; - } - // Single atomic read-modify-write: capture the task's bound session AND - // remove it in one cycle, closing the TOCTOU window a separate - // read-then-remove would open (and cutting three file reads to one). The - // dedicated session exists only to run this task, so it's torn down after. - let boundSessionId: string | undefined; - let removed = false; - let rollbackBefore: DurableCronTask[] | undefined; - let rollbackAfter: DurableCronTask[] | undefined; - try { - await runWithScheduledTaskTarget(target, () => - updateCronTasks( - workspaceCwd, - (tasks) => { - const idx = tasks.findIndex((t) => t.id === id); - if (idx === -1) return tasks; // not found → no write - const match = tasks[idx]!.sessionId; - if (typeof match === 'string' && match.length > 0) { - boundSessionId = match; - } - removed = true; - rollbackBefore = tasks; - rollbackAfter = tasks.filter((_, i) => i !== idx); - return rollbackAfter; - }, - { assertCanCommit: target.assertGenerationOpen }, - ), - ); - } catch (err) { - if (sendGenerationClosedError(res, err)) return; - writeStderrLine( - `qwen serve: DELETE ${base}/${id} failed: ${err instanceof Error ? err.message : String(err)}`, - ); - res.status(500).json({ - error: 'Failed to delete scheduled task', - code: 'scheduled_tasks_write_failed', - }); - return; - } - if (rollbackBefore && rollbackAfter) { + app.delete( + `${base}/:id`, + mutate(), + withTarget(async (req, res, target) => { + const { workspaceCwd, bridge } = target; + const id = typeof req.params['id'] === 'string' ? req.params['id'] : ''; + if (id.length === 0) { + res + .status(400) + .json({ error: 'Task id is required', code: 'invalid_id' }); + return; + } + // Single atomic read-modify-write: capture the task's bound session AND + // remove it in one cycle, closing the TOCTOU window a separate + // read-then-remove would open (and cutting three file reads to one). The + // dedicated session exists only to run this task, so it's torn down after. + let boundSessionId: string | undefined; + let removed = false; + let rollbackBefore: DurableCronTask[] | undefined; + let rollbackAfter: DurableCronTask[] | undefined; try { - target.assertGenerationOpen?.(); - } catch (error) { - await rollbackCronMutation( - target, - rollbackBefore, - rollbackAfter, - `DELETE ${base}/${id}`, + await runWithScheduledTaskTarget(target, () => + updateCronTasks( + workspaceCwd, + (tasks) => { + const idx = tasks.findIndex((t) => t.id === id); + if (idx === -1) return tasks; // not found → no write + const match = tasks[idx]!.sessionId; + if (typeof match === 'string' && match.length > 0) { + boundSessionId = match; + } + removed = true; + rollbackBefore = tasks; + rollbackAfter = tasks.filter((_, i) => i !== idx); + return rollbackAfter; + }, + { assertCanCommit: target.assertGenerationOpen }, + ), ); - if (sendGenerationClosedError(res, error)) return; - throw error; + } catch (err) { + if (sendActivityGateError(res, err)) return; + if (sendGenerationClosedError(res, err)) return; + writeStderrLine( + `qwen serve: DELETE ${base}/${id} failed: ${err instanceof Error ? err.message : String(err)}`, + ); + res.status(500).json({ + error: 'Failed to delete scheduled task', + code: 'scheduled_tasks_write_failed', + }); + return; } - } - if (!removed) { - res.status(404).json({ error: 'Task not found', code: 'task_not_found' }); - return; - } - // Stop the now-orphaned session (keeps its transcript on disk as history). - if (boundSessionId && bridge) { - await bridge.closeSession(boundSessionId).catch(() => {}); - } - if (boundSessionId) { - channelDeliveryAuthorizations?.revokeScheduledTask( - workspaceCwd, - boundSessionId, - id, - ); - } - res.status(200).json({ deleted: true, id }); - }); + if (rollbackBefore && rollbackAfter) { + try { + target.assertGenerationOpen?.(); + } catch (error) { + await rollbackCronMutation( + target, + rollbackBefore, + rollbackAfter, + `DELETE ${base}/${id}`, + ); + if (sendGenerationClosedError(res, error)) return; + throw error; + } + } + if (!removed) { + res + .status(404) + .json({ error: 'Task not found', code: 'task_not_found' }); + return; + } + // Stop the now-orphaned session (keeps its transcript on disk as history). + if (boundSessionId && bridge) { + try { + await runWithScheduledTaskTarget(target, () => + bridge.closeSession(boundSessionId!), + ); + } catch (error) { + if (sendActivityGateError(res, error)) return; + } + } + if (boundSessionId) { + channelDeliveryAuthorizations?.revokeScheduledTask( + workspaceCwd, + boundSessionId, + id, + ); + } + res.status(200).json({ deleted: true, id }); + }), + ); // ── Record a manual run ─────────────────────────────────────────── // Marks the task as run *now* (updates lastFiredAt + appends a 'manual' run @@ -1029,135 +1113,148 @@ function registerScheduledTaskCrudRoutes( // prompt itself is executed by the client in the task's bound session; this // route only records that a run happened, keeping manual and scheduled runs // consistent in the history. - app.post(`${base}/:id/run`, mutate(), async (req, res) => { - const target = resolveTarget(req, res); - if (!target) return; - if (!requireOpenGeneration(target, res)) return; - const { workspaceCwd } = target; - const id = typeof req.params['id'] === 'string' ? req.params['id'] : ''; - if (id.length === 0) { - res - .status(400) - .json({ error: 'Task id is required', code: 'invalid_id' }); - return; - } - // A manual run is stamped at its exact instant (not minute-rounded like a - // scheduler fire): the scheduler compares slots as `slot > lastFiredAt`, so - // a precise timestamp behaves correctly, and — unlike rounding — it can't - // collide with the creation-minute anchor that describeLastRun reads as - // "never run" when a task is run manually within its creation minute. - const now = Date.now(); - let found = false; - let blockedDisabled = false; - let blockedLegacy = false; - let updated: DurableCronTask | undefined; - let rollbackBefore: DurableCronTask[] | undefined; - let rollbackAfter: DurableCronTask[] | undefined; - try { - await runWithScheduledTaskTarget(target, () => - updateCronTasks( - workspaceCwd, - (tasks) => { - const idx = tasks.findIndex((t) => t.id === id); - if (idx === -1) return tasks; // not found → no write - found = true; - const current = tasks[idx]!; - // A legacy guarded task (isolated + precondition, both removed) must not - // run from ANY path. The scheduler already skips it and the list view - // reports it disabled; reject a direct `/run` too — its on-disk - // `enabled` may still be true, so the disabled check below is not enough. - // Executing it here would run the prompt with its safety gate ignored, - // which is exactly what the removal must never allow. - if (taskHasLegacyCondition(current)) { - blockedLegacy = true; - return tasks; // no write - } - // A disabled task must not record a manual run: it's paused (and if it - // was disabled by archiving its session, that session can't even fire), - // so stamping lastFiredAt + a 'manual' entry would write a phantom "ran" - // record. Mirrors the PATCH route's refusal to re-enable such tasks and - // the UI, where onRunPrompt already rejects before recording. - if (current.enabled === false) { - blockedDisabled = true; - return tasks; // no write - } - const next: DurableCronTask = { - ...current, - lastFiredAt: now, - runs: appendCronRun(current.runs, { - at: now, - kind: 'manual', - ...(current.sessionId ? { sessionId: current.sessionId } : {}), - }), - }; - updated = next; - // A one-shot's manual run IS its single fire — remove it from the store - // so the scheduler doesn't ALSO fire it at its original scheduled time - // (its slot is still in the future, so stamping lastFiredAt=now wouldn't - // stop that fire). The response still returns the recorded run. - rollbackBefore = tasks; - const nextTasks = !current.recurring - ? tasks.filter((_, i) => i !== idx) - : tasks.map((t, i) => (i === idx ? next : t)); - rollbackAfter = nextTasks; - return nextTasks; - }, - { assertCanCommit: target.assertGenerationOpen }, - ), - ); - } catch (err) { - if (sendGenerationClosedError(res, err)) return; - writeStderrLine( - `qwen serve: POST ${base}/${id}/run failed: ${err instanceof Error ? err.message : String(err)}`, - ); - res.status(500).json({ - error: 'Failed to record scheduled task run', - code: 'scheduled_tasks_write_failed', - }); - return; - } - if (rollbackBefore && rollbackAfter) { + app.post( + `${base}/:id/run`, + mutate(), + withTarget(async (req, res, target) => { + const { workspaceCwd } = target; + const id = typeof req.params['id'] === 'string' ? req.params['id'] : ''; + if (id.length === 0) { + res + .status(400) + .json({ error: 'Task id is required', code: 'invalid_id' }); + return; + } + // A manual run is stamped at its exact instant (not minute-rounded like a + // scheduler fire): the scheduler compares slots as `slot > lastFiredAt`, so + // a precise timestamp behaves correctly, and — unlike rounding — it can't + // collide with the creation-minute anchor that describeLastRun reads as + // "never run" when a task is run manually within its creation minute. + const now = Date.now(); + let found = false; + let blockedDisabled = false; + let blockedLegacy = false; + let updated: DurableCronTask | undefined; + let rollbackBefore: DurableCronTask[] | undefined; + let rollbackAfter: DurableCronTask[] | undefined; try { - target.assertGenerationOpen?.(); - } catch (error) { - await rollbackCronMutation( - target, - rollbackBefore, - rollbackAfter, - `POST ${base}/${id}/run`, + await runWithScheduledTaskTarget(target, () => + updateCronTasks( + workspaceCwd, + (tasks) => { + const idx = tasks.findIndex((t) => t.id === id); + if (idx === -1) return tasks; // not found → no write + found = true; + const current = tasks[idx]!; + // A legacy guarded task (isolated + precondition, both removed) must not + // run from ANY path. The scheduler already skips it and the list view + // reports it disabled; reject a direct `/run` too — its on-disk + // `enabled` may still be true, so the disabled check below is not enough. + // Executing it here would run the prompt with its safety gate ignored, + // which is exactly what the removal must never allow. + if (taskHasLegacyCondition(current)) { + blockedLegacy = true; + return tasks; // no write + } + // A disabled task must not record a manual run: it's paused (and if it + // was disabled by archiving its session, that session can't even fire), + // so stamping lastFiredAt + a 'manual' entry would write a phantom "ran" + // record. Mirrors the PATCH route's refusal to re-enable such tasks and + // the UI, where onRunPrompt already rejects before recording. + if (current.enabled === false) { + blockedDisabled = true; + return tasks; // no write + } + const next: DurableCronTask = { + ...current, + lastFiredAt: now, + runs: appendCronRun(current.runs, { + at: now, + kind: 'manual', + ...(current.sessionId + ? { sessionId: current.sessionId } + : {}), + }), + }; + updated = next; + // A one-shot's manual run IS its single fire — remove it from the store + // so the scheduler doesn't ALSO fire it at its original scheduled time + // (its slot is still in the future, so stamping lastFiredAt=now wouldn't + // stop that fire). The response still returns the recorded run. + rollbackBefore = tasks; + const nextTasks = !current.recurring + ? tasks.filter((_, i) => i !== idx) + : tasks.map((t, i) => (i === idx ? next : t)); + rollbackAfter = nextTasks; + return nextTasks; + }, + { assertCanCommit: target.assertGenerationOpen }, + ), ); - if (sendGenerationClosedError(res, error)) return; - throw error; + } catch (err) { + if (sendActivityGateError(res, err)) return; + if (sendGenerationClosedError(res, err)) return; + writeStderrLine( + `qwen serve: POST ${base}/${id}/run failed: ${err instanceof Error ? err.message : String(err)}`, + ); + res.status(500).json({ + error: 'Failed to record scheduled task run', + code: 'scheduled_tasks_write_failed', + }); + return; } - } - if (blockedLegacy) { - res.status(409).json({ - error: - 'This task uses the removed isolated run mode with a precondition and can no longer run. Recreate it (and call the `create_sub_session` tool from the prompt if you need per-run isolation).', - code: 'task_legacy_unsupported', - }); - return; - } - if (blockedDisabled) { - res.status(409).json({ - error: - 'Cannot run a disabled task; enable it first (unarchive its session if it was archived).', - code: 'task_disabled', - }); - return; - } - if (!found || !updated) { - res.status(404).json({ error: 'Task not found', code: 'task_not_found' }); - return; - } - const view = toView(updated); - // A consumed one-shot was removed from the store — its manual run WAS its - // single fire, so the returned view must not advertise a future nextRunAt on - // an entity the next GET omits (the shipped dialog reloads, but an embedder - // gets this object from the SDK). - if (!updated.recurring) view.nextRunAt = null; - res.status(200).json(view); - }); + if (rollbackBefore && rollbackAfter) { + try { + target.assertGenerationOpen?.(); + } catch (error) { + await rollbackCronMutation( + target, + rollbackBefore, + rollbackAfter, + `POST ${base}/${id}/run`, + ); + if (sendGenerationClosedError(res, error)) return; + throw error; + } + } + if (blockedLegacy) { + res.status(409).json({ + error: + 'This task uses the removed isolated run mode with a precondition and can no longer run. Recreate it (and call the `create_sub_session` tool from the prompt if you need per-run isolation).', + code: 'task_legacy_unsupported', + }); + return; + } + if (blockedDisabled) { + res.status(409).json({ + error: + 'Cannot run a disabled task; enable it first (unarchive its session if it was archived).', + code: 'task_disabled', + }); + return; + } + if (!found || !updated) { + res + .status(404) + .json({ error: 'Task not found', code: 'task_not_found' }); + return; + } + if (!updated.recurring && updated.sessionId) { + channelDeliveryAuthorizations?.revokeScheduledTask( + workspaceCwd, + updated.sessionId, + updated.id, + ); + } + const view = toView(updated); + // A consumed one-shot was removed from the store — its manual run WAS its + // single fire, so the returned view must not advertise a future nextRunAt on + // an entity the next GET omits (the shipped dialog reloads, but an embedder + // gets this object from the SDK). + if (!updated.recurring) view.nextRunAt = null; + res.status(200).json(view); + }), + ); } /** @@ -1238,13 +1335,20 @@ export function registerWorkspaceQualifiedScheduledTasksRoutes( registerScheduledTaskCrudRoutes(app, { prefix: '/workspaces/:workspace', resolveTarget: (req, res) => { - const runtime = resolveWorkspaceRuntimeFromParam( + const runtime = resolveWorkspaceRuntimeWithLiveCompatibilityFromParam( workspaceRegistry, req, res, ); if (!runtime) return null; if (!requireTrustedWorkspaceRuntime(runtime, res)) return null; + if ( + runtime.provenance === 'live-conversation' && + !deps.conversationRuntimeActivity + ) { + sendConversationRuntimeUnavailable(res); + return null; + } if ( runtime.provenance === 'live-conversation' && req.method === 'POST' && @@ -1260,6 +1364,10 @@ export function registerWorkspaceQualifiedScheduledTasksRoutes( return { workspaceCwd: runtime.workspaceCwd, runtimeBaseDir: runtime.sessionRuntimeBaseDir, + ...(runtime.provenance === 'live-conversation' && + deps.conversationRuntimeActivity + ? { activity: deps.conversationRuntimeActivity } + : {}), ...(cleanupSession ? { cleanupSession: (sessionId: string) => diff --git a/packages/cli/src/serve/routes/session-runtime.test.ts b/packages/cli/src/serve/routes/session-runtime.test.ts index a1553346581..063d1d2a23c 100644 --- a/packages/cli/src/serve/routes/session-runtime.test.ts +++ b/packages/cli/src/serve/routes/session-runtime.test.ts @@ -39,6 +39,7 @@ function response(): Response { res.statusCode = statusCode; return res; }), + set: vi.fn(() => res), json: vi.fn(() => res), }; return res as unknown as Response; @@ -183,4 +184,90 @@ describe('requireSessionRuntime telemetry attribution', () => { expect(telemetryMocks.setDaemonTelemetryWorkspace).not.toHaveBeenCalled(); }, ); + + it('redacts internal workspace ids from ambiguous ownership responses', () => { + const primary = runtime('/workspace/primary', { primary: true }); + const internal = { + ...runtime('/workspace/conversations'), + provenance: 'live-conversation' as const, + }; + const setup = registry({ + primary, + runtimes: [primary, internal], + resolution: { kind: 'ambiguous', runtimes: [primary, internal] }, + }); + const res = response(); + + expect( + requireSessionRuntime({ + sessionId: 'duplicate-session', + route: 'GET /session/:id/events', + res, + workspaceRegistry: setup.registry, + }), + ).toBeUndefined(); + expect(res.statusCode).toBe(500); + expect(res.json).toHaveBeenCalledWith( + expect.objectContaining({ + code: 'ambiguous_session_owner', + sessionId: 'duplicate-session', + }), + ); + expect(vi.mocked(res.json).mock.calls[0]?.[0]).not.toHaveProperty( + 'workspaceIds', + ); + }); + + it('reports an unavailable primary while an internal runtime is registered', () => { + const primary = runtime('/workspace/primary', { primary: true }); + const internal = { + ...runtime('/workspace/conversations'), + provenance: 'live-conversation' as const, + }; + const setup = registry({ + primary, + runtimes: [primary, internal], + resolution: { kind: 'not_found' }, + }); + expect( + setup.registry.beginReplacement(setup.registry.primaryEntry, 'next'), + ).toBe(true); + const res = response(); + + expect( + requireSessionRuntime({ + sessionId: 'primary-session', + route: 'POST /session/:id/prompt', + res, + workspaceRegistry: setup.registry, + }), + ).toBeUndefined(); + expect(res.statusCode).toBe(503); + expect(res.set).toHaveBeenCalledWith('Retry-After', '1'); + expect(res.json).toHaveBeenCalledWith( + expect.objectContaining({ code: 'workspace_runtime_unavailable' }), + ); + }); + + it('keeps ordinary workspace ids in ambiguous ownership responses', () => { + const primary = runtime('/workspace/primary', { primary: true }); + const secondary = runtime('/workspace/secondary'); + const setup = registry({ + primary, + runtimes: [primary, secondary], + resolution: { kind: 'ambiguous', runtimes: [primary, secondary] }, + }); + const res = response(); + + requireSessionRuntime({ + sessionId: 'duplicate-session', + route: 'GET /session/:id/events', + res, + workspaceRegistry: setup.registry, + }); + + expect(vi.mocked(res.json).mock.calls[0]?.[0]).toMatchObject({ + workspaceIds: ['primary', 'secondary'], + }); + }); }); diff --git a/packages/cli/src/serve/routes/session-runtime.ts b/packages/cli/src/serve/routes/session-runtime.ts index 45468858304..f27478b2db0 100644 --- a/packages/cli/src/serve/routes/session-runtime.ts +++ b/packages/cli/src/serve/routes/session-runtime.ts @@ -10,6 +10,7 @@ import type { WorkspaceRegistry, WorkspaceRuntime, } from '../workspace-registry.js'; +import { isInternalWorkspaceRuntime } from '../workspace-runtime-visibility.js'; import { sendUntrustedWorkspaceResponse, sendWorkspaceRuntimeUnavailable, @@ -46,11 +47,21 @@ export function requireSessionRuntime(opts: { daemonLog, details = {}, } = opts; - if (workspaceRegistry.listEntries().length === 1) { + if (workspaceRegistry.listAllEntries().length === 1) { return requirePrimarySessionRuntime(workspaceRegistry, res); } const resolution = workspaceRegistry.resolveLiveSessionOwner(sessionId); + if (resolution.kind === 'unavailable') { + daemonLog?.warn('session routing failed', { + route, + resolutionKind: 'workspace_runtime_unavailable', + sessionId, + ...details, + }); + sendWorkspaceRuntimeUnavailable(res); + return undefined; + } if (resolution.kind === 'found') { const runtime = resolution.runtime; setDaemonTelemetryWorkspace(res, runtime.workspaceCwd); @@ -74,6 +85,12 @@ export function requireSessionRuntime(opts: { } if (resolution.kind === 'not_found') { + if ( + workspaceRegistry.primaryEntry.state !== 'active' || + !workspaceRegistry.primaryEntry.current + ) { + return requirePrimarySessionRuntime(workspaceRegistry, res); + } daemonLog?.warn('session routing failed', { route, resolutionKind: 'not_found', @@ -103,7 +120,11 @@ export function requireSessionRuntime(opts: { code: 'ambiguous_session_owner', sessionId, route, - workspaceIds, + ...(resolution.runtimes.every( + (runtime) => !isInternalWorkspaceRuntime(runtime), + ) + ? { workspaceIds } + : {}), }); return undefined; } diff --git a/packages/cli/src/serve/routes/session.ts b/packages/cli/src/serve/routes/session.ts index f9dbed0b88b..0814a647859 100644 --- a/packages/cli/src/serve/routes/session.ts +++ b/packages/cli/src/serve/routes/session.ts @@ -40,6 +40,8 @@ import { isReservedLiveSessionSource, readLoadableLiveConversationMetadata, } from '../conversations/session-source.js'; +import type { ConversationRuntimeActivityGate } from '../conversations/conversation-runtime-activity.js'; +import { ConversationRuntimeOwnershipError } from '../conversations/conversation-runtime-errors.js'; import type { Application, Request, RequestHandler, Response } from 'express'; import { writeStderrLine } from '../../utils/stdioHelpers.js'; import { parseCallerSuppliedSessionId } from '../../config/session-id.js'; @@ -123,9 +125,12 @@ import { sendWorkspaceRuntimeUnavailable, } from '../workspace-route-runtime.js'; import type { + WorkspaceEntry, WorkspaceRegistry, WorkspaceRuntime, + WorkspaceRuntimeGeneration, } from '../workspace-registry.js'; +import { isInternalWorkspaceRuntime } from '../workspace-runtime-visibility.js'; import { createWorkspaceRuntimeSessionService, runWithWorkspaceRuntimeStorage, @@ -175,6 +180,9 @@ interface RegisterSessionRoutesDeps { virtualSubagentSessions?: VirtualSubagentSessions; materializeLiveConversationDirectory?: (sessionId: string) => Promise; isLiveSessionActive?: (sessionId: string) => boolean; + ensureConversationRuntime?: () => Promise; + liveConversationRootPath?: string; + conversationRuntimeActivity?: ConversationRuntimeActivityGate; } // Chosen cap for one serialized transcript response, kept proportional to @@ -191,14 +199,19 @@ const TRANSCRIPT_CURSOR_TOO_LARGE_REPLAY_ERROR = 'Transcript pagination state exceeds the safe limit'; // Must exceed CHANNEL_DELIVERY_IPC_TIMEOUT_MS (30 s, channel-delivery-ipc.ts) plus scheduling slack. const CHANNEL_DELIVERY_AUTHORIZATION_GRACE_MS = 60_000; -const PRIMARY_ONLY_LIVE_SESSION_ROUTES = [ +const PRIMARY_ONLY_LIVE_SESSION_ROUTES = ['POST /session/:id/cd'] as const; +const PRIMARY_OR_INTERNAL_LIVE_SESSION_ROUTES = [ 'POST /session/:id/branch', 'POST /session/:id/side-task', 'POST /session/:id/fork', - 'POST /session/:id/cd', ] as const; type PrimaryOnlyLiveSessionRoute = (typeof PRIMARY_ONLY_LIVE_SESSION_ROUTES)[number]; +type PrimaryOrInternalLiveSessionRoute = + (typeof PRIMARY_OR_INTERNAL_LIVE_SESSION_ROUTES)[number]; +type RestrictedLiveSessionRoute = + | PrimaryOnlyLiveSessionRoute + | PrimaryOrInternalLiveSessionRoute; function isPrimaryOnlyLiveSessionRoute( route: string, @@ -208,6 +221,14 @@ function isPrimaryOnlyLiveSessionRoute( ); } +function isPrimaryOrInternalLiveSessionRoute( + route: string, +): route is PrimaryOrInternalLiveSessionRoute { + return ( + PRIMARY_OR_INTERNAL_LIVE_SESSION_ROUTES as readonly string[] + ).includes(route); +} + function isReadOnlyWorkspaceInspection(runtime: WorkspaceRuntime): boolean { return !runtime.primary && !runtime.trusted; } @@ -453,7 +474,6 @@ export function registerSessionRoutes( ): void { const { boundWorkspace, - bridge, workspaceRegistry, archiveCoordinator, mutate, @@ -497,14 +517,14 @@ export function registerSessionRoutes( })), getBridgeWorkspaceId: (bridge) => workspaceRegistry - .listEntries() + .listAllEntries() .find((entry) => entry.current?.runtime.bridge === bridge) ?.workspaceId, }); const captureRuntimeGenerationAssertion = ( runtime: WorkspaceRuntime, ): (() => void) | undefined => { - const registeredGeneration = workspaceRegistry.getEntryByWorkspaceId( + const registeredGeneration = workspaceRegistry.getManagedEntryByWorkspaceId( runtime.workspaceId, )?.current; const guard = @@ -639,11 +659,23 @@ export function registerSessionRoutes( }); const status = error.code === 'session_id_admission_unavailable' ? 503 : 409; + const internalIdentities = new Set( + workspaceRegistry + .listAllEntries() + .filter((entry) => entry.internal) + .flatMap((entry) => [entry.workspaceCwd, entry.workspaceId]), + ); + const publicDetails = Object.fromEntries( + Object.entries(error.details).filter( + ([, value]) => + typeof value !== 'string' || !internalIdentities.has(value), + ), + ); res.status(status).json({ error: error.message, code: error.code, sessionId: error.sessionId, - ...error.details, + ...publicDetails, }); }; @@ -676,9 +708,62 @@ export function registerSessionRoutes( ): { runtime: WorkspaceRuntime; workspaceCwd: string } | undefined => { const cwd = parseOptionalWorkspaceCwd(body, boundWorkspace, res); if (cwd === undefined) return undefined; + const isWithinConversationRoot = (root: string, candidate: string) => { + const relative = path.relative(root, candidate); + return ( + relative === '' || + (relative !== '..' && + !relative.startsWith(`..${path.sep}`) && + !path.isAbsolute(relative)) + ); + }; + const canonicalizeIfPresent = (candidate: string): string => { + const resolved = path.resolve(candidate); + try { + return fs.realpathSync.native(resolved); + } catch { + return resolved; + } + }; + const canonicalizeExistingAncestor = (candidate: string): string => { + let ancestor = path.resolve(candidate); + const missingTail: string[] = []; + while (true) { + try { + return path.join(fs.realpathSync.native(ancestor), ...missingTail); + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error; + const parent = path.dirname(ancestor); + if (parent === ancestor) throw error; + missingTail.unshift(path.basename(ancestor)); + ancestor = parent; + } + } + }; + const rejectReservedConversationRoot = (): undefined => { + res.status(400).json({ + error: + 'Generic session creation is unavailable in the Conversations workspace.', + code: 'live_session_creation_reserved', + }); + return undefined; + }; + if ( + 'cwd' in body && + deps.liveConversationRootPath && + isWithinConversationRoot( + path.resolve(deps.liveConversationRootPath), + path.resolve(cwd), + ) + ) { + return rejectReservedConversationRoot(); + } let key: string; + let reservedCheckKey: string; try { key = canonicalizeWorkspace(cwd); + reservedCheckKey = + 'cwd' in body ? canonicalizeExistingAncestor(key) : key; } catch (err) { if (workspaceRegistry.listEntries().length > 1 && 'cwd' in body) { logSessionRoutingFailure('POST /session', 'workspace_mismatch', { @@ -690,6 +775,24 @@ export function registerSessionRoutes( sendBridgeError(res, err, { route: 'POST /session' }); return undefined; } + const liveRoots = [ + ...workspaceRegistry + .listAllEntries() + .filter((entry) => entry.internal) + .map((entry) => entry.workspaceCwd), + ...(deps.liveConversationRootPath + ? [ + path.resolve(deps.liveConversationRootPath), + canonicalizeIfPresent(deps.liveConversationRootPath), + ] + : []), + ]; + if ( + 'cwd' in body && + liveRoots.some((root) => isWithinConversationRoot(root, reservedCheckKey)) + ) { + return rejectReservedConversationRoot(); + } if (workspaceRegistry.listEntries().length === 1) { const runtime = requirePrimarySessionRuntime(workspaceRegistry, res); if (!runtime) return undefined; @@ -788,6 +891,176 @@ export function registerSessionRoutes( return runtime; }; + const sendConversationRuntimeError = ( + res: Response, + error: unknown, + ): boolean => { + if (!(error instanceof ConversationRuntimeOwnershipError)) return false; + res.status(error.status).json({ + error: error.message, + code: error.code, + retryable: error.retryable, + }); + return true; + }; + + const resolveLiveCatalogRuntime = async ( + req: Request, + res: Response, + paramName: 'id' | 'workspace', + ): Promise => { + if ( + req.query['sourceType'] !== 'default' || + req.query['sourceId'] !== undefined || + !deps.ensureConversationRuntime + ) { + return undefined; + } + + const selector = req.params[paramName] ?? ''; + let entry = workspaceRegistry.getManagedEntryByWorkspaceId(selector); + if (!entry && path.isAbsolute(selector)) { + entry = workspaceRegistry.getManagedEntryByWorkspaceCwd(selector); + } + const configuredRoot = deps.liveConversationRootPath + ? path.resolve(deps.liveConversationRootPath) + : undefined; + const matchesConfiguredRoot = + configuredRoot !== undefined && + selector === configuredRoot && + path.resolve(selector) === selector; + if ((!entry || !entry.internal) && !matchesConfiguredRoot) { + return undefined; + } + if (entry?.internal && entry.state !== 'active') { + res.status(503).json({ + error: 'The Conversations runtime is temporarily unavailable.', + code: 'conversation_runtime_unavailable', + retryable: true, + }); + return null; + } + + try { + const runtime = await deps.ensureConversationRuntime(); + const activeEntry = workspaceRegistry.getManagedEntryByWorkspaceCwd( + runtime.workspaceCwd, + ); + const selectorMatchesRuntime = + matchesConfiguredRoot || + selector === runtime.workspaceId || + selector === runtime.workspaceCwd; + if ( + !selectorMatchesRuntime || + !activeEntry?.internal || + activeEntry.state !== 'active' || + activeEntry.current?.runtime !== runtime + ) { + return undefined; + } + return runtime; + } catch (error) { + if (sendConversationRuntimeError(res, error)) return null; + throw error; + } + }; + + const resolveQualifiedSessionTarget = ( + req: Request, + res: Response, + options: { allowUntrustedSecondary?: boolean } = {}, + ): + | { kind: 'internal'; entry: WorkspaceEntry } + | { kind: 'ordinary'; runtime: WorkspaceRuntime } + | undefined => { + const selector = req.params['workspace'] ?? ''; + const entry = + workspaceRegistry.getManagedEntryByWorkspaceId(selector) ?? + (path.isAbsolute(selector) + ? workspaceRegistry.getManagedEntryByWorkspaceCwd(selector) + : undefined); + if (entry?.internal) return { kind: 'internal', entry }; + const runtime = resolveWorkspaceRuntimeFromParam( + workspaceRegistry, + req, + res, + ); + if (!runtime) return undefined; + if ( + !runtime.trusted && + (!options.allowUntrustedSecondary || runtime.primary) + ) { + sendUntrustedWorkspaceResponse(res, { + workspaceCwd: runtime.workspaceCwd, + workspaceId: runtime.workspaceId, + }); + return undefined; + } + setDaemonTelemetryWorkspace(res, runtime.workspaceCwd); + return { kind: 'ordinary', runtime }; + }; + + const assertCurrentInternalGeneration = ( + entry: WorkspaceEntry, + generation: WorkspaceRuntimeGeneration, + res: Response, + ): boolean => { + if ( + entry.state !== 'active' || + entry.current !== generation || + generation.guard.closed + ) { + sendWorkspaceRuntimeUnavailable(res); + return false; + } + generation.guard.assertOpen(); + return true; + }; + + const resolveQualifiedSessionRuntime = async ( + req: Request, + res: Response, + route: string, + sessionIds: readonly string[], + archiveState: SessionArchiveState | 'any', + ): Promise => { + const target = resolveQualifiedSessionTarget(req, res); + if (!target) return undefined; + if (target.kind === 'ordinary') return target.runtime; + const internalEntry = target.entry; + const generation = + internalEntry.state === 'active' ? internalEntry.current : undefined; + if (!generation) { + sendWorkspaceRuntimeUnavailable(res); + return undefined; + } + const runtime = generation.runtime; + const service = createWorkspaceRuntimeSessionService(runtime); + for (const sessionId of sessionIds) { + const location = await service.getSessionLocation(sessionId); + if (location === 'conflict') throw new SessionConflictError(sessionId); + if ( + location === undefined || + (archiveState !== 'any' && location !== archiveState) + ) { + throw new SessionNotFoundError(sessionId); + } + const metadata = await readLoadableLiveConversationMetadata( + sessionId, + (candidateId) => service.readCreationMetadata(candidateId), + ); + if (!metadata) throw new SessionNotFoundError(sessionId); + } + if (!assertCurrentInternalGeneration(internalEntry, generation, res)) { + return undefined; + } + if (!assertTrustedSessionOwner(res, route, sessionIds[0] ?? '', runtime)) { + return undefined; + } + setDaemonTelemetryWorkspace(res, runtime.workspaceCwd); + return runtime; + }; + const resolveLegacyPrimaryRuntimeFromParam = ( req: Request, res: Response, @@ -871,13 +1144,24 @@ export function registerSessionRoutes( res: Response, target: { route: string; - runtime: WorkspaceRuntime; + runtime?: WorkspaceRuntime; + resolveRuntime?: ( + sessionId: string, + ) => Promise; workspaceQualified?: boolean; archiveState?: SessionArchiveState; }, ): Promise => { const sessionId = requireSessionId(req, res); if (sessionId === null) return; + let preResolvedRuntime = target.runtime; + if (target.workspaceQualified && !preResolvedRuntime) { + const qualifiedTarget = resolveQualifiedSessionTarget(req, res); + if (!qualifiedTarget) return; + if (qualifiedTarget.kind === 'ordinary') { + preResolvedRuntime = qualifiedTarget.runtime; + } + } const rawFormat = req.query['format']; const format = parseSessionExportFormat(rawFormat); if (!format) { @@ -891,29 +1175,39 @@ export function registerSessionRoutes( } try { const result = await archiveCoordinator.runSharedMany([sessionId], () => - runWithWorkspaceRuntimeStorage(target.runtime, async () => { - if (target.archiveState === 'archived') { - await assertSessionArchived( - target.runtime.workspaceCwd, - sessionId, - target.runtime.sessionRuntimeBaseDir, - ); - } else { - await assertSessionLoadable( - target.runtime.workspaceCwd, + (async () => { + const runtime = + preResolvedRuntime ?? (await target.resolveRuntime?.(sessionId)); + if (!runtime) return undefined; + const assertRuntimeGenerationOpen = + captureRuntimeGenerationAssertion(runtime); + assertRuntimeGenerationOpen?.(); + return runWithWorkspaceRuntimeStorage(runtime, async () => { + if (target.archiveState === 'archived') { + await assertSessionArchived( + runtime.workspaceCwd, + sessionId, + runtime.sessionRuntimeBaseDir, + ); + } else { + await assertSessionLoadable( + runtime.workspaceCwd, + sessionId, + runtime.sessionRuntimeBaseDir, + ); + } + assertRuntimeGenerationOpen?.(); + return exportSessionTranscript({ + workspaceCwd: runtime.workspaceCwd, sessionId, - target.runtime.sessionRuntimeBaseDir, - ); - } - return exportSessionTranscript({ - workspaceCwd: target.runtime.workspaceCwd, - sessionId, - format, - archiveState: target.archiveState, - config: { getChannel: () => 'daemon' }, + format, + archiveState: target.archiveState, + config: { getChannel: () => 'daemon' }, + }); }); - }), + })(), ); + if (!result) return; const filename = result.filename.replace(/["\\\r\n]/g, '_'); res .status(200) @@ -935,7 +1229,7 @@ export function registerSessionRoutes( route: target.route, sessionId, ...(target.workspaceQualified - ? { workspaceCwd: target.runtime.workspaceCwd } + ? { workspaceCwd: target.runtime?.workspaceCwd } : {}), }); } @@ -957,7 +1251,9 @@ export function registerSessionRoutes( code: 'ambiguous_session_owner', sessionId, route, - workspaceIds, + ...(runtimes.every((runtime) => !isInternalWorkspaceRuntime(runtime)) + ? { workspaceIds } + : {}), }); }; @@ -998,8 +1294,14 @@ export function registerSessionRoutes( res: Response, route: string, sessionId: string, - runtime: Pick, - liveRuntime: Pick, + runtime: Pick< + WorkspaceRuntime, + 'workspaceCwd' | 'workspaceId' | 'provenance' + >, + liveRuntime: Pick< + WorkspaceRuntime, + 'workspaceCwd' | 'workspaceId' | 'provenance' + >, ): void => { logSessionRoutingFailure(route, 'workspace_conflict', { sessionId, @@ -1012,21 +1314,39 @@ export function registerSessionRoutes( error: `Session "${sessionId}" is already live or restoring in another workspace runtime.`, code: 'session_workspace_conflict', sessionId, - workspaceCwd: runtime.workspaceCwd, - workspaceId: runtime.workspaceId, - liveWorkspaceCwd: liveRuntime.workspaceCwd, - liveWorkspaceId: liveRuntime.workspaceId, + ...(!isInternalWorkspaceRuntime(runtime) && + !isInternalWorkspaceRuntime(liveRuntime) + ? { + workspaceId: runtime.workspaceId, + workspaceCwd: runtime.workspaceCwd, + liveWorkspaceId: liveRuntime.workspaceId, + liveWorkspaceCwd: liveRuntime.workspaceCwd, + } + : {}), }); }; - const resolveRuntimeForSessionRestore = ( + const resolveRuntimeForSessionRestore = async ( body: Record, res: Response, route: string, sessionId: string, - ): { runtime: WorkspaceRuntime; workspaceCwd: string } | undefined => { + ): Promise< + { runtime: WorkspaceRuntime; workspaceCwd: string } | undefined + > => { const cwd = parseOptionalWorkspaceCwd(body, boundWorkspace, res); if (cwd === undefined) return undefined; + const configuredRoot = deps.liveConversationRootPath + ? path.resolve(deps.liveConversationRootPath) + : undefined; + const bootstrappedRuntime = + 'cwd' in body && + configuredRoot !== undefined && + path.isAbsolute(cwd) && + path.resolve(cwd) === configuredRoot && + deps.ensureConversationRuntime + ? await deps.ensureConversationRuntime() + : undefined; let key: string; try { key = canonicalizeWorkspace(cwd); @@ -1042,9 +1362,29 @@ export function registerSessionRoutes( return undefined; } - const runtime = workspaceRegistry.resolveWorkspaceCwd( - 'cwd' in body ? key : undefined, - ); + const managedEntry = + 'cwd' in body + ? workspaceRegistry.getManagedEntryByWorkspaceCwd(key) + : undefined; + if ( + bootstrappedRuntime && + (!managedEntry?.internal || + managedEntry.state !== 'active' || + managedEntry.current?.runtime !== bootstrappedRuntime) + ) { + sendWorkspaceRuntimeUnavailable(res); + return undefined; + } + if ( + managedEntry?.internal && + (managedEntry.state !== 'active' || !managedEntry.current) + ) { + sendWorkspaceRuntimeUnavailable(res); + return undefined; + } + const runtime = managedEntry?.internal + ? managedEntry.current?.runtime + : workspaceRegistry.resolveWorkspaceCwd('cwd' in body ? key : undefined); if (!runtime) { logSessionRoutingFailure(route, 'workspace_mismatch', { requestedWorkspace: key, @@ -1052,7 +1392,9 @@ export function registerSessionRoutes( sendWorkspaceMismatch(res, key); return undefined; } - setDaemonTelemetryWorkspace(res, runtime.workspaceCwd); + if (!managedEntry?.internal) { + setDaemonTelemetryWorkspace(res, runtime.workspaceCwd); + } if (!runtime.primary && !runtime.trusted) { logSessionRoutingFailure(route, 'untrusted_workspace', { workspaceId: runtime.workspaceId, @@ -1066,6 +1408,10 @@ export function registerSessionRoutes( } const liveOwner = workspaceRegistry.resolveLiveSessionOwner(sessionId); + if (liveOwner.kind === 'unavailable') { + sendWorkspaceRuntimeUnavailable(res); + return undefined; + } if (liveOwner.kind === 'ambiguous') { sendAmbiguousSessionOwner(res, route, sessionId, liveOwner.runtimes); return undefined; @@ -1102,7 +1448,7 @@ export function registerSessionRoutes( const sendNonPrimarySessionRouteUnsupported = ( res: Response, - route: PrimaryOnlyLiveSessionRoute, + route: RestrictedLiveSessionRoute, sessionId: string, runtime: WorkspaceRuntime, ): void => { @@ -1110,8 +1456,12 @@ export function registerSessionRoutes( error: `Route "${route}" is only available for primary workspace sessions.`, code: 'non_primary_session_route_not_supported', sessionId, - workspaceId: runtime.workspaceId, - workspaceCwd: runtime.workspaceCwd, + ...(!isInternalWorkspaceRuntime(runtime) + ? { + workspaceId: runtime.workspaceId, + workspaceCwd: runtime.workspaceCwd, + } + : {}), route, }); }; @@ -1168,6 +1518,7 @@ export function registerSessionRoutes( route: string, sessionId: string, hasCursor: boolean, + legacyPrimaryFallback = false, ): Promise => { const activeInRuntime = async ( runtime: WorkspaceRuntime, @@ -1177,7 +1528,14 @@ export function registerSessionRoutes( sessionId, runtime.sessionRuntimeBaseDir, ); - return location === 'active'; + if (location !== 'active') return false; + if (!isInternalWorkspaceRuntime(runtime)) return true; + const service = createWorkspaceRuntimeSessionService(runtime); + return ( + (await readLoadableLiveConversationMetadata(sessionId, (candidateId) => + service.readCreationMetadata(candidateId), + )) !== undefined + ); }; const throwMissingActiveTranscript = (): never => { if (hasCursor) { @@ -1185,68 +1543,169 @@ export function registerSessionRoutes( } throw new SessionNotFoundError(sessionId); }; + let loadError: unknown; + const recordLoadError = (err: unknown): void => { + if ( + loadError !== undefined && + shouldPreserveTranscriptResolutionError(loadError) && + shouldPreserveTranscriptResolutionError(err) + ) { + // Rare (a session id usually resolves to one workspace): two + // workspaces each raised a structured error. We keep the later one + // but log the superseded error so it is not lost silently. + logSessionRoutingFailure( + route, + 'transcript_resolution_error_superseded', + { + sessionId, + supersededError: + loadError instanceof Error ? loadError.name : String(loadError), + newError: err instanceof Error ? err.name : String(err), + }, + ); + } + if ( + loadError === undefined || + shouldPreserveTranscriptResolutionError(err) + ) { + loadError = err; + } + }; - if (workspaceRegistry.listEntries().length === 1) { - const runtime = requirePrimarySessionRuntime(workspaceRegistry, res); - if (!runtime) return undefined; - if (await activeInRuntime(runtime)) { - return runtime; + for (const entry of workspaceRegistry.listAllEntries()) { + const generation = entry.current; + if (!entry.internal || !generation) continue; + if (!assertCurrentInternalGeneration(entry, generation, res)) { + return undefined; } - return throwMissingActiveTranscript(); + const runtime = generation.runtime; + let active: boolean; + try { + active = await activeInRuntime(runtime); + } catch (err) { + if (!assertCurrentInternalGeneration(entry, generation, res)) { + return undefined; + } + recordLoadError(err); + continue; + } + if (!assertCurrentInternalGeneration(entry, generation, res)) { + return undefined; + } + if (!active) continue; + const ordinaryCollisions: WorkspaceRuntime[] = []; + for (const ordinaryRuntime of workspaceRegistry.list()) { + const ordinaryService = + createWorkspaceRuntimeSessionService(ordinaryRuntime); + if (await ordinaryService.sessionExistsInAnyState(sessionId)) { + ordinaryCollisions.push(ordinaryRuntime); + } + } + if (ordinaryCollisions.length > 0) { + sendAmbiguousSessionOwner(res, route, sessionId, [ + runtime, + ...ordinaryCollisions, + ]); + return undefined; + } + if (!assertTrustedSessionOwner(res, route, sessionId, runtime)) { + return undefined; + } + setDaemonTelemetryWorkspace(res, runtime.workspaceCwd); + return runtime; + } + + if (legacyPrimaryFallback) { + const runtime = workspaceRegistry.primary; + if (loadError === undefined) return runtime; + try { + if (await activeInRuntime(runtime)) return runtime; + } catch (err) { + recordLoadError(err); + } + throw loadError; } const liveOwner = workspaceRegistry.resolveLiveSessionOwner(sessionId); + if (liveOwner.kind === 'unavailable') { + sendWorkspaceRuntimeUnavailable(res); + return undefined; + } if (liveOwner.kind === 'ambiguous') { sendAmbiguousSessionOwner(res, route, sessionId, liveOwner.runtimes); return undefined; } if (liveOwner.kind === 'found') { - setDaemonTelemetryWorkspace(res, liveOwner.runtime.workspaceCwd); + const internalEntry = isInternalWorkspaceRuntime(liveOwner.runtime) + ? workspaceRegistry.getManagedEntryByWorkspaceCwd( + liveOwner.runtime.workspaceCwd, + ) + : undefined; + const internalGeneration = internalEntry?.current; + if ( + isInternalWorkspaceRuntime(liveOwner.runtime) && + (!internalEntry?.internal || + !internalGeneration || + internalGeneration.runtime !== liveOwner.runtime) + ) { + sendWorkspaceRuntimeUnavailable(res); + return undefined; + } + if ( + internalEntry && + internalGeneration && + !assertCurrentInternalGeneration(internalEntry, internalGeneration, res) + ) { + return undefined; + } if ( !assertTrustedSessionOwner(res, route, sessionId, liveOwner.runtime) ) { return undefined; } - if (await activeInRuntime(liveOwner.runtime)) { + let active = false; + try { + active = await activeInRuntime(liveOwner.runtime); + } catch (err) { + recordLoadError(err); + } + if ( + internalEntry && + internalGeneration && + !assertCurrentInternalGeneration(internalEntry, internalGeneration, res) + ) { + return undefined; + } + if (active) { + setDaemonTelemetryWorkspace(res, liveOwner.runtime.workspaceCwd); return liveOwner.runtime; } + if (loadError !== undefined) throw loadError; + return throwMissingActiveTranscript(); + } + + if (workspaceRegistry.listEntries().length === 1) { + const runtime = requirePrimarySessionRuntime(workspaceRegistry, res); + if (!runtime) return undefined; + try { + if (await activeInRuntime(runtime)) { + return runtime; + } + } catch (err) { + recordLoadError(err); + } + if (loadError !== undefined) throw loadError; return throwMissingActiveTranscript(); } const activeRuntimes: WorkspaceRuntime[] = []; - let loadError: unknown; for (const runtime of workspaceRegistry.list()) { try { if (await activeInRuntime(runtime)) { activeRuntimes.push(runtime); } } catch (err) { - if ( - loadError === undefined || - shouldPreserveTranscriptResolutionError(err) - ) { - if ( - loadError !== undefined && - shouldPreserveTranscriptResolutionError(loadError) - ) { - // Rare (a session id usually resolves to one workspace): two - // workspaces each raised a structured error. We keep the later one - // but log the superseded error so it is not lost silently. - logSessionRoutingFailure( - route, - 'transcript_resolution_error_superseded', - { - sessionId, - supersededError: - loadError instanceof Error - ? loadError.name - : String(loadError), - newError: err instanceof Error ? err.name : String(err), - }, - ); - } - loadError = err; - } + recordLoadError(err); } } if (activeRuntimes.length === 1) { @@ -1277,40 +1736,395 @@ export function registerSessionRoutes( throw new Error( `Transcript session resolution failed across ${runtimes.length} workspace(s)`, ); - } - return throwMissingActiveTranscript(); + } + return throwMissingActiveTranscript(); + }; + + const resolveSessionAnyStateRuntime = async ( + res: Response, + route: string, + sessionId: string, + ): Promise => { + const owner = workspaceRegistry.resolveLiveSessionOwner(sessionId); + if (owner.kind === 'unavailable') { + sendWorkspaceRuntimeUnavailable(res); + return undefined; + } + if (owner.kind === 'ambiguous') { + sendAmbiguousSessionOwner(res, route, sessionId, owner.runtimes); + return undefined; + } + const matches = new Set(); + if (owner.kind === 'found') matches.add(owner.runtime); + for (const entry of workspaceRegistry.listAllEntries()) { + const generation = entry.current; + if (!entry.internal || !generation) continue; + if (!assertCurrentInternalGeneration(entry, generation, res)) { + return undefined; + } + const runtime = generation.runtime; + const service = createWorkspaceRuntimeSessionService(runtime); + const exists = await service.sessionExistsInAnyState(sessionId); + if (!assertCurrentInternalGeneration(entry, generation, res)) { + return undefined; + } + if (!exists) continue; + const metadata = await readLoadableLiveConversationMetadata( + sessionId, + (candidateId) => service.readCreationMetadata(candidateId), + ); + if (!assertCurrentInternalGeneration(entry, generation, res)) { + return undefined; + } + if (metadata === undefined) continue; + matches.add(runtime); + } + if (owner.kind !== 'found' || isInternalWorkspaceRuntime(owner.runtime)) { + for (const runtime of workspaceRegistry.list()) { + const service = createWorkspaceRuntimeSessionService(runtime); + if (await service.sessionExistsInAnyState(sessionId)) { + matches.add(runtime); + } + } + } + if (matches.size > 1) { + sendAmbiguousSessionOwner(res, route, sessionId, [...matches]); + return undefined; + } + const runtime = [...matches][0]; + if (!runtime) { + res.status(404).json({ + error: `No session with id "${sessionId}"`, + code: 'session_not_found', + sessionId, + }); + return undefined; + } + if (!assertTrustedSessionOwner(res, route, sessionId, runtime)) { + return undefined; + } + setDaemonTelemetryWorkspace(res, runtime.workspaceCwd); + return runtime; + }; + + const resolveSessionBatchRuntime = async ( + req: Request | undefined, + res: Response, + route: string, + sessionIds: readonly string[], + ): Promise => { + if (req) { + return resolveQualifiedSessionRuntime(req, res, route, sessionIds, 'any'); + } + + let internalRuntime: WorkspaceRuntime | undefined; + let hasInternalSession = false; + const hasOrdinarySession = async (sessionId: string): Promise => { + for (const runtime of workspaceRegistry.list()) { + if ( + await createWorkspaceRuntimeSessionService( + runtime, + ).sessionExistsInAnyState(sessionId) + ) { + return true; + } + } + return false; + }; + const sendBatchWorkspaceConflict = (): void => { + res.status(409).json({ + error: 'All sessions in this operation must share one workspace.', + code: 'session_workspace_conflict', + }); + }; + for (const sessionId of sessionIds) { + const candidates = new Set(); + const owner = workspaceRegistry.resolveLiveSessionOwner(sessionId); + if (owner.kind === 'unavailable') { + sendWorkspaceRuntimeUnavailable(res); + return undefined; + } + if (owner.kind === 'found' && isInternalWorkspaceRuntime(owner.runtime)) { + candidates.add(owner.runtime); + } + if (owner.kind === 'ambiguous') { + for (const runtime of owner.runtimes) { + if (isInternalWorkspaceRuntime(runtime)) candidates.add(runtime); + } + } + for (const entry of workspaceRegistry.listAllEntries()) { + const generation = entry.current; + if (!entry.internal || !generation) continue; + if (!assertCurrentInternalGeneration(entry, generation, res)) { + return undefined; + } + const runtime = generation.runtime; + const service = createWorkspaceRuntimeSessionService(runtime); + const exists = await service.sessionExistsInAnyState(sessionId); + if (!assertCurrentInternalGeneration(entry, generation, res)) { + return undefined; + } + if (!exists) continue; + const metadata = await readLoadableLiveConversationMetadata( + sessionId, + (candidateId) => service.readCreationMetadata(candidateId), + ); + if (!assertCurrentInternalGeneration(entry, generation, res)) { + return undefined; + } + if (!metadata) continue; + candidates.add(runtime); + } + if (candidates.size > 0) { + for (const runtime of workspaceRegistry.list()) { + const service = createWorkspaceRuntimeSessionService(runtime); + if (await service.sessionExistsInAnyState(sessionId)) { + candidates.add(runtime); + } + } + } + if (candidates.size === 0) { + if (hasInternalSession) { + if (await hasOrdinarySession(sessionId)) { + sendBatchWorkspaceConflict(); + return undefined; + } + throw new SessionNotFoundError(sessionId); + } + continue; + } + if (candidates.size !== 1) { + sendAmbiguousSessionOwner(res, route, sessionId, [...candidates]); + return undefined; + } + const candidate = [...candidates][0]!; + if (internalRuntime && internalRuntime !== candidate) { + sendBatchWorkspaceConflict(); + return undefined; + } + internalRuntime = candidate; + hasInternalSession = true; + } + if (!internalRuntime) { + const runtime = workspaceRegistry.primary; + setDaemonTelemetryWorkspace(res, runtime.workspaceCwd); + return runtime; + } + + for (const sessionId of sessionIds) { + const service = createWorkspaceRuntimeSessionService(internalRuntime); + if (!(await service.sessionExistsInAnyState(sessionId))) { + if (await hasOrdinarySession(sessionId)) { + sendBatchWorkspaceConflict(); + return undefined; + } + throw new SessionNotFoundError(sessionId); + } + const metadata = await readLoadableLiveConversationMetadata( + sessionId, + (candidateId) => service.readCreationMetadata(candidateId), + ); + if (!metadata) { + throw new SessionNotFoundError(sessionId); + } + } + const internalEntry = workspaceRegistry.getManagedEntryByWorkspaceCwd( + internalRuntime.workspaceCwd, + ); + const generation = internalEntry?.current; + if ( + !internalEntry?.internal || + !generation || + generation.runtime !== internalRuntime || + !assertCurrentInternalGeneration(internalEntry, generation, res) + ) { + if (!res.headersSent) sendWorkspaceRuntimeUnavailable(res); + return undefined; + } + if ( + !assertTrustedSessionOwner( + res, + route, + sessionIds[0] ?? '', + internalRuntime, + ) + ) { + return undefined; + } + setDaemonTelemetryWorkspace(res, internalRuntime.workspaceCwd); + return internalRuntime; + }; + + const parseSessionIdsBody = ( + req: Request, + res: Response, + ): string[] | undefined => { + const body = safeBody(req); + const sessionIds: unknown = body['sessionIds']; + if ( + !Array.isArray(sessionIds) || + sessionIds.length === 0 || + sessionIds.length > 100 || + !sessionIds.every((id) => typeof id === 'string') + ) { + res.status(400).json({ + error: '`sessionIds` must be a non-empty string array (max 100)', + code: 'invalid_request', + }); + return undefined; + } + return [...new Set(sessionIds as string[])]; + }; + + const serializeSessionErrors = ( + errors: Array<{ sessionId: string; error: unknown }>, + redactDetails = false, + ): Array<{ sessionId: string; error: string }> => + errors.map((e) => ({ + sessionId: e.sessionId, + error: redactDetails + ? 'Session operation failed.' + : e.error instanceof Error + ? e.error.message + : String(e.error), + })); + + const runResolvedSessionBatch = async (params: { + req: Request | undefined; + res: Response; + route: string; + sessionIds: string[]; + run: ( + runtime: WorkspaceRuntime, + coordinatorLockHeld: boolean, + ) => Promise; + }): Promise<{ result: T; internal: boolean } | undefined> => { + const { req, res, route, sessionIds, run } = params; + const runtime = await resolveSessionBatchRuntime( + req, + res, + route, + sessionIds, + ); + if (!runtime) return undefined; + if (!isInternalWorkspaceRuntime(runtime)) { + return { result: await run(runtime, false), internal: false }; + } + return archiveCoordinator.runExclusiveMany(sessionIds, async () => { + const verifiedRuntime = await resolveSessionBatchRuntime( + req, + res, + route, + sessionIds, + ); + if (!verifiedRuntime) return undefined; + if (verifiedRuntime !== runtime) { + sendWorkspaceRuntimeUnavailable(res); + return undefined; + } + return { result: await run(verifiedRuntime, true), internal: true }; + }); + }; + + const deleteSessions = ( + req: Request | undefined, + res: Response, + route: string, + sessionIds: string[], + ) => { + const run = async ( + runtime: WorkspaceRuntime, + coordinatorLockHeld: boolean, + ) => { + captureRuntimeGenerationAssertion(runtime)?.(); + const service = createWorkspaceRuntimeSessionService(runtime); + return runWithSessionListInvalidation( + runtime, + ['active', 'archived'], + () => + runWithWorkspaceRuntimeStorage(runtime, () => + deleteDaemonSessions({ + sessionIds, + service, + bridge: runtime.bridge, + coordinator: archiveCoordinator, + coordinatorLockHeld, + onError: ({ phase, sessionId, error }) => { + writeStderrLine( + `qwen serve: ${phase}Session failed for ${safeLogValue(sessionId)}: ${safeLogValue(error)}`, + ); + }, + }), + ), + ); + }; + return runResolvedSessionBatch({ req, res, route, sessionIds, run }); + }; + + const archiveSessions = ( + req: Request | undefined, + res: Response, + route: string, + sessionIds: string[], + ) => { + const run = async ( + runtime: WorkspaceRuntime, + coordinatorLockHeld: boolean, + ) => { + captureRuntimeGenerationAssertion(runtime)?.(); + const service = createWorkspaceRuntimeSessionService(runtime, { + onWarning: logSessionArchiveWarning, + }); + return runWithSessionListInvalidation( + runtime, + ['active', 'archived'], + () => + runWithWorkspaceRuntimeStorage(runtime, () => + archiveDaemonSessions({ + sessionIds, + service, + bridge: runtime.bridge, + coordinator: archiveCoordinator, + coordinatorLockHeld, + }), + ), + ); + }; + return runResolvedSessionBatch({ req, res, route, sessionIds, run }); }; - const parseSessionIdsBody = ( - req: Request, + const unarchiveSessions = ( + req: Request | undefined, res: Response, - ): string[] | undefined => { - const body = safeBody(req); - const sessionIds: unknown = body['sessionIds']; - if ( - !Array.isArray(sessionIds) || - sessionIds.length === 0 || - sessionIds.length > 100 || - !sessionIds.every((id) => typeof id === 'string') - ) { - res.status(400).json({ - error: '`sessionIds` must be a non-empty string array (max 100)', - code: 'invalid_request', + route: string, + sessionIds: string[], + ) => { + const run = async ( + runtime: WorkspaceRuntime, + coordinatorLockHeld: boolean, + ) => { + captureRuntimeGenerationAssertion(runtime)?.(); + const service = createWorkspaceRuntimeSessionService(runtime, { + onWarning: logSessionArchiveWarning, }); - return undefined; - } - return [...new Set(sessionIds as string[])]; + return runWithSessionListInvalidation( + runtime, + ['active', 'archived'], + () => + runWithWorkspaceRuntimeStorage(runtime, () => + unarchiveDaemonSessions({ + sessionIds, + service, + coordinator: archiveCoordinator, + coordinatorLockHeld, + }), + ), + ); + }; + return runResolvedSessionBatch({ req, res, route, sessionIds, run }); }; - const serializeSessionErrors = ( - errors: Array<{ sessionId: string; error: unknown }>, - ): Array<{ sessionId: string; error: string }> => - errors.map((e) => ({ - sessionId: e.sessionId, - error: e.error instanceof Error ? e.error.message : String(e.error), - })); - - const withPrimaryOnlyMutableSession = ( + const withRestrictedMutableSession = ( route: string, handler: ( req: Request, @@ -1319,15 +2133,19 @@ export function registerSessionRoutes( runtime: WorkspaceRuntime, ) => Promise | void, ): RequestHandler => { - if (!isPrimaryOnlyLiveSessionRoute(route)) { - throw new Error(`Unregistered primary-only session route: ${route}`); + const primaryOnly = isPrimaryOnlyLiveSessionRoute(route); + if (!primaryOnly && !isPrimaryOrInternalLiveSessionRoute(route)) { + throw new Error(`Unregistered restricted session route: ${route}`); } return async (req, res) => { const sessionId = requireSessionId(req, res); if (sessionId === null) return; const runtime = resolveLiveSessionRuntime(sessionId, res, route); if (!runtime) return; - if (!runtime.primary) { + if ( + !runtime.primary && + (primaryOnly || !isInternalWorkspaceRuntime(runtime)) + ) { logSessionRoutingFailure( route, 'non_primary_session_route_not_supported', @@ -2111,13 +2929,14 @@ export function registerSessionRoutes( | { runtime: WorkspaceRuntime; workspaceCwd: string } | undefined; try { - resolvedRuntime = resolveRuntimeForSessionRestore( + resolvedRuntime = await resolveRuntimeForSessionRestore( body, res, route, sessionId, ); } catch (err) { + if (sendConversationRuntimeError(res, err)) return; sendBridgeError(res, err, { route, sessionId }); return; } @@ -2134,32 +2953,37 @@ export function registerSessionRoutes( if (liveReplayMode === null) return; const clientId = parseClientIdHeader(req, res); if (clientId === null) return; - let sessionIdReservation: RequestedSessionIdReservation; - try { - sessionIdReservation = requestedSessionIdAdmission.reserveRestore( - sessionId, - { - bridge: runtime.bridge, - workspaceCwd, - workspaceId: runtime.workspaceId, - }, - ); - } catch (error) { - if (error instanceof RequestedSessionIdAdmissionError) { - sendRequestedSessionIdAdmissionError(res, error, route); - return; + let sessionIdReservation: RequestedSessionIdReservation | undefined; + if (!isInternalWorkspaceRuntime(runtime)) { + try { + sessionIdReservation = requestedSessionIdAdmission.reserveRestore( + sessionId, + { + bridge: runtime.bridge, + workspaceCwd, + workspaceId: runtime.workspaceId, + }, + ); + } catch (error) { + if (error instanceof RequestedSessionIdAdmissionError) { + sendRequestedSessionIdAdmissionError(res, error, route); + return; + } + throw error; } - throw error; } try { const session = await archiveCoordinator.runSharedMany( [sessionId], async () => { - await assertSessionLoadable( + const location = await assertSessionLoadable( workspaceCwd, sessionId, runtime.sessionRuntimeBaseDir, ); + if (location === undefined && isInternalWorkspaceRuntime(runtime)) { + throw new SessionNotFoundError(sessionId); + } // Recover the persisted parent lineage so the restored live entry // reports it (the bridge otherwise creates the entry without it, and // status calls would show a restored sub-session as top-level). @@ -2176,6 +3000,18 @@ export function registerSessionRoutes( if (metadata === undefined) { throw new SessionNotFoundError(sessionId); } + assertRuntimeGenerationOpen?.(); + if (isInternalWorkspaceRuntime(runtime)) { + sessionIdReservation = requestedSessionIdAdmission.reserveRestore( + sessionId, + { + bridge: runtime.bridge, + workspaceCwd, + workspaceId: runtime.workspaceId, + }, + ); + setDaemonTelemetryWorkspace(res, runtime.workspaceCwd); + } let liveConversationCwd: string | undefined; if (runtime.provenance === 'live-conversation') { const materialize = deps.materializeLiveConversationDirectory; @@ -2414,12 +3250,16 @@ export function registerSessionRoutes( } res.status(200).json(session); } catch (err) { + if (err instanceof RequestedSessionIdAdmissionError) { + sendRequestedSessionIdAdmissionError(res, err, route); + return; + } sendBridgeError(res, err, { route, sessionId, }); } finally { - sessionIdReservation.release(); + sessionIdReservation?.release(); } }; @@ -2545,7 +3385,7 @@ export function registerSessionRoutes( app.post( '/session/:id/branch', mutate(), - withPrimaryOnlyMutableSession( + withRestrictedMutableSession( 'POST /session/:id/branch', async (req, res, sessionId, runtime) => { const body = safeBody(req); @@ -2613,7 +3453,7 @@ export function registerSessionRoutes( app.post( '/session/:id/side-task', mutate(), - withPrimaryOnlyMutableSession( + withRestrictedMutableSession( 'POST /session/:id/side-task', async (req, res, sessionId, runtime) => { const body = safeBody(req); @@ -2677,7 +3517,7 @@ export function registerSessionRoutes( app.post( '/session/:id/fork', mutate(), - withPrimaryOnlyMutableSession( + withRestrictedMutableSession( 'POST /session/:id/fork', async (req, res, sessionId, runtime) => { const body = safeBody(req); @@ -2714,7 +3554,7 @@ export function registerSessionRoutes( app.post( '/session/:id/cd', mutate(), - withPrimaryOnlyMutableSession( + withRestrictedMutableSession( 'POST /session/:id/cd', async (req, res, sessionId, runtime) => { const body = safeBody(req); @@ -2766,17 +3606,23 @@ export function registerSessionRoutes( app.get('/session/:id/export', async (req, res) => { await handleSessionExport(req, res, { route: 'GET /session/:id/export', - runtime: workspaceRegistry.primary, + resolveRuntime: (sessionId) => + resolveTranscriptSessionRuntime( + res, + 'GET /session/:id/export', + sessionId, + false, + true, + ), }); }); app.get('/workspaces/:workspace/session/:id/export', async (req, res) => { const route = 'GET /workspaces/:workspace/session/:id/export'; - const runtime = requireTrustedRuntimeForWorkspaceRoute(req, res, route); - if (!runtime) return; await handleSessionExport(req, res, { route, - runtime, + resolveRuntime: (sessionId) => + resolveQualifiedSessionRuntime(req, res, route, [sessionId], 'active'), workspaceQualified: true, }); }); @@ -2785,11 +3631,16 @@ export function registerSessionRoutes( '/workspaces/:workspace/session/:id/archive/export', async (req, res) => { const route = 'GET /workspaces/:workspace/session/:id/archive/export'; - const runtime = requireTrustedRuntimeForWorkspaceRoute(req, res, route); - if (!runtime) return; await handleSessionExport(req, res, { route, - runtime, + resolveRuntime: (sessionId) => + resolveQualifiedSessionRuntime( + req, + res, + route, + [sessionId], + 'archived', + ), workspaceQualified: true, archiveState: 'archived', }); @@ -2828,6 +3679,7 @@ export function registerSessionRoutes( cursor !== undefined, ); if (!runtime) return undefined; + captureRuntimeGenerationAssertion(runtime)?.(); return runtime.bridge.getSessionTranscriptPage({ sessionId, ...(limit !== undefined ? { limit } : {}), @@ -2850,20 +3702,12 @@ export function registerSessionRoutes( const route = 'GET /workspaces/:workspace/session/:id/transcript'; const sessionId = requireSessionId(req, res); if (sessionId === null) return; - const runtime = resolveWorkspaceRuntimeFromParam( - workspaceRegistry, - req, - res, - ); - if (!runtime) return; - if (!runtime.trusted && runtime.primary) { - sendUntrustedWorkspaceResponse(res, { - sessionId, - workspaceCwd: runtime.workspaceCwd, - workspaceId: runtime.workspaceId, - }); - return; - } + const qualifiedTarget = resolveQualifiedSessionTarget(req, res, { + allowUntrustedSecondary: true, + }); + if (!qualifiedTarget) return; + const preResolvedRuntime = + qualifiedTarget.kind === 'ordinary' ? qualifiedTarget.runtime : undefined; const limit = parseTranscriptLimitQuery(req.query['limit'], res); if (limit === null) return; const cursor = parseTranscriptCursorQuery(req.query['cursor'], res); @@ -2890,8 +3734,21 @@ export function registerSessionRoutes( try { const result = await runWithoutDebugLogSession(() => - archiveCoordinator.runSharedMany([sessionId], () => - runWithWorkspaceRuntimeStorage(runtime, async () => { + archiveCoordinator.runSharedMany([sessionId], async () => { + const runtime = + preResolvedRuntime ?? + (await resolveQualifiedSessionRuntime( + req, + res, + route, + [sessionId], + 'active', + )); + if (!runtime) return undefined; + const assertRuntimeGenerationOpen = + captureRuntimeGenerationAssertion(runtime); + assertRuntimeGenerationOpen?.(); + return runWithWorkspaceRuntimeStorage(runtime, async () => { const service = createWorkspaceRuntimeSessionService(runtime); if (cursor === undefined) { await assertSessionLoadable( @@ -2950,6 +3807,7 @@ export function registerSessionRoutes( !activePromptBeforeRead && !activePromptAfterRead, encodeCursor: (state) => codec.encode(state), }); + assertRuntimeGenerationOpen?.(); const cursorTooLarge = replay.nextCursor !== undefined && Buffer.byteLength(replay.nextCursor) > @@ -2977,9 +3835,10 @@ export function registerSessionRoutes( } : {}), }; - }), - ), + }); + }), ); + if (result === undefined) return; const serialized = serializeWorkspaceTranscriptResponse( result, sessionId, @@ -3782,26 +4641,19 @@ export function registerSessionRoutes( if (uniqueIds === undefined) return; if (rejectActiveLiveSessionMutation(res, uniqueIds)) return; try { - const runtime = workspaceRegistry.primary; - const service = createWorkspaceRuntimeSessionService(runtime); - const result = await runWithSessionListInvalidation( - runtime, - ['active', 'archived'], - () => - runWithWorkspaceRuntimeStorage(runtime, () => - deleteDaemonSessions({ - sessionIds: uniqueIds, - service, - bridge, - coordinator: archiveCoordinator, - onError: ({ phase, sessionId, error }) => { - writeStderrLine( - `qwen serve: ${phase}Session failed for ${safeLogValue(sessionId)}: ${safeLogValue(error)}`, - ); - }, - }), - ), + const operation = await deleteSessions( + undefined, + res, + 'POST /sessions/delete', + uniqueIds, ); + if (!operation) return; + const result = operation.internal + ? { + ...operation.result, + errors: serializeSessionErrors(operation.result.errors, true), + } + : operation.result; for (const removedId of result.removed) { clearBranchSessionEntry(removedId); } @@ -3816,30 +4668,20 @@ export function registerSessionRoutes( if (uniqueIds === undefined) return; if (rejectActiveLiveSessionMutation(res, uniqueIds)) return; - const runtime = workspaceRegistry.primary; - const service = createWorkspaceRuntimeSessionService(runtime, { - onWarning: logSessionArchiveWarning, - }); - try { - const result = await runWithSessionListInvalidation( - runtime, - ['active', 'archived'], - () => - runWithWorkspaceRuntimeStorage(runtime, () => - archiveDaemonSessions({ - sessionIds: uniqueIds, - service, - bridge, - coordinator: archiveCoordinator, - }), - ), + const operation = await archiveSessions( + undefined, + res, + 'POST /sessions/archive', + uniqueIds, ); + if (!operation) return; + const { result } = operation; res.status(200).json({ archived: result.archived, alreadyArchived: result.alreadyArchived, notFound: result.notFound, - errors: serializeSessionErrors(result.errors), + errors: serializeSessionErrors(result.errors, operation.internal), }); } catch (err) { sendBridgeError(res, err, { route: 'POST /sessions/archive' }); @@ -3850,29 +4692,20 @@ export function registerSessionRoutes( const uniqueIds = parseSessionIdsBody(req, res); if (uniqueIds === undefined) return; - const runtime = workspaceRegistry.primary; - const service = createWorkspaceRuntimeSessionService(runtime, { - onWarning: logSessionArchiveWarning, - }); - try { - const result = await runWithSessionListInvalidation( - runtime, - ['active', 'archived'], - () => - runWithWorkspaceRuntimeStorage(runtime, () => - unarchiveDaemonSessions({ - sessionIds: uniqueIds, - service, - coordinator: archiveCoordinator, - }), - ), + const operation = await unarchiveSessions( + undefined, + res, + 'POST /sessions/unarchive', + uniqueIds, ); + if (!operation) return; + const { result } = operation; res.status(200).json({ unarchived: result.unarchived, alreadyActive: result.alreadyActive, notFound: result.notFound, - errors: serializeSessionErrors(result.errors), + errors: serializeSessionErrors(result.errors, operation.internal), }); } catch (err) { sendBridgeError(res, err, { route: 'POST /sessions/unarchive' }); @@ -3884,33 +4717,20 @@ export function registerSessionRoutes( mutate(), async (req, res) => { const route = 'POST /workspaces/:workspace/sessions/delete'; - const runtime = requireTrustedRuntimeForWorkspaceRoute(req, res, route); - if (!runtime) return; const clientId = parseClientIdHeader(req, res); if (clientId === null) return; const uniqueIds = parseSessionIdsBody(req, res); if (uniqueIds === undefined) return; if (rejectActiveLiveSessionMutation(res, uniqueIds)) return; try { - const service = createWorkspaceRuntimeSessionService(runtime); - const result = await runWithSessionListInvalidation( - runtime, - ['active', 'archived'], - () => - runWithWorkspaceRuntimeStorage(runtime, () => - deleteDaemonSessions({ - sessionIds: uniqueIds, - service, - bridge: runtime.bridge, - coordinator: archiveCoordinator, - onError: ({ phase, sessionId, error }) => { - writeStderrLine( - `qwen serve: ${phase}Session failed for ${safeLogValue(sessionId)}: ${safeLogValue(error)}`, - ); - }, - }), - ), - ); + const operation = await deleteSessions(req, res, route, uniqueIds); + if (!operation) return; + const result = operation.internal + ? { + ...operation.result, + errors: serializeSessionErrors(operation.result.errors, true), + } + : operation.result; for (const removedId of result.removed) { clearBranchSessionEntry(removedId); } @@ -3926,33 +4746,18 @@ export function registerSessionRoutes( mutate(), async (req, res) => { const route = 'POST /workspaces/:workspace/sessions/archive'; - const runtime = requireTrustedRuntimeForWorkspaceRoute(req, res, route); - if (!runtime) return; const uniqueIds = parseSessionIdsBody(req, res); if (uniqueIds === undefined) return; if (rejectActiveLiveSessionMutation(res, uniqueIds)) return; - const service = createWorkspaceRuntimeSessionService(runtime, { - onWarning: logSessionArchiveWarning, - }); try { - const result = await runWithSessionListInvalidation( - runtime, - ['active', 'archived'], - () => - runWithWorkspaceRuntimeStorage(runtime, () => - archiveDaemonSessions({ - sessionIds: uniqueIds, - service, - bridge: runtime.bridge, - coordinator: archiveCoordinator, - }), - ), - ); + const operation = await archiveSessions(req, res, route, uniqueIds); + if (!operation) return; + const { result } = operation; res.status(200).json({ archived: result.archived, alreadyArchived: result.alreadyArchived, notFound: result.notFound, - errors: serializeSessionErrors(result.errors), + errors: serializeSessionErrors(result.errors, operation.internal), }); } catch (err) { sendBridgeError(res, err, { route }); @@ -3965,31 +4770,17 @@ export function registerSessionRoutes( mutate(), async (req, res) => { const route = 'POST /workspaces/:workspace/sessions/unarchive'; - const runtime = requireTrustedRuntimeForWorkspaceRoute(req, res, route); - if (!runtime) return; const uniqueIds = parseSessionIdsBody(req, res); if (uniqueIds === undefined) return; - const service = createWorkspaceRuntimeSessionService(runtime, { - onWarning: logSessionArchiveWarning, - }); try { - const result = await runWithSessionListInvalidation( - runtime, - ['active', 'archived'], - () => - runWithWorkspaceRuntimeStorage(runtime, () => - unarchiveDaemonSessions({ - sessionIds: uniqueIds, - service, - coordinator: archiveCoordinator, - }), - ), - ); + const operation = await unarchiveSessions(req, res, route, uniqueIds); + if (!operation) return; + const { result } = operation; res.status(200).json({ unarchived: result.unarchived, alreadyActive: result.alreadyActive, notFound: result.notFound, - errors: serializeSessionErrors(result.errors), + errors: serializeSessionErrors(result.errors, operation.internal), }); } catch (err) { sendBridgeError(res, err, { route }); @@ -4038,7 +4829,10 @@ export function registerSessionRoutes( ); type SessionOrganizationTarget = { - runtime: WorkspaceRuntime; + runtime?: WorkspaceRuntime; + resolveRuntime?: ( + sessionId: string, + ) => Promise; route: string; }; @@ -4051,98 +4845,113 @@ export function registerSessionRoutes( if (sessionId === null) return; try { await archiveCoordinator.runSharedMany([sessionId], () => - runWithWorkspaceRuntimeStorage(target.runtime, async () => { - // Organization is workspace-scoped sidecar state, not live-session - // metadata. It intentionally applies to persisted and archived sessions. - const sessionService = createWorkspaceRuntimeSessionService( - target.runtime, - ); - let exists = await sessionService.sessionExistsInAnyState(sessionId); - if (!exists) { - try { - const summary = - target.runtime.bridge.getSessionSummary(sessionId); - exists = summary.workspaceCwd === target.runtime.workspaceCwd; - } catch { - exists = false; + (async () => { + const runtime = + target.runtime ?? (await target.resolveRuntime?.(sessionId)); + if (!runtime) return; + const assertRuntimeGenerationOpen = + captureRuntimeGenerationAssertion(runtime); + assertRuntimeGenerationOpen?.(); + return runWithWorkspaceRuntimeStorage(runtime, async () => { + // Organization is workspace-scoped sidecar state, not live-session + // metadata. It intentionally applies to persisted and archived sessions. + const sessionService = + createWorkspaceRuntimeSessionService(runtime); + let exists = + await sessionService.sessionExistsInAnyState(sessionId); + if (!exists) { + try { + const summary = runtime.bridge.getSessionSummary(sessionId); + exists = summary.workspaceCwd === runtime.workspaceCwd; + } catch { + exists = false; + } } - } - if (!exists) { - res.status(404).json({ - error: `No session with id "${sessionId}"`, - sessionId, - }); - return; - } + if (!exists) { + res.status(404).json({ + error: `No session with id "${sessionId}"`, + sessionId, + }); + return; + } + assertRuntimeGenerationOpen?.(); - const body = safeBody(req); - const rawIsPinned = body['isPinned']; - if (rawIsPinned !== undefined && typeof rawIsPinned !== 'boolean') { - res.status(400).json({ - error: '`isPinned` must be a boolean', - code: 'invalid_session_organization', - field: 'isPinned', - }); - return; - } - const rawGroupId = body['groupId']; - if ( - rawGroupId !== undefined && - rawGroupId !== null && - typeof rawGroupId !== 'string' - ) { - res.status(400).json({ - error: '`groupId` must be a string or null', - code: 'invalid_session_organization', - field: 'groupId', - }); - return; - } - const rawColor = body['color']; - if ( - rawColor !== undefined && - rawColor !== null && - (typeof rawColor !== 'string' || - !GROUP_COLOR_OPTIONS.includes( - rawColor as SessionGroupPresetColor, - )) - ) { - res.status(400).json({ - error: '`color` must be a supported color or null', - code: 'invalid_session_organization', - field: 'color', - }); - return; - } + const body = safeBody(req); + const rawIsPinned = body['isPinned']; + if (rawIsPinned !== undefined && typeof rawIsPinned !== 'boolean') { + res.status(400).json({ + error: '`isPinned` must be a boolean', + code: 'invalid_session_organization', + field: 'isPinned', + }); + return; + } + const rawGroupId = body['groupId']; + if ( + rawGroupId !== undefined && + rawGroupId !== null && + typeof rawGroupId !== 'string' + ) { + res.status(400).json({ + error: '`groupId` must be a string or null', + code: 'invalid_session_organization', + field: 'groupId', + }); + return; + } + const rawColor = body['color']; + if ( + rawColor !== undefined && + rawColor !== null && + (typeof rawColor !== 'string' || + !GROUP_COLOR_OPTIONS.includes( + rawColor as SessionGroupPresetColor, + )) + ) { + res.status(400).json({ + error: '`color` must be a supported color or null', + code: 'invalid_session_organization', + field: 'color', + }); + return; + } - const organization = await createSessionOrganizationService( - target.runtime.workspaceCwd, - ).updateSessionOrganization(sessionId, { - ...(rawIsPinned !== undefined ? { isPinned: rawIsPinned } : {}), - ...(rawGroupId !== undefined - ? { groupId: rawGroupId as string | null } - : {}), - ...(rawColor !== undefined - ? { color: rawColor as SessionGroupPresetColor | null } - : {}), + const organization = await createSessionOrganizationService( + runtime.workspaceCwd, + ).updateSessionOrganization(sessionId, { + ...(rawIsPinned !== undefined ? { isPinned: rawIsPinned } : {}), + ...(rawGroupId !== undefined + ? { groupId: rawGroupId as string | null } + : {}), + ...(rawColor !== undefined + ? { color: rawColor as SessionGroupPresetColor | null } + : {}), + }); + res.status(200).json({ sessionId, ...organization }); }); - res.status(200).json({ sessionId, ...organization }); - }), + })(), ); } catch (err) { if (sendSessionOrganizationError(res, err)) return; sendBridgeError(res, err, { route: target.route, sessionId, - workspaceCwd: target.runtime.workspaceCwd, + ...(target.runtime + ? { workspaceCwd: target.runtime.workspaceCwd } + : {}), }); } }; app.patch('/session/:id/organization', mutate(), async (req, res) => { await handleSessionOrganizationUpdate(req, res, { - runtime: workspaceRegistry.primary, route: 'PATCH /session/:id/organization', + resolveRuntime: (sessionId) => + resolveSessionAnyStateRuntime( + res, + 'PATCH /session/:id/organization', + sessionId, + ), }); }); @@ -4151,11 +4960,10 @@ export function registerSessionRoutes( mutate(), async (req, res) => { const route = 'PATCH /workspaces/:workspace/session/:id/organization'; - const runtime = requireTrustedRuntimeForWorkspaceRoute(req, res, route); - if (!runtime) return; await handleSessionOrganizationUpdate(req, res, { - runtime, route, + resolveRuntime: (sessionId) => + resolveQualifiedSessionRuntime(req, res, route, [sessionId], 'any'), }); }, ); @@ -4360,7 +5168,11 @@ export function registerSessionRoutes( // Express decodes URL-encoded path params automatically; clients pass // the absolute workspace cwd encoded (e.g. // GET /workspace/%2Fwork%2Fa/sessions). - const runtime = resolveRuntimeForCatalogRoute(req, res, paramName, route); + const liveRuntime = await resolveLiveCatalogRuntime(req, res, paramName); + if (liveRuntime === null) return; + const runtime = + liveRuntime ?? + resolveRuntimeForCatalogRoute(req, res, paramName, route); if (runtime === null) return; const key = runtime.workspaceCwd; const readOnlySecondary = isReadOnlyWorkspaceInspection(runtime); @@ -4494,19 +5306,31 @@ export function registerSessionRoutes( 'session list live path received persisted-only options', ); } - const result = usePersisted - ? await runWorkspaceInspectionWithLogPolicy(runtime, () => - listWorkspaceSessionsForResponse(runtime.bridge, key, options, { - mergeLive: !readOnlySecondary, - runtimeBaseDir: runtime.sessionRuntimeBaseDir, - signal: controller.signal, - }), - ) - : listLiveWorkspaceSessionsForResponse( - runtime.bridge, - key, - options, - ); + const listSessions = () => + usePersisted + ? runWorkspaceInspectionWithLogPolicy(runtime, () => + listWorkspaceSessionsForResponse( + runtime.bridge, + key, + options, + { + mergeLive: !readOnlySecondary, + runtimeBaseDir: runtime.sessionRuntimeBaseDir, + signal: controller.signal, + }, + ), + ) + : Promise.resolve( + listLiveWorkspaceSessionsForResponse( + runtime.bridge, + key, + options, + ), + ); + const result = + liveRuntime && deps.conversationRuntimeActivity + ? await deps.conversationRuntimeActivity.run(listSessions) + : await listSessions(); controller.signal.throwIfAborted(); if (res.destroyed) return; res.status(200).json({ @@ -4527,6 +5351,17 @@ export function registerSessionRoutes( res.off('close', onResponseClosed); } } catch (err) { + if ( + err && + typeof err === 'object' && + (err as { code?: unknown }).code === 'daemon_draining' + ) { + res.status(503).json({ + error: 'The daemon is draining and no longer accepts work.', + code: 'daemon_draining', + }); + return; + } if (err instanceof InvalidCursorError) { res.status(400).json({ error: err.message, diff --git a/packages/cli/src/serve/routes/workspace-channel-management.test.ts b/packages/cli/src/serve/routes/workspace-channel-management.test.ts index f6fc2fede69..0b6a044fb24 100644 --- a/packages/cli/src/serve/routes/workspace-channel-management.test.ts +++ b/packages/cli/src/serve/routes/workspace-channel-management.test.ts @@ -304,6 +304,37 @@ describe('workspace Channel management routes', () => { expect(liveService.upsert).not.toHaveBeenCalled(); }); + it('fails closed before internal channel reads when the activity gate is absent', async () => { + const primary = runtime('primary', '/work/primary'); + const live = runtime( + 'conversations', + '/work/Conversations', + true, + 'live-conversation', + ); + const liveService = service(); + const resolveService = vi.fn(() => liveService); + const app = express(); + app.use(express.json()); + registerWorkspaceChannelManagementRoutes(app, { + primaryRuntime: primary, + workspaceRegistry: createWorkspaceRegistry([primary, live]), + resolveService, + mutate: () => (_req, _res, next) => next(), + safeBody: (req) => (req.body ?? {}) as Record, + parseAndValidateClientId: () => undefined, + }); + + const response = await request(app).get( + '/workspaces/conversations/channels', + ); + + expect(response.status).toBe(503); + expect(response.body.code).toBe('conversation_runtime_unavailable'); + expect(resolveService).not.toHaveBeenCalled(); + expect(liveService.list).not.toHaveBeenCalled(); + }); + it('fails closed for an untrusted secondary workspace', async () => { const { app, primaryService, secondaryService } = mount(false); diff --git a/packages/cli/src/serve/routes/workspace-channel-management.ts b/packages/cli/src/serve/routes/workspace-channel-management.ts index 425f00b148c..27cb866c0d0 100644 --- a/packages/cli/src/serve/routes/workspace-channel-management.ts +++ b/packages/cli/src/serve/routes/workspace-channel-management.ts @@ -18,8 +18,10 @@ import type { import { assertValidChannelSecretUpdates } from '../channel-settings-store.js'; import { requireTrustedWorkspaceRuntime, - resolveWorkspaceRuntimeFromParam, + resolveWorkspaceRuntimeWithLiveCompatibilityFromParam, + sendConversationRuntimeUnavailable, } from '../workspace-route-runtime.js'; +import type { ConversationRuntimeActivityGate } from '../conversations/conversation-runtime-activity.js'; import type { WorkspaceRegistry, WorkspaceRuntime, @@ -45,6 +47,7 @@ interface RegisterWorkspaceChannelManagementRoutesDeps { res: Response, runtime: WorkspaceRuntime, ) => string | undefined | null; + conversationRuntimeActivity?: ConversationRuntimeActivityGate; } type RuntimeResolver = (req: Request, res: Response) => WorkspaceRuntime | null; @@ -282,8 +285,14 @@ async function resolveTarget( res: Response, resolveRuntime: RuntimeResolver, resolveService: RegisterWorkspaceChannelManagementRoutesDeps['resolveService'], + activity: ConversationRuntimeActivityGate | undefined, ): Promise< - { runtime: WorkspaceRuntime; service: ChannelManagementService } | undefined + | { + runtime: WorkspaceRuntime; + service: ChannelManagementService; + run: (operation: () => Promise) => Promise; + } + | undefined > { const runtime = resolveRuntime(req, res); if (!runtime || !requireTrustedWorkspaceRuntime(runtime, res)) return; @@ -295,8 +304,16 @@ async function resolveTarget( }); return; } + if (runtime.provenance === 'live-conversation' && !activity) { + sendConversationRuntimeUnavailable(res); + return; + } try { - const service = await resolveService(runtime); + const run = (operation: () => Promise): Promise => + runtime.provenance === 'live-conversation' + ? activity!.run(operation) + : operation(); + const service = await run(async () => resolveService(runtime)); if (!service) { res.status(503).json({ error: 'Channel management is unavailable.', @@ -304,7 +321,7 @@ async function resolveTarget( }); return; } - return { runtime, service }; + return { runtime, service, run }; } catch (error) { sendManagementError(res, error); return; @@ -317,7 +334,11 @@ export function registerWorkspaceChannelManagementRoutes( ): void { const primary: RuntimeResolver = () => deps.primaryRuntime; const qualified: RuntimeResolver = (req, res) => - resolveWorkspaceRuntimeFromParam(deps.workspaceRegistry, req, res); + resolveWorkspaceRuntimeWithLiveCompatibilityFromParam( + deps.workspaceRegistry, + req, + res, + ); const register = (prefix: string, resolveRuntime: RuntimeResolver) => { const pairingRead = deps.mutate({ strict: true }); @@ -332,7 +353,13 @@ export function registerWorkspaceChannelManagementRoutes( const restart = deps.mutate({ strict: true }); const target = (req: Request, res: Response) => - resolveTarget(req, res, resolveRuntime, deps.resolveService); + resolveTarget( + req, + res, + resolveRuntime, + deps.resolveService, + deps.conversationRuntimeActivity, + ); const validateClient = ( req: Request, res: Response, @@ -342,9 +369,23 @@ export function registerWorkspaceChannelManagementRoutes( app.get(`${prefix}/channel-types`, async (req, res) => { const runtime = resolveRuntime(req, res); if (!runtime || !requireTrustedWorkspaceRuntime(runtime, res)) return; + if ( + runtime.provenance === 'live-conversation' && + !deps.conversationRuntimeActivity + ) { + sendConversationRuntimeUnavailable(res); + return; + } try { noStore(res); - res.status(200).json(await supportedChannelCatalog()); + const catalog = + runtime.provenance === 'live-conversation' && + deps.conversationRuntimeActivity + ? await deps.conversationRuntimeActivity.run(() => + supportedChannelCatalog(), + ) + : await supportedChannelCatalog(); + res.status(200).json(catalog); } catch (error) { sendManagementError(res, error); } @@ -355,7 +396,7 @@ export function registerWorkspaceChannelManagementRoutes( if (!resolved || !validateClient(req, res, resolved.runtime)) return; try { noStore(res); - res.status(200).json(await resolved.service.list()); + res.status(200).json(await resolved.run(() => resolved.service.list())); } catch (error) { sendManagementError(res, error); } @@ -371,7 +412,11 @@ export function registerWorkspaceChannelManagementRoutes( if (!name) return; try { noStore(res); - res.status(200).json(await resolved.service.pairingRequests(name)); + res + .status(200) + .json( + await resolved.run(() => resolved.service.pairingRequests(name)), + ); } catch (error) { sendManagementError(res, error); } @@ -392,7 +437,11 @@ export function registerWorkspaceChannelManagementRoutes( noStore(res); res .status(200) - .json(await resolved.service.approvePairing(name, code)); + .json( + await resolved.run(() => + resolved.service.approvePairing(name, code), + ), + ); } catch (error) { sendManagementError(res, error); } @@ -409,7 +458,11 @@ export function registerWorkspaceChannelManagementRoutes( if (!name) return; try { noStore(res); - res.status(200).json(await resolved.service.pairingApprovals(name)); + res + .status(200) + .json( + await resolved.run(() => resolved.service.pairingApprovals(name)), + ); } catch (error) { sendManagementError(res, error); } @@ -430,7 +483,11 @@ export function registerWorkspaceChannelManagementRoutes( noStore(res); res .status(200) - .json(await resolved.service.revokePairingApproval(name, subject)); + .json( + await resolved.run(() => + resolved.service.revokePairingApproval(name, subject), + ), + ); } catch (error) { sendManagementError(res, error); } @@ -446,7 +503,11 @@ export function registerWorkspaceChannelManagementRoutes( if (!request) return; try { noStore(res); - res.status(200).json(await resolved.service.upsert(name, request)); + res + .status(200) + .json( + await resolved.run(() => resolved.service.upsert(name, request)), + ); } catch (error) { sendManagementError(res, error); } @@ -461,7 +522,11 @@ export function registerWorkspaceChannelManagementRoutes( if (!request) return; try { noStore(res); - res.status(200).json(await resolved.service.remove(name, request)); + res + .status(200) + .json( + await resolved.run(() => resolved.service.remove(name, request)), + ); } catch (error) { sendManagementError(res, error); } @@ -476,7 +541,13 @@ export function registerWorkspaceChannelManagementRoutes( if (!request) return; try { noStore(res); - res.status(200).json(await resolved.service.setStartup(name, request)); + res + .status(200) + .json( + await resolved.run(() => + resolved.service.setStartup(name, request), + ), + ); } catch (error) { sendManagementError(res, error); } @@ -496,7 +567,11 @@ export function registerWorkspaceChannelManagementRoutes( if (!name) return; try { noStore(res); - res.status(200).json(await resolved.service[operation](name)); + res + .status(200) + .json( + await resolved.run(() => resolved.service[operation](name)), + ); } catch (error) { sendManagementError(res, error); } diff --git a/packages/cli/src/serve/routes/workspace-channel-observed-contacts.test.ts b/packages/cli/src/serve/routes/workspace-channel-observed-contacts.test.ts index 59b0ccf6145..c2f83a4f466 100644 --- a/packages/cli/src/serve/routes/workspace-channel-observed-contacts.test.ts +++ b/packages/cli/src/serve/routes/workspace-channel-observed-contacts.test.ts @@ -160,6 +160,26 @@ describe('workspace observed channel contact routes', () => { expect(empty.body).toEqual({ users: [], groups: [] }); }); + it('fails closed before internal contact reads when the activity gate is absent', async () => { + const primary = runtime('primary', '/work/main'); + const live = { + ...runtime('conversations', '/work/Conversations'), + provenance: 'live-conversation' as const, + }; + const app = express(); + registerWorkspaceChannelObservedContactRoutes(app, { + primaryWorkspace: primary.workspaceCwd, + workspaceRegistry: registry([primary, live]), + }); + + const response = await request(app).get( + '/workspaces/conversations/channel/observed-contacts', + ); + + expect(response.status).toBe(503); + expect(response.body.code).toBe('conversation_runtime_unavailable'); + }); + it('rejects legacy reads when the live primary workspace is untrusted', async () => { const primary = runtime('primary', '/work/main'); const app = express(); diff --git a/packages/cli/src/serve/routes/workspace-channel-observed-contacts.ts b/packages/cli/src/serve/routes/workspace-channel-observed-contacts.ts index 64c64baf350..98fda06eef0 100644 --- a/packages/cli/src/serve/routes/workspace-channel-observed-contacts.ts +++ b/packages/cli/src/serve/routes/workspace-channel-observed-contacts.ts @@ -12,10 +12,12 @@ import { } from '../../commands/channel/observed-contact-store.js'; import { requireTrustedWorkspaceRuntime, - resolveWorkspaceRuntimeFromParam, + resolveWorkspaceRuntimeWithLiveCompatibilityFromParam, + sendConversationRuntimeUnavailable, sendGenerationClosedError, sendUntrustedWorkspaceResponse, } from '../workspace-route-runtime.js'; +import type { ConversationRuntimeActivityGate } from '../conversations/conversation-runtime-activity.js'; import type { WorkspaceRegistry } from '../workspace-registry.js'; interface RegisterWorkspaceChannelObservedContactRoutesDeps { @@ -23,6 +25,7 @@ interface RegisterWorkspaceChannelObservedContactRoutesDeps { workspaceRegistry: WorkspaceRegistry; isWorkspaceTrusted?: () => boolean; captureGenerationAssertion?: () => (() => void) | undefined; + conversationRuntimeActivity?: ConversationRuntimeActivityGate; } const DEFAULT_FRESH_WITHIN_SECONDS = 7 * 24 * 60 * 60; @@ -95,15 +98,49 @@ export function registerWorkspaceChannelObservedContactRoutes( ); }); - app.get('/workspaces/:workspace/channel/observed-contacts', (req, res) => { - const runtime = resolveWorkspaceRuntimeFromParam( - deps.workspaceRegistry, - req, - res, - ); - if (!runtime || !requireTrustedWorkspaceRuntime(runtime, res)) return; - sendContacts(req, res, runtime.workspaceCwd, () => - runtime.generationGuard?.assertOpen(), - ); - }); + app.get( + '/workspaces/:workspace/channel/observed-contacts', + async (req, res) => { + const runtime = resolveWorkspaceRuntimeWithLiveCompatibilityFromParam( + deps.workspaceRegistry, + req, + res, + ); + if (!runtime || !requireTrustedWorkspaceRuntime(runtime, res)) return; + if ( + runtime.provenance === 'live-conversation' && + !deps.conversationRuntimeActivity + ) { + sendConversationRuntimeUnavailable(res); + return; + } + const send = async () => + sendContacts(req, res, runtime.workspaceCwd, () => + runtime.generationGuard?.assertOpen(), + ); + if ( + runtime.provenance === 'live-conversation' && + deps.conversationRuntimeActivity + ) { + try { + await deps.conversationRuntimeActivity.run(send); + } catch (error) { + if ( + error && + typeof error === 'object' && + (error as { code?: unknown }).code === 'daemon_draining' + ) { + res.status(503).json({ + error: 'The daemon is draining and no longer accepts work.', + code: 'daemon_draining', + }); + return; + } + throw error; + } + return; + } + await send(); + }, + ); } diff --git a/packages/cli/src/serve/routes/workspace-extensions.ts b/packages/cli/src/serve/routes/workspace-extensions.ts index b9d2954e7c4..349327b8fea 100644 --- a/packages/cli/src/serve/routes/workspace-extensions.ts +++ b/packages/cli/src/serve/routes/workspace-extensions.ts @@ -41,6 +41,7 @@ import type { WorkspaceRegistry, WorkspaceRuntime, } from '../workspace-registry.js'; +import type { ConversationRuntimeActivityGate } from '../conversations/conversation-runtime-activity.js'; import type { DaemonWorkspaceService } from '../workspace-service/index.js'; import { createExtensionsController, @@ -233,6 +234,7 @@ interface RegisterWorkspaceExtensionRoutesDeps { captureGenerationAssertion?: () => (() => void) | undefined; // Enables V2 workspace projection and targeted reconciliation routes. workspaceRegistry?: WorkspaceRegistry; + conversationRuntimeActivity?: ConversationRuntimeActivityGate; } /** @@ -435,7 +437,11 @@ export function registerWorkspaceExtensionRoutes( run: async (run: () => Promise): Promise => { if (used) throw new Error('Runtime reconciliation already released'); used = true; - provideTask(run); + provideTask( + deps.conversationRuntimeActivity + ? () => deps.conversationRuntimeActivity!.run(run) + : run, + ); return (await queued) as T; }, release: () => { @@ -455,7 +461,7 @@ export function registerWorkspaceExtensionRoutes( const globalReconciliationOptions = () => workspaceRegistry ? { - refreshRuntimes: () => workspaceRegistry.list(), + refreshRuntimes: () => workspaceRegistry.listAll(), reserveRuntimeReconciliation, onRuntimeReconciled, } @@ -475,7 +481,7 @@ export function registerWorkspaceExtensionRoutes( ): readonly AcpSessionBridge[] => (typeof runtimes === 'function' ? runtimes() - : (runtimes ?? workspaceRegistry?.list()) + : (runtimes ?? workspaceRegistry?.listAll()) )?.map((runtime) => runtime.bridge) ?? [bridge]; if (workspaceRegistry) { @@ -492,7 +498,7 @@ export function registerWorkspaceExtensionRoutes( const generation = (await manager.getExtensionStoreSnapshot()) .generation; const pendingRuntimes = workspaceRegistry - .list() + .listAll() .filter( (runtime) => (appliedGenerationByWorkspaceId.get(runtime.workspaceId) ?? 0) !== @@ -502,25 +508,29 @@ export function registerWorkspaceExtensionRoutes( return; const runtimes = pendingRuntimes; if (runtimes.length === 0) return; - const results = await runtimeReconciliationQueue.run( - async () => - await Promise.allSettled( - runtimes.map(async (runtime) => { - runtime.workspaceService.invalidateWorkspaceSkillsStatus(); - try { - const result = - await runtime.bridge.refreshExtensionsForAllSessions(); - if (result.failed > 0) { - throw new Error( - `${result.failed} extension session refresh(es) failed`, - ); - } - } finally { + const reconcile = () => + runtimeReconciliationQueue.run( + async () => + await Promise.allSettled( + runtimes.map(async (runtime) => { runtime.workspaceService.invalidateWorkspaceSkillsStatus(); - } - }), - ), - ); + try { + const result = + await runtime.bridge.refreshExtensionsForAllSessions(); + if (result.failed > 0) { + throw new Error( + `${result.failed} extension session refresh(es) failed`, + ); + } + } finally { + runtime.workspaceService.invalidateWorkspaceSkillsStatus(); + } + }), + ), + ); + const results = deps.conversationRuntimeActivity + ? await deps.conversationRuntimeActivity.run(reconcile) + : await reconcile(); results.forEach((result, index) => { if (result.status === 'fulfilled') { const workspaceId = runtimes[index]!.workspaceId; @@ -1620,7 +1630,7 @@ export function registerWorkspaceExtensionRoutes( }, { ...(workspaceRegistry - ? { refreshRuntimes: () => workspaceRegistry.list() } + ? { refreshRuntimes: () => workspaceRegistry.listAll() } : {}), }, ); @@ -1780,7 +1790,7 @@ export function registerWorkspaceExtensionRoutes( { deadlineMs: EXTENSION_PREPARE_DEADLINE_MS, ...(workspaceRegistry - ? { refreshRuntimes: () => workspaceRegistry.list() } + ? { refreshRuntimes: () => workspaceRegistry.listAll() } : {}), }, ); @@ -1889,7 +1899,7 @@ export function registerWorkspaceExtensionRoutes( { deadlineMs: EXTENSION_PREPARE_DEADLINE_MS, ...(workspaceRegistry - ? { refreshRuntimes: () => workspaceRegistry.list() } + ? { refreshRuntimes: () => workspaceRegistry.listAll() } : {}), }, ); @@ -1943,7 +1953,7 @@ export function registerWorkspaceExtensionRoutes( }, { ...(workspaceRegistry - ? { refreshRuntimes: () => workspaceRegistry.list() } + ? { refreshRuntimes: () => workspaceRegistry.listAll() } : {}), }, ); diff --git a/packages/cli/src/serve/routes/workspace-management.test.ts b/packages/cli/src/serve/routes/workspace-management.test.ts index 373dd776ba9..913363d0b59 100644 --- a/packages/cli/src/serve/routes/workspace-management.test.ts +++ b/packages/cli/src/serve/routes/workspace-management.test.ts @@ -425,6 +425,41 @@ describe('POST /workspaces', () => { vi.clearAllMocks(); }); + it('reserves the Conversations root and its children but allows its parent', async () => { + const parent = await mkdtemp(join(REAL_DIR, 'qws-conversations-reserved-')); + const reserved = join(parent, 'conversations'); + const child = join(reserved, 'child'); + const missingChild = join(reserved, 'missing'); + const alias = join(parent, 'conversation-alias'); + await mkdir(child, { recursive: true }); + await symlink(reserved, alias, 'dir'); + try { + const { app } = createApp({ + workspaceRegistry: createMockRegistry([ + makeRuntime('/unrelated-primary', { primary: true }), + ]), + reservedWorkspaceRoots: [reserved], + runtimeRemoval: createRemovalController(), + }); + + for (const cwd of [reserved, child, missingChild, alias]) { + const response = await request(app) + .post('/workspaces') + .send({ cwd, persist: false }); + expect(response.status).toBe(409); + expect(response.body).toMatchObject({ + code: 'conversation_workspace_reserved', + }); + } + + await expect( + request(app).post('/workspaces').send({ cwd: parent, persist: false }), + ).resolves.toMatchObject({ status: 201 }); + } finally { + await rm(parent, { recursive: true, force: true }); + } + }); + it('allows scratch creation but protects existing paths in loopback development', async () => { const parent = await mkdtemp(join(REAL_DIR, 'qws-scratch-route-')); try { @@ -462,6 +497,33 @@ describe('POST /workspaces', () => { } }); + it('does not let the internal Conversations runtime block scratch creation', async () => { + const parent = await mkdtemp(join(REAL_DIR, 'qws-scratch-live-route-')); + try { + const root = prepareManagedScratchRoot(join(parent, 'root'), []); + const internal = makeRuntime(root.canonicalRoot, { + provenance: 'live-conversation', + }); + const { app } = createApp({ + workspaceRegistry: createMockRegistry([ + makeRuntime('/workspace', { primary: true }), + internal, + ]), + managedScratchRoot: root, + runtimeRemoval: createRemovalController(), + }); + + const response = await request(app) + .post('/workspaces') + .send({ kind: 'scratch' }); + + expect(response.status).toBe(201); + expect(response.body.cwd).toMatch(/scratch-/u); + } finally { + await rm(parent, { recursive: true, force: true }); + } + }); + it('returns 501 when createWorkspaceRuntime is not provided', async () => { const { app } = createApp({ createWorkspaceRuntime: undefined }); const res = await request(app) @@ -1569,6 +1631,24 @@ describe('PATCH /workspaces/:workspace', () => { expect(setDisplayNameByIds).not.toHaveBeenCalled(); }); + it('does not expose the internal Conversations runtime by id', async () => { + const runtime = makeRuntime(REAL_DIR, { + provenance: 'live-conversation', + displayName: 'Live', + }); + const { app } = createApp({ + workspaceRegistry: createMockRegistry([runtime]), + }); + + const res = await request(app) + .patch(`/workspaces/${encodeURIComponent(runtime.workspaceId)}`) + .send({ displayName: 'Renamed' }); + + expect(res.status).toBe(400); + expect(res.body.code).toBe('workspace_mismatch'); + expect(runtime.displayName).toBe('Live'); + }); + it('clears a workspace display name by cwd', async () => { const runtime = makeRuntime(REAL_DIR, { displayName: 'Payments' }); const { app } = createApp({ @@ -1790,6 +1870,25 @@ describe('DELETE /workspaces/:workspace', () => { expect(res.body).not.toHaveProperty('workspaceCount'); }); + it('does not expose the internal Conversations runtime by id', async () => { + const runtime = makeRuntime(REAL_DIR, { + provenance: 'live-conversation', + }); + const runtimeRemoval = createRemovalController(); + const { app } = createApp({ + workspaceRegistry: createMockRegistry([runtime]), + runtimeRemoval, + }); + + const res = await request(app).delete( + `/workspaces/${encodeURIComponent(runtime.workspaceId)}`, + ); + + expect(res.status).toBe(400); + expect(res.body.code).toBe('workspace_mismatch'); + expect(runtimeRemoval.beginDrain).not.toHaveBeenCalled(); + }); + it('returns the fast busy snapshot without disturbing runtime gates', async () => { const runtime = makeRuntime(REAL_DIR); Object.assign(runtime.bridge, { sessionCount: 1, activePromptCount: 1 }); @@ -2382,6 +2481,56 @@ describe('persistent workspace registrations', () => { ]); }); + it('keeps legacy Conversations registrations inactive and only forgets the stored record', async () => { + const reserved = '/reserved/qwen-code/conversations'; + const registrationId = workspaceRegistrationId(reserved); + const internal = makeRuntime(reserved, { + provenance: 'live-conversation', + removable: false, + registrationIds: [registrationId], + }); + const registry = createMockRegistry([internal]); + const removeById = vi.fn().mockResolvedValue(true); + const store = { + read: vi.fn().mockResolvedValue({ + schemaVersion: 1, + primaryWorkspace: '/primary', + workspaces: [reserved], + }), + removeById, + } as unknown as WorkspaceRegistrationStore; + const { app } = createApp({ + workspaceRegistry: registry, + workspaceRegistrationStore: store, + reservedWorkspaceRoots: [reserved], + }); + + const listed = await request(app).get('/workspace-registrations'); + expect(listed.status).toBe(200); + expect(listed.body.entries).toEqual([ + expect.objectContaining({ + id: registrationId, + cwd: reserved, + active: false, + persisted: true, + }), + ]); + + const removed = await request(app).delete( + `/workspace-registrations/${registrationId}`, + ); + expect(removed.status).toBe(200); + expect(removed.body).toEqual({ + removed: true, + active: false, + restartRequired: false, + }); + expect(removeById).toHaveBeenCalledWith(registrationId); + expect(internal.registrationIds).toEqual([registrationId]); + expect(registry.listManaged()).toContain(internal); + expect(registry.syncRuntimeMetadata).not.toHaveBeenCalled(); + }); + it('forgets persistence without unloading an active runtime', async () => { const aliasId = workspaceRegistrationId('/raw/symlink-alias'); const active = makeRuntime(REAL_DIR, { registrationIds: [aliasId] }); diff --git a/packages/cli/src/serve/routes/workspace-management.ts b/packages/cli/src/serve/routes/workspace-management.ts index 2d900dc43c9..8d3aba38288 100644 --- a/packages/cli/src/serve/routes/workspace-management.ts +++ b/packages/cli/src/serve/routes/workspace-management.ts @@ -19,6 +19,7 @@ import type { WorkspaceRegistry, WorkspaceRuntime, } from '../workspace-registry.js'; +import { isInternalWorkspaceRuntime } from '../workspace-runtime-visibility.js'; import type { AcpHttpHandle } from '../acp-http/index.js'; import { isPortableAbsolutePath, @@ -73,6 +74,7 @@ export interface WorkspaceManagementRouteDeps { pickWorkspaceDirectory?: ( signal?: AbortSignal, ) => Promise; + reservedWorkspaceRoots?: readonly string[]; } export interface WorkspaceRemovalActivity { @@ -128,9 +130,32 @@ export function registerWorkspaceManagementRoutes( getAcpHandle, runtimeRemoval, pickWorkspaceDirectory: pickWorkspaceDirectoryOverride, + reservedWorkspaceRoots = [], } = deps; const pickWorkspaceDirectory = pickWorkspaceDirectoryOverride ?? pickNativeDirectory; + const canonicalizeIfPresent = (candidate: string): string => { + const resolved = resolve(candidate); + try { + return realpathSync.native(resolved); + } catch { + return resolved; + } + }; + const isReservedWorkspacePath = (candidate: string): boolean => { + const resolvedCandidate = resolve(candidate); + const canonicalCandidate = canonicalizeIfPresent(candidate); + return reservedWorkspaceRoots.some((configuredRoot) => { + const resolvedRoot = resolve(configuredRoot); + const canonicalRoot = canonicalizeIfPresent(configuredRoot); + return ( + resolvedCandidate === resolvedRoot || + isWithinRoot(resolvedCandidate, resolvedRoot) || + canonicalCandidate === canonicalRoot || + isWithinRoot(canonicalCandidate, canonicalRoot) + ); + }); + }; // Serialize runtime addition, persistence promotion/forget, updates, and // removal by canonical cwd so conflicting management mutations cannot cross // their validation and persistence commit points concurrently. @@ -345,6 +370,7 @@ export function registerWorkspaceManagementRoutes( .listManaged() .some( (runtime) => + !isInternalWorkspaceRuntime(runtime) && !isScratchRootCompatible( runtime.workspaceCwd, managedScratchRoot.canonicalRoot, @@ -391,6 +417,7 @@ export function registerWorkspaceManagementRoutes( const boundCwds = workspaceRegistry .listManaged() + .filter((entry) => !isInternalWorkspaceRuntime(entry)) .map((entry) => entry.workspaceCwd); for (const [cwd, operation] of inFlight) { if (operation === 'addition' && cwd !== canonical) boundCwds.push(cwd); @@ -707,6 +734,14 @@ export function registerWorkspaceManagementRoutes( return; } + if (isReservedWorkspacePath(sandboxCwd)) { + res.status(409).json({ + error: 'Workspace path is reserved for Conversations.', + code: 'conversation_workspace_reserved', + }); + return; + } + // Canonicalize with the OS-native syscall, the same call startup // registration uses (canonicalizeWorkspace -> realpathSync.native). The // POSIX JS realpath() can differ on case-insensitive filesystems @@ -723,6 +758,14 @@ export function registerWorkspaceManagementRoutes( return; } + if (isReservedWorkspacePath(canonical)) { + res.status(409).json({ + error: 'Workspace path is reserved for Conversations.', + code: 'conversation_workspace_reserved', + }); + return; + } + if ( managedScratchRoot && !isScratchRootCompatible(canonical, managedScratchRoot.canonicalRoot) @@ -1143,7 +1186,7 @@ export function registerWorkspaceManagementRoutes( ): WorkspaceRuntime | undefined => { const selector = String(req.params['workspace'] ?? ''); const byId = workspaceRegistry.getManagedByWorkspaceId(selector); - if (byId) return byId; + if (byId && !isInternalWorkspaceRuntime(byId)) return byId; if (!isPortableAbsolutePath(selector)) { res.status(400).json({ error: '`workspace` must decode to a workspace id or absolute path', @@ -1571,7 +1614,10 @@ export function registerWorkspaceManagementRoutes( primaryWorkspace: snapshot.primaryWorkspace, entries: snapshot.workspaces.map((cwd) => { const registrationId = workspaceRegistrationId(cwd); - const runtime = workspaceRegistry.getByWorkspaceCwd(cwd); + const reserved = isReservedWorkspacePath(cwd); + const runtime = reserved + ? undefined + : workspaceRegistry.getByWorkspaceCwd(cwd); return { id: registrationId, cwd, @@ -1579,7 +1625,8 @@ export function registerWorkspaceManagementRoutes( ? { displayName: snapshot.displayNames[registrationId] } : {}), active: - runtime !== undefined || registrationIsActive(registrationId), + !reserved && + (runtime !== undefined || registrationIsActive(registrationId)), persisted: true, }; }), @@ -1625,7 +1672,11 @@ export function registerWorkspaceManagementRoutes( registrationId || candidate.registrationIds?.includes(registrationId) === true, ); + if (runtime && isInternalWorkspaceRuntime(runtime)) { + runtime = undefined; + } operationCwd = runtime?.workspaceCwd; + let reservedRegistration = false; if (!operationCwd) { let storedCwd: string | undefined; try { @@ -1647,6 +1698,7 @@ export function registerWorkspaceManagementRoutes( return; } if (storedCwd) { + reservedRegistration = isReservedWorkspacePath(storedCwd); try { operationCwd = realpathSync.native(resolve(storedCwd)); } catch { @@ -1673,10 +1725,14 @@ export function registerWorkspaceManagementRoutes( ownsInFlight = true; } runtime = - (operationCwd + (!reservedRegistration && operationCwd ? workspaceRegistry.getManagedByWorkspaceCwd(operationCwd) : undefined) ?? runtime; - const active = registrationIsActive(registrationId); + if (runtime && isInternalWorkspaceRuntime(runtime)) { + runtime = undefined; + } + const active = + !reservedRegistration && registrationIsActive(registrationId); let removed: boolean; try { removed = await workspaceRegistrationStore.removeById(registrationId); diff --git a/packages/cli/src/serve/routes/workspace-qualified-extensions.test.ts b/packages/cli/src/serve/routes/workspace-qualified-extensions.test.ts index c9d78232db8..387dc3210ed 100644 --- a/packages/cli/src/serve/routes/workspace-qualified-extensions.test.ts +++ b/packages/cli/src/serve/routes/workspace-qualified-extensions.test.ts @@ -27,6 +27,7 @@ import { type WorkspaceRuntime, } from '../workspace-registry.js'; import type { AcpSessionBridge } from '../acp-session-bridge.js'; +import { ConversationWorkspace } from '../conversations/conversation-workspace.js'; import type { DaemonWorkspaceService } from '../workspace-service/types.js'; const extensionId = 'a'.repeat(64); @@ -92,7 +93,13 @@ function makeWorkspaceService(): DaemonWorkspaceService { function makeRuntime( workspaceCwd: string, - opts: { primary: boolean; trusted: boolean; workspaceId: string }, + opts: { + primary: boolean; + trusted: boolean; + workspaceId: string; + provenance?: 'live-conversation'; + removable?: boolean; + }, ): WorkspaceRuntime { return { workspaceId: opts.workspaceId, @@ -100,6 +107,8 @@ function makeRuntime( sessionRuntimeBaseDir: path.join(workspaceCwd, '.runtime'), primary: opts.primary, trusted: opts.trusted, + ...(opts.provenance ? { provenance: opts.provenance } : {}), + ...(opts.removable === undefined ? {} : { removable: opts.removable }), env: { mode: 'parent-process', overlayKeys: [] }, bridge: makeBridge(), workspaceService: makeWorkspaceService(), @@ -113,6 +122,7 @@ function makeRuntime( } async function makeHarness(opts?: { + internalRuntime?: boolean; secondaryTrusted?: boolean; singleWorkspace?: boolean; }) { @@ -121,6 +131,9 @@ async function makeHarness(opts?: { ); const primaryCwd = path.join(scratch, 'primary'); const secondaryCwd = path.join(scratch, 'secondary'); + const conversationWorkspace = opts?.internalRuntime + ? new ConversationWorkspace({ homeDir: scratch }) + : undefined; await fsp.mkdir(primaryCwd, { recursive: true }); await fsp.mkdir(secondaryCwd, { recursive: true }); const canonicalPrimary = canonicalizeWorkspace(primaryCwd); @@ -135,18 +148,41 @@ async function makeHarness(opts?: { trusted: opts?.secondaryTrusted ?? true, workspaceId: hashDaemonWorkspace(canonicalSecondary), }); + const conversationRoot = conversationWorkspace + ? (await conversationWorkspace.getRoot()).canonicalRoot + : undefined; + const internal = conversationRoot + ? makeRuntime(conversationRoot, { + primary: false, + trusted: true, + workspaceId: hashDaemonWorkspace(conversationRoot), + provenance: 'live-conversation', + removable: false, + }) + : undefined; const registry = createWorkspaceRegistry( - opts?.singleWorkspace ? [primary] : [primary, secondary], + opts?.singleWorkspace + ? [primary] + : [primary, secondary, ...(internal ? [internal] : [])], ); const app = createServeApp( { ...baseOpts, workspace: canonicalPrimary, token: 'secret' }, undefined, { workspaceRegistry: registry, + ...(conversationWorkspace + ? { + liveConversationWorkspace: conversationWorkspace, + conversationRuntimeOwnershipFactory: () => ({ + acquire: vi.fn(async () => ({ reclaimed: false })), + release: vi.fn(async () => false), + }), + } + : {}), }, ); activeApps.add(app); - return { app, scratch, primary, secondary, registry }; + return { app, scratch, primary, secondary, internal, registry }; } function auth(pending: request.Test): request.Test { @@ -827,8 +863,8 @@ describe('extension management v2 REST', () => { } }); - it('fans a global default change out to every registered workspace', async () => { - const h = await makeHarness(); + it('fans a global default change out to every registered runtime', async () => { + const h = await makeHarness({ internalRuntime: true }); mockExtensionManager(); try { const started = await auth( @@ -845,11 +881,45 @@ describe('extension management v2 REST', () => { expect( h.secondary.bridge.refreshExtensionsForAllSessions, ).toHaveBeenCalledOnce(); + expect( + h.internal?.bridge.refreshExtensionsForAllSessions, + ).toHaveBeenCalledOnce(); } finally { await fsp.rm(h.scratch, { recursive: true, force: true }); } }); + it('does not reconcile extension generations after runtime activity seals', async () => { + vi.useFakeTimers(); + const h = await makeHarness({ internalRuntime: true }); + mockExtensionManager(); + vi.spyOn(process.stderr, 'write').mockReturnValue(true); + const activity = ( + h.app.locals as { + conversationRuntimeActivity?: { sealAndWait(): Promise }; + } + ).conversationRuntimeActivity; + try { + expect(activity).toBeDefined(); + await activity!.sealAndWait(); + + await vi.advanceTimersByTimeAsync(30_000); + + expect( + h.primary.bridge.refreshExtensionsForAllSessions, + ).not.toHaveBeenCalled(); + expect( + h.secondary.bridge.refreshExtensionsForAllSessions, + ).not.toHaveBeenCalled(); + expect( + h.internal?.bridge.refreshExtensionsForAllSessions, + ).not.toHaveBeenCalled(); + } finally { + vi.useRealTimers(); + await fsp.rm(h.scratch, { recursive: true, force: true }); + } + }); + it('includes runtimes registered while a global mutation is committing', async () => { const h = await makeHarness(); mockExtensionManager(); diff --git a/packages/cli/src/serve/routes/workspace-qualified-voice.test.ts b/packages/cli/src/serve/routes/workspace-qualified-voice.test.ts index f2e08e2b2bd..2d575e6acc7 100644 --- a/packages/cli/src/serve/routes/workspace-qualified-voice.test.ts +++ b/packages/cli/src/serve/routes/workspace-qualified-voice.test.ts @@ -34,6 +34,7 @@ function runtime( trusted?: boolean; envMode?: 'parent-process' | 'runtime-overlay'; effectiveEnv?: Readonly>; + provenance?: 'live-conversation'; } = {}, ): WorkspaceRuntime { return { @@ -50,6 +51,7 @@ function runtime( effectiveEnv: opts.effectiveEnv ?? {}, }, bridge: { publishWorkspaceEvent: vi.fn() }, + ...(opts.provenance ? { provenance: opts.provenance } : {}), } as unknown as WorkspaceRuntime; } @@ -223,6 +225,26 @@ describe('workspace-qualified Voice routes', () => { expect(acquireVoiceLease).not.toHaveBeenCalled(); }); + it('rejects the internal runtime by id and cwd without falling back', async () => { + const { app, registry, acquireVoiceLease, transcribe } = await createApp(); + const liveCwd = path.join(homes.at(-1)!, 'conversations'); + await fsp.mkdir(liveCwd, { recursive: true }); + registry.add( + runtime('live-id', liveCwd, { provenance: 'live-conversation' }), + ); + + for (const selector of ['live-id', encodeURIComponent(liveCwd)]) { + await expect( + request(app).get(`/workspaces/${selector}/voice`), + ).resolves.toMatchObject({ + status: 400, + body: { code: 'workspace_mismatch' }, + }); + } + expect(acquireVoiceLease).not.toHaveBeenCalled(); + expect(transcribe).not.toHaveBeenCalled(); + }); + it('transcribes with the selected runtime cwd and effective environment', async () => { const { app, secondary, transcribe } = await createApp(); await enableSecondaryVoice(secondary); diff --git a/packages/cli/src/serve/routes/workspace-trust.test.ts b/packages/cli/src/serve/routes/workspace-trust.test.ts index 69de6fc54f6..5d9a58f1c8f 100644 --- a/packages/cli/src/serve/routes/workspace-trust.test.ts +++ b/packages/cli/src/serve/routes/workspace-trust.test.ts @@ -39,17 +39,19 @@ describe('workspace trust routes', () => { it.each([ [ 'managed-scratch', + 409, 'managed_scratch_trust_fixed', 'Managed scratch workspace trust cannot be changed', ], [ 'live-conversation', - 'live_conversation_trust_fixed', - 'Live conversation workspace trust cannot be changed', + 400, + 'workspace_mismatch', + '`:workspace` must decode to a workspace id or absolute path', ], ] as const)( 'rejects manual trust changes for %s provenance', - async (provenance, code, error) => { + async (provenance, status, code, error) => { const selected = runtime(provenance); const primary = runtime('existing', true); const app = express(); @@ -66,7 +68,7 @@ describe('workspace trust routes', () => { ) .send({ desiredState: 'untrusted' }); - expect(response.status).toBe(409); + expect(response.status).toBe(status); expect(response.body).toEqual({ code, error }); expect( selected.workspaceService.requestWorkspaceTrustChange, diff --git a/packages/cli/src/serve/run-qwen-serve.test.ts b/packages/cli/src/serve/run-qwen-serve.test.ts index 3bd7f46ce57..f9f51fe877b 100644 --- a/packages/cli/src/serve/run-qwen-serve.test.ts +++ b/packages/cli/src/serve/run-qwen-serve.test.ts @@ -73,6 +73,7 @@ import { } from './workspace-registration-store.js'; import { getDeferredRuntimeRequestTiming } from './server/request-helpers.js'; import type { WorkspaceFileSystemFactory } from './fs/workspace-file-system.js'; +import { ConversationWorkspace } from './conversations/conversation-workspace.js'; const originalTestRuntimeDir = process.env['QWEN_RUNTIME_DIR']; const isolatedTestRuntimeDir = fs.realpathSync( @@ -3733,6 +3734,57 @@ describe('runQwenServe pre-listen bridge option validation', () => { expect(stdoutWrites.join('')).not.toContain('qwen serve listening on'); }); + it.each(['root', 'child', 'missing-child', 'alias'] as const)( + 'rejects an explicit Conversations reserved %s before listening', + async (candidateKind) => { + tmpDir = fs.realpathSync( + fs.mkdtempSync(path.join(os.tmpdir(), 'qws-reserved-workspace-')), + ); + const liveConversationWorkspace = new ConversationWorkspace({ + homeDir: tmpDir, + }); + fs.mkdirSync(liveConversationWorkspace.rootPath, { recursive: true }); + const child = path.join(liveConversationWorkspace.rootPath, 'session-1'); + fs.mkdirSync(child); + const missingChild = path.join( + liveConversationWorkspace.rootPath, + 'missing-session', + ); + const alias = path.join(tmpDir, 'conversation-alias'); + fs.symlinkSync( + liveConversationWorkspace.rootPath, + alias, + process.platform === 'win32' ? 'junction' : 'dir', + ); + const workspace = + candidateKind === 'root' + ? liveConversationWorkspace.rootPath + : candidateKind === 'child' + ? child + : candidateKind === 'missing-child' + ? missingChild + : alias; + const stdoutWrites: string[] = []; + vi.spyOn(process.stdout, 'write').mockImplementation((chunk) => { + stdoutWrites.push(String(chunk)); + return true; + }); + + await expect( + runQwenServe( + { + port: 0, + hostname: '127.0.0.1', + mode: 'http-bridge', + workspace, + }, + { liveConversationWorkspace }, + ), + ).rejects.toThrow(/reserved for Conversations/); + expect(stdoutWrites.join('')).not.toContain('qwen serve listening on'); + }, + ); + it('rejects an unknown embedded external Tool Guard mode before listening', async () => { tmpDir = fs.realpathSync( fs.mkdtempSync(path.join(os.tmpdir(), 'qws-guard-opt-')), @@ -4931,10 +4983,22 @@ describe('runQwenServe runtime startup failures', () => { const explicitSecondary = path.join(tmpDir, 'explicit-secondary'); const restoredSecondary = path.join(tmpDir, 'restored-secondary'); const nestedSecondary = path.join(explicitSecondary, 'nested'); + const liveConversationWorkspace = new ConversationWorkspace({ + homeDir: tmpDir, + }); + const reservedConversationChild = path.join( + liveConversationWorkspace.rootPath, + 'legacy-child', + ); + const missingReservedConversationChild = path.join( + liveConversationWorkspace.rootPath, + 'missing-legacy-child', + ); fs.mkdirSync(primary); fs.mkdirSync(explicitSecondary); fs.mkdirSync(restoredSecondary); fs.mkdirSync(nestedSecondary); + fs.mkdirSync(reservedConversationChild, { recursive: true }); const restoredSecondaryAlias = path.join( tmpDir, 'restored-secondary-alias', @@ -5000,6 +5064,9 @@ describe('runQwenServe runtime startup failures', () => { nestedSecondary, restoredSecondaryAlias, canonicalRestoredSecondary, + liveConversationWorkspace.rootPath, + reservedConversationChild, + missingReservedConversationChild, ], displayNames: { [workspaceRegistrationId(canonicalExplicitSecondary)]: @@ -5023,6 +5090,7 @@ describe('runQwenServe runtime startup failures', () => { }, { workspaceRegistrationStore: store, + liveConversationWorkspace, daemonLogBaseDir: path.join(tmpDir, 'debug'), resolveOnListen: true, }, @@ -5066,6 +5134,11 @@ describe('runQwenServe runtime startup failures', () => { String(message).includes('path nests with an explicit'), ), ).toBe(true); + expect( + stderrWrite.mock.calls.filter(([message]) => + String(message).includes('path is reserved for Conversations'), + ), + ).toHaveLength(3); } finally { await handle.close(); } @@ -7171,6 +7244,47 @@ describe('runQwenServe runtime startup failures', () => { expect(bridge.shutdown).toHaveBeenCalledOnce(); }); + it('propagates an admitted session maintenance drain failure', async () => { + tmpDir = fs.realpathSync( + fs.mkdtempSync(path.join(os.tmpdir(), 'qws-maintenance-failure-')), + ); + vi.spyOn(qwenCore, 'resolveTelemetrySettings').mockResolvedValue({ + enabled: false, + sensitiveSpanAttributeMaxLength: 1024 * 1024, + }); + const bridge = makeRuntimeBridge(); + vi.spyOn(acpBridge, 'createAcpSessionBridge').mockReturnValue( + bridge as ReturnType, + ); + const sealMaintenanceAndWait = vi.fn(async () => { + throw new Error('maintenance drain failed'); + }); + vi.spyOn(serverModule, 'createServeApp').mockImplementation(() => { + const runtimeApp = express(); + runtimeApp.locals['sessionArchiveCoordinator'] = { + sealMaintenanceAndWait, + }; + return runtimeApp; + }); + + const handle = await runQwenServe( + { + port: 0, + hostname: '127.0.0.1', + mode: 'http-bridge', + workspace: tmpDir, + maxSessions: 1, + serveWebShell: false, + }, + { resolveOnListen: true }, + ); + await handle.runtimeReady; + + await expect(handle.close()).rejects.toThrow('maintenance drain failed'); + expect(sealMaintenanceAndWait).toHaveBeenCalledOnce(); + expect(bridge.shutdown).not.toHaveBeenCalled(); + }); + it('does not cancel deferred runtime once startup is already running', async () => { tmpDir = fs.realpathSync( fs.mkdtempSync(path.join(os.tmpdir(), 'qws-health-close-running-')), @@ -10438,6 +10552,80 @@ describe('runQwenServe channel worker supervisor', () => { } }); + it('keeps the channel lease when lifecycle aggregation wraps the retryable worker error', async () => { + tmpDir = fs.realpathSync( + fs.mkdtempSync(path.join(os.tmpdir(), 'qws-channel-aggregate-error-')), + ); + const worker = makeWorker({ + enabled: true, + state: 'failed', + pid: 1234, + channels: ['telegram'], + error: 'Channel worker did not exit after SIGKILL.', + }); + worker.stop + .mockRejectedValueOnce( + new Error('Channel worker did not exit after SIGKILL.'), + ) + .mockResolvedValueOnce(undefined); + const exitSpy = vi + .spyOn(process, 'exit') + .mockImplementation((() => undefined) as never); + const existingSigintListeners = new Set(process.rawListeners('SIGINT')); + const existingSigtermListeners = new Set(process.rawListeners('SIGTERM')); + + const handle = await runQwenServe( + { + port: 0, + hostname: '127.0.0.1', + mode: 'http-bridge', + workspace: tmpDir, + serveWebShell: false, + channelSelection: { mode: 'names', names: ['telegram'] }, + }, + { + bridge: makeFakeBridge(), + channelWorkerSupervisorFactory: vi.fn(() => worker), + channelServicePidfile: makePidfileDeps(), + }, + ); + + mockChannelWorkerEnabledState.value = true; + const signalListener = process + .rawListeners('SIGTERM') + .find( + (listener) => + !existingSigtermListeners.has(listener) && + listener.name === 'onSignal', + ) as ((signal: NodeJS.Signals) => Promise) | undefined; + const originalServerClose = handle.server.close; + handle.server.close = vi.fn((callback) => { + setImmediate(() => callback?.(new Error('listener close failed'))); + return handle.server; + }) as typeof handle.server.close; + try { + expect(signalListener).toBeDefined(); + await signalListener!('SIGTERM'); + + expect(worker.stop).toHaveBeenCalledOnce(); + expect(exitSpy).not.toHaveBeenCalled(); + } finally { + handle.server.close = originalServerClose; + await handle.close().catch(() => undefined); + for (const listener of process.rawListeners('SIGINT')) { + if (!existingSigintListeners.has(listener)) { + process.removeListener('SIGINT', listener as never); + } + } + for (const listener of process.rawListeners('SIGTERM')) { + if (!existingSigtermListeners.has(listener)) { + process.removeListener('SIGTERM', listener as never); + } + } + exitSpy.mockRestore(); + } + }); + it('bounds the logger flush before allowing a retryable close to reject', async () => { tmpDir = fs.realpathSync( fs.mkdtempSync(path.join(os.tmpdir(), 'qws-channel-log-stuck-')), @@ -11052,6 +11240,76 @@ describe('runQwenServe channel worker supervisor', () => { expect(pidfile.removeServeServiceInfo).toHaveBeenCalledWith(process.pid); }); + it('drains the runtime when channel grouping rejects startup after listen', async () => { + tmpDir = fs.realpathSync( + fs.mkdtempSync(path.join(os.tmpdir(), 'qws-channel-untrusted-')), + ); + const bridge = makeFakeBridge(); + const pidfile = makePidfileDeps(); + + await expect( + runQwenServe( + { + port: 0, + hostname: '127.0.0.1', + mode: 'http-bridge', + workspace: tmpDir, + serveWebShell: false, + channelSelection: { mode: 'names', names: ['telegram'] }, + }, + { + bridge, + trustedWorkspace: false, + channelServicePidfile: pidfile, + }, + ), + ).rejects.toThrow('not trusted; cannot host channels'); + + expect(bridge.shutdown).toHaveBeenCalledTimes(1); + expect(pidfile.removeServeServiceInfo).toHaveBeenCalledWith(process.pid); + }); + + it('preserves the startup reason when channel cleanup also fails', async () => { + tmpDir = fs.realpathSync( + fs.mkdtempSync(path.join(os.tmpdir(), 'qws-channel-startup-cleanup-')), + ); + const cleanupError = new Error('session maintenance drain failed'); + const bridge = makeFakeBridge(); + const originalCreateServeApp = serverModule.createServeApp; + vi.spyOn(serverModule, 'createServeApp').mockImplementation((...args) => { + const app = originalCreateServeApp(...args); + app.locals['sessionArchiveCoordinator'] = { + sealMaintenanceAndWait: vi.fn().mockRejectedValue(cleanupError), + }; + return app; + }); + + try { + await runQwenServe( + { + port: 0, + hostname: '127.0.0.1', + mode: 'http-bridge', + workspace: tmpDir, + serveWebShell: false, + channelSelection: { mode: 'names', names: ['telegram'] }, + }, + { + bridge, + trustedWorkspace: false, + channelServicePidfile: makePidfileDeps(), + }, + ); + expect.fail('Expected serve startup to reject.'); + } catch (error) { + expect(error).toBeInstanceOf(AggregateError); + expect((error as AggregateError).message).toContain( + 'is not trusted; cannot host channels', + ); + expect((error as AggregateError).errors).toContain(cleanupError); + } + }); + it('keeps the serve owner alive when failed startup cannot confirm worker exit', async () => { tmpDir = fs.realpathSync( fs.mkdtempSync(path.join(os.tmpdir(), 'qws-channel-worker-retained-')), @@ -11276,22 +11534,20 @@ describe('runQwenServe channel worker supervisor', () => { }); vi.spyOn(serverModule, 'createServeApp').mockReturnValue({ locals: {}, - listen: vi.fn((port, _host, cb) => { - portsAttempted.push(port); - const srv = createServer(); - if (typeof cb === 'function') { - srv.once('error', cb); - } - if (portsAttempted.length === 1) { - const err = new Error('address in use') as NodeJS.ErrnoException; - err.code = 'EADDRINUSE'; - setImmediate(() => srv.emit('error', err)); - } else { - srv.listen(0, '127.0.0.1', cb); - } - return srv; - }), } as unknown as express.Application); + const testServer = createServer(); + const nativeListen = testServer.listen.bind(testServer); + testServer.listen = vi.fn((port: number) => { + portsAttempted.push(port); + if (portsAttempted.length === 1) { + const err = new Error('address in use') as NodeJS.ErrnoException; + err.code = 'EADDRINUSE'; + setImmediate(() => testServer.emit('error', err)); + } else { + nativeListen(0, '127.0.0.1'); + } + return testServer; + }) as unknown as typeof testServer.listen; const handle = await runQwenServe( { @@ -11303,6 +11559,7 @@ describe('runQwenServe channel worker supervisor', () => { }, { bridge: makeFakeBridge(), + httpServerFactory: () => testServer, resolveOnListen: true, }, ); @@ -11335,13 +11592,13 @@ describe('runQwenServe channel worker supervisor', () => { listenError.code = 'EADDRINUSE'; vi.spyOn(serverModule, 'createServeApp').mockReturnValue({ locals: {}, - listen: vi.fn((port) => { - portsAttempted.push(port); - const srv = createServer(); - setImmediate(() => srv.emit('error', listenError)); - return srv; - }), } as unknown as express.Application); + const testServer = createServer(); + testServer.listen = vi.fn((port: number) => { + portsAttempted.push(port); + setImmediate(() => testServer.emit('error', listenError)); + return testServer; + }) as unknown as typeof testServer.listen; await expect( runQwenServe( @@ -11353,7 +11610,10 @@ describe('runQwenServe channel worker supervisor', () => { workspace: tmpDir, serveWebShell: false, }, - { bridge: makeFakeBridge() }, + { + bridge: makeFakeBridge(), + httpServerFactory: () => testServer, + }, ), ).rejects.toBe(listenError); @@ -11369,13 +11629,13 @@ describe('runQwenServe channel worker supervisor', () => { listenError.code = 'EACCES'; vi.spyOn(serverModule, 'createServeApp').mockReturnValue({ locals: {}, - listen: vi.fn((port, _host, _cb) => { - portsAttempted.push(port); - const srv = createServer(); - setImmediate(() => srv.emit('error', listenError)); - return srv; - }), } as unknown as express.Application); + const testServer = createServer(); + testServer.listen = vi.fn((port: number) => { + portsAttempted.push(port); + setImmediate(() => testServer.emit('error', listenError)); + return testServer; + }) as unknown as typeof testServer.listen; await expect( runQwenServe( @@ -11386,7 +11646,10 @@ describe('runQwenServe channel worker supervisor', () => { workspace: tmpDir, serveWebShell: false, }, - { bridge: makeFakeBridge() }, + { + bridge: makeFakeBridge(), + httpServerFactory: () => testServer, + }, ), ).rejects.toBe(listenError); @@ -11409,13 +11672,13 @@ describe('runQwenServe channel worker supervisor', () => { listenError.code = 'EADDRINUSE'; vi.spyOn(serverModule, 'createServeApp').mockReturnValue({ locals: {}, - listen: vi.fn((port) => { - portsAttempted.push(port); - const srv = createServer(); - setImmediate(() => srv.emit('error', listenError)); - return srv; - }), } as unknown as express.Application); + const testServer = createServer(); + testServer.listen = vi.fn((port: number) => { + portsAttempted.push(port); + setImmediate(() => testServer.emit('error', listenError)); + return testServer; + }) as unknown as typeof testServer.listen; await expect( runQwenServe( @@ -11426,7 +11689,10 @@ describe('runQwenServe channel worker supervisor', () => { workspace: tmpDir, serveWebShell: false, }, - { bridge: makeFakeBridge() }, + { + bridge: makeFakeBridge(), + httpServerFactory: () => testServer, + }, ), ).rejects.toBe(listenError); @@ -11448,13 +11714,13 @@ describe('runQwenServe channel worker supervisor', () => { listenError.code = 'EADDRINUSE'; vi.spyOn(serverModule, 'createServeApp').mockReturnValue({ locals: {}, - listen: vi.fn((port) => { - portsAttempted.push(port); - const srv = createServer(); - setImmediate(() => srv.emit('error', listenError)); - return srv; - }), } as unknown as express.Application); + const testServer = createServer(); + testServer.listen = vi.fn((port: number) => { + portsAttempted.push(port); + setImmediate(() => testServer.emit('error', listenError)); + return testServer; + }) as unknown as typeof testServer.listen; await expect( runQwenServe( @@ -11465,7 +11731,10 @@ describe('runQwenServe channel worker supervisor', () => { workspace: tmpDir, serveWebShell: false, }, - { bridge: makeFakeBridge() }, + { + bridge: makeFakeBridge(), + httpServerFactory: () => testServer, + }, ), ).rejects.toBe(listenError); diff --git a/packages/cli/src/serve/run-qwen-serve.ts b/packages/cli/src/serve/run-qwen-serve.ts index adc554aba64..b54ae723f95 100644 --- a/packages/cli/src/serve/run-qwen-serve.ts +++ b/packages/cli/src/serve/run-qwen-serve.ts @@ -6,7 +6,7 @@ import { X509Certificate, createHash, timingSafeEqual } from 'node:crypto'; import * as fs from 'node:fs'; -import type { Server } from 'node:http'; +import { createServer, type Server } from 'node:http'; import * as https from 'node:https'; import * as path from 'node:path'; import * as os from 'node:os'; @@ -135,8 +135,10 @@ import { type ManagedScratchRoot, type WorkspaceRuntimeProvenance, } from './managed-scratch-workspace.js'; +import { ConversationRuntimeOwnershipError } from './conversations/conversation-runtime-errors.js'; import { ConversationWorkspace } from './conversations/conversation-workspace.js'; import { LIVE_HOST_PROTOCOL_VERSION } from './live/types.js'; +import { ServeAppLifecycleController } from './serve-app-lifecycle.js'; import { workspaceRegistrationId, type WorkspaceRegistrationStore, @@ -813,6 +815,15 @@ export interface RunHandle { const retryableChannelWorkerShutdownErrors = new WeakSet(); +function hasRetryableChannelWorkerShutdownError(error: unknown): boolean { + if (error instanceof AggregateError) { + return error.errors.some(hasRetryableChannelWorkerShutdownError); + } + return ( + error instanceof Error && retryableChannelWorkerShutdownErrors.has(error) + ); +} + type CoreRuntime = typeof import('./core-runtime.js'); type LiveDiscoveryRuntime = typeof import('./live/discovery.js'); type ProviderConfig = NonNullable>; @@ -976,6 +987,8 @@ function buildProviderSetupInputs( export interface RunQwenServeDeps { /** Bridge instance; tests inject a fake. Defaults to a fresh real one. */ bridge?: AcpSessionBridge; + /** Test/embed override for the plain HTTP server constructor. */ + httpServerFactory?: (app: Application) => Server; /** * Whether to start the real ACP child eagerly after listen. Production * keeps this on; tests can disable it so boot-path assertions do not wait @@ -2511,6 +2524,34 @@ async function runQwenServeImpl( // Resolve the bound workspace list. The first explicit workspace remains the // primary workspace for legacy APIs; later workspaces are isolated secondary // runtimes. + const liveConversationWorkspace = + deps.liveConversationWorkspace ?? new ConversationWorkspace(); + const isReservedConversationWorkspace = (candidate: string): boolean => { + const resolvedCandidate = path.resolve(candidate); + const resolvedRoot = path.resolve(liveConversationWorkspace.rootPath); + let canonicalRoot = resolvedRoot; + try { + canonicalRoot = fs.realpathSync.native(resolvedRoot); + } catch { + // The reserved root is intentionally not materialized during startup. + } + return ( + resolvedCandidate === resolvedRoot || + isWithinRoot(resolvedCandidate, resolvedRoot) || + resolvedCandidate === canonicalRoot || + isWithinRoot(resolvedCandidate, canonicalRoot) + ); + }; + const reservedRawWorkspace = rawWorkspaces.find((workspace) => + isReservedConversationWorkspace(workspace), + ); + if (reservedRawWorkspace) { + throw new Error( + `Workspace ${JSON.stringify( + reservedRawWorkspace, + )} is reserved for Conversations.`, + ); + } const workspaceInputs = rawWorkspaces.map((workspace) => ({ raw: workspace, cwd: validateAndCanonicalizeWorkspace(workspace), @@ -2622,6 +2663,16 @@ async function runQwenServeImpl( journalGrowthSessionLimitProviders.delete(provider); }; }; + const reservedStartupWorkspace = workspaceInputs.find((workspace) => + isReservedConversationWorkspace(workspace.cwd), + ); + if (reservedStartupWorkspace) { + throw new Error( + `Workspace ${JSON.stringify( + reservedStartupWorkspace.raw, + )} is reserved for Conversations.`, + ); + } let workspaceRegistrationStore = deps.workspaceRegistrationStore; if ( workspaceRegistrationStore === undefined && @@ -2638,6 +2689,14 @@ async function runQwenServeImpl( for (const storedWorkspace of stored.workspaces) { const registrationId = workspaceRegistrationId(storedWorkspace); const displayName = stored.displayNames?.[registrationId]; + if (isReservedConversationWorkspace(storedWorkspace)) { + writeStderrLine( + `qwen serve: skipping persisted workspace registration ${JSON.stringify( + storedWorkspace, + )}: path is reserved for Conversations`, + ); + continue; + } let cwd: string; try { cwd = validateAndCanonicalizeWorkspace(storedWorkspace); @@ -2649,6 +2708,14 @@ async function runQwenServeImpl( ); continue; } + if (isReservedConversationWorkspace(cwd)) { + writeStderrLine( + `qwen serve: skipping persisted workspace registration ${JSON.stringify( + storedWorkspace, + )}: path is reserved for Conversations`, + ); + continue; + } const existingInput = workspaceInputs.find( (workspace) => workspace.cwd === cwd, ); @@ -3064,6 +3131,28 @@ async function runQwenServeImpl( // webShellDir is already undefined whenever serveWebShell === false, so this // collapses to "did we resolve real assets". const webShellMounted = !!webShellDir; + const serveAppLifecycle = new ServeAppLifecycleController(); + const liveDiscoveryStableBaseDir = path.resolve( + deps.liveDiscoveryStableBaseDir ?? path.join(os.homedir(), '.qwen'), + ); + let resolveServeAppStartup!: () => void; + let rejectServeAppStartup!: (error: Error) => void; + let serveAppStartupSettled = false; + const serveAppStartupReady = new Promise((resolve, reject) => { + resolveServeAppStartup = resolve; + rejectServeAppStartup = reject; + }); + void serveAppStartupReady.catch(() => undefined); + const markServeAppStartupReady = (): void => { + if (serveAppStartupSettled) return; + serveAppStartupSettled = true; + resolveServeAppStartup(); + }; + const markServeAppStartupFailed = (error: Error): void => { + if (serveAppStartupSettled) return; + serveAppStartupSettled = true; + rejectServeAppStartup(error); + }; let runtimeApp: Application | undefined; let runtimeAppForCleanup: Application | undefined; let bridgeRef: AcpSessionBridge | undefined = deps.bridge; @@ -3168,7 +3257,6 @@ async function runQwenServeImpl( }; }; let closeServerAfterChannelWorkerStartupFailure = false; - let runtimeFailureListenerClose: Promise | undefined; const getChannelWorkerSnapshot = (): ChannelWorkerSnapshot => channelWorkerManager?.primarySnapshot() ?? { enabled: false, @@ -3455,8 +3543,6 @@ async function runQwenServeImpl( }`, ); } - const liveConversationWorkspace = - deps.liveConversationWorkspace ?? new ConversationWorkspace(); let runtimeBootSettings: | ReturnType | undefined; @@ -4800,7 +4886,7 @@ async function runQwenServeImpl( core.registerDaemonGaugeCallbacks({ sessionCount: () => workspaceRegistry - .list() + .listAll() .reduce((sum, item) => sum + item.bridge.sessionCount, 0), sseCount: () => runtime.getActiveSseCount(), heapUsed: () => process.memoryUsage().heapUsed, @@ -4920,16 +5006,16 @@ async function runQwenServeImpl( rssBytes: mem.rss, heapUsedBytes: mem.heapUsed, activeSessions: workspaceRegistry - .list() + .listAll() .reduce((sum, item) => sum + item.bridge.sessionCount, 0), activePrompts: workspaceRegistry - .list() + .listAll() .reduce( (sum, item) => sum + (item.bridge.activePromptCount ?? 0), 0, ), queuedPrompts: workspaceRegistry - .list() + .listAll() .reduce( (sum, item) => sum + (item.bridge.pendingPromptTotal ?? 0), 0, @@ -5788,6 +5874,8 @@ async function runQwenServeImpl( }; const app = runtime.createServeApp(opts, () => actualPort, { + serveAppLifecycle, + liveDiscoveryStableBaseDir, workspaceRegistry, getSessionBridges: () => runtimeBridges, createWorkspaceRuntime: createDynamicWorkspaceRuntime, @@ -6402,8 +6490,9 @@ async function runQwenServeImpl( // When TLS is configured, wrap the Express app in an HTTPS listener // (`https.Server extends http.Server`, so everything downstream — // `server.maxConnections`, `server.address()`, `attachServer(server)`, - // graceful close — is unchanged). Otherwise `app.listen()` keeps the - // existing plain-HTTP path bit-for-bit. + // graceful close — is unchanged). Plain HTTP uses the same explicitly + // lifecycle-bound server shape. + let closeHost: (() => Promise) | undefined; const onListening = (error?: Error) => { // Error handling (retry/reject) is owned by tryListen's // server.once('error') handler. @@ -6417,10 +6506,9 @@ async function runQwenServeImpl( profileCheckpoint('serve_listener_ready'); finalizeStartupProfile(`serve-${process.pid}`); - // Listener-level connection cap, set inside the listen callback - // because Node only exposes the underlying `Server` after - // `app.listen()` returns. Each session's `EventBus` already - // refuses to admit more than `DEFAULT_MAX_SUBSCRIBERS` (64), but + // Listener-level connection cap, set inside the listen callback after + // Node has opened the underlying `Server`. Each session's `EventBus` + // already refuses to admit more than `DEFAULT_MAX_SUBSCRIBERS` (64), but // an attacker can still open *connections* that never finish // their headers, never reach the bus, and just sit consuming // socket descriptors. The default of 256 leaves room for many @@ -6452,15 +6540,34 @@ async function runQwenServeImpl( instanceNonce: string; pid: number; }> = []; + const rememberLiveDiscoveryOwner = (owner: { + runtimeBaseDir: string; + instanceNonce: string; + pid: number; + }): void => { + if ( + liveDiscoveryOwners.some( + (candidate) => + candidate.runtimeBaseDir === owner.runtimeBaseDir && + candidate.instanceNonce === owner.instanceNonce && + candidate.pid === owner.pid, + ) + ) { + return; + } + liveDiscoveryOwners.push(owner); + }; let liveDiscoveryPublish: Promise | undefined; let liveDiscoveryRetryTimer: NodeJS.Timeout | undefined; let liveDiscoveryRetryTask: Promise | undefined; + let liveDiscoveryBootRetryApp: Application | undefined; let liveDiscoveryEnabled = false; let liveDiscoveryShuttingDown = false; let liveDiscoveryToggle: Promise = Promise.resolve(); let attemptPendingLiveDiscovery: (() => Promise) | undefined; const pendingLiveDiscoveryBaseDirs = new Set(); const warnedLiveDiscoveryOwners = new Set(); + let warnedLiveDiscoveryBootFailure = false; const liveDiscoveryRetryDelayMs = deps.liveDiscoveryRetryDelayMs !== undefined && Number.isFinite(deps.liveDiscoveryRetryDelayMs) && @@ -6473,21 +6580,24 @@ async function runQwenServeImpl( !liveDiscoveryEnabled || liveDiscoveryRetryTimer || liveDiscoveryRetryTask || - pendingLiveDiscoveryBaseDirs.size === 0 || - !attemptPendingLiveDiscovery + (!liveDiscoveryBootRetryApp && + (pendingLiveDiscoveryBaseDirs.size === 0 || + !attemptPendingLiveDiscovery)) ) { return; } liveDiscoveryRetryTimer = setTimeout(() => { liveDiscoveryRetryTimer = undefined; - if ( - liveDiscoveryShuttingDown || - !liveDiscoveryEnabled || - !attemptPendingLiveDiscovery - ) { + if (liveDiscoveryShuttingDown || !liveDiscoveryEnabled) { return; } - const retry = attemptPendingLiveDiscovery().finally(() => { + const retryApp = liveDiscoveryBootRetryApp; + liveDiscoveryBootRetryApp = undefined; + const retryOperation = retryApp + ? publishLiveDiscovery(retryApp) + : attemptPendingLiveDiscovery?.(); + if (!retryOperation) return; + const retry = retryOperation.finally(() => { if (liveDiscoveryRetryTask === retry) { liveDiscoveryRetryTask = undefined; } @@ -6516,18 +6626,23 @@ async function runQwenServeImpl( | undefined; const instanceNonce = coordinator?.daemonInstanceNonce; if (typeof instanceNonce !== 'string') return Promise.resolve(); - liveDiscoveryPublish = loadLiveDiscoveryRuntime() + let publicationFailed = false; + let publicationRetryable = false; + const publication = serveAppLifecycle + .startBoot() + .then(() => loadLiveDiscoveryRuntime()) .then( async ({ - getStableLiveDiscoveryBaseDir, + handoffLiveDiscoveryOwner, LiveDiscoveryOwnerActiveError, + LiveDiscoveryPublicationError, + removeLiveDiscoveryFile, writeLiveDiscoveryFile, }) => { if (liveDiscoveryShuttingDown || !liveDiscoveryEnabled) return; - const stableBaseDir = path.resolve( - deps.liveDiscoveryStableBaseDir ?? - getStableLiveDiscoveryBaseDir(), - ); + liveDiscoveryBootRetryApp = undefined; + warnedLiveDiscoveryBootFailure = false; + const stableBaseDir = liveDiscoveryStableBaseDir; const runtimeBaseDir = path.resolve(liveRuntimeBaseDir); const targetBaseDirs = new Set(); if (runtimeBaseDir !== stableBaseDir) { @@ -6545,21 +6660,62 @@ async function runQwenServeImpl( instanceNonce, }; attemptPendingLiveDiscovery = async () => { - for (const runtimeBaseDir of [ - ...pendingLiveDiscoveryBaseDirs, - ]) { - if (liveDiscoveryShuttingDown || !liveDiscoveryEnabled) + const targets = [...pendingLiveDiscoveryBaseDirs]; + const published: Array<{ + runtimeBaseDir: string; + instanceNonce: string; + pid: number; + }> = []; + const rollbackPublished = async (): Promise => { + for (const owner of published.splice(0)) { + try { + await removeLiveDiscoveryFile( + owner.runtimeBaseDir, + owner, + ); + } catch (cleanupError) { + rememberLiveDiscoveryOwner(owner); + daemonLog.warn( + `failed to roll back Live Host discovery at ${owner.runtimeBaseDir}: ${ + cleanupError instanceof Error + ? cleanupError.message + : String(cleanupError) + }`, + ); + } + } + }; + for (const runtimeBaseDir of targets) { + if (liveDiscoveryShuttingDown || !liveDiscoveryEnabled) { + await rollbackPublished(); return; + } try { + if (runtimeBaseDir !== stableBaseDir) { + await handoffLiveDiscoveryOwner( + runtimeBaseDir, + record, + async () => undefined, + ); + } await writeLiveDiscoveryFile(runtimeBaseDir, record); - pendingLiveDiscoveryBaseDirs.delete(runtimeBaseDir); - warnedLiveDiscoveryOwners.delete(runtimeBaseDir); - liveDiscoveryOwners.push({ + published.push({ runtimeBaseDir, instanceNonce, pid: process.pid, }); } catch (err) { + if ( + err instanceof LiveDiscoveryPublicationError && + err.published + ) { + published.push({ + runtimeBaseDir, + instanceNonce, + pid: process.pid, + }); + } + await rollbackPublished(); if (err instanceof LiveDiscoveryOwnerActiveError) { if (!warnedLiveDiscoveryOwners.has(runtimeBaseDir)) { warnedLiveDiscoveryOwners.add(runtimeBaseDir); @@ -6567,58 +6723,90 @@ async function runQwenServeImpl( `failed to publish Live Host discovery at ${runtimeBaseDir}: ${err.message}`, ); } - continue; + return; } - pendingLiveDiscoveryBaseDirs.delete(runtimeBaseDir); daemonLog.warn( `failed to publish Live Host discovery at ${runtimeBaseDir}: ${ err instanceof Error ? err.message : String(err) }`, ); + return; } } + for (const owner of published) { + pendingLiveDiscoveryBaseDirs.delete(owner.runtimeBaseDir); + warnedLiveDiscoveryOwners.delete(owner.runtimeBaseDir); + rememberLiveDiscoveryOwner(owner); + } }; await attemptPendingLiveDiscovery(); scheduleLiveDiscoveryRetry(); }, ) .catch((err) => { - daemonLog.warn( - `failed to publish Live Host discovery: ${ - err instanceof Error ? err.message : String(err) - }`, - ); + publicationFailed = true; + publicationRetryable = + err instanceof ConversationRuntimeOwnershipError && err.retryable; + if (!publicationRetryable || !warnedLiveDiscoveryBootFailure) { + warnedLiveDiscoveryBootFailure = publicationRetryable; + daemonLog.warn( + `failed to publish Live Host discovery: ${ + err instanceof Error ? err.message : String(err) + }`, + ); + } }); + const trackedPublication = publication.finally(() => { + if ( + publicationFailed && + liveDiscoveryPublish === trackedPublication + ) { + liveDiscoveryPublish = undefined; + if ( + publicationRetryable && + !liveDiscoveryShuttingDown && + liveDiscoveryEnabled + ) { + liveDiscoveryBootRetryApp = candidateApp; + scheduleLiveDiscoveryRetry(); + } + } + }); + liveDiscoveryPublish = trackedPublication; return liveDiscoveryPublish; }; const removeLiveDiscoveryOwners = async (): Promise => { - const owners = liveDiscoveryOwners.splice(0); + const owners = [...liveDiscoveryOwners]; if (owners.length === 0) return; let removeLiveDiscoveryFile: LiveDiscoveryRuntime['removeLiveDiscoveryFile']; try { ({ removeLiveDiscoveryFile } = await loadLiveDiscoveryRuntime()); } catch (err) { - daemonLog.warn( - `failed to load Live discovery runtime for cleanup: ${ - err instanceof Error ? err.message : String(err) - }`, - ); - return; + throw new Error('Failed to load Live discovery cleanup support.', { + cause: err, + }); } + const errors: unknown[] = []; for (const owner of owners) { try { await removeLiveDiscoveryFile(owner.runtimeBaseDir, owner); + const index = liveDiscoveryOwners.indexOf(owner); + if (index >= 0) liveDiscoveryOwners.splice(index, 1); } catch (err) { - daemonLog.warn( - `failed to remove Live Host discovery at ${owner.runtimeBaseDir}: ${ - err instanceof Error ? err.message : String(err) - }`, - ); + errors.push(err); } } + if (errors.length > 0) { + throw new AggregateError( + errors, + 'Live Host discovery cleanup is incomplete.', + ); + } }; const unpublishLiveDiscovery = async (): Promise => { liveDiscoveryEnabled = false; + liveDiscoveryBootRetryApp = undefined; + warnedLiveDiscoveryBootFailure = false; cancelLiveDiscoveryRetry(); pendingLiveDiscoveryBaseDirs.clear(); attemptPendingLiveDiscovery = undefined; @@ -6632,6 +6820,7 @@ async function runQwenServeImpl( ( candidateApp.locals as { setLiveDiscoveryEnabled?: (enabled: boolean) => Promise; + onConversationRuntimeReady?: () => void; } ).setLiveDiscoveryEnabled = (enabled) => { const operation = liveDiscoveryToggle.then(() => @@ -6642,6 +6831,13 @@ async function runQwenServeImpl( liveDiscoveryToggle = operation.catch(() => undefined); return operation; }; + ( + candidateApp.locals as { + onConversationRuntimeReady?: () => void; + } + ).onConversationRuntimeReady = () => { + void publishLiveDiscovery(candidateApp); + }; }; const cleanupLiveDiscovery = async (): Promise => { liveDiscoveryShuttingDown = true; @@ -6649,81 +6845,6 @@ async function runQwenServeImpl( await liveDiscoveryToggle; await unpublishLiveDiscovery(); }; - try { - channelWorkspaceGroups = resolveChannelWorkspaceGroupsAtListen(); - } catch (err) { - removeCurrentServePidfile(); - const error = err instanceof Error ? err : new Error(String(err)); - server.close((closeErr) => { - if (closeErr) { - daemonLog.error( - 'server close after channel worker validation error failed', - closeErr, - ); - } - reject(error); - }); - return; - } - if (channelWorkspaceGroups) { - for (const group of channelWorkspaceGroups) { - daemonLog.info('channel worker group assigned', { - workspace: group.workspaceCwd, - channels: - group.selection.mode === 'all' ? ['all'] : group.selection.names, - }); - } - if (opts.channelSelection?.mode === 'all') { - writeStderrLine( - 'qwen serve: --channel all is primary-workspace only; non-primary workspace channels are not hosted.', - ); - } - } - writeStdoutLine( - `qwen serve listening on ${url} (mode=${opts.mode}, ` + - `workspace=${boundWorkspace})`, - ); - // Operator log on stderr too (systemd/docker/k8s default - // captures only stderr for service diagnostics, and the - // workspace= breadcrumb is the single piece of information - // operators need most when triaging migration issues — - // "did the daemon bind to the right workspace?"). The stdout - // line above stays put so integration tests + scripts that - // parse stdout for the listening URL keep working; - // `JSON.stringify(boundWorkspace)` quotes the value - // symmetrically with the workspace_mismatch log (defends - // against control-char log injection if `boundWorkspace` - // somehow contained one — operator-controlled today, but - // cheap defense-in-depth). - writeStderrLine( - `qwen serve: bound to workspace ${JSON.stringify(boundWorkspace)}`, - ); - writeStderrLine( - `qwen serve: startup timing: processToListenMs=${startup.processToListenMs} ` + - `runQwenServeToListenMs=${startup.runQwenServeToListenMs}`, - ); - if (!token) { - writeStderrLine( - `qwen serve: bearer auth disabled (loopback default). Set ${QWEN_SERVER_TOKEN_ENV} to enable.`, - ); - if (opts.clientMcpOverWs === true) { - writeStderrLine( - `qwen serve: client-hosted MCP tools are accepted over the WebSocket without auth. ` + - `Set ${QWEN_SERVE_CLIENT_MCP_OVER_WS_ENV}=0 to disable.`, - ); - } - } else if (opts.requireAuth) { - // The boot check above guarantees `token` is set whenever - // `--require-auth` is on, so this branch only fires alongside - // a successfully-authenticated daemon. The log line lets - // operators confirm the hardening is active without parsing - // `/capabilities` (and is a useful breadcrumb when triaging - // "why is loopback returning 401" tickets). - writeStderrLine( - 'qwen serve: --require-auth enabled (bearer token mandatory ' + - 'on every route, including loopback /health).', - ); - } let shuttingDown = false; let closePromise: Promise | undefined; let runtimeStartupTimer: NodeJS.Timeout | undefined; @@ -6800,6 +6921,7 @@ async function runQwenServeImpl( bridgeForCleanup?: AcpSessionBridge, ): Promise => { const error = err instanceof Error ? err : new Error(String(err)); + markServeAppStartupFailed(error); if (runtimeStartupSettled) { disposeRuntimeAppResources(runtimeApp ?? runtimeAppForCleanup); await shutdownBridgeAfterFailedStartup(bridgeForCleanup); @@ -6822,16 +6944,13 @@ async function runQwenServeImpl( daemonLog.error('runtime startup failed', error); markRuntimeFailed(error); if (closeServerAfterChannelWorkerStartupFailure && server.listening) { - runtimeFailureListenerClose = new Promise((resolve) => { - server.close((closeErr) => { - if (closeErr) { - daemonLog.error( - 'server close after runtime startup error failed', - closeErr, - ); - } - resolve(); - }); + server.close((closeErr) => { + if (closeErr) { + daemonLog.error( + 'server close after runtime startup error failed', + closeErr, + ); + } }); server.closeAllConnections(); } @@ -7058,6 +7177,7 @@ async function runQwenServeImpl( if (runtimeStartupSettled) return; } if (runtimeStartupSettled) return; + markServeAppStartupReady(); await publishLiveDiscovery(candidateApp); runtimeStartupSettled = true; clearRuntimeStartupTimer(); @@ -7192,10 +7312,7 @@ async function runQwenServeImpl( process.exit(runtimeStartupError === undefined ? 0 : 1); } catch (err) { daemonLog.error('shutdown error', err instanceof Error ? err : null); - if ( - err instanceof Error && - retryableChannelWorkerShutdownErrors.has(err) - ) { + if (hasRetryableChannelWorkerShutdownError(err)) { daemonLog.error( 'refusing to exit while a channel worker or service lease remains; signal again to retry after the child exits (another signal during that retry forces exit)', ); @@ -7244,6 +7361,10 @@ async function runQwenServeImpl( ?.locals?.['sessionArchiveCoordinator'] as | { sealMaintenanceAndWait?: () => Promise } | undefined; + const initiallyMountedConversationActivity = initiallyMountedApp + ?.locals?.['conversationRuntimeActivity'] as + | { sealAndWait?: () => Promise } + | undefined; // Calling an async function runs through its first await // synchronously. Seal an already-mounted runtime before close() // yields so no management request can enter the shutdown window. @@ -7256,6 +7377,8 @@ async function runQwenServeImpl( initiallyMountedLive?.sealAndWaitLiveCoordinator?.(); const initialSessionMaintenanceWait = initiallyMountedSessionMaintenance?.sealMaintenanceAndWait?.(); + const initialConversationActivityWait = + initiallyMountedConversationActivity?.sealAndWait?.(); let processRegistryShutdown: Promise | undefined; const startProcessRegistryShutdown = () => { processRegistryShutdown ??= managedProcessRegistry @@ -7276,24 +7399,9 @@ async function runQwenServeImpl( // behavior in charge and could orphan agent children. We detach // AFTER drain completes (`finish` below). - // Two-phase shutdown: - // 1. The shared process registry starts every agent child's - // 5s TERM/KILL and 10s raw-exit timeline before slower worker - // or bridge cleanup. `bridge.shutdown()` then drains its - // in-flight state against those same terminal promises. - // 2. `server.close()` — drains in-flight HTTP connections - // (long-lived SSE subscribers especially). This is - // what `SHUTDOWN_FORCE_CLOSE_MS` actually protects: - // a single hung SSE consumer would otherwise pin - // the listener open forever. - // - // Crucially, the force timer is armed AFTER bridge.shutdown - // resolves, not at the start of the whole sequence. An - // earlier version raced both phases against the same 5s - // timer; if the bridge took 5–10s to kill its children - // (e.g. SIGTERM grace period), the timer fired first, - // resolved this promise, and `process.exit(0)` ran while - // the bridge was still tearing children down. + // The shared lifecycle closes the listener in parallel with this + // host drain, then releases Conversations ownership only after both + // the listener callback and every child/bridge drain are proven. let settled = false; // Track bridge.shutdown failures so close() // doesn't silently report success when the bridge @@ -7341,7 +7449,7 @@ async function runQwenServeImpl( // Server.close error takes precedence (operator-visible // listener problem); fall back to the bridge error // captured during shutdown if any. - const finalErr = + let finalErr = err ?? bridgeShutdownError ?? channelWorkerShutdownError; const retryableChannelClose = channelWorkerShutdownError !== undefined && @@ -7359,7 +7467,23 @@ async function runQwenServeImpl( rej(retryableError); return; } - await cleanupLiveDiscovery(); + try { + await cleanupLiveDiscovery(); + } catch (cleanupError) { + const normalizedCleanupError = + cleanupError instanceof Error + ? cleanupError + : new Error(String(cleanupError)); + if (finalErr) { + writeDaemonLifecycleBestEffort(() => { + daemonLog.error( + 'Live Host discovery cleanup failed during shutdown', + normalizedCleanupError, + ); + }); + } + finalErr ??= normalizedCleanupError; + } if (loggerPublished || loggerSignalOwned) { writeDaemonLifecycleBestEffort(() => { if (finalErr) { @@ -7408,6 +7532,9 @@ async function runQwenServeImpl( ] as | { sealMaintenanceAndWait?: () => Promise } | undefined; + const conversationActivity = appForCleanup?.locals?.[ + 'conversationRuntimeActivity' + ] as { sealAndWait?: () => Promise } | undefined; await initialManagementWait; if (workspaceManagementHandle !== initiallyMountedManagement) { await workspaceManagementHandle?.sealAndWait?.(); @@ -7423,6 +7550,12 @@ async function runQwenServeImpl( if (sessionMaintenance !== initiallyMountedSessionMaintenance) { await sessionMaintenance?.sealMaintenanceAndWait?.(); } + await initialConversationActivityWait; + if ( + conversationActivity !== initiallyMountedConversationActivity + ) { + await conversationActivity?.sealAndWait?.(); + } stopTrustPolicyMonitor(appForCleanup); const waitForTrustPolicyIdle = appForCleanup?.locals?.[ 'waitForTrustPolicyIdle' @@ -7505,72 +7638,103 @@ async function runQwenServeImpl( bridgeShutdownError ??= processRegistryError; } }) - .finally(() => { - if (!server.listening) { - void (runtimeFailureListenerClose ?? Promise.resolve()).then( - () => finish(), - ); - return; - } - // Phase 2: arm the force timer NOW so it only races - // server.close, not the bridge tear-down above. - // `RunHandle.close()` contract says "fully - // closed and bridge drained" — the previous code - // resolved on a 100ms shortcut AFTER - // `closeAllConnections()` without waiting for - // `server.close`'s callback, so embedders/tests - // could observe a "closed" handle while the server - // was still finalizing. Now: force-close just - // accelerates `server.close` by killing the - // sockets, but we still wait for `server.close`'s - // callback to fire. A secondary deadline catches - // the pathological case where `server.close` never - // resolves at all (kernel-stuck socket etc.) so - // shutdown is still bounded. - const SECONDARY_DEADLINE_MS = 2_000; - let secondaryTimer: NodeJS.Timeout | undefined; - const forceTimer = setTimeout(() => { - daemonLog.warn( - `${SHUTDOWN_FORCE_CLOSE_MS}ms listener-drain timeout reached; force-closing remaining connections`, - ); - server.closeAllConnections(); - // After force-close, server.close's callback - // SHOULD fire promptly. Give it `SECONDARY_DEADLINE_MS` - // before we resolve anyway with a warning — much - // longer than the previous 100ms shortcut, and - // logged so the operator knows the contract was - // bent. - secondaryTimer = setTimeout(() => { - daemonLog.warn( - `server.close did not fire ${SECONDARY_DEADLINE_MS}ms after force-close; resolving anyway`, - ); - finish(); - }, SECONDARY_DEADLINE_MS); - secondaryTimer.unref(); - }, SHUTDOWN_FORCE_CLOSE_MS); - forceTimer.unref(); - server.close((err) => { - clearTimeout(forceTimer); - if (secondaryTimer) clearTimeout(secondaryTimer); - finish(err); - }); - }); + .then( + () => finish(), + (error: unknown) => + finish( + error instanceof Error ? error : new Error(String(error)), + ), + ); }); return closePromise; }, }; + closeHost = handle.close; + handle.close = () => serveAppLifecycle.close(); + + try { + channelWorkspaceGroups = resolveChannelWorkspaceGroupsAtListen(); + } catch (err) { + removeCurrentServePidfile(); + const error = err instanceof Error ? err : new Error(String(err)); + markServeAppStartupFailed(error); + void serveAppLifecycle.close().then( + () => reject(error), + (closeError: unknown) => + reject( + closeError instanceof Error + ? new AggregateError([error, closeError], error.message) + : error, + ), + ); + return; + } + if (channelWorkspaceGroups) { + for (const group of channelWorkspaceGroups) { + daemonLog.info('channel worker group assigned', { + workspace: group.workspaceCwd, + channels: + group.selection.mode === 'all' ? ['all'] : group.selection.names, + }); + } + if (opts.channelSelection?.mode === 'all') { + writeStderrLine( + 'qwen serve: --channel all is primary-workspace only; non-primary workspace channels are not hosted.', + ); + } + } + writeStdoutLine( + `qwen serve listening on ${url} (mode=${opts.mode}, ` + + `workspace=${boundWorkspace})`, + ); + // Operator log on stderr too (systemd/docker/k8s default + // captures only stderr for service diagnostics, and the + // workspace= breadcrumb is the single piece of information + // operators need most when triaging migration issues — + // "did the daemon bind to the right workspace?"). The stdout + // line above stays put so integration tests + scripts that + // parse stdout for the listening URL keep working; + // `JSON.stringify(boundWorkspace)` quotes the value + // symmetrically with the workspace_mismatch log (defends + // against control-char log injection if `boundWorkspace` + // somehow contained one — operator-controlled today, but + // cheap defense-in-depth). + writeStderrLine( + `qwen serve: bound to workspace ${JSON.stringify(boundWorkspace)}`, + ); + writeStderrLine( + `qwen serve: startup timing: processToListenMs=${startup.processToListenMs} ` + + `runQwenServeToListenMs=${startup.runQwenServeToListenMs}`, + ); + if (!token) { + writeStderrLine( + `qwen serve: bearer auth disabled (loopback default). Set ${QWEN_SERVER_TOKEN_ENV} to enable.`, + ); + if (opts.clientMcpOverWs === true) { + writeStderrLine( + `qwen serve: client-hosted MCP tools are accepted over the WebSocket without auth. ` + + `Set ${QWEN_SERVE_CLIENT_MCP_OVER_WS_ENV}=0 to disable.`, + ); + } + } else if (opts.requireAuth) { + // The boot check above guarantees `token` is set whenever + // `--require-auth` is on, so this branch only fires alongside + // a successfully-authenticated daemon. The log line lets + // operators confirm the hardening is active without parsing + // `/capabilities` (and is a useful breadcrumb when triaging + // "why is loopback returning 401" tickets). + writeStderrLine( + 'qwen serve: --require-auth enabled (bearer token mandatory ' + + 'on every route, including loopback /health).', + ); + } process.on('SIGINT', onSignal); process.on('SIGTERM', onSignal); process.on('uncaughtExceptionMonitor', onUncaughtExceptionMonitor); - // Swap the boot-error listener for a runtime-error one - // before resolving. `tryListen`'s `server.once('error', ...)` - // only catches errors BEFORE listening; post-listen errors - // (EMFILE after FD exhaustion, runtime errors on the listener) - // would be unhandled and crash the daemon. Use a persistent - // listener that logs to stderr instead. - server.removeAllListeners('error'); + // The per-attempt boot-error listener was removed by handleListening. + // Keep the lifecycle listener and add persistent runtime diagnostics. server.on('error', (err) => { daemonLog.error('server error', err instanceof Error ? err : null); }); @@ -7590,6 +7754,7 @@ async function runQwenServeImpl( | AcpHttpHandle | undefined; acpHandle?.attachServer?.(server); + markServeAppStartupReady(); void publishLiveDiscovery(preparedRuntimeApp); } } else if (deferRuntimeUntilFirstHealth) { @@ -7624,10 +7789,7 @@ async function runQwenServeImpl( closeErr instanceof Error ? closeErr : null, ), ); - if ( - closeErr instanceof Error && - retryableChannelWorkerShutdownErrors.has(closeErr) - ) { + if (hasRetryableChannelWorkerShutdownError(closeErr)) { writeDaemonLifecycleBestEffort(() => daemonLog.error( 'runtime startup failed, but qwen serve remains alive to retain the channel service lease until worker exit is confirmed', @@ -7643,10 +7805,9 @@ async function runQwenServeImpl( } }; let server: Server; - let httpsServer: https.Server | undefined; if (tlsOptions) { try { - httpsServer = https.createServer(tlsOptions, app); + server = https.createServer(tlsOptions, app); } catch (err) { // createSecureContext throws a raw OpenSSL string (e.g. // "error:0B080074:...key values mismatch") when cert/key don't pair. @@ -7661,30 +7822,30 @@ async function runQwenServeImpl( ); return; } + } else { + server = deps.httpServerFactory?.(app) ?? createServer(app); } + serveAppLifecycle.bindServer(server, { + startupReady: serveAppStartupReady, + drainHost: () => { + if (closeHost) return closeHost(); + if (!server.listening) return Promise.resolve(); + return new Promise((resolve, rejectClose) => { + server.close((error) => { + if (error) rejectClose(error); + else resolve(); + }); + }); + }, + }); const tryListen = (attemptPort: number, attempt: number): void => { - try { - if (httpsServer) { - // server.listen(port, host, cb) registers `cb` as a one-time - // `listening` listener. On failed attempts (EADDRINUSE), - // `listening` never fires so the listener accumulates. Clear - // stale listeners before each retry. - httpsServer.removeAllListeners('listening'); - server = httpsServer.listen(attemptPort, listenHostname, onListening); - } else { - server = app.listen(attemptPort, listenHostname, onListening); - } - } catch (err) { - // Synchronous listen failure (e.g. invalid address) — not - // recoverable via port bump. - removeCurrentServePidfile(); - reject(err instanceof Error ? err : new Error(String(err))); - return; - } - - server.once('error', (err: NodeJS.ErrnoException) => { - server.close(); + const handleListening = (): void => { + server.removeListener('error', handleError); + onListening(); + }; + const handleError = (err: NodeJS.ErrnoException): void => { + server.removeListener('listening', handleListening); const nextPort = attemptPort + 1; if ( err.code === 'EADDRINUSE' && @@ -7697,16 +7858,48 @@ async function runQwenServeImpl( `qwen serve: port ${attemptPort} is in use, trying ${nextPort}...`, ); tryListen(nextPort, attempt + 1); - } else { - if (err.code === 'EADDRINUSE' && attempt > 0) { - writeStderrLine( - `qwen serve: all ports ${opts.port}–${attemptPort} are in use`, - ); - } - removeCurrentServePidfile(); - reject(err); + return; } - }); + if (err.code === 'EADDRINUSE' && attempt > 0) { + writeStderrLine( + `qwen serve: all ports ${opts.port}–${attemptPort} are in use`, + ); + } + removeCurrentServePidfile(); + markServeAppStartupFailed(err); + void serveAppLifecycle.close().then( + () => reject(err), + (closeError: unknown) => + reject( + closeError instanceof Error + ? new AggregateError([err, closeError], err.message) + : err, + ), + ); + }; + try { + server.once('listening', handleListening); + server.once('error', handleError); + server.listen(attemptPort, listenHostname); + } catch (err) { + // Synchronous listen failure (e.g. invalid address) — not + // recoverable via port bump. + removeCurrentServePidfile(); + server.removeListener('listening', handleListening); + server.removeListener('error', handleError); + const error = err instanceof Error ? err : new Error(String(err)); + markServeAppStartupFailed(error); + void serveAppLifecycle.close().then( + () => reject(error), + (closeError: unknown) => + reject( + closeError instanceof Error + ? new AggregateError([error, closeError], error.message) + : error, + ), + ); + return; + } }; tryListen(opts.port, 0); diff --git a/packages/cli/src/serve/serve-app-lifecycle.test.ts b/packages/cli/src/serve/serve-app-lifecycle.test.ts new file mode 100644 index 00000000000..2f76ad9225c --- /dev/null +++ b/packages/cli/src/serve/serve-app-lifecycle.test.ts @@ -0,0 +1,303 @@ +/** + * @license + * Copyright 2026 Qwen Team + * SPDX-License-Identifier: Apache-2.0 + */ + +import { createServer, get } from 'node:http'; +import express from 'express'; +import { afterEach, describe, expect, it, vi } from 'vitest'; +import { + getServeAppLifecycle, + installServeAppLifecycle, +} from './serve-app-lifecycle.js'; + +const servers = new Set>(); + +afterEach(async () => { + await Promise.all( + [...servers].map( + (server) => + new Promise((resolve) => { + if (!server.listening) { + resolve(); + return; + } + server.close(() => resolve()); + }), + ), + ); + servers.clear(); +}); + +describe('ServeAppLifecycle', () => { + it('opens boot only after the bound listener and host startup are ready', async () => { + const app = express(); + const lifecycle = installServeAppLifecycle(app); + let markReady: (() => void) | undefined; + const startupReady = new Promise((resolve) => { + markReady = resolve; + }); + const server = createServer(app); + servers.add(server); + lifecycle.bindServer(server, { startupReady }); + const boot = vi.fn(async () => undefined); + lifecycle.setBootStarter(boot); + + server.listen(0, '127.0.0.1'); + await new Promise((resolve) => server.once('listening', resolve)); + expect(boot).not.toHaveBeenCalled(); + markReady?.(); + await lifecycle.awaitBootAdmission(); + await vi.waitFor(() => expect(boot).toHaveBeenCalledOnce()); + }); + + it('rejects unbound, already-listening, and duplicate binding', async () => { + const app = express(); + const lifecycle = installServeAppLifecycle(app); + await expect(lifecycle.awaitBootAdmission()).rejects.toMatchObject({ + code: 'conversation_runtime_unavailable', + }); + + const listening = createServer(app); + servers.add(listening); + listening.listen(0, '127.0.0.1'); + await new Promise((resolve) => listening.once('listening', resolve)); + expect(() => lifecycle.bindServer(listening)).toThrow(/before its first/); + + const otherApp = express(); + const otherLifecycle = installServeAppLifecycle(otherApp); + const first = createServer(otherApp); + const second = createServer(otherApp); + servers.add(first); + servers.add(second); + otherLifecycle.bindServer(first); + expect(() => otherLifecycle.bindServer(second)).toThrow(/one server/); + + const closedLifecycle = installServeAppLifecycle(express()); + await closedLifecycle.close(); + expect(() => closedLifecycle.bindServer(createServer())).toThrow( + /before its first listen/, + ); + }); + + it('runs direct-embed cleanup after a pre-listen server error', async () => { + const app = express(); + const lifecycle = installServeAppLifecycle(app); + const server = createServer(app); + servers.add(server); + const drain = vi.fn(async () => undefined); + const boot = vi.fn(async () => undefined); + lifecycle.setAppDrain(drain); + lifecycle.setBootStarter(boot); + lifecycle.bindServer(server); + + server.emit('error', new Error('listen failed')); + await lifecycle.close(); + + expect(drain).toHaveBeenCalledOnce(); + expect(boot).not.toHaveBeenCalled(); + }); + + it('waits for listener, app, and host drain before releasing ownership', async () => { + const app = express(); + const lifecycle = installServeAppLifecycle(app); + const server = createServer(app); + servers.add(server); + let releaseApp: (() => void) | undefined; + let releaseHost: (() => void) | undefined; + const appDrain = new Promise((resolve) => { + releaseApp = resolve; + }); + const hostDrain = new Promise((resolve) => { + releaseHost = resolve; + }); + const release = vi.fn(async () => true); + lifecycle.setOwnership({ + acquire: vi.fn(async () => ({ reclaimed: false })), + release, + }); + lifecycle.setAppDrain(() => appDrain); + lifecycle.bindServer(server, { drainHost: () => hostDrain }); + server.listen(0, '127.0.0.1'); + await new Promise((resolve) => server.once('listening', resolve)); + + const close = lifecycle.close(); + await vi.waitFor(() => expect(server.listening).toBe(false)); + expect(release).not.toHaveBeenCalled(); + releaseApp?.(); + await Promise.resolve(); + expect(release).not.toHaveBeenCalled(); + releaseHost?.(); + await close; + expect(release).toHaveBeenCalledOnce(); + }); + + it('uses the same cleanup when the embed closes the server directly', async () => { + const app = express(); + const lifecycle = installServeAppLifecycle(app); + const server = createServer(app); + servers.add(server); + const drain = vi.fn(async () => undefined); + const release = vi.fn(async () => true); + lifecycle.setAppDrain(drain); + lifecycle.setOwnership({ + acquire: vi.fn(async () => ({ reclaimed: false })), + release, + }); + lifecycle.bindServer(server); + server.listen(0, '127.0.0.1'); + await new Promise((resolve) => server.once('listening', resolve)); + + server.close(); + await lifecycle.close(); + expect(drain).toHaveBeenCalledOnce(); + expect(release).toHaveBeenCalledOnce(); + expect(getServeAppLifecycle(app)).toBe(lifecycle); + }); + + it('force-closes active connections when an embed listener is already closing', async () => { + const app = express(); + let requestStarted: (() => void) | undefined; + const started = new Promise((resolve) => { + requestStarted = resolve; + }); + app.get('/hold', () => requestStarted?.()); + const lifecycle = installServeAppLifecycle(app); + const server = createServer(app); + servers.add(server); + const release = vi.fn(async () => true); + lifecycle.setOwnership({ + acquire: vi.fn(async () => ({ reclaimed: false })), + release, + }); + lifecycle.bindServer(server); + server.listen(0, '127.0.0.1'); + await new Promise((resolve) => server.once('listening', resolve)); + const address = server.address(); + if (!address || typeof address === 'string') throw new Error('No address'); + const client = get(`http://127.0.0.1:${address.port}/hold`); + client.on('error', () => undefined); + await started; + + server.close(); + expect(server.listening).toBe(false); + await lifecycle.close({ timeoutMs: 0 }); + + expect(release).toHaveBeenCalledOnce(); + client.destroy(); + }); + + it('starts every drain and listener close when one drain throws synchronously', async () => { + const app = express(); + const lifecycle = installServeAppLifecycle(app); + const server = createServer(app); + servers.add(server); + const hostDrain = vi.fn(async () => undefined); + const release = vi.fn(async () => true); + lifecycle.setAppDrain(() => { + throw new Error('app drain failed'); + }); + lifecycle.setOwnership({ + acquire: vi.fn(async () => ({ reclaimed: false })), + release, + }); + lifecycle.bindServer(server, { drainHost: hostDrain }); + server.listen(0, '127.0.0.1'); + await new Promise((resolve) => server.once('listening', resolve)); + + await expect(lifecycle.close()).rejects.toThrow('app drain failed'); + expect(hostDrain).toHaveBeenCalledOnce(); + expect(server.listening).toBe(false); + expect(release).not.toHaveBeenCalled(); + }); + + it('tracks an explicit boot retry after the automatic attempt fails', async () => { + const app = express(); + const lifecycle = installServeAppLifecycle(app); + const server = createServer(app); + servers.add(server); + const automaticBoot = vi + .fn<() => Promise>() + .mockRejectedValueOnce(new Error('first boot failed')) + .mockResolvedValue(undefined); + lifecycle.setBootStarter(automaticBoot); + lifecycle.bindServer(server); + server.listen(0, '127.0.0.1'); + await new Promise((resolve) => server.once('listening', resolve)); + await vi.waitFor(() => expect(automaticBoot).toHaveBeenCalledOnce()); + await vi.waitFor(() => expect(lifecycle.getBootPromise()).toBeUndefined()); + + await expect(lifecycle.startBoot(automaticBoot)).resolves.toBeUndefined(); + expect(automaticBoot).toHaveBeenCalledTimes(2); + }); + + it('waits for an explicit boot retry before releasing ownership', async () => { + const app = express(); + const lifecycle = installServeAppLifecycle(app); + const server = createServer(app); + servers.add(server); + let finishBoot: (() => void) | undefined; + const bootPending = new Promise((resolve) => { + finishBoot = resolve; + }); + const boot = vi.fn(() => bootPending); + const release = vi.fn(async () => true); + lifecycle.setOwnership({ + acquire: vi.fn(async () => ({ reclaimed: false })), + release, + }); + lifecycle.bindServer(server); + server.listen(0, '127.0.0.1'); + await new Promise((resolve) => server.once('listening', resolve)); + + const retry = lifecycle.startBoot(boot); + await vi.waitFor(() => expect(boot).toHaveBeenCalledOnce()); + const close = lifecycle.close(); + await vi.waitFor(() => expect(server.listening).toBe(false)); + expect(release).not.toHaveBeenCalled(); + finishBoot?.(); + await retry; + await close; + expect(release).toHaveBeenCalledOnce(); + }); + + it('rejects a late boot after shutdown has sealed admission', async () => { + const app = express(); + const lifecycle = installServeAppLifecycle(app); + const server = createServer(app); + servers.add(server); + lifecycle.bindServer(server); + server.listen(0, '127.0.0.1'); + await new Promise((resolve) => server.once('listening', resolve)); + + await lifecycle.close(); + const boot = vi.fn(async () => undefined); + await expect(lifecycle.startBoot(boot)).rejects.toMatchObject({ + code: 'conversation_runtime_unavailable', + }); + expect(boot).not.toHaveBeenCalled(); + }); + + it('maps host startup failure to the structured unavailable error', async () => { + const app = express(); + const lifecycle = installServeAppLifecycle(app); + const server = createServer(app); + servers.add(server); + let rejectStartup: ((error: Error) => void) | undefined; + const startupReady = new Promise((_resolve, reject) => { + rejectStartup = reject; + }); + lifecycle.bindServer(server, { startupReady }); + server.listen(0, '127.0.0.1'); + await new Promise((resolve) => server.once('listening', resolve)); + + const admission = lifecycle.awaitBootAdmission(); + rejectStartup?.(new Error('private startup detail')); + + await expect(admission).rejects.toMatchObject({ + code: 'conversation_runtime_unavailable', + retryable: true, + }); + }); +}); diff --git a/packages/cli/src/serve/serve-app-lifecycle.ts b/packages/cli/src/serve/serve-app-lifecycle.ts new file mode 100644 index 00000000000..eabe3d55e92 --- /dev/null +++ b/packages/cli/src/serve/serve-app-lifecycle.ts @@ -0,0 +1,334 @@ +/** + * @license + * Copyright 2026 Qwen Team + * SPDX-License-Identifier: Apache-2.0 + */ + +import type { Server } from 'node:http'; +import type { Application } from 'express'; +import type { ConversationRuntimeOwnership } from './conversations/conversation-runtime-ownership.js'; +import { conversationRuntimeUnavailableError } from './conversations/conversation-runtime-errors.js'; + +const SERVE_APP_LIFECYCLE = Symbol('qwen.serveAppLifecycle'); +const DEFAULT_CLOSE_TIMEOUT_MS = 5_000; +const SECONDARY_CLOSE_TIMEOUT_MS = 2_000; + +export interface ServeAppLifecycleBindingOptions { + startupReady?: Promise; + drainHost?: () => Promise; +} + +export interface ServeAppLifecycle { + bindServer(server: Server, options?: ServeAppLifecycleBindingOptions): void; + close(options?: { timeoutMs?: number }): Promise; +} + +interface ServeAppLifecycleLocals { + [SERVE_APP_LIFECYCLE]?: ServeAppLifecycleController; +} + +function deferred(): { + promise: Promise; + resolve: () => void; + reject: (error: Error) => void; +} { + let resolve!: () => void; + let reject!: (error: Error) => void; + const promise = new Promise((resolvePromise, rejectPromise) => { + resolve = resolvePromise; + reject = rejectPromise; + }); + void promise.catch(() => undefined); + return { promise, resolve, reject }; +} + +export class ServeAppLifecycleController implements ServeAppLifecycle { + private readonly admission = deferred(); + private server?: Server; + private startupReady = true; + private listenerReady = false; + private listenerClosed = false; + private sealed = false; + private closePending?: Promise; + private ownership?: ConversationRuntimeOwnership; + private appDrain?: () => Promise; + private hostDrain?: () => Promise; + private bootStarter?: () => Promise | void; + private bootPending?: Promise; + private bootStarted = false; + + bindServer( + server: Server, + options: ServeAppLifecycleBindingOptions = {}, + ): void { + if (this.server || server.listening || this.bootStarted || this.sealed) { + throw new Error( + 'Serve app lifecycle must bind one server before its first listen.', + ); + } + this.server = server; + this.hostDrain = options.drainHost; + this.startupReady = options.startupReady === undefined; + server.once('listening', () => { + this.listenerReady = true; + if (this.sealed) { + server.close(); + server.closeAllConnections(); + return; + } + this.openAdmissionIfReady(); + }); + server.on('close', () => { + this.listenerClosed = true; + this.seal(); + void this.close().catch(() => undefined); + }); + server.on('error', (error) => { + if (!this.listenerReady && options.startupReady === undefined) { + this.seal(error); + void this.close().catch(() => undefined); + } + }); + if (options.startupReady) { + void options.startupReady.then( + () => { + this.startupReady = true; + this.openAdmissionIfReady(); + }, + (error: unknown) => { + this.seal(error instanceof Error ? error : new Error(String(error))); + }, + ); + } + } + + close(options: { timeoutMs?: number } = {}): Promise { + if (this.closePending) return this.closePending; + const pending = this.closeOnce(options); + this.closePending = pending; + void pending.catch(() => { + if (this.closePending === pending) this.closePending = undefined; + }); + return pending; + } + + setOwnership(ownership: ConversationRuntimeOwnership): void { + if (this.ownership) { + throw new Error('Serve app lifecycle ownership is already configured.'); + } + this.ownership = ownership; + } + + setAppDrain(drain: () => Promise): void { + this.appDrain = drain; + } + + setBootStarter(starter: () => Promise | void): void { + this.bootStarter = starter; + this.openAdmissionIfReady(); + } + + async awaitBootAdmission(): Promise { + if (!this.server || this.sealed) { + throw conversationRuntimeUnavailableError(); + } + try { + await this.admission.promise; + } catch (error) { + throw conversationRuntimeUnavailableError(error); + } + if (this.sealed) { + throw conversationRuntimeUnavailableError(); + } + } + + async startBoot(starter = this.bootStarter): Promise { + await this.awaitBootAdmission(); + await this.beginBoot(starter); + } + + getBootPromise(): Promise | undefined { + return this.bootPending; + } + + isBootStarted(): boolean { + return this.bootStarted; + } + + sealBoot(): void { + this.seal(); + } + + private openAdmissionIfReady(): void { + if ( + this.sealed || + !this.server || + !this.listenerReady || + !this.startupReady + ) { + return; + } + this.admission.resolve(); + const bootStarter = this.bootStarter; + if (bootStarter && !this.bootStarted) { + void this.beginBoot(bootStarter)?.catch(() => undefined); + } + } + + private beginBoot( + starter: (() => Promise | void) | undefined, + ): Promise | undefined { + if (!starter) return undefined; + if (this.sealed) { + return Promise.reject(conversationRuntimeUnavailableError()); + } + if (this.bootPending) return this.bootPending; + this.bootStarted = true; + const pending = Promise.resolve().then(starter); + const tracked = pending.finally(() => { + if (this.bootPending === tracked) this.bootPending = undefined; + }); + this.bootPending = tracked; + void tracked.catch(() => undefined); + return tracked; + } + + private seal(error?: Error): void { + if (this.sealed) return; + this.sealed = true; + this.admission.reject(error ?? conversationRuntimeUnavailableError()); + } + + private async closeOnce(options: { timeoutMs?: number }): Promise { + this.seal(); + const drains = [ + this.startDrain(this.appDrain), + this.startDrain(this.hostDrain), + this.bootPending?.catch(() => undefined), + ].filter((value): value is Promise => value !== undefined); + const listenerClose = this.closeListener( + options.timeoutMs ?? DEFAULT_CLOSE_TIMEOUT_MS, + ); + const results = await Promise.allSettled([...drains, listenerClose]); + const errors = results + .filter( + (result): result is PromiseRejectedResult => + result.status === 'rejected', + ) + .map((result) => result.reason as unknown); + if (errors.length === 1) { + throw errors[0]; + } + if (errors.length > 1) { + throw new AggregateError(errors, 'Serve app shutdown is incomplete.'); + } + await this.ownership?.release(); + } + + private startDrain( + drain: (() => Promise) | undefined, + ): Promise | undefined { + if (!drain) return undefined; + try { + return drain(); + } catch (error) { + return Promise.reject(error); + } + } + + private closeListener(timeoutMs: number): Promise { + if ( + !this.server || + this.listenerClosed || + (!this.listenerReady && !this.server.listening) + ) { + return Promise.resolve(); + } + const server = this.server; + if (!server.listening) { + return this.waitForClosingListener(server, timeoutMs); + } + return new Promise((resolve, reject) => { + let secondaryTimer: NodeJS.Timeout | undefined; + const forceTimer = setTimeout( + () => { + server.closeAllConnections(); + secondaryTimer = setTimeout(() => { + reject(new Error('The serve listener did not confirm shutdown.')); + }, SECONDARY_CLOSE_TIMEOUT_MS); + secondaryTimer.unref(); + }, + Math.max(0, timeoutMs), + ); + forceTimer.unref(); + server.close((error) => { + clearTimeout(forceTimer); + if (secondaryTimer) clearTimeout(secondaryTimer); + if (error) reject(error); + else { + this.listenerClosed = true; + resolve(); + } + }); + }); + } + + private waitForClosingListener( + server: Server, + timeoutMs: number, + ): Promise { + return new Promise((resolve, reject) => { + let secondaryTimer: NodeJS.Timeout | undefined; + const onClose = () => { + server.off('close', onClose); + clearTimeout(forceTimer); + if (secondaryTimer) clearTimeout(secondaryTimer); + this.listenerClosed = true; + resolve(); + }; + const forceTimer = setTimeout( + () => { + server.closeAllConnections(); + secondaryTimer = setTimeout(() => { + server.off('close', onClose); + reject(new Error('The serve listener did not confirm shutdown.')); + }, SECONDARY_CLOSE_TIMEOUT_MS); + secondaryTimer.unref(); + }, + Math.max(0, timeoutMs), + ); + forceTimer.unref(); + server.once('close', onClose); + if (this.listenerClosed) onClose(); + }); + } +} + +export function installServeAppLifecycle( + app: Application, + lifecycle = new ServeAppLifecycleController(), +): ServeAppLifecycleController { + const locals = app.locals as ServeAppLifecycleLocals; + if (locals[SERVE_APP_LIFECYCLE]) { + throw new Error('Serve app lifecycle is already installed.'); + } + locals[SERVE_APP_LIFECYCLE] = lifecycle; + return lifecycle; +} + +export function getServeAppLifecycle(app: Application): ServeAppLifecycle { + const lifecycle = (app.locals as ServeAppLifecycleLocals)[ + SERVE_APP_LIFECYCLE + ]; + if (!lifecycle) { + throw new Error('Application was not created by createServeApp.'); + } + return lifecycle; +} + +export function getServeAppLifecycleController( + app: Application, +): ServeAppLifecycleController { + const lifecycle = getServeAppLifecycle(app); + return lifecycle as ServeAppLifecycleController; +} diff --git a/packages/cli/src/serve/server.test.ts b/packages/cli/src/serve/server.test.ts index a8fc7828d0b..a7a44f2aa9f 100644 --- a/packages/cli/src/serve/server.test.ts +++ b/packages/cli/src/serve/server.test.ts @@ -6,7 +6,7 @@ import { existsSync, realpathSync, promises as fsp } from 'node:fs'; import { EventEmitter } from 'node:events'; -import type { ServerResponse } from 'node:http'; +import { createServer, type ServerResponse } from 'node:http'; import type { AddressInfo } from 'node:net'; import * as os from 'node:os'; import * as path from 'node:path'; @@ -21,7 +21,7 @@ import { beforeEach, vi, } from 'vitest'; -import request from 'supertest'; +import supertest from 'supertest'; import { WebSocket } from 'ws'; import { trace, type Span } from '@opentelemetry/api'; import { @@ -38,6 +38,10 @@ import { type ChannelWorkerControlState, } from './channel-worker-manager.js'; import { runQwenServe, type RunHandle } from './run-qwen-serve.js'; +import { + getServeAppLifecycle, + type ServeAppLifecycle, +} from './serve-app-lifecycle.js'; import { ChannelDeliveryAuthorizationStore } from './channel-delivery-authorization.js'; import { CHANNEL_WORKER_PROMPT_AUTHORIZATION_META_KEY, @@ -297,36 +301,120 @@ const baseOpts: ServeOptions = { // Direct app tests bypass runQwenServe's reconciler cleanup. const createdApps = new Set>(); +const createdAppLifecycles = new Map< + ReturnType, + { lifecycle: ServeAppLifecycle; server: ReturnType } +>(); + +function request(target: Parameters[0]) { + const bound = createdAppLifecycles.get( + target as ReturnType, + ); + return supertest(bound?.server ?? target); +} function createServeApp(...args: Parameters) { - const app = createServeAppImpl(...args); + const deps = args[2]; + const app = createServeAppImpl( + args[0], + args[1], + deps?.liveConversationWorkspace + ? { + ...deps, + conversationRuntimeOwnershipFactory: + deps.conversationRuntimeOwnershipFactory ?? + (() => ({ + acquire: vi.fn(async () => ({ reclaimed: false })), + release: vi.fn(async () => false), + })), + } + : deps, + ); createdApps.add(app); + if (deps?.liveConversationWorkspace) { + const lifecycle = getServeAppLifecycle(app); + const server = createServer(app); + lifecycle.bindServer(server); + server.listen(0); + server.unref(); + createdAppLifecycles.set(app, { lifecycle, server }); + } return app; } -function stopCreatedApps() { +async function stopCreatedApps() { for (const app of createdApps) { ( app.locals as { stopExtensionGenerationReconciler?: () => void } ).stopExtensionGenerationReconciler?.(); + const bound = createdAppLifecycles.get(app); + if (bound) { + await bound.lifecycle.close().catch(() => undefined); + } } + createdAppLifecycles.clear(); createdApps.clear(); } afterEach(stopCreatedApps); -it('stops extension generation reconcilers for direct app tests', () => { +it('stops extension generation reconcilers for direct app tests', async () => { const stopExtensionGenerationReconciler = vi.fn(); createdApps.add({ locals: { stopExtensionGenerationReconciler }, } as ReturnType); - stopCreatedApps(); + await stopCreatedApps(); expect(stopExtensionGenerationReconciler).toHaveBeenCalledOnce(); expect(createdApps.size).toBe(0); }); +it('disposes app-owned resources during direct lifecycle shutdown', async () => { + const deviceFlowRegistry = new DeviceFlowRegistry({ + events: { publish: () => {} }, + resolveProvider: () => undefined, + }); + const disposeDeviceFlows = vi.spyOn(deviceFlowRegistry, 'dispose'); + const app = createServeAppImpl({ ...baseOpts, rateLimit: true }, undefined, { + bridge: fakeBridge(), + deviceFlowRegistry, + }); + const rateLimiter = getRateLimiter(app)!; + const disposeRateLimiter = vi.spyOn(rateLimiter, 'dispose'); + const originalStopExtensionGenerationReconciler = app.locals[ + 'stopExtensionGenerationReconciler' + ] as (() => void) | undefined; + const stopExtensionGenerationReconciler = vi.fn(() => + originalStopExtensionGenerationReconciler?.(), + ); + app.locals['stopExtensionGenerationReconciler'] = + stopExtensionGenerationReconciler; + const lifecycle = getServeAppLifecycle(app); + const server = createServer(app); + lifecycle.bindServer(server); + + try { + await new Promise((resolve, reject) => { + server.once('error', reject); + server.listen(0, '127.0.0.1', resolve); + }); + + await lifecycle.close(); + await lifecycle.close(); + + expect(disposeDeviceFlows).toHaveBeenCalledOnce(); + expect(disposeRateLimiter).toHaveBeenCalledOnce(); + expect(stopExtensionGenerationReconciler).toHaveBeenCalledOnce(); + expect(server.listening).toBe(false); + } finally { + if (server.listening) await lifecycle.close().catch(() => undefined); + if (disposeDeviceFlows.mock.calls.length === 0) { + deviceFlowRegistry.dispose(); + } + } +}); + function fakeDaemonLog(): DaemonLogger { return { info: vi.fn(), @@ -3865,17 +3953,18 @@ describe('createServeApp', () => { displayName: 'Conversations', provenance: 'live-conversation', }; + const registry = createWorkspaceRegistry([ + makeWorkspaceRuntimeForTest({ + workspaceId: 'primary-id', + workspaceCwd: WS_BOUND, + primary: true, + bridge: primaryBridge, + }), + liveRuntime, + ]); const app = createServeApp(baseOpts, undefined, { bridge: primaryBridge, - workspaceRegistry: createWorkspaceRegistry([ - makeWorkspaceRuntimeForTest({ - workspaceId: 'primary-id', - workspaceCwd: WS_BOUND, - primary: true, - bridge: primaryBridge, - }), - liveRuntime, - ]), + workspaceRegistry: registry, }); const response = await request(app) @@ -3895,6 +3984,18 @@ describe('createServeApp', () => { trusted: true, kind: 'live', }); + expect(response.body.features).not.toContain('multi_workspace_sessions'); + expect(response.body.limits).toHaveProperty('maxSessionsPerWorkspace'); + expect(response.body.limits).toHaveProperty('maxTotalSessions'); + + expect(registry.beginDrain(liveRuntime)).toBe(true); + const draining = await request(app) + .get('/capabilities') + .set('Host', `127.0.0.1:${baseOpts.port}`); + expect(draining.status).toBe(200); + expect(draining.body.workspaces).toEqual([ + expect.objectContaining({ id: 'primary-id' }), + ]); }); it('reports the current primary runtime permission policy after replacement', async () => { @@ -21159,6 +21260,92 @@ describe('createServeApp', () => { expect(secondaryBridge.sessionTranscriptCalls).toEqual([]); }); + it('prefers active ordinary sessions without losing internal archive errors', async () => { + const archivedSid = '55555555-bbbb-cccc-dddd-b0b0b0b0b0b0'; + const conflictedSid = '55555555-bbbb-cccc-dddd-b1b1b1b1b1b1'; + const internalOnlyArchivedSid = '55555555-bbbb-cccc-dddd-b2b2b2b2b2b2'; + const internalOnlyConflictedSid = '55555555-bbbb-cccc-dddd-b3b3b3b3b3b3'; + const internalDir = path.join(runtimeDir, 'internal-conversations'); + await fsp.mkdir(internalDir, { recursive: true }); + const internalWs = realpathSync(internalDir); + await writeTranscriptSession(archivedSid, 'active', wsDir); + await writeTranscriptSession(archivedSid, 'archived', internalWs); + await writeTranscriptSession(conflictedSid, 'active', wsDir); + await writeTranscriptSession(conflictedSid, 'active', internalWs); + await writeTranscriptSession(conflictedSid, 'archived', internalWs); + await writeTranscriptSession( + internalOnlyArchivedSid, + 'archived', + internalWs, + ); + await writeTranscriptSession( + internalOnlyConflictedSid, + 'active', + internalWs, + ); + await writeTranscriptSession( + internalOnlyConflictedSid, + 'archived', + internalWs, + ); + const primaryBridge = fakeBridge({ + sessionTranscriptImpl: async (req) => ({ + v: 1, + sessionId: req.sessionId, + events: [], + hasMore: false, + }), + }); + const internalBridge = fakeBridge(); + const internalRuntime: WorkspaceRuntime = { + ...makeWorkspaceRuntimeForTest({ + workspaceId: 'internal-conversations', + workspaceCwd: internalWs, + primary: false, + bridge: internalBridge, + }), + provenance: 'live-conversation', + removable: false, + }; + const registry = createWorkspaceRegistry([ + makeWorkspaceRuntimeForTest({ + workspaceId: 'primary', + workspaceCwd: wsDir, + primary: true, + bridge: primaryBridge, + }), + internalRuntime, + ]); + const app = createServeApp({ ...baseOpts, workspace: wsDir }, undefined, { + workspaceRegistry: registry, + }); + + const transcript = await request(app) + .get(`/session/${archivedSid}/transcript`) + .set('Host', `127.0.0.1:${baseOpts.port}`); + const exported = await request(app) + .get(`/session/${conflictedSid}/export?format=json`) + .set('Host', `127.0.0.1:${baseOpts.port}`); + const internalOnlyTranscript = await request(app) + .get(`/session/${internalOnlyArchivedSid}/transcript`) + .set('Host', `127.0.0.1:${baseOpts.port}`); + const internalOnlyExport = await request(app) + .get(`/session/${internalOnlyConflictedSid}/export?format=json`) + .set('Host', `127.0.0.1:${baseOpts.port}`); + + expect(transcript.status).toBe(200); + expect(exported.status).toBe(200); + expect(exported.text).toContain(conflictedSid); + expect(internalOnlyTranscript.status).toBe(409); + expect(internalOnlyTranscript.body.code).toBe('session_archived'); + expect(internalOnlyExport.status).toBe(409); + expect(internalOnlyExport.body.code).toBe('session_conflict'); + expect(primaryBridge.sessionTranscriptCalls).toEqual([ + { sessionId: archivedSid }, + ]); + expect(internalBridge.sessionTranscriptCalls).toEqual([]); + }); + it('prefers structured transcript errors found after generic scan failures', async () => { const sid = '55555555-bbbb-cccc-dddd-afafafafafaf'; const secondaryDir = path.join(runtimeDir, 'archived-after-failure'); @@ -29465,7 +29652,15 @@ describe('Live conversation runtime lifecycle', () => { }; } - function setupLiveRuntime(liveBridgeOptions: FakeBridgeOpts = {}) { + function setupLiveRuntime( + liveBridgeOptions: FakeBridgeOpts = {}, + rootOverrides: Partial<{ + configuredRoot: string; + canonicalRoot: string; + device: number; + inode: number; + }> = {}, + ) { const primaryBridge = fakeBridge(); const registry = createWorkspaceRegistry([ makeWorkspaceRuntimeForTest({ @@ -29480,8 +29675,10 @@ describe('Live conversation runtime lifecycle', () => { canonicalRoot: '/work/live-conversations', device: 1, inode: 2, + ...rootOverrides, }; const conversationWorkspace = { + rootPath: root.configuredRoot, revalidate: vi.fn(async () => root), assertExactRoot: vi.fn(async () => root), materializeConversationDirectory: vi.fn( @@ -29547,6 +29744,7 @@ describe('Live conversation runtime lifecycle', () => { coordinator, registry, root, + primaryBridge, liveRuntime, liveBridge, conversationWorkspace, @@ -29673,9 +29871,9 @@ describe('Live conversation runtime lifecycle', () => { setup.resolveCreation(); const capabilities = await capabilitiesPromise; - expect(setup.registry.getByWorkspaceCwd(setup.root.canonicalRoot)).toBe( - setup.liveRuntime, - ); + expect( + setup.registry.getManagedByWorkspaceCwd(setup.root.canonicalRoot), + ).toBe(setup.liveRuntime); expect(capabilities.body.features).toContain('realtime_voice'); expect(capabilities.body.workspaces).toContainEqual( expect.objectContaining({ @@ -29696,6 +29894,340 @@ describe('Live conversation runtime lifecycle', () => { } }); + it('boots only an exact default-source Conversations catalog request', async () => { + const restoreLiveSettings = await disableLiveVoiceAtBoot(); + const setup = setupLiveRuntime(); + const rootSelector = encodeURIComponent(setup.root.configuredRoot); + try { + const arbitrary = await request(setup.app) + .get('/workspaces/not-conversations/sessions?sourceType=default') + .set('Host', `127.0.0.1:${baseOpts.port}`); + expect(arbitrary.status).toBe(400); + + const unfiltered = await request(setup.app) + .get(`/workspaces/${rootSelector}/sessions`) + .set('Host', `127.0.0.1:${baseOpts.port}`); + expect(unfiltered.status).toBe(400); + + const sourceId = await request(setup.app) + .get( + `/workspaces/${rootSelector}/sessions?sourceType=default&sourceId=unexpected`, + ) + .set('Host', `127.0.0.1:${baseOpts.port}`); + expect(sourceId.status).toBe(400); + expect(setup.createWorkspaceRuntime).not.toHaveBeenCalled(); + + const catalogPromise = request(setup.app) + .get(`/workspaces/${rootSelector}/sessions?sourceType=default`) + .set('Host', `127.0.0.1:${baseOpts.port}`) + .then((response) => response); + await vi.waitFor(() => { + expect(setup.createWorkspaceRuntime).toHaveBeenCalledOnce(); + }); + setup.resolveCreation(); + + const catalog = await catalogPromise; + expect(catalog.status).toBe(200); + expect(catalog.body.sessions).toEqual([]); + expect(setup.registry.getByWorkspaceCwd(setup.root.canonicalRoot)).toBe( + undefined, + ); + expect( + setup.registry.getManagedByWorkspaceCwd(setup.root.canonicalRoot), + ).toBe(setup.liveRuntime); + } finally { + await restoreLiveSettings(); + } + }); + + it('boots the singular Conversations catalog under exact default-source proof', async () => { + const restoreLiveSettings = await disableLiveVoiceAtBoot(); + const setup = setupLiveRuntime(); + const rootSelector = encodeURIComponent(setup.root.configuredRoot); + try { + const unfiltered = await request(setup.app) + .get(`/workspace/${rootSelector}/sessions`) + .set('Host', `127.0.0.1:${baseOpts.port}`); + expect(unfiltered.status).toBe(400); + const sourceId = await request(setup.app) + .get( + `/workspace/${rootSelector}/sessions?sourceType=default&sourceId=unexpected`, + ) + .set('Host', `127.0.0.1:${baseOpts.port}`); + expect(sourceId.status).toBe(400); + expect(setup.createWorkspaceRuntime).not.toHaveBeenCalled(); + + const catalogPromise = request(setup.app) + .get(`/workspace/${rootSelector}/sessions?sourceType=default`) + .set('Host', `127.0.0.1:${baseOpts.port}`) + .then((response) => response); + await vi.waitFor(() => { + expect(setup.createWorkspaceRuntime).toHaveBeenCalledOnce(); + }); + setup.resolveCreation(); + + const catalog = await catalogPromise; + expect(catalog.status).toBe(200); + expect(catalog.body.sessions).toEqual([]); + expect( + setup.registry.getManagedByWorkspaceCwd(setup.root.canonicalRoot), + ).toBe(setup.liveRuntime); + } finally { + await restoreLiveSettings(); + } + }); + + it('classifies post-publication root revalidation failure as terminal', async () => { + const restoreLiveSettings = await disableLiveVoiceAtBoot(); + const setup = setupLiveRuntime(); + vi.mocked(setup.conversationWorkspace.revalidate) + .mockResolvedValueOnce(setup.root) + .mockRejectedValueOnce(new Error('/private/root changed')); + const rootSelector = encodeURIComponent(setup.root.configuredRoot); + try { + const catalogPromise = request(setup.app) + .get(`/workspaces/${rootSelector}/sessions?sourceType=default`) + .set('Host', `127.0.0.1:${baseOpts.port}`) + .then((response) => response); + await vi.waitFor(() => { + expect(setup.createWorkspaceRuntime).toHaveBeenCalledOnce(); + }); + setup.resolveCreation(); + + const catalog = await catalogPromise; + expect(catalog.status).toBe(503); + expect(catalog.body).toEqual({ + error: 'The Conversations root could not be verified.', + code: 'conversation_root_compromised', + retryable: false, + }); + expect(JSON.stringify(catalog.body)).not.toContain('/private/root'); + expect( + setup.registry.getManagedByWorkspaceCwd(setup.root.canonicalRoot), + ).toBeUndefined(); + } finally { + await restoreLiveSettings(); + } + }); + + it('serializes catalog boot failure and allows an explicit retry', async () => { + const restoreLiveSettings = await disableLiveVoiceAtBoot(); + const setup = setupLiveRuntime(); + const onConversationRuntimeReady = vi.fn(); + setup.app.locals['onConversationRuntimeReady'] = onConversationRuntimeReady; + const route = `/workspaces/${encodeURIComponent( + setup.root.configuredRoot, + )}/sessions?sourceType=default`; + try { + const failedPromise = request(setup.app) + .get(route) + .set('Host', `127.0.0.1:${baseOpts.port}`) + .then((response) => response); + await vi.waitFor(() => { + expect(setup.createWorkspaceRuntime).toHaveBeenCalledOnce(); + }); + setup.rejectCreation(new Error('/private/runtime publication failed')); + + const failed = await failedPromise; + expect(failed.status).toBe(503); + expect(failed.body).toEqual({ + error: 'The Conversations runtime is temporarily unavailable.', + code: 'conversation_runtime_unavailable', + retryable: true, + }); + expect(JSON.stringify(failed.body)).not.toContain('/private/runtime'); + expect(onConversationRuntimeReady).not.toHaveBeenCalled(); + + const retryPromise = request(setup.app) + .get(route) + .set('Host', `127.0.0.1:${baseOpts.port}`) + .then((response) => response); + await vi.waitFor(() => { + expect(setup.createWorkspaceRuntime).toHaveBeenCalledTimes(2); + }); + setup.resolveCreation(); + await expect(retryPromise).resolves.toMatchObject({ status: 200 }); + expect(onConversationRuntimeReady).toHaveBeenCalledOnce(); + } finally { + await restoreLiveSettings(); + } + }); + + it('boots an exact Conversations restore target before source proof', async () => { + const restoreLiveSettings = await disableLiveVoiceAtBoot(); + const setup = setupLiveRuntime(); + const getLocation = vi + .spyOn(SessionService.prototype, 'getSessionLocation') + .mockResolvedValue(undefined); + mockWt.realpath = (candidate) => candidate; + try { + const restorePromise = request(setup.app) + .post('/session/live-cold-session/load') + .set('Host', `127.0.0.1:${baseOpts.port}`) + .send({ cwd: setup.root.configuredRoot }) + .then((response) => response); + await vi.waitFor(() => { + expect(setup.createWorkspaceRuntime).toHaveBeenCalledOnce(); + }); + setup.resolveCreation(); + + const response = await restorePromise; + expect(response.status).toBe(404); + expect(response.body.code).toBe('session_not_found'); + expect(setup.liveBridge.loadCalls).toHaveLength(0); + expect(getLocation).toHaveBeenCalled(); + } finally { + mockWt.realpath = undefined; + getLocation.mockRestore(); + await restoreLiveSettings(); + } + }); + + it('keeps non-catalog routes hidden after the Live runtime is active', async () => { + const restoreLiveSettings = await disableLiveVoiceAtBoot(); + const setup = setupLiveRuntime(); + setup.registry.add(setup.liveRuntime); + try { + const catalog = await request(setup.app) + .get('/workspaces/live-conversations/sessions?sourceType=default') + .set('Host', `127.0.0.1:${baseOpts.port}`); + expect(catalog.status).toBe(200); + expect(setup.createWorkspaceRuntime).not.toHaveBeenCalled(); + + const unfiltered = await request(setup.app) + .get('/workspaces/live-conversations/sessions') + .set('Host', `127.0.0.1:${baseOpts.port}`); + expect(unfiltered.status).toBe(400); + + const aggregate = await request(setup.app) + .get('/workspaces/live-conversations/session-info') + .set('Host', `127.0.0.1:${baseOpts.port}`); + expect(aggregate.status).toBe(400); + } finally { + await restoreLiveSettings(); + } + }); + + it('allows exact organization updates only after persisted Live source proof', async () => { + const setup = setupLiveRuntime(); + setup.registry.add(setup.liveRuntime); + const sessionId = 'live-persisted-organization'; + const getLocation = vi + .spyOn(SessionService.prototype, 'getSessionLocation') + .mockResolvedValue('active'); + const sessionExists = vi + .spyOn(SessionService.prototype, 'sessionExistsInAnyState') + .mockImplementation(async function (candidateId) { + if (candidateId === 'ordinary-session') { + return this.getProjectRoot() === '/work/live-primary'; + } + return ( + this.getProjectRoot() === setup.root.canonicalRoot && + candidateId !== 'missing-live-session' + ); + }); + const readMetadata = vi + .spyOn(SessionService.prototype, 'readCreationMetadata') + .mockResolvedValue({ + sourceType: 'default', + sourceId: `realtime_voice:p1:h1:a1:${sessionId}`, + }); + const updateOrganization = vi + .spyOn( + qwenCore.SessionOrganizationService.prototype, + 'updateSessionOrganization', + ) + .mockResolvedValue({ + isPinned: true, + groupId: null, + color: null, + }); + try { + const updated = await request(setup.app) + .patch( + `/workspaces/${setup.liveRuntime.workspaceId}/session/${sessionId}/organization`, + ) + .set('Host', `127.0.0.1:${baseOpts.port}`) + .send({ isPinned: true }); + expect(updated.status).toBe(200); + expect(updated.body).toMatchObject({ sessionId, isPinned: true }); + expect(updateOrganization).toHaveBeenCalledOnce(); + + readMetadata.mockImplementation(async (candidateId) => + candidateId === 'not-live' + ? { sourceType: 'channel' } + : { + sourceType: 'default', + sourceId: `realtime_voice:p1:h1:a1:${candidateId}`, + }, + ); + const rejectedBatch = await request(setup.app) + .post(`/workspaces/${setup.liveRuntime.workspaceId}/sessions/delete`) + .set('Host', `127.0.0.1:${baseOpts.port}`) + .send({ sessionIds: [sessionId, 'not-live'] }); + expect(rejectedBatch.status).toBe(404); + expect(setup.liveBridge.closeCalls).toHaveLength(0); + + for (const rejectedSessionId of ['not-live', 'missing-live-session']) { + const rejectedLegacyBatch = await request(setup.app) + .post('/sessions/delete') + .set('Host', `127.0.0.1:${baseOpts.port}`) + .send({ sessionIds: [sessionId, rejectedSessionId] }); + expect(rejectedLegacyBatch.status).toBe(404); + expect(rejectedLegacyBatch.body).toMatchObject({ + code: 'session_not_found', + sessionId: rejectedSessionId, + }); + } + expect(setup.liveBridge.closeCalls).toHaveLength(0); + + for (const sessionIds of [ + [sessionId, 'ordinary-session'], + ['ordinary-session', sessionId], + ]) { + const rejectedMixedBatch = await request(setup.app) + .post('/sessions/delete') + .set('Host', `127.0.0.1:${baseOpts.port}`) + .send({ sessionIds }); + expect(rejectedMixedBatch.status).toBe(409); + expect(rejectedMixedBatch.body).toMatchObject({ + code: 'session_workspace_conflict', + }); + } + expect(setup.liveBridge.closeCalls).toHaveLength(0); + + readMetadata.mockResolvedValue({ sourceType: 'channel' }); + const rejected = await request(setup.app) + .patch( + `/workspaces/${setup.liveRuntime.workspaceId}/session/not-live/organization`, + ) + .set('Host', `127.0.0.1:${baseOpts.port}`) + .send({ isPinned: true }); + expect(rejected.status).toBe(404); + expect(updateOrganization).toHaveBeenCalledOnce(); + + const entry = setup.registry.getManagedEntryByWorkspaceId( + setup.liveRuntime.workspaceId, + ); + expect(entry).toBeDefined(); + setup.registry.beginReplacement(entry!, 'policy-2'); + const unavailable = await request(setup.app) + .patch( + `/workspaces/${setup.liveRuntime.workspaceId}/session/${sessionId}/organization`, + ) + .set('Host', `127.0.0.1:${baseOpts.port}`) + .send({ isPinned: true }); + expect(unavailable.status).toBe(503); + expect(unavailable.body.code).toBe('workspace_runtime_unavailable'); + expect(updateOrganization).toHaveBeenCalledOnce(); + } finally { + getLocation.mockRestore(); + sessionExists.mockRestore(); + readMetadata.mockRestore(); + updateOrganization.mockRestore(); + } + }); + it('shares the boot publication with a concurrent Live Host bind', async () => { const restoreLiveSettings = await enableLiveVoiceAtBoot(); const setup = setupLiveRuntime(); @@ -29755,8 +30287,21 @@ describe('Live conversation runtime lifecycle', () => { await vi.waitFor(() => { expect(setup.createWorkspaceRuntime).toHaveBeenCalledOnce(); }); + let capabilitiesSettled = false; + const capabilitiesDuringBoot = request(setup.app) + .get('/capabilities') + .set('Host', `127.0.0.1:${baseOpts.port}`) + .then((response) => { + capabilitiesSettled = true; + return response; + }); + await new Promise((resolve) => setImmediate(resolve)); + expect(capabilitiesSettled).toBe(false); setup.resolveCreation(); await enabling; + await expect(capabilitiesDuringBoot).resolves.toMatchObject({ + status: 200, + }); expect(setup.app.locals['liveVoiceEnabled']).toBe(true); const enabledCapabilities = await request(setup.app) @@ -29812,6 +30357,9 @@ describe('Live conversation runtime lifecycle', () => { } : {}; }); + const getLocation = vi + .spyOn(SessionService.prototype, 'getSessionLocation') + .mockResolvedValue('active'); try { const rejectedNew = await request(setup.app) .post('/session') @@ -29821,6 +30369,17 @@ describe('Live conversation runtime lifecycle', () => { expect(rejectedNew.body.code).toBe('live_session_creation_reserved'); expect(setup.liveBridge.calls).toHaveLength(0); + const rejectedDotPrefixedChild = await request(setup.app) + .post('/session') + .set('Host', `127.0.0.1:${baseOpts.port}`) + .send({ cwd: `${setup.root.canonicalRoot}/..hidden` }); + expect(rejectedDotPrefixedChild.status).toBe(400); + expect(rejectedDotPrefixedChild.body.code).toBe( + 'live_session_creation_reserved', + ); + expect(setup.primaryBridge.calls).toHaveLength(0); + expect(setup.liveBridge.calls).toHaveLength(0); + const standaloneRestore = await request(setup.app) .post('/session/generic-session/load') .set('Host', `127.0.0.1:${baseOpts.port}`) @@ -29878,6 +30437,7 @@ describe('Live conversation runtime lifecycle', () => { expect(setup.liveBridge.loadCalls).toHaveLength(5); expect(setup.liveBridge.resumeCalls).toHaveLength(1); } finally { + getLocation.mockRestore(); readCreationMetadata.mockRestore(); await ( setup.app.locals['sealAndWaitLiveCoordinator'] as () => Promise @@ -29885,6 +30445,87 @@ describe('Live conversation runtime lifecycle', () => { } }); + it('rejects canonical aliases of the configured Live root before publication', async () => { + const tmp = await fsp.mkdtemp( + path.join(os.tmpdir(), 'qwen-live-reserved-root-'), + ); + const realHome = path.join(tmp, 'real-home'); + const linkedHome = path.join(tmp, 'linked-home'); + const alternateRootAlias = path.join(tmp, 'alternate-root-alias'); + const relativeRoot = path.join('Documents', 'Qwen Code', 'Conversations'); + const realRoot = path.join(realHome, relativeRoot); + const realChild = path.join(realRoot, 'conversation-probe'); + await fsp.mkdir(realChild, { recursive: true }); + await fsp.symlink( + realHome, + linkedHome, + process.platform === 'win32' ? 'junction' : 'dir', + ); + await fsp.symlink( + realRoot, + alternateRootAlias, + process.platform === 'win32' ? 'junction' : 'dir', + ); + const setup = setupLiveRuntime( + {}, + { + configuredRoot: path.join(linkedHome, relativeRoot), + canonicalRoot: realpathSync(realRoot), + }, + ); + try { + for (const cwd of [ + realpathSync(realRoot), + realpathSync(realChild), + path.join(alternateRootAlias, 'new-conversation'), + ]) { + const response = await request(setup.app) + .post('/session') + .set('Host', `127.0.0.1:${baseOpts.port}`) + .send({ cwd }); + + expect(response.status).toBe(400); + expect(response.body.code).toBe('live_session_creation_reserved'); + } + expect(setup.primaryBridge.calls).toHaveLength(0); + expect(setup.liveBridge.calls).toHaveLength(0); + } finally { + await fsp.rm(tmp, { recursive: true, force: true }); + } + }); + + it.skipIf(process.platform === 'win32')( + 'keeps ordinary creation available when the Live root cannot be inspected', + async () => { + const tmp = await fsp.mkdtemp( + path.join(os.tmpdir(), 'qwen-live-unreadable-root-'), + ); + const ordinaryCwd = path.join(tmp, 'ordinary-workspace'); + const configuredRoot = path.join(tmp, 'looped-conversations'); + await fsp.mkdir(ordinaryCwd, { recursive: true }); + await fsp.symlink(configuredRoot, configuredRoot); + const canonicalOrdinaryCwd = realpathSync(ordinaryCwd); + const setup = setupLiveRuntime( + {}, + { configuredRoot, canonicalRoot: configuredRoot }, + ); + try { + const response = await request(setup.app) + .post('/session') + .set('Host', `127.0.0.1:${baseOpts.port}`) + .send({ cwd: canonicalOrdinaryCwd }); + + expect(response.status).toBe(200); + expect(setup.primaryBridge.calls).toEqual([ + expect.objectContaining({ workspaceCwd: canonicalOrdinaryCwd }), + ]); + expect(setup.liveBridge.calls).toHaveLength(0); + } finally { + await fsp.rm(tmp, { recursive: true, force: true }); + } + }, + ); + it('retries a failed boot publication when the Host connects', async () => { const restoreLiveSettings = await enableLiveVoiceAtBoot(); const setup = setupLiveRuntime(); @@ -29910,9 +30551,9 @@ describe('Live conversation runtime lifecycle', () => { setup.resolveCreation(); await vi.waitFor(() => { - expect(setup.registry.getByWorkspaceCwd(setup.root.canonicalRoot)).toBe( - setup.liveRuntime, - ); + expect( + setup.registry.getManagedByWorkspaceCwd(setup.root.canonicalRoot), + ).toBe(setup.liveRuntime); expect(setup.liveBridge.workspaceToolsCalls).toBe(0); expect(setup.liveBridge.liveScreenContextHandler).toEqual( expect.any(Function), @@ -29959,9 +30600,9 @@ describe('Live conversation runtime lifecycle', () => { `${failedChannel} handler bind failed`, ); - expect(setup.registry.getByWorkspaceCwd(setup.root.canonicalRoot)).toBe( - setup.liveRuntime, - ); + expect( + setup.registry.getManagedByWorkspaceCwd(setup.root.canonicalRoot), + ).toBe(setup.liveRuntime); expect(setup.liveBridge.liveScreenContextHandler).toBeUndefined(); expect(setup.liveBridge.liveTaskToolRequestHandler).toBeUndefined(); expect(setup.liveBridge.liveSpeakToUserHandler).toBeUndefined(); @@ -30015,9 +30656,9 @@ describe('Live conversation runtime lifecycle', () => { setup.resolveCreation(); await sealed; expect(settled).toBe(true); - expect(setup.registry.getByWorkspaceCwd(setup.root.canonicalRoot)).toBe( - setup.liveRuntime, - ); + expect( + setup.registry.getManagedByWorkspaceCwd(setup.root.canonicalRoot), + ).toBe(setup.liveRuntime); } finally { await restoreLiveSettings(); } @@ -30421,7 +31062,7 @@ describe('Live Appshot server integration', () => { } }); - it('binds the dedicated Appshot channel after Host hello and gates start until ready', async () => { + it('binds the dedicated Appshot channel after Host hello and waits to start until ready', async () => { const channelGate = deferred(); const setup = await setupAppshotProbe({ beforeRevalidate: () => channelGate.promise, @@ -30438,26 +31079,29 @@ describe('Live Appshot server integration', () => { blocker: 'appshot', requirements: { appshot: 'checking' }, }); - const blocked = await request(setup.app) + let startSettled = false; + const started = request(setup.app) .post('/live/start') .set('Host', `127.0.0.1:${baseOpts.port}`) - .send({}); - expect(blocked.status).toBe(503); - expect(blocked.body).toMatchObject({ - code: 'live_unavailable', - status: { - blocker: 'appshot', - requirements: { appshot: 'checking' }, - }, - }); + .send({}) + .then((response) => { + startSettled = true; + return response; + }); + await new Promise((resolve) => setImmediate(resolve)); + expect(startSettled).toBe(false); channelGate.resolve(undefined); + expect(await started).toMatchObject({ + status: 200, + body: { state: 'starting' }, + }); await vi.waitFor(() => { expect(setup.captureHandler).toEqual(expect.any(Function)); expect(setup.speakHandler).toEqual(expect.any(Function)); expect(setup.coordinator.getStatus()).toMatchObject({ available: true, - state: 'idle', + state: 'starting', requirements: { appshot: 'ready' }, }); }); diff --git a/packages/cli/src/serve/server.ts b/packages/cli/src/serve/server.ts index ad57f33b1c5..47caf4e124f 100644 --- a/packages/cli/src/serve/server.ts +++ b/packages/cli/src/serve/server.ts @@ -6,6 +6,7 @@ import express from 'express'; import type { Application } from 'express'; +import * as path from 'node:path'; import type { DaemonStatusProvider } from '@qwen-code/acp-bridge'; import { hashDaemonWorkspace, @@ -196,6 +197,7 @@ import { type WorkspaceRuntime, type WorkspaceRuntimeEnvMetadata, } from './workspace-registry.js'; +import { isInternalWorkspaceRuntime } from './workspace-runtime-visibility.js'; import { createWorkspaceRuntimeSessionService, runWithWorkspaceRuntimeStorage, @@ -277,7 +279,18 @@ import { LiveSessionCoordinator } from './live/live-session-coordinator.js'; import { LiveSetupController } from './live/live-setup-controller.js'; import { LiveTaskService } from './live/live-task-service.js'; import type { ConversationWorkspace } from './conversations/conversation-workspace.js'; +import { ConversationRuntimeActivityGate } from './conversations/conversation-runtime-activity.js'; +import { conversationRootCompromisedError } from './conversations/conversation-runtime-errors.js'; import { ConversationRuntimeManager } from './conversations/conversation-runtime-manager.js'; +import { + createConversationRuntimeOwnership, + type ConversationRuntimeOwnership, +} from './conversations/conversation-runtime-ownership.js'; +import { getStableLiveDiscoveryBaseDir } from './live/discovery.js'; +import { + installServeAppLifecycle, + type ServeAppLifecycleController, +} from './serve-app-lifecycle.js'; import { LiveProviderConfigError, readLiveVoiceConfiguration, @@ -587,6 +600,13 @@ export interface ServeAppDeps { liveHostInstaller?: LiveHostInstaller; liveSessionCoordinator?: LiveSessionCoordinator; liveConversationWorkspace?: ConversationWorkspace; + liveDiscoveryStableBaseDir?: string; + conversationRuntimeOwnershipFactory?: ( + pid: number, + instanceNonce: string, + stableBaseDir: string, + ) => ConversationRuntimeOwnership; + serveAppLifecycle?: ServeAppLifecycleController; validateLiveProviderCredential?: ( credential: LiveProviderCredential, ) => Promise; @@ -706,6 +726,10 @@ export function createServeApp( ); } const app = express(); + const serveAppLifecycle = installServeAppLifecycle( + app, + deps.serveAppLifecycle, + ); // Forward `maxSessions` into the default-constructed bridge so // direct callers of `createServeApp` (tests, embeds) get the same // cap they configured via `ServeOptions`. Previously the default @@ -877,7 +901,7 @@ export function createServeApp( sessionArtifactsPersistenceAvailable: deps.sessionArtifactsPersistenceAvailable !== false, sessionGenerationAvailable: () => { - const runtimes = workspaceRegistry.list(); + const runtimes = workspaceRegistry.listAll(); return ( runtimes.length > 0 && runtimes.every( @@ -931,6 +955,7 @@ export function createServeApp( deps.workspaceRuntimeRemoval !== undefined && workspaceRegistry .listManaged() + .filter((runtime) => !isInternalWorkspaceRuntime(runtime)) .every((runtime) => isScratchRootCompatible( runtime.workspaceCwd, @@ -1025,11 +1050,21 @@ export function createServeApp( defaultBridgeForAdmission = bridge; } const archiveCoordinator = new SessionArchiveCoordinator(); + const conversationRuntimeActivity = deps.liveConversationWorkspace + ? new ConversationRuntimeActivityGate() + : undefined; ( app.locals as { sessionArchiveCoordinator?: SessionArchiveCoordinator; } ).sessionArchiveCoordinator = archiveCoordinator; + if (conversationRuntimeActivity) { + ( + app.locals as { + conversationRuntimeActivity?: ConversationRuntimeActivityGate; + } + ).conversationRuntimeActivity = conversationRuntimeActivity; + } const cleanupSession = (runtime: WorkspaceRuntime, sessionId: string) => runWithWorkspaceRuntimeStorage(runtime, () => @@ -1068,7 +1103,7 @@ export function createServeApp( .workspaceRegistry; if (!reg) return [bridge]; return reg - .list() + .listAll() .filter((rt) => rt.trusted) .map((rt) => rt.bridge); }, @@ -1183,7 +1218,7 @@ export function createServeApp( })), getBridgeWorkspaceId: (bridge) => workspaceRegistry - .listEntries() + .listAllEntries() .find((entry) => entry.current?.runtime.bridge === bridge)?.workspaceId, }); primaryTrustRegistry = workspaceRegistry; @@ -1275,6 +1310,25 @@ export function createServeApp( } }, }); + const conversationRuntimeOwnership = deps.liveConversationWorkspace + ? (deps.conversationRuntimeOwnershipFactory?.( + process.pid, + liveCoordinator.daemonInstanceNonce, + path.resolve( + deps.liveDiscoveryStableBaseDir ?? getStableLiveDiscoveryBaseDir(), + ), + ) ?? + createConversationRuntimeOwnership({ + pid: process.pid, + instanceNonce: liveCoordinator.daemonInstanceNonce, + stableBaseDir: path.resolve( + deps.liveDiscoveryStableBaseDir ?? getStableLiveDiscoveryBaseDir(), + ), + })) + : undefined; + if (conversationRuntimeOwnership) { + serveAppLifecycle.setOwnership(conversationRuntimeOwnership); + } liveCoordinator.setAppshotReadiness( liveVoiceEnabled && deps.liveConversationWorkspace ? { @@ -1288,6 +1342,7 @@ export function createServeApp( ); const conversationRuntimeManager = deps.liveConversationWorkspace ? new ConversationRuntimeManager({ + ownership: conversationRuntimeOwnership!, workspace: deps.liveConversationWorkspace, registry: workspaceRegistry, publishRuntime: async (canonicalRoot, validate) => { @@ -1296,7 +1351,11 @@ export function createServeApp( 'live-conversation', async (candidate) => { await validate(candidate); - await deps.liveConversationWorkspace!.revalidate(); + try { + await deps.liveConversationWorkspace!.revalidate(); + } catch (error) { + throw conversationRootCompromisedError(error); + } }, ); invalidateServeFeaturesCache(); @@ -1307,6 +1366,7 @@ export function createServeApp( let liveBoundRuntime: WorkspaceRuntime | undefined; let liveBindingPromise: Promise | undefined; let liveRuntimeBootPromise: Promise | undefined; + let liveRuntimeBootResult: WorkspaceRuntime | undefined; let liveAppshotChannelPromise: Promise | undefined; let liveCoordinatorSealed = false; const clearLiveRuntimeHandlers = (runtime: WorkspaceRuntime): void => { @@ -1361,6 +1421,7 @@ export function createServeApp( } if (liveBindingPromise) return liveBindingPromise; const pending = (async (): Promise => { + await serveAppLifecycle.awaitBootAdmission(); if (!conversationRuntimeManager) { throw new Error('Live conversation runtime is unavailable.'); } @@ -1369,6 +1430,16 @@ export function createServeApp( throw new Error('Live Voice is shutting down.'); } bindLiveRuntimeHandlers(runtime); + const notifyRuntimeReady = ( + app.locals as { + onConversationRuntimeReady?: () => void; + } + ).onConversationRuntimeReady; + if (notifyRuntimeReady) { + void Promise.resolve() + .then(notifyRuntimeReady) + .catch(() => undefined); + } return runtime; })().finally(() => { if (liveBindingPromise === pending) liveBindingPromise = undefined; @@ -1376,9 +1447,33 @@ export function createServeApp( liveBindingPromise = pending; return pending; }; + const startConversationRuntimeBoot = (): Promise => { + if (liveRuntimeBootPromise) return liveRuntimeBootPromise; + const pending = ensureLiveConversationRuntime() + .then((runtime) => { + liveRuntimeBootResult = runtime; + }) + .finally(() => { + if (liveRuntimeBootPromise === pending) { + liveRuntimeBootPromise = undefined; + } + }); + liveRuntimeBootPromise = pending; + return pending; + }; + if (liveVoiceEnabled) { + serveAppLifecycle.setBootStarter(startConversationRuntimeBoot); + } + const ensureConversationRuntimeWithLifecycle = async () => { + await serveAppLifecycle.startBoot(startConversationRuntimeBoot); + if (!liveRuntimeBootResult) { + throw new Error('Live conversation runtime is unavailable.'); + } + return liveRuntimeBootResult; + }; const verifyLiveAppshotChannel = (): Promise => { if (liveAppshotChannelPromise) return liveAppshotChannelPromise; - const pending = ensureLiveConversationRuntime() + const pending = ensureConversationRuntimeWithLifecycle() .then(() => { if (!liveCoordinatorSealed) { liveCoordinator.setAppshotReadiness({ state: 'ready' }); @@ -1402,7 +1497,7 @@ export function createServeApp( }; const liveTaskService = new LiveTaskService({ workspaceRegistry, - ensureConversationRuntime: ensureLiveConversationRuntime, + ensureConversationRuntime: ensureConversationRuntimeWithLifecycle, materializeConversationDirectory: async (sessionId) => { const conversationWorkspace = deps.liveConversationWorkspace; if (!conversationWorkspace) { @@ -1420,7 +1515,7 @@ export function createServeApp( deps.liveSessionCoordinator ?? new LiveSessionCoordinator({ host: liveCoordinator, - ensureConversationRuntime: ensureLiveConversationRuntime, + ensureConversationRuntime: ensureConversationRuntimeWithLifecycle, workspaceRegistry, getProviderCredential: resolveLiveCredential, materializeConversationDirectory: async (sessionId) => { @@ -1472,7 +1567,7 @@ export function createServeApp( } if (enabled === liveVoiceEnabled) return; if (enabled) { - await ensureLiveConversationRuntime(); + await ensureConversationRuntimeWithLifecycle(); liveVoiceEnabled = true; (app.locals as { liveVoiceEnabled?: boolean }).liveVoiceEnabled = true; liveCoordinator.setAppshotReadiness({ @@ -1710,8 +1805,11 @@ export function createServeApp( app.use( daemonTelemetryMiddleware((req) => { - const match = req.path.match(/^\/workspaces\/([^/]+)/); - const rawSelector = match?.[1]; + const pluralMatch = req.path.match(/^\/workspaces\/([^/]+)/); + const singularCatalogMatch = req.path.match( + /^\/workspace\/([^/]+)\/(?:sessions|session-info)(?:\/|$)/, + ); + const rawSelector = pluralMatch?.[1] ?? singularCatalogMatch?.[1]; if (rawSelector) { try { const selector = decodeURIComponent(rawSelector); @@ -1725,8 +1823,9 @@ export function createServeApp( if (runtime) return runtime.workspaceCwd; } } catch { - return primaryBoundWorkspace; + return undefined; } + return undefined; } return primaryBoundWorkspace; }, deps.recordDaemonRequest), @@ -1771,9 +1870,17 @@ export function createServeApp( getChildHeapPolicySnapshot: deps.getChildHeapPolicySnapshot, }); - if (liveVoiceEnabled) { + if (conversationRuntimeManager) { app.use('/capabilities', (_req, _res, next) => { - void (liveRuntimeBootPromise ?? Promise.resolve()).then(() => next()); + const boot = serveAppLifecycle.getBootPromise(); + if (!boot) { + next(); + return; + } + void boot.then( + () => next(), + () => next(), + ); }); } registerCapabilitiesRoutes(app, { @@ -1793,6 +1900,10 @@ export function createServeApp( if (liveVoiceSurfaceAvailable) { registerLiveRoutes(app, { coordinator: liveCoordinator, + ensureRuntimeReady: async () => { + await ensureConversationRuntimeWithLifecycle(); + await publishLiveVoiceEnabled(true); + }, mutate, ...(deps.persistSetting ? { @@ -2040,6 +2151,7 @@ export function createServeApp( safeBody, sendBridgeError, workspaceRegistry, + conversationRuntimeActivity, ...(primaryRuntimeTrustAuthoritative ? { isWorkspaceTrusted: isPrimaryWorkspaceTrusted } : {}), @@ -2145,19 +2257,13 @@ export function createServeApp( workspaceRegistrationStore: deps.workspaceRegistrationStore, getAcpHandle: () => acpHandleRef.current, runtimeRemoval: deps.workspaceRuntimeRemoval, + ...(deps.liveConversationWorkspace + ? { reservedWorkspaceRoots: [deps.liveConversationWorkspace.rootPath] } + : {}), }); ( app.locals as { workspaceManagementHandle?: WorkspaceManagementHandle } ).workspaceManagementHandle = workspaceManagementHandle; - if (liveVoiceEnabled) { - // Publish the daemon-owned runtime at boot so persisted Live sessions are - // visible in the Web Shell even before the native Host connects. Runtime - // construction is lazy with respect to ACP/Appshot/provider initialization; - // Host readiness remains the only path that starts those probes. - liveRuntimeBootPromise = ensureLiveConversationRuntime() - .then(() => undefined) - .catch(() => undefined); - } ( app.locals as { workspaceRuntimeRemoval?: WorkspaceRuntimeRemovalController; @@ -2317,6 +2423,9 @@ export function createServeApp( liveCoordinator.isActiveSession(sessionId), ...(liveConversationWorkspaceForRoutes ? { + ensureConversationRuntime: ensureConversationRuntimeWithLifecycle, + liveConversationRootPath: liveConversationWorkspaceForRoutes.rootPath, + conversationRuntimeActivity: conversationRuntimeActivity!, materializeLiveConversationDirectory: (sessionId: string) => liveConversationWorkspaceForRoutes.materializeConversationDirectory( sessionId, @@ -2398,6 +2507,7 @@ export function createServeApp( safeBody, parseAndValidateClientId: (req, res, runtime) => parseAndValidateWorkspaceClientId(req, res, runtime.bridge), + conversationRuntimeActivity, }); } registerWorkspaceChannelObservedContactRoutes(app, { @@ -2405,6 +2515,7 @@ export function createServeApp( workspaceRegistry, isWorkspaceTrusted: isPrimaryWorkspaceTrusted, captureGenerationAssertion: capturePrimaryGenerationAssertion, + conversationRuntimeActivity, }); registerWorkspaceLifecycleRoutes(app, { boundWorkspace: primaryBoundWorkspace, @@ -2502,6 +2613,7 @@ export function createServeApp( manageScheduledTaskSessions: deps.manageScheduledTaskSessions === true, channelDeliveryAuthorizations: deps.channelDeliveryAuthorizations, cleanupSession, + conversationRuntimeActivity, }); // Read-only token-usage dashboard (Daemon Status "统计" tab). Aggregate local @@ -2767,5 +2879,60 @@ export function createServeApp( setRateLimiter(app, rateLimiter); } + let appDrainComplete = false; + if (!deps.serveAppLifecycle) + serveAppLifecycle.setAppDrain(async () => { + if (appDrainComplete) return; + const pendingDrains = [ + workspaceManagementHandle.sealAndWait(), + ( + app.locals as { + sealAndWaitLiveCoordinator?: () => Promise; + } + ).sealAndWaitLiveCoordinator?.() ?? Promise.resolve(), + archiveCoordinator.sealMaintenanceAndWait(), + conversationRuntimeActivity?.sealAndWait() ?? Promise.resolve(), + ]; + const cleanupErrors: unknown[] = []; + const stopAppResource = (stop: (() => void) | undefined) => { + try { + stop?.(); + } catch (error) { + cleanupErrors.push(error); + } + }; + const locals = app.locals as { + stopScheduledTaskKeepalive?: () => void; + stopWorkspaceGitState?: () => void; + stopExtensionGenerationReconciler?: () => void; + }; + stopAppResource(locals.stopScheduledTaskKeepalive); + stopAppResource(locals.stopWorkspaceGitState); + stopAppResource(locals.stopExtensionGenerationReconciler); + stopAppResource(() => deviceFlowRegistry.dispose()); + stopAppResource(() => rateLimiter?.setDraining(true)); + stopAppResource(() => rateLimiter?.dispose()); + const drains = await Promise.allSettled(pendingDrains); + stopAppResource(() => acpHandleRef.current?.dispose()); + const bridgeDrains = await Promise.allSettled( + workspaceRegistry + .listManaged() + .map((runtime) => runtime.bridge.shutdown()), + ); + const errors = [ + ...cleanupErrors, + ...[...drains, ...bridgeDrains] + .filter( + (result): result is PromiseRejectedResult => + result.status === 'rejected', + ) + .map((result) => result.reason as unknown), + ]; + if (errors.length > 0) { + throw new AggregateError(errors, 'Serve app drain is incomplete.'); + } + appDrainComplete = true; + }); + return app; } diff --git a/packages/cli/src/serve/server/session-archive.test.ts b/packages/cli/src/serve/server/session-archive.test.ts index beaa275f2ec..5b489ca8f21 100644 --- a/packages/cli/src/serve/server/session-archive.test.ts +++ b/packages/cli/src/serve/server/session-archive.test.ts @@ -220,7 +220,7 @@ describe('SessionArchiveCoordinator', () => { expect(drained).toBe(true); }); - it('does not wait for shared transcript reads when sealed', async () => { + it('seals new shared maintenance and waits for admitted reads', async () => { const coordinator = new SessionArchiveCoordinator(); let finish!: () => void; const shared = coordinator.runSharedMany( @@ -231,9 +231,21 @@ describe('SessionArchiveCoordinator', () => { }), ); - await expect(coordinator.sealMaintenanceAndWait()).resolves.toBeUndefined(); + const drain = coordinator.sealMaintenanceAndWait(); + await expect( + coordinator.runSharedMany(['session-b'], async () => undefined), + ).rejects.toMatchObject({ code: 'daemon_draining' }); + let drained = false; + void drain.then(() => { + drained = true; + }); + await Promise.resolve(); + expect(drained).toBe(false); + finish(); await shared; + await drain; + expect(drained).toBe(true); }); }); diff --git a/packages/cli/src/serve/server/session-archive.ts b/packages/cli/src/serve/server/session-archive.ts index a580c3f52de..6dd59f3d1d1 100644 --- a/packages/cli/src/serve/server/session-archive.ts +++ b/packages/cli/src/serve/server/session-archive.ts @@ -121,6 +121,9 @@ export class SessionArchiveCoordinator { sessionIds: string[], fn: () => Promise, ): Promise { + if (this.maintenanceSealed) { + throw new DaemonDrainingError(); + } const uniqueSessionIds = [...new Set(sessionIds)]; for (const sessionId of uniqueSessionIds) { this.assertNotTransitioning(sessionId); @@ -128,6 +131,7 @@ export class SessionArchiveCoordinator { for (const sessionId of uniqueSessionIds) { this.shared.set(sessionId, (this.shared.get(sessionId) ?? 0) + 1); } + this.activeMaintenance++; try { return await fn(); } finally { @@ -139,6 +143,11 @@ export class SessionArchiveCoordinator { this.shared.set(sessionId, count); } } + this.activeMaintenance--; + if (this.activeMaintenance === 0) { + this.maintenanceDrain?.resolve(); + this.maintenanceDrain = undefined; + } } } } @@ -367,21 +376,31 @@ export async function deleteDaemonSessions(params: { service: SessionService; bridge: Pick; coordinator: SessionArchiveCoordinator; + coordinatorLockHeld?: boolean; onError?: (entry: { phase: DaemonDeleteErrorPhase; sessionId: string; error: string; }) => void; }): Promise { - const { sessionIds, service, bridge, coordinator, onError } = params; + const { + sessionIds, + service, + bridge, + coordinator, + coordinatorLockHeld = false, + onError, + } = params; const uniqueSessionIds = [...new Set(sessionIds)]; - for (const sessionId of uniqueSessionIds) { - coordinator.assertNotTransitioning(sessionId); + if (!coordinatorLockHeld) { + for (const sessionId of uniqueSessionIds) { + coordinator.assertNotTransitioning(sessionId); + } } const results = await Promise.all( uniqueSessionIds.map(async (sessionId) => { try { - return await coordinator.runExclusiveMany([sessionId], async () => { + const mutateSession = async () => { try { await bridge.closeSession(sessionId); } catch (error) { @@ -423,7 +442,10 @@ export async function deleteDaemonSessions(params: { }); } return result; - }); + }; + return await (coordinatorLockHeld + ? mutateSession() + : coordinator.runExclusiveMany([sessionId], mutateSession)); } catch (error) { if (error instanceof DaemonDrainingError) { throw error; @@ -596,16 +618,25 @@ export async function archiveDaemonSessions(params: { service: SessionService; bridge: Pick; coordinator: SessionArchiveCoordinator; + coordinatorLockHeld?: boolean; }): Promise { - const { sessionIds, service, bridge, coordinator } = params; + const { + sessionIds, + service, + bridge, + coordinator, + coordinatorLockHeld = false, + } = params; const uniqueSessionIds = [...new Set(sessionIds)]; - for (const sessionId of uniqueSessionIds) { - coordinator.assertNotTransitioning(sessionId); + if (!coordinatorLockHeld) { + for (const sessionId of uniqueSessionIds) { + coordinator.assertNotTransitioning(sessionId); + } } const results = await Promise.all( uniqueSessionIds.map(async (sessionId) => { try { - return await coordinator.runExclusiveMany([sessionId], async () => { + const mutateSession = async () => { try { await bridge.closeSession(sessionId, undefined, { requireAgentClose: true, @@ -701,7 +732,10 @@ export async function archiveDaemonSessions(params: { kind: mutation.value ?? 'notFound', mutationApplied: mutation.mutationApplied, }; - }); + }; + return await (coordinatorLockHeld + ? mutateSession() + : coordinator.runExclusiveMany([sessionId], mutateSession)); } catch (error) { if (error instanceof DaemonDrainingError) { throw error; @@ -745,16 +779,24 @@ export async function unarchiveDaemonSessions(params: { sessionIds: string[]; service: SessionService; coordinator: SessionArchiveCoordinator; + coordinatorLockHeld?: boolean; }): Promise { - const { sessionIds, service, coordinator } = params; + const { + sessionIds, + service, + coordinator, + coordinatorLockHeld = false, + } = params; const uniqueSessionIds = [...new Set(sessionIds)]; - for (const sessionId of uniqueSessionIds) { - coordinator.assertNotTransitioning(sessionId); + if (!coordinatorLockHeld) { + for (const sessionId of uniqueSessionIds) { + coordinator.assertNotTransitioning(sessionId); + } } const results = await Promise.all( uniqueSessionIds.map(async (sessionId) => { try { - return await coordinator.runExclusiveMany([sessionId], async () => { + const mutateSession = async () => { const initialLocation = await classifySessionLocation( service, sessionId, @@ -858,7 +900,10 @@ export async function unarchiveDaemonSessions(params: { mutationApplied: mutation.mutationApplied, maintenanceError: mutation.maintenanceError, }; - }); + }; + return await (coordinatorLockHeld + ? mutateSession() + : coordinator.runExclusiveMany([sessionId], mutateSession)); } catch (error) { if (error instanceof DaemonDrainingError) { throw error; diff --git a/packages/cli/src/serve/server/telemetry.test.ts b/packages/cli/src/serve/server/telemetry.test.ts index 2cd66082597..47353dcd6f1 100644 --- a/packages/cli/src/serve/server/telemetry.test.ts +++ b/packages/cli/src/serve/server/telemetry.test.ts @@ -812,7 +812,7 @@ describe('daemonTelemetryMiddleware — recordRequest seam', () => { }); describe('legacy session telemetry route catalog', () => { - it('contains 54 unique routes with the audited 47/7 attribution split', () => { + it('contains 54 unique routes with the audited 52/2 attribution split', () => { const keys = legacySessionTelemetryRoutes.map( ({ method, path }) => `${method} ${path}`, ); @@ -822,27 +822,19 @@ describe('legacy session telemetry route catalog', () => { legacySessionTelemetryRoutes.filter( ({ attribution }) => attribution === 'handler_resolved', ), - ).toHaveLength(47); + ).toHaveLength(52); expect( legacySessionTelemetryRoutes.filter( ({ attribution }) => attribution === 'pre_resolved', ), - ).toHaveLength(7); + ).toHaveLength(2); expect( legacySessionTelemetryRoutes .filter(({ attribution }) => attribution === 'pre_resolved') .map(({ method, path }) => `${method} ${path}`) .sort(), ).toEqual( - [ - 'GET /session/:id/export', - 'PATCH /session/:id/organization', - 'POST /permission/:requestId', - 'POST /session/:id/a2ui-action', - 'POST /sessions/archive', - 'POST /sessions/delete', - 'POST /sessions/unarchive', - ].sort(), + ['POST /permission/:requestId', 'POST /session/:id/a2ui-action'].sort(), ); for (const entry of legacySessionTelemetryRoutes) { expect(entry.route).toBe(`${entry.method} ${entry.path}`); diff --git a/packages/cli/src/serve/server/telemetry.ts b/packages/cli/src/serve/server/telemetry.ts index 6fa40f10441..4c3e0167f33 100644 --- a/packages/cli/src/serve/server/telemetry.ts +++ b/packages/cli/src/serve/server/telemetry.ts @@ -80,7 +80,7 @@ export const legacySessionTelemetryRoutes = [ { method: 'GET', path: '/session/:id/export', - attribution: 'pre_resolved', + attribution: 'handler_resolved', route: 'GET /session/:id/export', }, { @@ -218,19 +218,19 @@ export const legacySessionTelemetryRoutes = [ { method: 'POST', path: '/sessions/delete', - attribution: 'pre_resolved', + attribution: 'handler_resolved', route: 'POST /sessions/delete', }, { method: 'POST', path: '/sessions/archive', - attribution: 'pre_resolved', + attribution: 'handler_resolved', route: 'POST /sessions/archive', }, { method: 'POST', path: '/sessions/unarchive', - attribution: 'pre_resolved', + attribution: 'handler_resolved', route: 'POST /sessions/unarchive', }, { @@ -242,7 +242,7 @@ export const legacySessionTelemetryRoutes = [ { method: 'PATCH', path: '/session/:id/organization', - attribution: 'pre_resolved', + attribution: 'handler_resolved', route: 'PATCH /session/:id/organization', }, { @@ -672,7 +672,7 @@ export function resolveDaemonTelemetryRoute( } export function daemonTelemetryMiddleware( - resolveWorkspaceCwd: (req: Request) => string, + resolveWorkspaceCwd: (req: Request) => string | undefined, // Optional in-process sink for the Daemon Status dashboard's time-series // charts. Fed the same (durationMs, statusCode) already computed for OTel, // so it adds no extra measurement — just a second consumer. Only known @@ -700,7 +700,10 @@ export function daemonTelemetryMiddleware( let workspaceHash: string | undefined; if (route.attribution !== 'handler_resolved') { try { - workspaceHash = resolveWorkspaceHash(resolveWorkspaceCwd(req)); + const workspaceCwd = resolveWorkspaceCwd(req); + if (workspaceCwd !== undefined) { + workspaceHash = resolveWorkspaceHash(workspaceCwd); + } } catch { // Telemetry must not affect request handling. } diff --git a/packages/cli/src/serve/workspace-registry.test.ts b/packages/cli/src/serve/workspace-registry.test.ts index 5eded9f3577..1464db86ce5 100644 --- a/packages/cli/src/serve/workspace-registry.test.ts +++ b/packages/cli/src/serve/workspace-registry.test.ts @@ -171,6 +171,105 @@ describe('createSingleWorkspaceRegistry', () => { }); describe('createWorkspaceRegistry', () => { + it('hides internal runtimes from ordinary selectors and preserves owner routing', () => { + const primary = makeRuntime('/work/primary', { + workspaceId: 'ws-primary', + primary: true, + }); + const internal = makeRuntime('/work/conversations', { + workspaceId: 'ws-conversations', + provenance: 'live-conversation', + removable: false, + bridge: bridgeWithSummary((sessionId: string) => ({ + sessionId, + workspaceCwd: '/work/conversations', + })), + }); + const sessionOwnerIndex = createWorkspaceSessionOwnerIndex(); + sessionOwnerIndex.register('live-session', internal.workspaceCwd); + const registry = createWorkspaceRegistry([primary, internal], { + sessionOwnerIndex, + scanUnindexedOwners: false, + }); + + expect(registry.list()).toEqual([primary]); + expect(registry.listEntries()).toEqual([registry.primaryEntry]); + expect(registry.getByWorkspaceId(internal.workspaceId)).toBeUndefined(); + expect(registry.getByWorkspaceCwd(internal.workspaceCwd)).toBeUndefined(); + expect( + registry.getEntryByWorkspaceId(internal.workspaceId), + ).toBeUndefined(); + expect(registry.resolveWorkspaceCwd(internal.workspaceCwd)).toBeUndefined(); + expect(registry.listAll()).toEqual([primary, internal]); + expect(registry.getManagedByWorkspaceId(internal.workspaceId)).toBe( + internal, + ); + expect(registry.resolveLiveSessionOwner('live-session')).toEqual({ + kind: 'found', + runtime: internal, + }); + + const entry = registry.getManagedEntryByWorkspaceId(internal.workspaceId)!; + expect(registry.beginReplacement(entry, 'policy-2')).toBe(true); + expect(registry.resolveLiveSessionOwner('live-session')).toEqual({ + kind: 'unavailable', + }); + expect(sessionOwnerIndex.getWorkspaceCwds('live-session')).toEqual([ + internal.workspaceCwd, + ]); + expect(() => + registry.activateReplacement( + entry, + makeRuntime(internal.workspaceCwd, { + workspaceId: internal.workspaceId, + }), + 'policy-2', + ), + ).toThrow(/identity does not match/); + }); + + it('does not fall back to an ordinary duplicate while an indexed internal owner is unavailable', () => { + const sessionId = 'duplicate-session'; + const primary = makeRuntime('/work/primary', { + workspaceId: 'ws-primary', + primary: true, + bridge: bridgeWithSummary(() => ({ + sessionId, + workspaceCwd: '/work/primary', + })), + }); + const internal = makeRuntime('/work/conversations', { + workspaceId: 'ws-conversations', + provenance: 'live-conversation', + removable: false, + bridge: bridgeWithSummary(() => ({ + sessionId, + workspaceCwd: '/work/conversations', + })), + }); + const sessionOwnerIndex = createWorkspaceSessionOwnerIndex(); + sessionOwnerIndex.register(sessionId, primary.workspaceCwd); + sessionOwnerIndex.register(sessionId, internal.workspaceCwd); + const registry = createWorkspaceRegistry([primary, internal], { + sessionOwnerIndex, + scanUnindexedOwners: true, + }); + + expect( + registry.beginReplacement( + registry.getManagedEntryByWorkspaceId(internal.workspaceId)!, + 'policy-2', + ), + ).toBe(true); + expect(registry.resolveLiveSessionOwner(sessionId)).toEqual({ + kind: 'unavailable', + }); + expect(sessionOwnerIndex.getWorkspaceCwds(sessionId)).toEqual([ + primary.workspaceCwd, + internal.workspaceCwd, + ]); + }); + it('keeps runtime order frozen and uses the marked primary runtime', () => { const primary = makeRuntime('/work/primary', { workspaceId: 'ws-primary', diff --git a/packages/cli/src/serve/workspace-registry.ts b/packages/cli/src/serve/workspace-registry.ts index b4026c2765e..e65248fb96b 100644 --- a/packages/cli/src/serve/workspace-registry.ts +++ b/packages/cli/src/serve/workspace-registry.ts @@ -12,6 +12,7 @@ import type { ClientMcpSenderRegistry } from './acp-http/client-mcp-sender-regis import type { WorkspaceFileSystemFactory } from './fs/index.js'; import type { WorkspaceRuntimeProvenance } from './managed-scratch-workspace.js'; import type { DaemonWorkspaceService } from './workspace-service/types.js'; +import { isInternalWorkspaceRuntime } from './workspace-runtime-visibility.js'; export interface WorkspaceRuntimeEnvMetadata { readonly mode: 'parent-process' | 'runtime-overlay'; @@ -99,6 +100,7 @@ export interface WorkspaceEntry { displayName?: string; readonly primary: boolean; readonly removable: boolean; + readonly internal: boolean; registrationIds: readonly string[]; lastGenerationId: number; state: WorkspaceEntryState; @@ -111,6 +113,7 @@ export interface WorkspaceEntry { export type WorkspaceSessionOwnerResolution = | { readonly kind: 'found'; readonly runtime: WorkspaceRuntime } | { readonly kind: 'not_found' } + | { readonly kind: 'unavailable' } | { readonly kind: 'ambiguous'; readonly runtimes: readonly WorkspaceRuntime[]; @@ -141,8 +144,14 @@ export interface WorkspaceRegistry { readonly primaryEntry: WorkspaceEntry; list(): readonly WorkspaceRuntime[]; listEntries(): readonly WorkspaceEntry[]; + listAll(): readonly WorkspaceRuntime[]; + listAllEntries(): readonly WorkspaceEntry[]; getEntryByWorkspaceCwd(workspaceCwd: string): WorkspaceEntry | undefined; getEntryByWorkspaceId(workspaceId: string): WorkspaceEntry | undefined; + getManagedEntryByWorkspaceCwd( + workspaceCwd: string, + ): WorkspaceEntry | undefined; + getManagedEntryByWorkspaceId(workspaceId: string): WorkspaceEntry | undefined; beginReplacement(entry: WorkspaceEntry, configuredRevision: string): boolean; activateReplacement( entry: WorkspaceEntry, @@ -248,6 +257,7 @@ export function createWorkspaceRegistry( : {}), primary: runtime.primary, removable: runtime.removable === true, + internal: isInternalWorkspaceRuntime(runtime), registrationIds: Object.freeze([...(runtime.registrationIds ?? [])]), lastGenerationId: generationId, state: 'active', @@ -340,10 +350,21 @@ export function createWorkspaceRegistry( Object.freeze( entries.flatMap((entry) => { const runtime = activeRuntime(entry); - return runtime ? [runtime] : []; + return runtime && !entry.internal ? [runtime] : []; }), ) as readonly WorkspaceRuntime[], listEntries: () => + Object.freeze( + entries.filter((entry) => entry.state !== 'removed' && !entry.internal), + ) as readonly WorkspaceEntry[], + listAll: () => + Object.freeze( + entries.flatMap((entry) => { + const runtime = activeRuntime(entry); + return runtime ? [runtime] : []; + }), + ) as readonly WorkspaceRuntime[], + listAllEntries: () => Object.freeze( entries.filter((entry) => entry.state !== 'removed'), ) as readonly WorkspaceEntry[], @@ -353,14 +374,24 @@ export function createWorkspaceRegistry( entry.current?.runtime ? [entry.current.runtime] : [], ), ) as readonly WorkspaceRuntime[], - getEntryByWorkspaceCwd: (workspaceCwd) => byCwd.get(workspaceCwd), - getEntryByWorkspaceId: (workspaceId) => byId.get(workspaceId), + getEntryByWorkspaceCwd: (workspaceCwd) => { + const entry = byCwd.get(workspaceCwd); + return entry?.internal ? undefined : entry; + }, + getEntryByWorkspaceId: (workspaceId) => { + const entry = byId.get(workspaceId); + return entry?.internal ? undefined : entry; + }, + getManagedEntryByWorkspaceCwd: (workspaceCwd) => byCwd.get(workspaceCwd), + getManagedEntryByWorkspaceId: (workspaceId) => byId.get(workspaceId), beginReplacement: (entry, configuredRevision) => { if (entry.state === 'blocked') { entry.configuredRevision = configuredRevision; entry.state = 'transitioning'; entry.current?.guard.close(); - sessionOwnerIndex?.removeWorkspace(entry.workspaceCwd); + if (!entry.internal) { + sessionOwnerIndex?.removeWorkspace(entry.workspaceCwd); + } delete entry.applyError; return true; } @@ -368,7 +399,9 @@ export function createWorkspaceRegistry( entry.configuredRevision = configuredRevision; entry.state = 'transitioning'; entry.current.guard.close(); - sessionOwnerIndex?.removeWorkspace(entry.workspaceCwd); + if (!entry.internal) { + sessionOwnerIndex?.removeWorkspace(entry.workspaceCwd); + } delete entry.applyError; return true; }, @@ -381,7 +414,8 @@ export function createWorkspaceRegistry( if ( runtime.workspaceId !== entry.workspaceId || runtime.workspaceCwd !== entry.workspaceCwd || - runtime.primary !== entry.primary + runtime.primary !== entry.primary || + isInternalWorkspaceRuntime(runtime) !== entry.internal ) { throw new Error('Replacement runtime identity does not match entry.'); } @@ -420,8 +454,14 @@ export function createWorkspaceRegistry( entry.appliedRevision = null; entry.applyError = error; }, - getByWorkspaceCwd: (workspaceCwd) => activeRuntime(byCwd.get(workspaceCwd)), - getByWorkspaceId: (workspaceId) => activeRuntime(byId.get(workspaceId)), + getByWorkspaceCwd: (workspaceCwd) => { + const entry = byCwd.get(workspaceCwd); + return entry?.internal ? undefined : activeRuntime(entry); + }, + getByWorkspaceId: (workspaceId) => { + const entry = byId.get(workspaceId); + return entry?.internal ? undefined : activeRuntime(entry); + }, getManagedByWorkspaceCwd: (workspaceCwd) => byCwd.get(workspaceCwd)?.current?.runtime, getManagedByWorkspaceId: (workspaceId) => @@ -449,7 +489,10 @@ export function createWorkspaceRegistry( resolveWorkspaceCwd: (workspaceCwd) => workspaceCwd === undefined ? activeRuntime(primaryEntry) - : activeRuntime(byCwd.get(workspaceCwd)), + : (() => { + const entry = byCwd.get(workspaceCwd); + return entry?.internal ? undefined : activeRuntime(entry); + })(), add: (runtime) => { if (byCwd.has(runtime.workspaceCwd)) { throw new Error( @@ -482,7 +525,9 @@ export function createWorkspaceRegistry( const entry = entryForRuntime(runtime); if (!entry || runtime.primary || entry.state !== 'draining') return; entry.current?.guard.close(); - sessionOwnerIndex?.removeWorkspace(runtime.workspaceCwd); + if (!entry.internal) { + sessionOwnerIndex?.removeWorkspace(runtime.workspaceCwd); + } }, completeDrain: (runtime) => { const entry = entryForRuntime(runtime); @@ -499,6 +544,7 @@ export function createWorkspaceRegistry( const indexedCwds = sessionOwnerIndex?.getWorkspaceCwds(sessionId) ?? []; if (indexedCwds.length > 0) { const matches: WorkspaceRuntime[] = []; + let internalUnavailable = false; for (const workspaceCwd of indexedCwds) { const entry = byCwd.get(workspaceCwd); const runtime = entry?.current?.runtime; @@ -506,7 +552,10 @@ export function createWorkspaceRegistry( sessionOwnerIndex?.remove(sessionId, workspaceCwd); continue; } - if (entry.state !== 'active') continue; + if (entry.state !== 'active') { + internalUnavailable ||= entry.internal; + continue; + } try { runtime.bridge.getSessionSummary(sessionId); matches.push(runtime); @@ -518,6 +567,7 @@ export function createWorkspaceRegistry( throw err; } } + if (internalUnavailable) return { kind: 'unavailable' }; if (matches.length === 1) { return { kind: 'found', runtime: matches[0]! }; } diff --git a/packages/cli/src/serve/workspace-route-runtime.test.ts b/packages/cli/src/serve/workspace-route-runtime.test.ts index ac92e324ad8..cbbe2fab2e3 100644 --- a/packages/cli/src/serve/workspace-route-runtime.test.ts +++ b/packages/cli/src/serve/workspace-route-runtime.test.ts @@ -11,12 +11,15 @@ import type { Request, Response } from 'express'; import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import { createSingleWorkspaceRegistry, + createWorkspaceRegistry, type WorkspaceRuntime, } from './workspace-registry.js'; import { resolveContainedCwd, resolveContainedCwdOrFail, + resolveRegisteredWorkspaceRuntimeByPathSelector, resolveWorkspaceRuntimeFromParam, + resolveWorkspaceRuntimeWithLiveCompatibilityFromParam, } from './workspace-route-runtime.js'; function fakeReq(cwd?: unknown): Request { @@ -161,6 +164,45 @@ function makeResponse(): Response { } describe('resolveWorkspaceRuntimeFromParam', () => { + it.each(['ws-live', '/work/conversations'])( + 'treats internal selector %s as an ordinary workspace mismatch', + (selector) => { + const primary = makeRuntime(); + const internal = { + ...makeRuntime(), + workspaceId: 'ws-live', + workspaceCwd: '/work/conversations', + primary: false, + provenance: 'live-conversation' as const, + removable: false, + }; + const registry = createWorkspaceRegistry([primary, internal]); + const response = makeResponse(); + const json = vi.mocked(response.json); + + expect( + resolveWorkspaceRuntimeFromParam( + registry, + { params: { workspace: selector } } as unknown as Request, + response, + ), + ).toBeNull(); + expect( + resolveRegisteredWorkspaceRuntimeByPathSelector( + registry, + internal.workspaceCwd, + ), + ).toBeUndefined(); + expect(response.status).toHaveBeenCalledWith(400); + expect(JSON.stringify(json.mock.calls)).not.toContain( + internal.workspaceCwd, + ); + expect(JSON.stringify(json.mock.calls)).not.toContain( + internal.workspaceId, + ); + }, + ); + it('returns retryable unavailable for a registered transitioning workspace', () => { const registry = createSingleWorkspaceRegistry(makeRuntime()); registry.beginReplacement(registry.primaryEntry, 'policy-2'); @@ -201,3 +243,72 @@ describe('resolveWorkspaceRuntimeFromParam', () => { }); }); }); + +describe('resolveWorkspaceRuntimeWithLiveCompatibilityFromParam', () => { + function setup() { + const primary = makeRuntime(); + const internal = { + ...makeRuntime(), + workspaceId: 'ws-live', + workspaceCwd: '/work/conversations', + primary: false, + provenance: 'live-conversation' as const, + removable: false, + }; + return { + internal, + registry: createWorkspaceRegistry([primary, internal]), + }; + } + + it.each(['ws-live', '/work/conversations'])( + 'allows the exact internal selector %s only through the explicit seam', + (selector) => { + const { internal, registry } = setup(); + + expect( + resolveWorkspaceRuntimeWithLiveCompatibilityFromParam( + registry, + { params: { workspace: selector } } as unknown as Request, + makeResponse(), + ), + ).toBe(internal); + }, + ); + + it('does not allow a path alias for the internal runtime', () => { + const { registry } = setup(); + const response = makeResponse(); + + expect( + resolveWorkspaceRuntimeWithLiveCompatibilityFromParam( + registry, + { + params: { workspace: '/work/conversations/.' }, + } as unknown as Request, + response, + ), + ).toBeNull(); + expect(response.status).toHaveBeenCalledWith(400); + }); + + it('returns a sanitized unavailable response for inactive internal state', () => { + const { internal, registry } = setup(); + registry.beginDrain(internal); + const response = makeResponse(); + + expect( + resolveWorkspaceRuntimeWithLiveCompatibilityFromParam( + registry, + { params: { workspace: internal.workspaceId } } as unknown as Request, + response, + ), + ).toBeNull(); + expect(response.status).toHaveBeenCalledWith(503); + expect(response.json).toHaveBeenCalledWith({ + error: 'The Conversations runtime is temporarily unavailable.', + code: 'conversation_runtime_unavailable', + retryable: true, + }); + }); +}); diff --git a/packages/cli/src/serve/workspace-route-runtime.ts b/packages/cli/src/serve/workspace-route-runtime.ts index a815a5066e8..2cd5577c442 100644 --- a/packages/cli/src/serve/workspace-route-runtime.ts +++ b/packages/cli/src/serve/workspace-route-runtime.ts @@ -13,6 +13,7 @@ import type { WorkspaceRegistry, WorkspaceRuntime, } from './workspace-registry.js'; +import { isInternalWorkspaceRuntime } from './workspace-runtime-visibility.js'; export interface WorkspaceRouteContext { readonly runtime: WorkspaceRuntime; @@ -127,15 +128,18 @@ export function resolveManagedWorkspaceRuntimeByPathSelector( selector: string, ): WorkspaceRuntime | undefined { const exact = registry.getManagedByWorkspaceCwd(selector); - if (exact) return exact; + if (exact && !isInternalWorkspaceRuntime(exact)) return exact; if (path.isAbsolute(selector) && !isUncPath(selector)) { try { const canonicalSelector = canonicalizeWorkspace(selector); const canonicalMatch = registry.getManagedByWorkspaceCwd(canonicalSelector); - if (canonicalMatch) return canonicalMatch; + if (canonicalMatch && !isInternalWorkspaceRuntime(canonicalMatch)) { + return canonicalMatch; + } for (const runtime of registry.listManaged()) { + if (isInternalWorkspaceRuntime(runtime)) continue; if (canonicalizeWorkspace(runtime.workspaceCwd) === canonicalSelector) { return runtime; } @@ -150,8 +154,9 @@ export function resolveManagedWorkspaceRuntimeByPathSelector( .listManaged() .find( (runtime) => + !isInternalWorkspaceRuntime(runtime) && normalizePortableAbsolutePath(runtime.workspaceCwd) === - normalizedSelector, + normalizedSelector, ); } @@ -171,6 +176,43 @@ export function resolveWorkspaceRuntimeFromParam( return runtime; } +export function resolveWorkspaceRuntimeWithLiveCompatibilityFromParam( + registry: WorkspaceRegistry, + req: Request, + res: Response, + paramName = 'workspace', +): WorkspaceRuntime | null { + if ( + typeof registry.getManagedEntryByWorkspaceId !== 'function' || + typeof registry.getManagedEntryByWorkspaceCwd !== 'function' + ) { + return resolveWorkspaceRuntimeFromParam(registry, req, res, paramName); + } + const selector = req.params[paramName] ?? ''; + let entry = registry.getManagedEntryByWorkspaceId(selector); + if (!entry && isPortableAbsolutePath(selector)) { + entry = registry.getManagedEntryByWorkspaceCwd(selector); + } + if (!entry?.internal) { + return resolveWorkspaceRuntimeFromParam(registry, req, res, paramName); + } + const runtime = entry.state === 'active' ? entry.current?.runtime : undefined; + if (!runtime) { + sendConversationRuntimeUnavailable(res); + return null; + } + return runtime; +} + +export function sendConversationRuntimeUnavailable(res: Response): void { + res.set('Retry-After', '1'); + res.status(503).json({ + error: 'The Conversations runtime is temporarily unavailable.', + code: 'conversation_runtime_unavailable', + retryable: true, + }); +} + export function sendWorkspaceRuntimeUnavailable( res: Response, entry?: Pick, @@ -211,7 +253,7 @@ export function resolveManagedWorkspaceRuntimeFromParam( ): WorkspaceRuntime | null { const selector = req.params[paramName] ?? ''; const byId = registry.getManagedByWorkspaceId(selector); - if (byId) return byId; + if (byId && !isInternalWorkspaceRuntime(byId)) return byId; if (!isPortableAbsolutePath(selector)) { res.status(400).json({ diff --git a/packages/cli/src/serve/workspace-runtime-visibility.ts b/packages/cli/src/serve/workspace-runtime-visibility.ts new file mode 100644 index 00000000000..457e2eee59d --- /dev/null +++ b/packages/cli/src/serve/workspace-runtime-visibility.ts @@ -0,0 +1,13 @@ +/** + * @license + * Copyright 2026 Qwen Team + * SPDX-License-Identifier: Apache-2.0 + */ + +import type { WorkspaceRuntimeProvenance } from './managed-scratch-workspace.js'; + +export function isInternalWorkspaceRuntime(runtime: { + readonly provenance?: WorkspaceRuntimeProvenance; +}): boolean { + return runtime.provenance === 'live-conversation'; +} diff --git a/packages/cli/src/serve/workspace-trust-reconciler.test.ts b/packages/cli/src/serve/workspace-trust-reconciler.test.ts index f54606e4302..839aec6509e 100644 --- a/packages/cli/src/serve/workspace-trust-reconciler.test.ts +++ b/packages/cli/src/serve/workspace-trust-reconciler.test.ts @@ -78,10 +78,13 @@ describe('workspace trust reconciler', () => { await reconciler.reconcile(nextPolicy); - expect(registry.getByWorkspaceCwd(scratch.workspaceCwd)).toBe(scratch); + expect(registry.getManagedByWorkspaceCwd(scratch.workspaceCwd)).toBe( + scratch, + ); expect(scratch.trusted).toBe(true); expect( - registry.getEntryByWorkspaceCwd(scratch.workspaceCwd)?.appliedRevision, + registry.getManagedEntryByWorkspaceCwd(scratch.workspaceCwd) + ?.appliedRevision, ).toBe('two'); expect(buildRuntime).not.toHaveBeenCalled(); expect(drainRuntime).not.toHaveBeenCalled(); diff --git a/packages/cli/src/serve/workspace-trust-reconciler.ts b/packages/cli/src/serve/workspace-trust-reconciler.ts index b937d13cf9a..209b42a89b8 100644 --- a/packages/cli/src/serve/workspace-trust-reconciler.ts +++ b/packages/cli/src/serve/workspace-trust-reconciler.ts @@ -243,7 +243,7 @@ export function createWorkspaceTrustReconciler( snapshot: DaemonTrustPolicySnapshot, ): Promise => { const planned: PlannedReplacement[] = []; - for (const entry of options.registry.listEntries()) { + for (const entry of options.registry.listAllEntries()) { const current = entry.current; if ( entry.state === 'active' && diff --git a/packages/web-shell/client/App.test.tsx b/packages/web-shell/client/App.test.tsx index 8b788531569..01dc9bc33df 100644 --- a/packages/web-shell/client/App.test.tsx +++ b/packages/web-shell/client/App.test.tsx @@ -9182,7 +9182,7 @@ describe('App session callbacks', () => { ); }); - it('labels the Live composer workspace without exposing its backing name', async () => { + it('keeps the Live runtime out of the ordinary composer workspace selector', async () => { mockWorkspace.capabilities = { workspaces: [ { @@ -9205,16 +9205,9 @@ describe('App session callbacks', () => { renderApp(); await flush(); - expect( - testState.latestChatEditorProps?.workspaces?.find( - (entry) => entry.id === 'live', - ), - ).toMatchObject({ label: 'Live' }); - expect( - testState.latestChatEditorProps?.workspaces?.some( - (entry) => entry.label === 'Conversations', - ), - ).toBe(false); + expect(testState.latestChatEditorProps?.workspaces).toEqual([ + expect.objectContaining({ id: 'primary', cwd: '/tmp/project' }), + ]); }); it('keeps composer git status stable across an equivalent refresh', async () => { diff --git a/packages/web-shell/client/App.tsx b/packages/web-shell/client/App.tsx index f36aa91c833..c6e31a1c0e0 100644 --- a/packages/web-shell/client/App.tsx +++ b/packages/web-shell/client/App.tsx @@ -2134,6 +2134,16 @@ export function App({ } return capabilityWorkspaces; }, [lockedWorkspaceCapability, workspace.capabilities?.workspaces]); + const ordinaryWorkspaces = useMemo( + () => workspaces.filter((entry) => entry.kind !== 'live'), + [workspaces], + ); + const isKnownLiveWorkspaceCwd = useCallback( + (cwd: string | undefined) => + cwd !== undefined && + workspaces.some((entry) => entry.kind === 'live' && entry.cwd === cwd), + [workspaces], + ); const composerWorkspacesRef = useRef< | Array<{ id: string; @@ -2145,11 +2155,10 @@ export function App({ | undefined >(undefined); const nextComposerWorkspaces = !lockedWorkspaceCwd - ? workspaces.map((entry) => ({ + ? ordinaryWorkspaces.map((entry) => ({ id: entry.id, cwd: entry.cwd, - label: - entry.kind === 'live' ? t('sidebar.live') : workspaceLabel(entry), + label: workspaceLabel(entry), primary: entry.primary, trusted: entry.trusted, })) @@ -2172,14 +2181,14 @@ export function App({ composerWorkspacesRef.current = nextComposerWorkspaces; } const composerWorkspaces = composerWorkspacesRef.current; - const workspacesRef = useRef(workspaces); - workspacesRef.current = workspaces; + const workspacesRef = useRef(ordinaryWorkspaces); + workspacesRef.current = ordinaryWorkspaces; const visibleWorkspaces = useMemo( () => lockedWorkspaceCwd - ? workspaces.filter((entry) => entry.cwd === lockedWorkspaceCwd) - : workspaces, - [lockedWorkspaceCwd, workspaces], + ? ordinaryWorkspaces.filter((entry) => entry.cwd === lockedWorkspaceCwd) + : ordinaryWorkspaces, + [lockedWorkspaceCwd, ordinaryWorkspaces], ); const sessionActions = useActions(); const reloadTranscript = useCallback( @@ -2250,14 +2259,14 @@ export function App({ useEffect(() => { if (!workspace.capabilities || !selectedWorkspaceCwd) return; - const selected = workspaces.find( + const selected = ordinaryWorkspaces.find( (entry) => entry.cwd === selectedWorkspaceCwd, ); if (selected?.trusted) return; composerSourceVersionRef.current += 1; selectedWorkspaceCwdRef.current = undefined; setSelectedWorkspaceCwd(undefined); - }, [selectedWorkspaceCwd, workspace.capabilities, workspaces]); + }, [ordinaryWorkspaces, selectedWorkspaceCwd, workspace.capabilities]); // The workspace the chip's status was last fetched for. On a workspace switch // we clear the status immediately so the chip never shows the previous repo's // branch/dirty counts while the new fetch is in flight; same-workspace @@ -2369,20 +2378,20 @@ export function App({ ? connection.workspaceCwd : (lockedWorkspaceCwd ?? selectedWorkspaceCwd ?? - workspaces.find((entry) => entry.primary)?.cwd), + ordinaryWorkspaces.find((entry) => entry.primary)?.cwd), [ connection.sessionId, connection.workspaceCwd, lockedWorkspaceCwd, selectedWorkspaceCwd, - workspaces, + ordinaryWorkspaces, ], ); // Worktree sessions query git status with the worktree path (?cwd= // parameter); the chip prefers the live branch from that status, falling // back to the creation-time sessionWorktree.branch. useEffect(() => { - if (!activeWorkspaceCwd) { + if (!activeWorkspaceCwd || isKnownLiveWorkspaceCwd(activeWorkspaceCwd)) { gitStatusWorkspaceCwdRef.current = undefined; setSelectedWorkspaceGitStatus(undefined); return; @@ -2450,6 +2459,7 @@ export function App({ }, [ activeWorkspaceCwd, connection.gitBranch, + isKnownLiveWorkspaceCwd, workspace.client, sessionWorktree, ]); @@ -5386,18 +5396,21 @@ export function App({ currentModelRef.current || connectionRef.current.currentModel; const modeId = currentModeRef.current || connectionRef.current.currentMode; - const primaryWorkspaceCwd = workspaces.find( + const primaryWorkspaceCwd = ordinaryWorkspaces.find( (entry) => entry.primary, )?.cwd; const requestedWorkspaceCwd = selectedWorkspaceCwdRef.current; const acceptedWorkspaceCwd = requestedWorkspaceCwd - ? workspaces.find( + ? ordinaryWorkspaces.find( (entry) => entry.cwd === requestedWorkspaceCwd && entry.trusted === true, )?.cwd : undefined; const targetWorkspaceCwd = - lockedWorkspaceCwd ?? acceptedWorkspaceCwd ?? primaryWorkspaceCwd; + ordinaryWorkspaces.find((entry) => entry.cwd === lockedWorkspaceCwd) + ?.cwd ?? + acceptedWorkspaceCwd ?? + primaryWorkspaceCwd; const catalogWorkspaceCwd = targetWorkspaceCwd ?? workspace.workspaceCwd ?? @@ -5471,7 +5484,7 @@ export function App({ sessionActions, sessionCatalogController, workspace.workspaceCwd, - workspaces, + ordinaryWorkspaces, ]); const onSubmitBeforeRef = useRef(onSubmitBefore); onSubmitBeforeRef.current = onSubmitBefore; @@ -5482,7 +5495,8 @@ export function App({ return connectionRef.current.workspaceCwd; } return ( - lockedWorkspaceCwd ?? + workspacesRef.current.find((entry) => entry.cwd === lockedWorkspaceCwd) + ?.cwd ?? selectedWorkspaceCwdRef.current ?? workspacesRef.current.find((entry) => entry.primary)?.cwd ); @@ -5804,10 +5818,13 @@ export function App({ // The workspace the Changes dialog reads — the same active workspace the // git-status effect targets (computed once above), so the chip and the // dialog always target the same repo. - const gitDiffWorkspaceCwd = activeWorkspaceCwd; + const gitDiffWorkspaceCwd = isKnownLiveWorkspaceCwd(activeWorkspaceCwd) + ? undefined + : activeWorkspaceCwd; const gitModeEligible = Boolean( !connection.sessionId && - workspaces.find((entry) => entry.cwd === activeWorkspaceCwd)?.trusted && + ordinaryWorkspaces.find((entry) => entry.cwd === activeWorkspaceCwd) + ?.trusted && selectedWorkspaceGitStatus?.branch, ); useEffect(() => { @@ -5871,7 +5888,7 @@ export function App({ sessionId: connection.sessionId, workspaces: workspace.capabilities?.workspaces || lockedWorkspaceCapability - ? workspaces + ? ordinaryWorkspaces : undefined, }), [ @@ -5879,7 +5896,7 @@ export function App({ connection.sessionId, lockedWorkspaceCapability, workspace.capabilities, - workspaces, + ordinaryWorkspaces, ], ); const [voiceUserRevision, setVoiceUserRevision] = useState(0); @@ -11159,7 +11176,7 @@ export function App({

{t('sidebar.scratchOutcomeUnknown')}

    - {workspaces.map((entry) => ( + {ordinaryWorkspaces.map((entry) => (
  • {entry.cwd}
    • ))}
@@ -11687,7 +11704,7 @@ export function App({ workspaces={ lockedWorkspaceCwd ? visibleWorkspaces - : workspaces + : ordinaryWorkspaces } lockedWorkspace={lockedWorkspaceCapability} onCreateViaChat={() => { @@ -11889,7 +11906,7 @@ export function App({ voiceWorkspaces={ workspace.capabilities?.workspaces || lockedWorkspaceCapability - ? workspaces + ? ordinaryWorkspaces : undefined } sessionWorkflowEnabled={sessionWorkflowEnabled} @@ -12415,21 +12432,28 @@ export function App({ workspaces={composerWorkspaces} selectedWorkspaceCwd={ connection.sessionId - ? workspaces.find( + ? ordinaryWorkspaces.find( (entry) => - entry.cwd === connection.workspaceCwd, - )?.primary - ? undefined - : connection.workspaceCwd + entry.cwd === connection.workspaceCwd && + !entry.primary, + )?.cwd : selectedWorkspaceCwd } workspaceSelectionDisabled={false} atWorkspaceCwd={ - lockedWorkspaceCwd ?? + ordinaryWorkspaces.find( + (entry) => entry.cwd === lockedWorkspaceCwd, + )?.cwd ?? (connection.sessionId - ? connection.workspaceCwd + ? isKnownLiveWorkspaceCwd( + connection.workspaceCwd, + ) + ? undefined + : connection.workspaceCwd : (selectedWorkspaceCwd ?? - workspaces.find((entry) => entry.primary)?.cwd)) + ordinaryWorkspaces.find( + (entry) => entry.primary, + )?.cwd)) } onSelectWorkspace={handleSelectComposerWorkspace} scratchWorkspaceSupported={ diff --git a/packages/web-shell/client/components/sidebar/WebShellSidebar.tsx b/packages/web-shell/client/components/sidebar/WebShellSidebar.tsx index d0a1f50dacb..26a4cfcc552 100644 --- a/packages/web-shell/client/components/sidebar/WebShellSidebar.tsx +++ b/packages/web-shell/client/components/sidebar/WebShellSidebar.tsx @@ -4360,8 +4360,8 @@ export function WebShellSidebar({ noSessionsLabel={t('sidebar.noSessions')} loadErrorLabel={t('sidebar.loadFailed')} organizationEnabled={false} - sourceType={selectedSessionSource} - channelGroupingEnabled={channelGroupingEnabled} + sourceType={sourceMetadataEnabled ? 'default' : undefined} + channelGroupingEnabled={false} ungroupedLabel={t('sidebar.groupUngrouped')} formatTime={(iso) => formatRelativeTime(iso, t)} autoExpandKey={ diff --git a/packages/web-shell/client/components/sidebar/WebShellSidebar.workspace-removal.test.tsx b/packages/web-shell/client/components/sidebar/WebShellSidebar.workspace-removal.test.tsx index 418bafc5272..a8a929e7e7b 100644 --- a/packages/web-shell/client/components/sidebar/WebShellSidebar.workspace-removal.test.tsx +++ b/packages/web-shell/client/components/sidebar/WebShellSidebar.workspace-removal.test.tsx @@ -662,7 +662,14 @@ function useWorkspaceSessionCatalog( cwd: string, options?: { archiveState?: string; group?: string }, ) => Promise, -): void { +): { + workspaceChannelTypes: ReturnType; + workspaceChannels: ReturnType; +} { + const workspaceChannelTypes = vi.fn().mockResolvedValue([]); + const workspaceChannels = vi + .fn() + .mockResolvedValue({ revision: '0', instances: {} }); workspace.client.workspaceByCwd.mockImplementation((cwd: string) => ({ listWorkspaceSessions: (options?: { archiveState?: string; @@ -672,6 +679,8 @@ function useWorkspaceSessionCatalog( return resolve(cwd, options); }, listSessionGroups: vi.fn().mockResolvedValue({ groups: [] }), + workspaceChannelTypes, + workspaceChannels, archiveSessionsData, unarchiveSessionsData, deleteSessionsData, @@ -679,6 +688,7 @@ function useWorkspaceSessionCatalog( exportSession, exportArchivedSession, })); + return { workspaceChannelTypes, workspaceChannels }; } function openRemoval(cwd: string): void { @@ -4146,6 +4156,7 @@ describe('WebShellSidebar session list notices', () => { describe('WebShellSidebar Live group', () => { it('shows Live sessions without exposing the backing Conversations workspace', async () => { + enableChannelOrganization(); const liveWorkspace: DaemonWorkspaceCapability = { id: 'live', cwd: '/Users/test/Documents/Qwen Code/Conversations', @@ -4154,7 +4165,7 @@ describe('WebShellSidebar Live group', () => { trusted: true, kind: 'live', }; - useWorkspaceSessionCatalog(async (cwd) => + const channelCatalog = useWorkspaceSessionCatalog(async (cwd) => cwd === liveWorkspace.cwd ? [{ sessionId: 'live-session', displayName: 'Voice check' }] : [], @@ -4178,8 +4189,30 @@ describe('WebShellSidebar Live group', () => { expect(container.textContent).toContain('Voice check'); expect(listWorkspaceSessions).toHaveBeenCalledWith( liveWorkspace.cwd, - expect.objectContaining({ archiveState: 'active' }), + expect.objectContaining({ + archiveState: 'active', + sourceType: 'default', + }), + ); + + await switchSessionSource('Channels'); + expect( + listWorkspaceSessions.mock.calls.findLast( + ([cwd]) => cwd === liveWorkspace.cwd, + )?.[1], + ).toEqual( + expect.objectContaining({ + archiveState: 'active', + sourceType: 'default', + }), ); + await act(async () => { + await new Promise((resolve) => setTimeout(resolve, 0)); + }); + expect(channelCatalog.workspaceChannelTypes).not.toHaveBeenCalled(); + expect(channelCatalog.workspaceChannels).not.toHaveBeenCalled(); + expect(container.textContent).not.toContain('Other channels'); + expect(container.textContent).toContain('Voice check'); }); }); diff --git a/packages/web-shell/client/voice/voice-workspace-target.test.ts b/packages/web-shell/client/voice/voice-workspace-target.test.ts index 46afbc5b39b..a231094f0c8 100644 --- a/packages/web-shell/client/voice/voice-workspace-target.test.ts +++ b/packages/web-shell/client/voice/voice-workspace-target.test.ts @@ -141,6 +141,22 @@ describe('resolveVoiceWorkspaceTarget', () => { ).toBeUndefined(); }); + it('does not expose the internal Live workspace as a voice target', () => { + const live = { + ...secondary, + id: 'live', + cwd: '/repo/conversations', + kind: 'live' as const, + }; + + expect( + resolveVoiceWorkspaceTarget({ + capabilities: capabilities({ workspaces: [primary, live] }), + intendedCwd: live.cwd, + }), + ).toBeUndefined(); + }); + it('fails closed for unknown and ambiguous secondary targets', () => { const duplicate = { ...secondary, id: 'other' }; const caps = capabilities({ diff --git a/packages/web-shell/client/voice/voice-workspace-target.ts b/packages/web-shell/client/voice/voice-workspace-target.ts index e7a775e16f5..4d296d98dd9 100644 --- a/packages/web-shell/client/voice/voice-workspace-target.ts +++ b/packages/web-shell/client/voice/voice-workspace-target.ts @@ -144,6 +144,7 @@ export function resolveVoiceWorkspaceTarget({ const matches = registered.filter((workspace) => workspace.cwd === cwd); if (matches.length !== 1) return undefined; const workspace = matches[0]; + if (workspace.kind === 'live') return undefined; if (workspace.primary) return legacyTarget(workspace.cwd, sessionId); if (!workspace.trusted) return undefined;