Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
cccf0c2
feat(serve): support runtime workspace removal
doudouOUC Jul 11, 2026
3495a84
fix(cli): address workspace removal review feedback
doudouOUC Jul 12, 2026
a1ad766
test(cli): strengthen workspace removal regressions
doudouOUC Jul 12, 2026
30567ab
test(webui): fix timeout assertion lint
doudouOUC Jul 12, 2026
664ab87
fix(cli): address workspace removal review feedback
doudouOUC Jul 12, 2026
83a29fa
fix(cli): update workspace Git test registry
doudouOUC Jul 12, 2026
9a4fd6e
fix(daemon): address workspace removal review feedback
doudouOUC Jul 12, 2026
b31ef40
codex: address PR review feedback (#6745)
doudouOUC Jul 12, 2026
b7a906e
codex: address PR review feedback (#6745)
doudouOUC Jul 12, 2026
516708f
codex: address PR review feedback (#6745)
doudouOUC Jul 12, 2026
974d617
codex: address PR review feedback (#6745)
doudouOUC Jul 12, 2026
f99ad09
codex: address PR review feedback (#6745)
doudouOUC Jul 12, 2026
02db446
codex: address PR review feedback (#6745)
doudouOUC Jul 12, 2026
c423a51
codex: address PR review feedback (#6745)
doudouOUC Jul 13, 2026
c7ebe12
codex: address PR review feedback (#6745)
doudouOUC Jul 13, 2026
09ea8e4
codex: address PR review feedback (#6745)
doudouOUC Jul 13, 2026
ccb4022
codex: fix CI failure on PR #6745
doudouOUC Jul 13, 2026
5afb065
codex: address PR review feedback (#6745)
doudouOUC Jul 13, 2026
3423f16
codex: address PR review feedback (#6745)
doudouOUC Jul 13, 2026
d509ff7
test(web-shell): cover workspace removal after sidebar rebase
doudouOUC Jul 13, 2026
8d5c9c9
codex: address PR review feedback (#6745)
doudouOUC Jul 13, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions docs/design/2026-07-11-daemon-workspace-runtime-removal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Daemon Workspace Runtime Removal

## Context

Runtime workspace registration and persistent registration are already available, but forgetting a persistent registration does not unload the live bridge, ACP mount, session admission state, or memory lane. This design adds synchronous hot removal for secondary runtimes while preserving the existing registration-forget API.

## Scope and invariants

- Only dynamically registered and persistence-restored secondary runtimes are removable. The primary and every `--workspace` runtime are static.
- `DELETE /workspaces/:workspace` removes the runtime and all known persistent aliases. It never removes workspace files, settings, transcripts, archives, or other project data.
- Non-force removal is observational: if the frozen runtime has activity, every gate is rolled back and the request returns `409 workspace_busy`. Force removal terminates that activity.
- Persistence is committed before destructive cleanup. A store failure restores the active runtime. Cleanup failures after the store commit cannot roll the operation back and use synchronous bridge kill as a fallback.
- A removed cwd remains reserved until cleanup completes, then may be registered again with a fresh bridge, ACP dispatcher, connection registry, and memory lane.

## Protocol

Production daemons advertise `workspace_runtime_removal` when the removal controller is installed. Capability workspace rows add optional `removable`; old clients and daemons remain compatible.

`DELETE /workspaces/:workspace` uses the existing workspace-id-or-canonical-cwd selector and accepts an optional JSON body containing a boolean `force`. Success returns the removed identity, whether force was requested, whether any persistent alias was removed, and the final post-drain activity snapshot. A non-force request that is already observably busy may return an earlier pre-drain snapshot without briefly gating the runtime. Existing `DELETE /workspace-registrations/:id` remains forget-only.

## Lifecycle

The registry tracks active, draining, and removed runtimes. Public resolution sees only active runtimes; management resolution retains draining runtimes for conflict reporting and cwd reservation.

Removal first takes a fast activity snapshot. It then synchronously marks the registry draining, closes per-workspace session admission, and drains the ACP mount and memory lane. The final snapshot reads pending session reservations before live bridge counts so a reservation-to-session transition cannot appear idle. A busy non-force request reverses the gates. Otherwise all known registration IDs are deleted atomically, queued memory work is failed, the sub-session launcher and bridge are stopped, the ACP mount is disposed, ownership indexes are cleared, and the registry entry is completed.

Runtime cleanup is memoized by runtime identity, not cwd, so a later runtime registered at the same path cannot reuse an old cleanup promise. Daemon shutdown seals management operations, waits for them to converge, stops launchers, and then uses the same bridge teardown path for the remaining managed runtimes.

## Persistence identity

Restoration records the ID of each raw stored path before canonicalization. Multiple raw aliases that resolve to one runtime are retained as one ID set, including aliases shadowed by an explicit startup workspace. Removal deletes that set plus the canonical registration ID under one store lock without changing the schema.

## UI

The Web Shell exposes removal only when both the feature tag and `removable: true` are present. The action remains available for untrusted workspaces. The first confirmation performs a non-force request; `workspace_busy` renders the activity counts and offers force removal. Force is disabled when the current session belongs to the target workspace. Success reconciles capabilities and session lists and falls back to the primary workspace when necessary.

## Failure and compatibility analysis

Client disconnects and SDK timeouts do not cancel server-side cleanup. Concurrent add, persistence promotion, and remove operations are serialized per canonical cwd. Shutdown rejects new management operations with `daemon_shutting_down` and waits for already-started work. Old clients ignore the optional capability field and feature; old daemons continue to produce a normal `DaemonHttpError` for the missing route.

The workspace-scoped channel worker group supplies activity and teardown through a thin adapter. Draining blocks reload and webhook routing for the target workspace; committed removal stops and unregisters only that worker so daemon status and pidfile metadata converge without affecting other workspaces.

## Verification

Unit coverage targets registry state transitions and owner cleanup, admission drain rollback, alias batch deletion, busy/force/store-failure route behavior, bridge shutdown reason idempotence, memory-lane cancellation, SDK request encoding, and Web Shell feature and force guards. The E2E plan lives at `.qwen/e2e-tests/workspace-runtime-removal.md`.
31 changes: 30 additions & 1 deletion docs/developers/qwen-serve-protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,8 @@ registry. Clients **must** gate UI off `features`, not off `mode` (per design

`persistent_workspace_registration` advertises durable registration for workspaces added at runtime. `POST /workspaces` accepts `{ "cwd": "/absolute/path", "persist": true }`; success includes `persisted: true`. Registrations are scoped to the daemon's canonical primary workspace under the user's Qwen home and are restored on the next daemon start. Omitting `persist` preserves process-local registration. `GET /workspace-registrations` lists the stored desired set, and `DELETE /workspace-registrations/:id` forgets an entry for the next restart without hot-removing an active runtime.

`workspace_runtime_removal` advertises synchronous hot removal through `DELETE /workspaces/:workspace`. Capability workspace entries add optional `removable`; only rows with `removable: true` may be removed. Removal also forgets every persistent registration alias for the runtime, but never deletes files, settings, transcripts, or archives.

`session_load` and `session_resume` advertise the explicit-restore routes (`POST /session/:id/load` and `POST /session/:id/resume`). Older daemons return `404` for these paths, so SDK clients should pre-flight `caps.features` before calling. `unstable_session_resume` is still advertised as a deprecated alias for compatibility with SDKs that shipped while the underlying ACP method was named `connection.unstable_resumeSession`; new clients should gate on `session_resume`.

`session_transcript` advertises `GET /session/:id/transcript`, a read-only paged replay view over the persisted active-session JSONL. It is separate from `/load`: it does not attach a client, seed the live EventBus, create a live session, or change the live replay window. Clients should use it when they need the complete on-disk transcript for a long session, and continue using `/load` only for bounded live replay during cold UI restore.
Expand Down Expand Up @@ -581,7 +583,7 @@ Stable contract: when `v` increments the frame layout has changed in a backwards

> **`workspaceCwd`** is the canonical absolute path for the daemon's primary workspace. Use it to omit `cwd` on `POST /session` (the route falls back to this primary path) and to keep old single-workspace clients compatible. Additive to v=1: pre-§02 v=1 daemons omit the field — clients that target older builds should null-check before consuming it.

> **`workspaces[]`** is present only when `features` contains `multi_workspace_sessions`. Each entry is `{ id, cwd, primary, trusted }`. The first/primary workspace remains mirrored by `workspaceCwd`; new clients choose a non-primary runtime by passing that entry's `cwd` to `POST /session`. Untrusted workspaces are advertised for diagnostics but reject fresh session creation with `403 untrusted_workspace` until trust changes.
> **`workspaces[]`** is present only when `features` contains `multi_workspace_sessions`. Each entry is `{ id, cwd, primary, trusted, removable? }`. The first/primary workspace remains mirrored by `workspaceCwd`; new clients choose a non-primary runtime by passing that entry's `cwd` to `POST /session`. Untrusted workspaces are advertised for diagnostics but reject fresh session creation with `403 untrusted_workspace` until trust changes. `removable` is present on daemons that support runtime removal and is true only for process-dynamic or persistence-restored secondary runtimes.

The workspace feature tags and `workspaces[]` are dynamic. Clients that add a workspace must fetch `/capabilities` again after the mutation completes; the daemon does not broadcast capability changes to clients that cached an earlier response. Forgetting persistence does not unload an active runtime, so that runtime remains advertised until restart.

Expand All @@ -607,9 +609,36 @@ A newly created runtime returns `201`; promoting an already-active secondary wor

Errors include `400 invalid_path` / `invalid_persist_flag` / `invalid_persist_target`, `409 workspace_exists` / `workspace_nested` / `workspace_limit_reached`, `500 workspace_registration_store_error` / `runtime_creation_failed`, and `501 persistence_not_available` / `not_implemented`.

### `DELETE /workspaces/:workspace`

Remove one removable secondary runtime. The selector follows the plural workspace routing rules and accepts either a workspace ID or a URL-encoded absolute cwd. The optional JSON body is `{ "force": boolean }`; omitting it requests non-force removal.

Non-force removal returns `409 workspace_busy` with an `activity` snapshot when the frozen runtime has sessions, prompts, pending starts, ACP connections, memory tasks, or workspace channel workers. Sending `{ "force": true }` terminates those resources. A successful response is:

```json
{
"removed": true,
"workspaceId": "stable-workspace-id",
"workspaceCwd": "/canonical/path/to/secondary-workspace",
"forced": true,
"persistedRegistrationRemoved": true,
"activity": {
"sessions": 2,
"activePrompts": 1,
"pendingSessionStarts": 0,
"acpConnections": 1,
"memoryTasks": 0,
"channelWorkers": 0
}
}
```

An immediately busy non-force request returns a fast pre-drain activity snapshot. Once drain starts, the busy or success response contains the final snapshot taken after admission and ACP drain gates close and before cleanup begins. Errors include `400 invalid_force_flag` / `workspace_mismatch`, `409 workspace_busy` / `primary_workspace_removal_forbidden` / `static_workspace_removal_forbidden` / `workspace_removal_in_progress` / `workspace_registration_in_progress`, `500 workspace_persist_failed` / `workspace_runtime_removal_failed`, `501 workspace_runtime_removal_unsupported`, and `503 daemon_shutting_down`.

### `GET /workspace-registrations`

List the persisted desired workspace set for this primary workspace. Entries remain visible with `active: false` when a stored directory could not be restored during the current start.
An entry remains `active: true` while its runtime is draining because the runtime still owns live resources until removal completes.

```json
{
Expand Down
3 changes: 3 additions & 0 deletions integration-tests/cli/qwen-serve-routes.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,7 @@ beforeAll(async () => {
'QWEN_SERVE_WRITER_IDLE_TIMEOUT_MS',
'QWEN_SERVE_RATE_LIMIT',
'QWEN_SERVE_NO_MCP_POOL',
'QWEN_SERVE_NO_PERSISTENT_REGISTRATION',
'QWEN_SERVE_CLIENT_MCP_OVER_WS',
'QWEN_SERVE_CDP_TUNNEL_OVER_WS',
].includes(k),
Expand Down Expand Up @@ -373,6 +374,8 @@ describe('qwen serve — capabilities envelope', () => {
'session_branch',
'workspace_reload',
'channel_control',
'persistent_workspace_registration',
'workspace_runtime_removal',
'workspace_qualified_rest_core',
'workspace_persisted_transcript',
'voice_transcribe',
Expand Down
45 changes: 45 additions & 0 deletions packages/acp-bridge/src/bridge.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,51 @@ describe('createAcpSessionBridge', () => {
await bridge.shutdown();
});

it('memoizes shutdown and keeps the first workspace-removal reason', async () => {
const lifecycle: Array<{ type: string; reason?: string }> = [];
const bridge = makeBridge({
channelFactory: async () => makeChannel().channel,
sessionLifecycle: (event) => lifecycle.push(event),
});
const session = await bridge.spawnOrAttach({ workspaceCwd: WS_A });
const iterator = bridge
.subscribeEvents(session.sessionId)
[Symbol.asyncIterator]();
const terminalEvent = iterator.next();

const first = bridge.shutdown({ reason: 'workspace_removed' });
const second = bridge.shutdown({ reason: 'daemon_shutdown' });

expect(second).toBe(first);
await first;
await expect(terminalEvent).resolves.toMatchObject({
value: {
type: 'session_died',
data: { reason: 'workspace_removed' },
},
});
expect(lifecycle.at(-1)).toMatchObject({
type: 'removed',
reason: 'workspace_removed',
});
});

it('publishes the shutdown promise before lifecycle callbacks can re-enter', async () => {
let reentered: Promise<void> | undefined;
const bridge = makeBridge({
channelFactory: async () => makeChannel().channel,
sessionLifecycle: (event) => {
if (event.type === 'removed') reentered = bridge.shutdown();
},
});
await bridge.spawnOrAttach({ workspaceCwd: WS_A });

const first = bridge.shutdown({ reason: 'workspace_removed' });

expect(reentered).toBe(first);
await first;
});

it('accepts a valid BridgeOptions.eventRingSize at construction time', () => {
// Smoke: positive finite integers are accepted; the underlying
// EventBus ring-size threading is exercised end-to-end in
Expand Down
Loading
Loading