docs(workflow-creator): add headless/background stage documentation

.agents/skills/workflow-creator/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: workflow-creator
-description: Create multi-agent workflows for Atomic CLI using defineWorkflow().run().compile() with ctx.stage() for session orchestration across Claude, Copilot, and OpenCode SDKs. Use whenever the user wants to create, edit, or debug workflows, build agent pipelines, define multi-stage automations, set up review loops, declare workflow inputs, or mentions .atomic/workflows/, defineWorkflow, ctx.stage, ctx.inputs, or the atomic workflow picker.
+description: Create multi-agent workflows for Atomic CLI using defineWorkflow().run().compile() with ctx.stage() for session orchestration across Claude, Copilot, and OpenCode SDKs. Use whenever the user wants to create, edit, or debug workflows, build agent pipelines, define multi-stage automations, set up review loops, declare workflow inputs, run background/headless stages, or mentions .atomic/workflows/, defineWorkflow, ctx.stage, ctx.inputs, headless, background stages, or the atomic workflow picker.
 ---
 
 # Workflow Creator
@@ -94,7 +94,7 @@ Workflow quality depends on two disciplines: **prompt engineering** (crafting cl
 
 ## How Workflows Work
 
-A workflow is a TypeScript file with a single `.run()` callback that orchestrates agent sessions dynamically. Inside the callback, `ctx.stage()` spawns sessions — each gets its own tmux window and graph node. Native TypeScript handles all control flow: loops, conditionals, `Promise.all()`, `try`/`catch`.
+A workflow is a TypeScript file with a single `.run()` callback that orchestrates agent sessions dynamically. Inside the callback, `ctx.stage()` spawns sessions — each gets its own tmux window and graph node (unless running in headless mode). Native TypeScript handles all control flow: loops, conditionals, `Promise.all()`, `try`/`catch`.
 
 ```ts
 import { defineWorkflow } from "@bastani/atomic/workflows";
@@ -109,6 +109,61 @@ export default defineWorkflow<"claude">({ name: "my-workflow", description: "...
 
 The runtime manages the full session lifecycle — callback return marks completion; throws mark errors. `.compile()` produces a branded `WorkflowDefinition` consumed by the CLI.
 
+### Background (headless) stages
+
+Stages can run in **headless mode** by passing `{ headless: true }` in `SessionRunOptions`. Headless stages execute the provider SDK **in-process** instead of spawning a tmux window — they are invisible in the workflow graph but tracked via a background task counter in the statusline.
+
+```ts
+// Headless stage — runs in-process, no tmux window, invisible in graph
+await ctx.stage(
+  { name: "background-analysis", headless: true },
+  {}, {},
+  async (s) => {
+    const result = await s.session.query("Analyze the codebase structure.");
+    s.save(s.sessionId);
+    return result.output;
+  },
+);
+```
+
+**When to use headless stages:**
+- Parallel data-gathering tasks that don't need a visible TUI (e.g., codebase research, infrastructure discovery)
+- Support tasks that should run alongside visible stages without cluttering the graph
+- Any stage where only the result matters, not the live TUI interaction
+
+**How they work per provider:**
+- **Claude**: Uses the Agent SDK `query()` API directly in-process (no tmux pane)
+- **Copilot**: SDK spawns its own CLI subprocess internally (no tmux pane needed)
+- **OpenCode**: Uses `createOpencode()` to start both server and client in-process
+
+**Key behaviors:**
+- The callback interface is **identical** to interactive stages — `s.client`, `s.session`, `s.save()`, `s.transcript()` all work the same way
+- Headless stages are **transparent to graph topology** — they don't consume or update the execution frontier, so `visible → [3 headless] → visible` renders as `visible → visible` in the graph
+- Errors in headless stages still fail the workflow — they are tracked and recorded identically to interactive stages
+- The `paneId` for headless stages is a virtual identifier: `headless-<name>-<sessionId>`
+
+**Common pattern — fan-out with headless background stages:**
+
+```ts
+// Visible stage seeds context
+const seed = await ctx.stage({ name: "seed" }, {}, {}, async (s) => { /* ... */ });
+
+// Three parallel headless stages gather data in the background
+const [a, b, c] = await Promise.all([
+  ctx.stage({ name: "gather-a", headless: true }, {}, {}, async (s) => { /* ... */ }),
+  ctx.stage({ name: "gather-b", headless: true }, {}, {}, async (s) => { /* ... */ }),
+  ctx.stage({ name: "gather-c", headless: true }, {}, {}, async (s) => { /* ... */ }),
+]);
+
+// Visible stage merges background results
+await ctx.stage({ name: "merge" }, {}, {}, async (s) => {
+  await s.session.query(`Merge:\n${a.result}\n${b.result}\n${c.result}`);
+  s.save(s.sessionId);
+});
+```
+
+See `references/control-flow.md` for full headless pattern details and `references/agent-sessions.md` for per-SDK headless session behavior.
+
 Workflows are SDK-specific. User-created workflows live in a project with `@bastani/atomic` installed as a dependency, along with the native agent SDK(s) for the provider(s) you target. Install only the SDK(s) you need:
 
 ```bash
@@ -159,8 +214,9 @@ Hard constraints enforced by the builder, loader, and runtime:
 3. **`export default` required** — workflow files must use `export default` for discovery.
 4. **Unique session names** — every `ctx.stage()` call must use a unique `name` across the workflow run.
 5. **Completed-only reads** — `transcript()` and `getMessages()` only access sessions whose callback has returned and saves have flushed. Attempting to read a still-running session throws.
-6. **Graph topology is auto-inferred** — the runtime derives parent-child edges from `await`/`Promise.all` patterns. Sequential `await` creates a chain; `Promise.all([...])` branches from the same parent; a stage after `Promise.all` receives all parallel stages as parents. See `references/control-flow.md` for full details.
+6. **Graph topology is auto-inferred** — the runtime derives parent-child edges from `await`/`Promise.all` patterns. Sequential `await` creates a chain; `Promise.all([...])` branches from the same parent; a stage after `Promise.all` receives all parallel stages as parents. Headless stages are **transparent** to the graph — they don't consume or update the execution frontier. See `references/control-flow.md` for full details.
 7. **Do not manually create clients or sessions** — the runtime auto-creates `s.client` and `s.session` from `clientOpts` and `sessionOpts`. Use `s.session.query()`, `s.session.send()`, and `s.client.session.prompt()` instead.
+8. **Headless stages share the same callback interface** — `s.client`, `s.session`, `s.save()`, `s.transcript()`, and return values all work identically in headless mode. The only differences are: no tmux window, no graph node, and a virtual `paneId`.
 
 ## Concept-to-Code Mapping
 
@@ -169,8 +225,10 @@ Every workflow pattern maps directly to TypeScript code:
 | Workflow Concept | Programmatic Pattern |
 |---|---|
 | Agent session (send prompt, get response) | `ctx.stage({ name }, {}, {}, async (s) => { /* use s.client, s.session */ })` |
+| Background (headless) session | `ctx.stage({ name, headless: true }, {}, {}, async (s) => { /* same API */ })` — invisible in graph, tracked by background counter |
 | Sequential execution | `await ctx.stage(...)` followed by `await ctx.stage(...)` |
 | Parallel execution | `Promise.all([ctx.stage(...), ctx.stage(...)])` |
+| Parallel background tasks | `Promise.all([ctx.stage({ name: "a", headless: true }, ...), ctx.stage({ name: "b", headless: true }, ...)])` |
 | Conditional branching | `if (...) { await ctx.stage({ name: "fix" }, {}, {}, ...) }` |
 | Bounded loops with visible graph nodes | `for (let i = 1; i <= N; i++) { await ctx.stage({ name: \`step-\${i}\` }, {}, {}, ...) }` |
 | Return data from session | `const h = await ctx.stage(opts, {}, {}, async (s) => { return value; }); h.result` |
@@ -191,6 +249,7 @@ Map the user's intent to sessions and patterns:
 |----------|---------|
 | What are the distinct steps? | Each step → `ctx.stage()` call |
 | Can any steps run in parallel? | `Promise.all([ctx.stage(...), ...])` |
+| Should any parallel steps run in the background? | `ctx.stage({ name, headless: true }, ...)` — invisible in graph, ideal for data-gathering |
 | Does any step need deterministic computation? | Plain TypeScript inside `.run()` or session callback |
 | Do any steps need to repeat? | `for`/`while` loop with `ctx.stage()` inside |
 | Are there conditional paths? | `if`/`else` wrapping `ctx.stage()` calls |

.agents/skills/workflow-creator/references/agent-sessions.md

@@ -201,6 +201,26 @@ Invoke named sub-agents by prefixing the prompt with `@"agent-name (agent)"`. Th
 })
 ```
 
+### Headless mode (background stages)
+
+Claude headless stages use the Agent SDK's `query()` API directly in-process instead of automating a tmux pane. Set `headless: true` in the stage options:
+
+```ts
+await ctx.stage(
+  { name: "background-analysis", headless: true },
+  {}, {},
+  async (s) => {
+    // s.session.query() works identically — the runtime uses
+    // HeadlessClaudeSessionWrapper which calls the Agent SDK directly
+    const result = await s.session.query("Analyze the codebase.");
+    s.save(s.sessionId);
+    return result.output;
+  },
+);
+```
+
+The callback interface is identical to interactive stages. Internally, the runtime uses `HeadlessClaudeClientWrapper` (no-op start/stop) and `HeadlessClaudeSessionWrapper` (calls `query()` from `@anthropic-ai/claude-agent-sdk` directly). No tmux pane is created, and the stage is invisible in the workflow graph.
+
 ## Copilot SDK
 
 Copilot uses a client-server architecture. The runtime auto-creates a `CopilotClient` (as `s.client`) and a `CopilotSession` (as `s.session`) before invoking your callback. Auto-cleanup (`session.disconnect()` and `client.stop()`) is handled by the runtime after the callback completes.
@@ -516,6 +536,24 @@ Pass the `agent` parameter in `sessionOpts` (3rd arg to `ctx.stage()`) to bind t
 })
 ```
 
+### Headless mode (background stages)
+
+Copilot headless stages let the SDK spawn its own CLI subprocess internally — no tmux pane is needed. Set `headless: true`:
+
+```ts
+await ctx.stage(
+  { name: "background-task", headless: true },
+  {}, {},
+  async (s) => {
+    // s.session.send() works identically
+    await s.session.send({ prompt: "Analyze the codebase." });
+    s.save(await s.session.getMessages());
+  },
+);
+```
+
+The SDK creates a `CopilotClient` without a `cliUrl` — it spawns its own CLI process internally rather than connecting to a tmux-hosted server. The callback interface is identical.
+
 ## OpenCode SDK
 
 OpenCode uses a client-server model. The runtime auto-creates an `OpencodeClient` (as `s.client`) and an OpenCode session (as `s.session`) before invoking your callback. Use `s.client.session.prompt({ sessionID: s.session.id, ... })` to send prompts.
@@ -728,3 +766,24 @@ Pass the `agent` parameter to `s.client.session.prompt()` to route a prompt to a
   );
 })
 ```
+
+### Headless mode (background stages)
+
+OpenCode headless stages use `createOpencode()` from the SDK to start both server and client in-process. Set `headless: true`:
+
+```ts
+await ctx.stage(
+  { name: "background-task", headless: true },
+  {}, { title: "background-task" },
+  async (s) => {
+    // s.client.session.prompt() works identically
+    const result = await s.client.session.prompt({
+      sessionID: s.session.id,
+      parts: [{ type: "text", text: "Analyze the codebase." }],
+    });
+    s.save(result.data!);
+  },
+);
+```
+
+Internally, the runtime uses `createOpencode({ port: 0 })` to start both the OpenCode server and client in-process. A cleanup callback closes the server when the stage completes. The callback interface is identical.

.agents/skills/workflow-creator/references/control-flow.md

@@ -287,6 +287,63 @@ In iterative loops each stage is naturally the successor of the last because `aw
 
 Each iteration's stages form a natural chain because each `await` follows the previous one. Conditional stages fit in seamlessly — the graph reflects whatever path was actually executed.
 
+### Headless (background) stages: transparent to graph topology
+
+Headless stages (`{ headless: true }`) are **invisible in the workflow graph** — they don't consume or update the execution frontier. This means they don't affect the parent-child edges inferred for visible stages.
+
+```ts
+// ✅ Graph renders: seed → merge (headless stages are transparent)
+.run(async (ctx) => {
+  const seed = await ctx.stage({ name: "seed" }, {}, {}, async (s) => {
+    const result = await s.session.query("Describe the project.");
+    s.save(s.sessionId);
+    return result.output;
+  });
+
+  // Three parallel headless stages — invisible in the graph
+  const [a, b, c] = await Promise.all([
+    ctx.stage({ name: "gather-a", headless: true }, {}, {}, async (s) => {
+      const result = await s.session.query(`List 3 pros:\n\n${seed.result}`);
+      s.save(s.sessionId);
+      return result.output;
+    }),
+    ctx.stage({ name: "gather-b", headless: true }, {}, {}, async (s) => {
+      const result = await s.session.query(`List 3 cons:\n\n${seed.result}`);
+      s.save(s.sessionId);
+      return result.output;
+    }),
+    ctx.stage({ name: "gather-c", headless: true }, {}, {}, async (s) => {
+      const result = await s.session.query(`List 3 uses:\n\n${seed.result}`);
+      s.save(s.sessionId);
+      return result.output;
+    }),
+  ]);
+
+  // Visible merge stage — chains from "seed" in the graph (not from headless stages)
+  await ctx.stage({ name: "merge" }, {}, {}, async (s) => {
+    await s.session.query(
+      `Combine:\n\n## Pros\n${a.result}\n\n## Cons\n${b.result}\n\n## Uses\n${c.result}`,
+    );
+    s.save(s.sessionId);
+  });
+})
+```
+
+**Key behaviors:**
+- Headless stages don't produce graph nodes — they are tracked by a background task counter in the statusline instead
+- The execution frontier is not updated when a headless stage spawns or settles, so the next visible stage chains from the last visible stage
+- Headless stages still participate in `Promise.all()` — the merge stage correctly awaits all three before running
+- Return values (`handle.result`) and transcript access (`s.transcript(handle)`) work identically
+
+**When to use headless vs. visible parallel stages:**
+
+| Concern | Use visible (`headless: false`) | Use headless (`headless: true`) |
+|---|---|---|
+| User needs to see the work | Yes — each stage gets a tmux window | No — tracked by counter only |
+| Debugging/monitoring | Yes — visible in graph + pane preview | No — errors tracked but no TUI |
+| Data-gathering/analysis | Possible but clutters the graph | Ideal — keeps graph clean |
+| Infrastructure discovery | Clutters graph for support work | Ideal — Ralph uses this pattern |
+
 ### Note on data flow vs. topology
 
 Graph topology (parent-child edges) is inferred from control flow. Data flow between sessions is separate: use `s.transcript(handle)` to read a prior session's saved output. The two concerns are independent — you do not need explicit dependency declarations to access another session's transcript; you just need that session's `await` to have completed before you read it.

.agents/skills/workflow-creator/references/failure-modes.md

@@ -45,6 +45,7 @@ Silent failures are catalogued first below. Loud failures are grouped at the end
 | [F13](#f13-parallel-siblings-read-each-others-transcripts) | Parallel siblings read each other's transcripts | all | loud |
 | [F14](#f14-forgetting-to-await-ctxstage) | Forgetting to `await` `ctx.stage()` | all | silent |
 | [F15](#f15-using-a-pending-sessionhandle-before-completion) | Using a pending `SessionHandle` before completion | all | silent |
+| [F16](#f16-headless-stage-errors-are-invisible-in-the-graph) | Headless stage errors are invisible in the graph | all | silent |
 
 ---
 
@@ -662,6 +663,57 @@ accessing `.result` without awaiting, the type will be `Promise`, not `T`.
 
 ---
 
+## F16. Headless stage errors are invisible in the graph
+
+**Symptom.** A workflow fails but the graph shows all visible stages as
+completed. The error message references a session name that doesn't appear
+in the graph panel.
+
+**Root cause.** Headless stages (`{ headless: true }`) are invisible in the
+workflow graph — they have no graph node, no tmux window, and no pane
+preview. When a headless stage throws, the error is recorded in the
+`failedRegistry` and the workflow halts, but the failure is only visible in
+the orchestrator's error output and the session's `error.txt` file on disk.
+
+**Affected SDKs.** All three — this is an executor-level behavior, not
+SDK-specific.
+
+### ❌ Wrong — no error context for headless stages
+
+```ts
+// Headless stage fails silently in the graph
+const [a, b, c] = await Promise.all([
+  ctx.stage({ name: "gather-a", headless: true }, {}, {}, async (s) => {
+    throw new Error("API key expired"); // Fails — no graph node to show red
+  }),
+  ctx.stage({ name: "gather-b", headless: true }, {}, {}, async (s) => { /* ... */ }),
+  ctx.stage({ name: "gather-c", headless: true }, {}, {}, async (s) => { /* ... */ }),
+]);
+```
+
+### ✅ Right — wrap headless stages with descriptive error context
+
+```ts
+const [a, b, c] = await Promise.all([
+  ctx.stage({ name: "gather-a", headless: true }, {}, {}, async (s) => {
+    try {
+      return await doWork(s);
+    } catch (error) {
+      throw new Error(`[gather-a] ${error instanceof Error ? error.message : String(error)}`);
+    }
+  }),
+  // ... same pattern for b, c
+]);
+```
+
+**Detection.** If a workflow fails and the graph shows no failed nodes,
+check the orchestrator log (`orchestrator.log` in the session directory)
+and look for `headless-<name>` in the error output. The session directory
+at `~/.atomic/sessions/<run-id>/<name>-<id>/error.txt` contains the
+full error for each failed headless stage.
+
+---
+
 ## Design checklist
 
 Before shipping a multi-session workflow, walk the list:
@@ -677,3 +729,4 @@ Before shipping a multi-session workflow, walk the list:
 - [ ] Every `ctx.stage()` call is `await`ed (F14)
 - [ ] `SessionHandle` values are only used after the promise resolves (F15)
 - [ ] If provider-level resume/fork is used at all, it stays within the same agent role (F12)
+- [ ] Headless stage callbacks include descriptive error context so failures can be diagnosed without a graph node (F16)