From f452e28b1abb82603d544167d245efed651cf3cd Mon Sep 17 00:00:00 2001 From: Ralph Bean Date: Fri, 27 Mar 2026 16:57:19 -0400 Subject: [PATCH 1/5] Add ADR 0005: Unidirectional control flow through the execution stack MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Establishes that control flows strictly downward through the execution stack (Dispatch → Infrastructure → Sandbox → Harness → Runtime) and no layer may influence layers above it. Distinguishes prohibited upward control flow from permitted upward data flow (telemetry, failure signals). Renames "Work Coordinator" to "Agent Dispatch and Coordination Layer" for clarity. Adds an "Execution Stack" section to architecture.md. Co-Authored-By: Claude Opus 4.6 --- docs/ADRs/0005-unidirectional-control-flow.md | 178 ++++++++++++++++++ docs/architecture.md | 18 +- 2 files changed, 194 insertions(+), 2 deletions(-) create mode 100644 docs/ADRs/0005-unidirectional-control-flow.md diff --git a/docs/ADRs/0005-unidirectional-control-flow.md b/docs/ADRs/0005-unidirectional-control-flow.md new file mode 100644 index 0000000000..f3950634a3 --- /dev/null +++ b/docs/ADRs/0005-unidirectional-control-flow.md @@ -0,0 +1,178 @@ +--- +title: "5. Unidirectional control flow through the execution stack" +status: Proposed +relates_to: + - agent-architecture + - agent-infrastructure + - security-threat-model +topics: + - architecture + - security + - portability +--- + +# 5. Unidirectional control flow through the execution stack + +Date: 2026-03-27 + +## Status + +Proposed + +## Context + +The [architecture document](../architecture.md) defines five components that +form the execution stack — the vertical path from event to agent action: + +1. **Agent Dispatch and Coordination Layer** — translates events into agent tasks +2. **Agent Infrastructure** — compute and orchestration that runs agents +3. **Agent Sandbox** — isolation boundary (network, filesystem) +4. **Agent Harness** — configuration and context layer (skills, prompts, tools) +5. **Agent Runtime** — the LLM in execution + +Other components (Policy Store, Intent Source, Identity Provider, Observability, +Agent Registry) exist alongside the stack but are not part of its vertical +control flow. + +Today, the architecture document names these components and their +responsibilities but does not state the structural relationship between them. +Without an explicit rule, it is ambiguous whether a lower layer may influence a +higher one — whether an agent runtime can modify its own harness, or a harness +can reconfigure its sandbox. + +This matters for four reasons: + +**Security.** A compromised agent runtime must not be able to weaken its own +sandbox. A poisoned skill must not be able to expand network access. Each layer +constrains the layers below it; those constraints must be immutable from below. +This directly supports the threat model's top priority (external prompt +injection) by ensuring that an injected instruction cannot escalate the agent's +own capabilities. + +**Portability.** Each layer can be swapped independently when control flows in +one direction. Replacing the infrastructure layer (GitHub Actions to Kubernetes) +requires re-implementing only that layer's interface to the layer below it. +Nothing in the sandbox, harness, or runtime changes. This is critical because +we intend to support multiple execution platforms. + +**Testability.** Each layer can be tested in isolation by mocking the layer +above it. A harness test does not need real infrastructure; a sandbox test does +not need a real dispatch layer. + +**Reasoning.** When debugging or auditing, control flow traces in one direction. +You never have to ask "did the agent change its own sandbox config?" or "did a +skill modify the dispatch layer?" The answer is always no. + +## Options + +### Option A: Unidirectional control flow (strict top-down) + +Control flows strictly downward through the execution stack. No layer may +influence, configure, or depend on layers above it. A layer that needs something +not provided by the layer above must fail or escalate — it cannot +self-provision. + +**Trade-offs:** +- Eliminates an entire class of security vulnerabilities (privilege escalation + from within the stack). +- Simple to reason about, audit, and test. +- Agents that need additional capabilities must fail and surface the gap, which + is the correct behavior in a zero-trust system. +- Slightly less flexible: an agent cannot dynamically request additional tools + or network access mid-execution. + +### Option B: Bidirectional control flow (allow upward requests) + +Lower layers may request changes from higher layers through a controlled +protocol — for example, the agent runtime could request an additional tool from +the harness, or the harness could request expanded network access from the +sandbox. + +**Trade-offs:** +- More flexible: agents can adapt to unanticipated needs at runtime. +- Introduces a request/approval protocol between layers, adding complexity. +- Every upward channel is an attack surface. A compromised runtime could use the + request mechanism to escalate its own capabilities. +- Violates zero-trust: the system must evaluate whether to grant requests from a + potentially compromised component. +- Makes reasoning harder: control flow becomes a graph, not a line. + +**Why we reject this:** The security risk and complexity outweigh the +flexibility gain. In a zero-trust model, a layer that can request changes to its +own constraints is a layer that can potentially weaken its own constraints. The +correct response to insufficient capabilities is failure and escalation to a +human or a higher-level process — not self-provisioning. + +## Decision + +Control flows strictly downward through the execution stack: + +``` +Agent Dispatch → Agent Infrastructure → Agent Sandbox → Agent Harness → Agent Runtime +``` + +No layer may influence, configure, or depend on layers above it: + +- The **agent runtime** cannot modify the harness (its own system prompt, + skills, tool definitions). +- The **agent harness** cannot modify the sandbox (network policy, filesystem + restrictions). +- The **agent sandbox** cannot modify the infrastructure (compute resources, + scheduling). +- The **agent infrastructure** cannot modify the agent dispatch and coordination + layer (what events cause agent invocations). + +### Control flow vs. data flow + +The unidirectional rule applies to **control flow** — configuration, behavior, +and constraints. It does not prohibit upward **data flow**: + +- **Prohibited (upward control flow):** A lower layer modifying the + configuration, behavior, or constraints of a higher layer. The agent runtime + cannot add tools to its own harness. The sandbox cannot expand its own network + policy. The harness cannot reconfigure infrastructure scheduling. +- **Permitted (upward data flow):** Telemetry, logs, traces, and failure signals + flowing from any layer to Observability. Exit codes and error messages + indicating failure. Forge comments explaining what an agent could not do. + +This distinction matters because Observability inherently collects data from +every layer in the stack — that is its job. The rule prohibits a layer from +*changing* layers above it, not from *emitting signals* that layers above +observe. A runtime that writes a structured log entry is emitting data upward; a +runtime that modifies its own system prompt is exerting control upward. Only the +latter is prohibited. + +Each component interface is a one-way contract: the layer above provides +configuration, the layer below consumes it. + +Cross-cutting concerns (Observability, Identity Provider, Policy Store) sit +alongside the stack and feed into layers from the side. They too follow the +unidirectional principle: an agent runtime cannot modify its own policy, +identity, or observability configuration. + +## Consequences + +- **Each layer boundary is a security boundary.** Compromise of a lower layer + cannot propagate upward. This is the stack's primary security property. +- **Agent runtimes that need capabilities not provided by their harness or + sandbox must fail or escalate to humans.** They cannot self-provision, and + they cannot request the missing capability from the harness — that would be + upward control flow. Instead, the runtime fails and the failure is surfaced + to humans through observability and the forge (e.g., posting a comment + explaining what it could not do and why). The human decides whether to + reconfigure the harness or sandbox for next time. This forces capability + gaps to surface during harness design and testing rather than at runtime + through ad-hoc self-provisioning. (See + [dual-interpretation escalation](../problems/code-review.md#dual-interpretation-escalation) + for the pattern of structured escalation to humans, and + [agent-architecture.md](../problems/agent-architecture.md#how-deadlocks-are-resolved) + for the principle that persistent disagreement escalates to humans.) +- **Swapping any layer requires only re-implementing that layer's interface to + the layer below it.** Moving from GitHub Actions to Kubernetes changes the + infrastructure layer; the sandbox, harness, and runtime are unaffected. +- **Cross-cutting concerns follow the same principle.** The Policy Store feeds + policy into the harness and sandbox from the side, but the runtime cannot + write back to the Policy Store. Observability collects signals from every + layer but no layer can modify its own observability configuration. +- **The architecture document gains an overarching structural principle** that + the individual component descriptions can reference. diff --git a/docs/architecture.md b/docs/architecture.md index 4cbd5525ed..7df94fa67f 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -6,6 +6,20 @@ This document names the parts of the system without deciding how they work. It e This is not exhaustive. Not every problem doc maps to a component here, and not every component here has a corresponding problem doc yet. +## Execution Stack + +Five components form the vertical execution path from event to agent action: + +1. **Agent Dispatch and Coordination Layer** — translates events into agent tasks +2. **Agent Infrastructure** — provisions and runs agent workloads +3. **Agent Sandbox** — enforces isolation (network, filesystem) +4. **Agent Harness** — assembles configuration and context (skills, prompts, tools) +5. **Agent Runtime** — the LLM in execution + +Control flows strictly downward through this stack. No layer may influence, configure, or depend on layers above it. This is the execution stack's primary structural invariant. (See [ADR 0005](ADRs/0005-unidirectional-control-flow.md).) + +The remaining components described in this document (Policy Store, Intent Source, Identity Provider, Observability, Agent Registry) are cross-cutting concerns that feed into the stack from the side. They are not part of the vertical control flow, but they follow the same principle: no component within the stack can modify the cross-cutting systems that constrain it. + ## Agent Infrastructure The compute and orchestration layer that runs agent workloads. Responsible for provisioning, scheduling, scaling, and lifecycle management of agent execution environments. @@ -75,11 +89,11 @@ Identity is not the same as trust. An agent's identity lets it authenticate to e - How are credentials rotated and revoked, and who has authority to do that? - Does the identity provider integrate with existing secrets management, or is it a new system? -## Work Coordinator +## Agent Dispatch and Coordination Layer The mechanism that assigns work to agents and prevents conflicts. Responsible for translating triggers (GitHub events, schedules, manual requests) into agent tasks and ensuring two agents don't work the same problem simultaneously. -The existing design principle is that [the repo is the coordinator](problems/agent-architecture.md#interaction-model-the-repo-as-coordinator) — branch protection, CODEOWNERS, status checks, and GitHub events provide coordination without a central orchestrator. The work coordinator component may be nothing more than the glue that connects GitHub webhooks to agent infrastructure. Or it may need to be more. +The existing design principle is that [the repo is the coordinator](problems/agent-architecture.md#interaction-model-the-repo-as-coordinator) — branch protection, CODEOWNERS, status checks, and GitHub events provide coordination without a central orchestrator. The agent dispatch and coordination layer may be nothing more than the glue that connects GitHub webhooks to agent infrastructure. Or it may need to be more. **Open questions:** From 7e9cc6046f2c30ab53826c9d77b52443f00afe18 Mon Sep 17 00:00:00 2001 From: Ralph Bean Date: Fri, 27 Mar 2026 16:58:00 -0400 Subject: [PATCH 2/5] Add ADR 0006: Forge abstraction layer MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Deterministic code paths (agent runtime wrapper and skill scripts) use a shared library (forgekit) for forge-portable operations. Agents themselves use native forge CLIs — LLMs adapt naturally to the forge they're working with. Updates architecture.md: adds Forge Abstraction Layer section, updates Identity Provider for forgekit credential issuance, generalizes dispatch layer to use forge-neutral language. Co-Authored-By: Claude Opus 4.6 --- docs/ADRs/0006-forge-abstraction-layer.md | 163 ++++++++++++++++++++++ docs/architecture.md | 20 ++- 2 files changed, 179 insertions(+), 4 deletions(-) create mode 100644 docs/ADRs/0006-forge-abstraction-layer.md diff --git a/docs/ADRs/0006-forge-abstraction-layer.md b/docs/ADRs/0006-forge-abstraction-layer.md new file mode 100644 index 0000000000..dc7efdb180 --- /dev/null +++ b/docs/ADRs/0006-forge-abstraction-layer.md @@ -0,0 +1,163 @@ +--- +title: "6. Forge abstraction layer" +status: Proposed +relates_to: + - agent-architecture + - agent-infrastructure +topics: + - portability + - architecture + - tooling +--- + +# 6. Forge abstraction layer + +Date: 2026-03-27 + +## Status + +Proposed + +## Context + +Fullsend currently targets GitHub-hosted organizations but intends to support +GitLab and eventually Forgejo. Many components interact with forge-specific +APIs: creating issues, opening pull/merge requests, applying labels, posting +status checks, and reading CODEOWNERS. Branch protection configuration varies +significantly across forges and is out of scope for the initial abstraction — +it remains an unsolved portability problem. + +These interactions happen in two distinct contexts: + +1. **Agent runtime (LLM-driven).** The agent decides to open a PR, comment on + an issue, or check labels. LLM-based agents are naturally good at detecting + which forge they're on and using its native CLI (`gh`, `glab`, etc.). + Forcing them through an abstraction adds friction without clear benefit — + the agent adapts. + +2. **Deterministic code paths (scripted).** Two specific places run + deterministic, non-LLM code that must work across forges: + - The **agent runtime wrapper** — the script that runs inside the sandbox, + configures the harness, and launches the agent runtime. It reads issue + metadata, posts status updates, and fetches configuration. This code + must work identically regardless of forge. + - **Skill scripts** — scripts embedded in `scripts/` directories within + skills that agents invoke as tools. These are shipped by fullsend and + must be portable. + +The forge abstraction belongs in the deterministic code, not in the agent's +mouth. + +## Options + +### Option 1: Abstraction everywhere + +A CLI tool that all forge interactions go through, including agent-initiated +ones. Agent prompts say `fullsend pr create` instead of `gh pr create`. + +**Pros:** +- Uniform interface everywhere. Easy to audit forge interactions. + +**Cons:** +- Fights the LLM's natural behavior. Agents are good at using native CLIs. +- Requires teaching every agent a non-standard CLI instead of leveraging + existing training data for `gh`, `glab`, etc. +- The abstraction is only valuable in deterministic code paths where we control + the source. In agent-generated commands, the LLM adapts naturally. + +### Option 2: Abstraction in deterministic code only + +A shared library/module used by the agent runtime wrapper and skill scripts. +Agents themselves use whatever forge CLI is available. + +**Pros:** +- Forge portability where it matters (our code), natural behavior where it + doesn't (agent-generated commands). +- Fewer moving parts — no CLI binary to distribute, just a library used by + code we already ship. +- Agents benefit from their training data on `gh`, `glab`, etc. + +**Cons:** +- Agents may use forge-specific features that don't exist on other forges. This + is acceptable — agent prompts can be tuned per-forge if needed, and the + harness can provide forge-appropriate context. + +### Option 3: No abstraction — accept GitHub coupling + +Use `gh` and GitHub APIs everywhere, including deterministic code. + +**Pros:** +- Simplest now. + +**Cons:** +- Porting the runtime wrapper and skill scripts to GitLab requires rewriting + every forge interaction in those code paths. + +## Decision + +Forge-specific interactions are abstracted in the two deterministic code paths +that fullsend controls: the **agent runtime wrapper** and **skill scripts**. +Agents themselves are free to use native forge CLIs. + +### Where the abstraction lives + +A shared library (working name: `forgekit`) provides functions for the forge +operations that deterministic code needs: + +- Issue operations: read metadata, apply labels, post comments +- PR/MR operations: create, update status, post review comments +- Status checks: post pass/fail results +- Code ownership: query CODEOWNERS / equivalent +- Repository metadata: default branch, permissions, clone URLs + +The agent runtime wrapper and skill scripts import this library. The library +detects the forge type from the repo's remote URL or from configuration in the +`.fullsend` repo, and dispatches to the appropriate backend. + +### What agents do + +Agents use whatever forge CLI is available in the sandbox (`gh`, `glab`, etc.). +The harness provides forge-appropriate context so agents know which system +they're on, but agents are not forced through an abstraction layer. LLMs are +naturally effective at using native CLIs based on their training data. + +### Key design points + +- **Labels are fullsend vocabulary.** Labels used as control signals (e.g., + "agent-ready", "not-reproducible") are part of the fullsend vocabulary. + The library maps them to the appropriate forge mechanism. Agents may also + apply these labels using native CLIs — the label names are the contract, + not the mechanism. +- **CODEOWNERS parsing is wrapped.** Different forges have different syntax + for code ownership. The library abstracts this behind a uniform query + interface for use by the runtime wrapper and review logic. +- **Skill scripts use the library, not forge CLIs.** Any `scripts/` shipped + with fullsend skills call `forgekit` functions, making skills portable + without rewriting. + +## Consequences + +- **Deterministic code is forge-portable.** The runtime wrapper and skill + scripts work across GitHub, GitLab, and Forgejo without modification. +- **Agent prompts are forge-aware, not forge-abstracted.** Agent definitions + may include forge-specific context (e.g., "you are working on a GitHub + repo, use `gh` for forge operations"), but this is a harness concern, not + an architectural constraint. +- **New forge backends require implementing the library adapter.** Adding + GitLab or Forgejo support means implementing `forgekit` backends and + ensuring the right forge CLI is available in the sandbox. +- **The library is a fullsend deliverable** that must be versioned and tested, + but it is simpler than a standalone CLI since it only needs to support the + operations used by deterministic code paths. +- **The agent dispatch and coordination layer uses this library.** It runs + deterministic code that interacts with forge APIs for event processing and + work assignment — it goes through `forgekit`, not forge APIs directly. +- **The Agent Identity Provider uses this library for credential issuance.** + `forgekit` is responsible for making agent identity credentials available to + the agent runtime (e.g., generating scoped tokens from a GitHub App or + equivalent). The sandbox is responsible for making those scoped tokens + available to the layers it controls (harness and runtime). +- **Branch protection remains an unsolved portability problem.** Branch + protection rules vary significantly across forges in both semantics and + configuration mechanisms. The initial `forgekit` abstraction does not attempt + to unify branch protection management. diff --git a/docs/architecture.md b/docs/architecture.md index 7df94fa67f..b0d5e540b5 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -78,7 +78,7 @@ This is the thing that actually reasons and acts. Everything else in this docume ## Agent Identity Provider -The system that gives agents credentials to act on external services. Responsible for issuing, scoping, rotating, and revoking the identities agents use to interact with GitHub, container registries, and other APIs. +The system that gives agents credentials to act on external services. Responsible for issuing, scoping, rotating, and revoking the identities agents use to interact with the hosting forge, container registries, and other APIs. Credential issuance is deterministic code; `forgekit` handles forge-portable token generation (e.g., GitHub App installation tokens vs. GitLab project access tokens). The sandbox makes scoped tokens available to the layers it controls (harness and runtime). (See [ADR 0006](ADRs/0006-forge-abstraction-layer.md).) Identity is not the same as trust. An agent's identity lets it authenticate to external services; the trust model is defined by repository permissions and CODEOWNERS, not by which credentials the agent holds. (See [agent-architecture.md](problems/agent-architecture.md) — "trust derives from repository permissions, not agent identity.") @@ -89,15 +89,27 @@ Identity is not the same as trust. An agent's identity lets it authenticate to e - How are credentials rotated and revoked, and who has authority to do that? - Does the identity provider integrate with existing secrets management, or is it a new system? +## Forge Abstraction Layer + +The boundary between fullsend's deterministic code and the hosting forge (GitHub, GitLab, Forgejo). A shared library (`forgekit`) provides a uniform interface to forge operations — issues, pull/merge requests, labels, status checks, and code ownership queries. + +The abstraction applies to two specific code paths: the **agent runtime wrapper** (the script inside the sandbox that configures the harness and launches the agent) and **skill scripts** (deterministic scripts embedded in fullsend-shipped skills). These are the code paths fullsend controls and must be forge-portable. Agents themselves use native forge CLIs (`gh`, `glab`, etc.) — LLMs are naturally effective at adapting to the forge they're working with. (See [ADR 0006](ADRs/0006-forge-abstraction-layer.md).) + +**Open questions:** + +- What is the right implementation language for the library? +- How does the library authenticate — does it receive credentials from the Agent Identity Provider, or discover them from the environment? +- How are forge-specific features that have no cross-forge equivalent handled — silently ignored, explicitly errored, or degraded gracefully? + ## Agent Dispatch and Coordination Layer -The mechanism that assigns work to agents and prevents conflicts. Responsible for translating triggers (GitHub events, schedules, manual requests) into agent tasks and ensuring two agents don't work the same problem simultaneously. +The mechanism that assigns work to agents and prevents conflicts. Responsible for translating triggers (forge events, schedules, manual requests) into agent tasks and ensuring two agents don't work the same problem simultaneously. -The existing design principle is that [the repo is the coordinator](problems/agent-architecture.md#interaction-model-the-repo-as-coordinator) — branch protection, CODEOWNERS, status checks, and GitHub events provide coordination without a central orchestrator. The agent dispatch and coordination layer may be nothing more than the glue that connects GitHub webhooks to agent infrastructure. Or it may need to be more. +The existing design principle is that [the repo is the coordinator](problems/agent-architecture.md#interaction-model-the-repo-as-coordinator) — branch protection, CODEOWNERS, status checks, and forge events provide coordination without a central orchestrator. The agent dispatch and coordination layer may be nothing more than the glue that connects forge webhooks to agent infrastructure. Or it may need to be more. **Open questions:** -- Is GitHub's event system sufficient, or do we need additional coordination logic (e.g. to prevent two implementation agents from picking up the same issue)? +- Is the forge's event system sufficient, or do we need additional coordination logic (e.g. to prevent two implementation agents from picking up the same issue)? - How does work assignment interact with the backlog/priority agent described in [agent-architecture.md](problems/agent-architecture.md)? - What happens when work needs to be cancelled, retried, or reassigned? - Does the coordinator need state (a queue, a lock, a claim system), or can it be stateless and event-driven? From 66504f94a7fede5fde69a7378532e87178940644 Mon Sep 17 00:00:00 2001 From: Ralph Bean Date: Fri, 27 Mar 2026 16:58:32 -0400 Subject: [PATCH 3/5] Add ADR 0007: GitHub Actions as initial execution platform GitHub Actions is the first execution platform, serving as both trigger and infrastructure. A platform-agnostic entry point ensures nothing below the infrastructure layer knows it's running on GH Actions. Kubernetes is the anticipated second platform. Updates architecture.md: resolves the initial platform question in Agent Infrastructure, adds GH Actions trigger note to Agent Dispatch and Coordination Layer. Co-Authored-By: Claude Opus 4.6 --- ...thub-actions-initial-execution-platform.md | 234 ++++++++++++++++++ docs/architecture.md | 6 +- 2 files changed, 238 insertions(+), 2 deletions(-) create mode 100644 docs/ADRs/0007-github-actions-initial-execution-platform.md diff --git a/docs/ADRs/0007-github-actions-initial-execution-platform.md b/docs/ADRs/0007-github-actions-initial-execution-platform.md new file mode 100644 index 0000000000..a88e21d4e4 --- /dev/null +++ b/docs/ADRs/0007-github-actions-initial-execution-platform.md @@ -0,0 +1,234 @@ +--- +title: "7. GitHub Actions as initial execution platform" +status: Proposed +relates_to: + - agent-infrastructure + - agent-architecture + - security-threat-model +topics: + - infrastructure + - execution + - portability +--- + +# 7. GitHub Actions as initial execution platform + +Date: 2026-03-27 + +## Status + +Proposed + +## Context + +Fullsend needs an execution platform — somewhere to run agent workloads when +triggered by events. The architecture doc identifies this as the "Agent +Infrastructure" component, and +[agent-infrastructure.md](../problems/agent-infrastructure.md) explores three +directions: adopt a 3rd party solution, use existing internal infrastructure, +or build our own. + +Two concurrent ADRs shape this decision: + +- **[ADR 0005](0005-unidirectional-control-flow.md)** establishes that the + execution stack has unidirectional control flow: Trigger → Infrastructure → + Sandbox → Harness → Runtime. The infrastructure layer must be swappable + without affecting layers below it. +- **[ADR 0006](0006-forge-abstraction-layer.md)** establishes a forge + abstraction layer so that agents do not call GitHub APIs directly. + +We need to choose an initial platform that lets us start running agents quickly +while preserving the ability to change platforms later. + +## Options + +### Option 1: GitHub Actions + +Use GitHub Actions for both the trigger layer and the infrastructure layer. +Workflows respond to GitHub events (issues, PRs, labels) and provision runners +that invoke fullsend's platform-agnostic entry point. + +**Pros:** +- Zero additional infrastructure to provision or operate. Orgs already have it. +- GitHub's event system provides the trigger layer for free. +- Runners provide compute with built-in secret management. +- The `.github/workflows/` mechanism is well-understood. +- Experiment #67 demonstrated that GitHub App token generation and scoped + `GH_TOKEN` passing work with Claude Code. (The experiment ran locally, not on + GitHub Actions runners — validating the full GH Actions environment remains + open work.) +- Fastest path to a working implementation. + +**Cons:** +- Couples the trigger and infrastructure layers (both are GitHub Actions). +- Runner resource limits: 6-hour job timeout, 20 concurrent jobs per org on the + free tier (jobs beyond this limit are **dropped, not queued**), 2,000 + minutes/month included. These limits are org-wide across all repos. +- Cost at scale — GitHub-hosted runners are billed per minute. +- Vendor lock-in risk — mitigated by ADR 0005's unidirectional rule and the + entry point contract described below. +- Self-hosted runners can relax resource and concurrency limits but add + operational burden. The initial implementation targets GitHub-hosted runners; + self-hosted runners are a viable optimization for orgs that hit limits. + +### Option 2: Kubernetes from day one + +Provision a Kubernetes cluster with an operator/controller that watches forge +events and runs agent workloads as pods. + +**Pros:** +- Full control over compute, isolation, and scaling. +- No vendor coupling at the infrastructure layer. + +**Cons:** +- Requires cluster provisioning, operator development, webhook ingestion, and + secret management — months of work before an agent runs. + +### Option 3: Hybrid from day one (GitHub Actions triggers, Kubernetes runs) + +GitHub Actions responds to events and dispatches work to a Kubernetes cluster +that runs the actual agent workloads. + +**Pros:** +- Easy triggers from GitHub's event system, flexible compute from Kubernetes. + +**Cons:** +- Premature complexity. Two systems to operate before we know what the workload + looks like. + +## Decision + +GitHub Actions is the initial execution platform for fullsend. It serves as +both the trigger layer and the infrastructure layer for the first +implementation. + +### Critical constraint: no GitHub Actions coupling below the infrastructure layer + +GitHub Actions is the infrastructure. It is NOT the sandbox, the harness, or +the runtime. The workflow YAML is infrastructure configuration — it provisions +compute and launches the sandbox. Nothing below the infrastructure layer should +know it is running on GitHub Actions. + +Concretely: + +- The workflow file passes the raw forge event to a **platform-agnostic entry + point**. The entry point consults `.fullsend` config, selects a sandbox + policy, harness configuration, and agent runtime — then orchestrates their + launch. The infrastructure layer (GitHub Actions) is not involved in those + choices. +- All configuration comes from the `.fullsend` repo (see + [ADR 0003](0003-org-config-repo-convention.md)), not from workflow YAML. +- GitHub Actions secrets bootstrap credentials (e.g., a GitHub App private + key), but credential issuance (generating ephemeral tokens) is handled by + the identity provider component, not by GitHub Actions-specific mechanisms. + +### The entry point contract + +The boundary between "infrastructure" and "everything below" is a +platform-agnostic entry point. On GitHub Actions, a workflow step invokes it. +On Kubernetes, a pod's entrypoint invokes it. The entry point receives the +**raw forge event** — e.g., "issue #123 was labeled with `agent-ready`" — not +a pre-processed task description. + +The entry point is responsible for: + +1. **Interpreting the event** — determining what happened and whether it + requires agent action. +2. **Consulting `.fullsend` configuration** — reading the org's `.fullsend` + repo to determine which sandbox policy, harness configuration, and agent + runtime to use for this event type. +3. **Orchestrating the launch** — setting up the sandbox with the selected + policy, assembling the harness with the selected agent definition, and + invoking the agent runtime. + +Control still flows strictly downward per +[ADR 0005](0005-unidirectional-control-flow.md) — the entry point configures +each layer top-down, and no layer can influence layers above it. The entry +point is the same regardless of execution platform. + +How bootstrap credentials (e.g., for fetching `.fullsend` config or issuing +ephemeral tokens) are provided to the entry point is an open question. On +GitHub Actions, the reusable workflow in `.fullsend` has access to secrets and +can pass them to the entry point. On Kubernetes, a mounted secret or service +account may serve the same role. The right approach will emerge from +implementation. + +### Future execution platforms + +The architecture anticipates additional execution platforms beyond GitHub +Actions: + +- **Kubernetes** — clusters with an independent trigger layer (e.g., a + controller/operator that watches forge events via webhooks or polling). +- **GitLab CI** — GitLab's native CI/CD runners, using GitLab CI pipeline + definitions as the trigger and infrastructure layer. +- **Forgejo Runners** — Forgejo's runner infrastructure, analogous to GitHub + Actions but for Forgejo-hosted organizations. + +In each case, the same principle applies: + +- The trigger layer changes to the platform's native event system. +- The infrastructure changes to the platform's native compute. +- Everything below (sandbox, harness, runtime) stays the same because of + ADR 0005's unidirectional rule. +- The forge abstraction layer (ADR 0006) means the entry point works unchanged. + +### Workflow file design + +The `.github/workflows/` file in enrolled repos should be as thin as possible — +a stub that calls a [reusable workflow](https://docs.github.com/en/actions/sharing-automations/reusing-workflows) +defined in the org's `.fullsend` repo. The reusable workflow in `.fullsend` +contains the actual entry point invocation, secret references, and sandbox +launch logic. The enrolled repo's workflow is just a `workflow_call` reference. + +This design has two benefits: + +- **Credential isolation.** The GitHub App private key and other secrets live + only in the `.fullsend` repo. Enrolled repos never have direct access to + these secrets — they invoke the reusable workflow, which has access. +- **Centralized updates.** Changing the entry point, sandbox image, or launch + logic requires updating only the `.fullsend` repo, not every enrolled repo. + +### Workflow file protection + +The fullsend workflow file in each enrolled repo **must be listed in +CODEOWNERS as human-owned.** If an agent could modify its own workflow file via +a PR, it would be modifying its own trigger and infrastructure layer — +violating [ADR 0005](0005-unidirectional-control-flow.md)'s unidirectional +rule. This is the same principle that makes CODEOWNERS itself always +human-owned: agents cannot modify their own guardrails. + +Which layer is responsible for verifying this? The agent dispatch and +coordination layer should perform a **pre-flight check** before launching +agent work: confirm that the enrolled repo's fullsend workflow file is +CODEOWNERS-protected. If it is not, the dispatch layer refuses to run and +surfaces the misconfiguration to humans. This keeps enforcement in the +topmost layer, consistent with unidirectional control flow — lower layers +don't need to worry about it because the dispatch layer has already verified +it. + +## Consequences + +- **Enrolled repos get a workflow file.** Each enrolled repo gets a + `.github/workflows/` file that invokes fullsend's entry point on relevant + events. This file is infrastructure, not application code. +- **The entry point is platform-agnostic.** It is a script or container, not a + GitHub Action. This is the portability boundary. +- **Kubernetes migration is additive.** When we add Kubernetes support, we + implement a new trigger layer and a new way to invoke the same entry point. + Nothing below the infrastructure layer changes. +- **Resource limits constrain initial agent capabilities.** GitHub Actions' + 6-hour job timeout, 20 concurrent jobs per org (free tier), and runner specs + limit what agents can do initially. Critically, jobs that exceed the + concurrency limit are dropped, not queued — a burst of events (e.g., many + issues labeled simultaneously) will lose work. The dispatch layer must handle + this, either by rate-limiting event processing or by retrying dropped work. +- **The fullsend workflow file in enrolled repos must be CODEOWNERS-protected.** + If agents could modify their own workflow file, they would be modifying their + own trigger and infrastructure layer, violating + [ADR 0005](0005-unidirectional-control-flow.md). The dispatch layer performs + a pre-flight check to verify this. +- **Trigger and infrastructure coupling is a known trade-off.** Both layers + are GitHub Actions initially. ADR 0005's layering ensures this coupling does + not leak below the infrastructure layer, so decoupling them later requires + no changes to the sandbox, harness, or runtime. diff --git a/docs/architecture.md b/docs/architecture.md index b0d5e540b5..7228984746 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -24,15 +24,15 @@ The remaining components described in this document (Policy Store, Intent Source The compute and orchestration layer that runs agent workloads. Responsible for provisioning, scheduling, scaling, and lifecycle management of agent execution environments. -This is the "where do agents physically run" question — whether that's a managed platform, internal Kubernetes, CI runners repurposed for agent work, or something purpose-built. +The initial execution platform is **GitHub Actions**. Enrolled repos contain a thin workflow stub that calls a reusable workflow in the org's `.fullsend` repo. The reusable workflow passes the raw forge event to a platform-agnostic entry point, which consults `.fullsend` config to select a sandbox policy, harness configuration, and agent runtime, then orchestrates their launch. Nothing below the infrastructure layer knows it is running on GitHub Actions. **Kubernetes**, **GitLab CI**, and **Forgejo Runners** are anticipated future platforms; when added, only the trigger and infrastructure layers change. (See [ADR 0007](ADRs/0007-github-actions-initial-execution-platform.md).) Infrastructure platform choice and configuration are specified in the org's `/.fullsend` repo. (See [ADR 0003](ADRs/0003-org-config-repo-convention.md).) **Open questions:** -- Do we adopt a 3rd party platform, use existing internal infrastructure, or build our own? (See [agent-infrastructure.md](problems/agent-infrastructure.md) for the three directions.) - Can different agent types (short-lived review vs. long-running implementation) run on different infrastructure? - Who in the org owns and operates this, and how does it relate to existing platform or CI ownership? +- What are the concrete resource limits (runner size, timeout, concurrency) that should be set as defaults for GitHub Actions runners? ## Agent Sandbox @@ -107,6 +107,8 @@ The mechanism that assigns work to agents and prevents conflicts. Responsible fo The existing design principle is that [the repo is the coordinator](problems/agent-architecture.md#interaction-model-the-repo-as-coordinator) — branch protection, CODEOWNERS, status checks, and forge events provide coordination without a central orchestrator. The agent dispatch and coordination layer may be nothing more than the glue that connects forge webhooks to agent infrastructure. Or it may need to be more. +For the initial implementation, GitHub Actions' event system (`on:` triggers in workflow YAML) serves as the trigger layer — translating forge events into agent workload invocations. This collapses the trigger and infrastructure into a single system initially; a future platform (Kubernetes, GitLab CI, or Forgejo Runners) would decouple them. (See [ADR 0007](ADRs/0007-github-actions-initial-execution-platform.md).) + **Open questions:** - Is the forge's event system sufficient, or do we need additional coordination logic (e.g. to prevent two implementation agents from picking up the same issue)? From fd54594eae45e34e5bf5ccfc757048ddcc8d291b Mon Sep 17 00:00:00 2001 From: Ralph Bean Date: Fri, 27 Mar 2026 17:32:52 -0400 Subject: [PATCH 4/5] Add ADR 0008: Reusable workflows for credential isolation (undecided) Proposes using GitHub reusable workflows so secrets (GitHub App private key) live only in the .fullsend repo, structurally inaccessible to enrolled repos. Marked undecided pending an experiment to verify that workflow_call actually provides this isolation. Co-Authored-By: Claude Opus 4.6 --- ...able-workflows-for-credential-isolation.md | 169 ++++++++++++++++++ 1 file changed, 169 insertions(+) create mode 100644 docs/ADRs/0008-reusable-workflows-for-credential-isolation.md diff --git a/docs/ADRs/0008-reusable-workflows-for-credential-isolation.md b/docs/ADRs/0008-reusable-workflows-for-credential-isolation.md new file mode 100644 index 0000000000..54f6a498d0 --- /dev/null +++ b/docs/ADRs/0008-reusable-workflows-for-credential-isolation.md @@ -0,0 +1,169 @@ +--- +title: "8. Reusable workflows for credential isolation" +status: Undecided +relates_to: + - agent-infrastructure + - security-threat-model + - agent-architecture +topics: + - security + - credentials + - infrastructure +--- + +# 8. Reusable workflows for credential isolation + +Date: 2026-03-27 + +## Status + +Undecided — the core premise (that GitHub reusable workflows prevent the +calling repo from accessing the called workflow's secrets) needs experimental +validation before this can be accepted. + +## Context + +[ADR 0007](0007-github-actions-initial-execution-platform.md) establishes +GitHub Actions as the initial execution platform. Enrolled repos get a thin +`.github/workflows/` stub that triggers on forge events. The question is how +secrets — particularly the GitHub App private key used to generate ephemeral +agent credentials — are kept out of the enrolled repo's reach. + +The threat: a contributor to an enrolled repo submits a PR that modifies the +workflow file to exfiltrate secrets. Even if the workflow file is +CODEOWNERS-protected (as ADR 0007 requires), the question is whether the +architecture can provide defense in depth — making secrets structurally +unavailable to the enrolled repo regardless of workflow file contents. + +GitHub's [reusable workflows](https://docs.github.com/en/actions/sharing-automations/reusing-workflows) +allow one workflow to call another via `workflow_call`. The called workflow +runs in the context of the calling repo but is defined in a different repo. +The premise of this ADR is that secrets defined in the `.fullsend` repo and +referenced in the reusable workflow are not accessible to the calling repo's +workflow — even if an attacker modifies the calling workflow. + +**This premise needs experimental proof.** Specifically: + +1. Can a calling workflow access secrets that are only available to the + reusable workflow's repo? (Expected: no.) +2. Can a calling workflow inject steps that run before or after the reusable + workflow and access its environment? (Expected: unclear.) +3. Can a calling workflow pass inputs that cause the reusable workflow to leak + secrets via outputs or logs? (Expected: possible — the reusable workflow + must be hardened against this.) +4. If the calling workflow is modified in a PR (not yet merged), does GitHub + Actions run the PR's version of the workflow or the base branch version? + (Expected: PR version for `pull_request` triggers, base version for + `push` triggers — but this matters for whether a malicious PR can + substitute a different reusable workflow reference.) + +## Options + +### Option 1: Reusable workflow in `.fullsend` repo + +The enrolled repo's workflow is a stub: + +```yaml +# .github/workflows/fullsend.yml in the enrolled repo +name: fullsend +on: + issues: + types: [labeled] +jobs: + dispatch: + uses: /.fullsend/.github/workflows/agent-dispatch.yml@main +``` + +The real workflow lives in `/.fullsend/.github/workflows/agent-dispatch.yml` +and has access to secrets defined in the `.fullsend` repo. + +**Pros:** +- Secrets never exist in the enrolled repo's settings. +- Centralized — updating the reusable workflow updates all enrolled repos. +- The enrolled repo's stub is trivially auditable. + +**Cons:** +- The isolation properties of `workflow_call` need experimental verification. +- Reusable workflows have constraints: they cannot use `strategy`, and the + calling workflow cannot add steps to the called workflow's jobs. +- Debugging is harder — the workflow definition is in a different repo from + the workflow run. + +### Option 2: Workflow in enrolled repo with org-level secrets + +The full workflow lives in each enrolled repo. Secrets are configured as +GitHub org-level secrets, scoped to repos that need them. + +**Pros:** +- Simpler — one repo, one workflow, one set of logs. +- Org-level secrets are a well-understood GitHub feature. + +**Cons:** +- Org-level secrets are available to any workflow run in the scoped repos. A + modified workflow file can access them. +- No structural isolation — defense depends entirely on CODEOWNERS preventing + workflow file changes, with no fallback. +- Each enrolled repo has its own copy of the workflow to maintain. + +### Option 3: External secret injection at runtime + +Secrets are not stored in GitHub at all. An external system (Vault, cloud KMS) +injects credentials at runtime via OIDC federation or a bootstrap token. + +**Pros:** +- Secrets never touch GitHub's secret storage. +- Fine-grained access control via the external system. + +**Cons:** +- Requires additional infrastructure (Vault, OIDC provider). +- Adds latency and a dependency on an external service's availability. +- The OIDC token or bootstrap token must still be available to the workflow + somehow — moves the problem rather than solving it. + +## Decision + +_Undecided pending experimental validation._ + +The reusable workflow approach (Option 1) is the leading candidate. If the +experiment confirms that secrets in the `.fullsend` repo are structurally +inaccessible to the calling repo's workflow, this provides meaningful defense +in depth beyond CODEOWNERS protection of the workflow file. + +### Experiment needed + +Create a test GitHub App with minimal permissions. Set up: + +1. A `.fullsend`-equivalent repo with a reusable workflow that accesses a + secret (the App's private key) and prints a confirmation (not the secret + itself). +2. An enrolled-equivalent repo with a stub workflow that calls the reusable + workflow. +3. Attempt to access the secret from the calling repo's workflow — via + additional jobs, modified inputs, environment inspection, and log + examination. +4. Test with both `push` and `pull_request` triggers to determine whether + PR-submitted workflow changes affect the reusable workflow reference. + +The experiment should be logged in `experiments/` following the project's +existing convention (see `experiments/67-claude-github-app-auth/`). + +## Consequences + +_Consequences depend on the experiment's outcome._ + +If the reusable workflow approach is validated: + +- The `.fullsend` repo becomes the only place where the GitHub App private key + is stored, reducing the attack surface to a single, tightly-governed repo. +- Enrolled repos never need secret configuration — they just reference the + reusable workflow. +- The enrolled repo's workflow stub is simple enough to be templated and + version-checked by a drift scanner. +- Contributors to enrolled repos cannot exfiltrate credentials by modifying + workflow files, even if CODEOWNERS review is somehow bypassed — structural + isolation provides defense in depth. + +If the experiment reveals that reusable workflows do not provide sufficient +isolation, Option 2 or Option 3 should be reconsidered, and CODEOWNERS +protection of the workflow file (per ADR 0007) becomes the primary defense +rather than a secondary one. From 5f8a58603d733908eef59d4a43d0f41b0489ae4b Mon Sep 17 00:00:00 2001 From: Ralph Bean Date: Mon, 30 Mar 2026 20:51:31 -0400 Subject: [PATCH 5/5] Reframe ADR 0008 around execution isolation, not credential format MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit OIDC federation vs stored secrets is orthogonal — the core decision is isolating the agent's execution namespace from enrolled repo owners. Credential provisioning is now a separate discussion section rather than a competing option. Co-Authored-By: Claude Opus 4.6 --- ...able-workflows-for-credential-isolation.md | 63 ++++++++++++------- 1 file changed, 40 insertions(+), 23 deletions(-) diff --git a/docs/ADRs/0008-reusable-workflows-for-credential-isolation.md b/docs/ADRs/0008-reusable-workflows-for-credential-isolation.md index 54f6a498d0..72a8c56038 100644 --- a/docs/ADRs/0008-reusable-workflows-for-credential-isolation.md +++ b/docs/ADRs/0008-reusable-workflows-for-credential-isolation.md @@ -26,21 +26,32 @@ validation before this can be accepted. [ADR 0007](0007-github-actions-initial-execution-platform.md) establishes GitHub Actions as the initial execution platform. Enrolled repos get a thin `.github/workflows/` stub that triggers on forge events. The question is how -secrets — particularly the GitHub App private key used to generate ephemeral -agent credentials — are kept out of the enrolled repo's reach. +to isolate the right to act as the agent — keeping it in a namespace where +the owners of individual enrolled repos cannot execute arbitrary code or +arbitrary workflows. + +This decision is about **execution isolation**, not credential format. Whether +the agent authenticates via an OIDC JWT, a long-lived secret, or a short-lived +token, the core requirement is the same: owners of enrolled repos must not be +able to access agent credentials directly. The choice between OIDC federation +and stored secrets is a complementary decision that can be layered on top of +this one. (And if an attacker must compromise the agent runtime itself to +exfiltrate credentials, that's a sandboxing concern — taken up separately.) The threat: a contributor to an enrolled repo submits a PR that modifies the -workflow file to exfiltrate secrets. Even if the workflow file is -CODEOWNERS-protected (as ADR 0007 requires), the question is whether the -architecture can provide defense in depth — making secrets structurally -unavailable to the enrolled repo regardless of workflow file contents. +workflow file to exfiltrate credentials or run arbitrary code in the agent's +execution context. Even if the workflow file is CODEOWNERS-protected (as +ADR 0007 requires), the question is whether the architecture can provide +defense in depth — making the agent's execution namespace structurally +inaccessible to enrolled repos regardless of workflow file contents. GitHub's [reusable workflows](https://docs.github.com/en/actions/sharing-automations/reusing-workflows) allow one workflow to call another via `workflow_call`. The called workflow runs in the context of the calling repo but is defined in a different repo. -The premise of this ADR is that secrets defined in the `.fullsend` repo and -referenced in the reusable workflow are not accessible to the calling repo's -workflow — even if an attacker modifies the calling workflow. +The premise of this ADR is that reusable workflows provide this execution +isolation — code and secrets defined in the `.fullsend` repo's workflow are +not accessible to the calling repo's workflow, even if an attacker modifies +the calling workflow. **This premise needs experimental proof.** Specifically: @@ -105,20 +116,26 @@ GitHub org-level secrets, scoped to repos that need them. workflow file changes, with no fallback. - Each enrolled repo has its own copy of the workflow to maintain. -### Option 3: External secret injection at runtime +## Credential provisioning (orthogonal) -Secrets are not stored in GitHub at all. An external system (Vault, cloud KMS) -injects credentials at runtime via OIDC federation or a bootstrap token. +How credentials are provisioned — OIDC federation, stored GitHub secrets, +external secret managers — is a separate decision that layers on top of this +one. OIDC federation determines *how* a credential is issued; the options +above determine *where the code that receives it runs*. In practice, OIDC +federation would be used *inside* whichever execution model is chosen, not +instead of it. -**Pros:** -- Secrets never touch GitHub's secret storage. -- Fine-grained access control via the external system. +Notably, GitHub's OIDC token includes a `job_workflow_ref` claim that +identifies the reusable workflow. An OIDC trust policy can restrict federation +to only the `.fullsend` repo's workflow, reinforcing Option 1's isolation. +This means the two approaches compose well — execution isolation prevents +enrolled repos from running arbitrary code in the agent namespace, and OIDC +federation ensures that even within that namespace, credentials are +short-lived and auditable. -**Cons:** -- Requires additional infrastructure (Vault, OIDC provider). -- Adds latency and a dependency on an external service's availability. -- The OIDC token or bootstrap token must still be available to the workflow - somehow — moves the problem rather than solving it. +OIDC federation does not solve execution isolation on its own: an enrolled +repo that can run arbitrary code in the workflow can also request the OIDC +token. ## Decision @@ -164,6 +181,6 @@ If the reusable workflow approach is validated: isolation provides defense in depth. If the experiment reveals that reusable workflows do not provide sufficient -isolation, Option 2 or Option 3 should be reconsidered, and CODEOWNERS -protection of the workflow file (per ADR 0007) becomes the primary defense -rather than a secondary one. +isolation, Option 2 should be reconsidered, and CODEOWNERS protection of the +workflow file (per ADR 0007) becomes the primary defense rather than a +secondary one.