From 97564926d827bd4ef538ca07e7a7129e3da48bb0 Mon Sep 17 00:00:00 2001 From: Adam Scerra Date: Tue, 7 Apr 2026 11:30:25 -0400 Subject: [PATCH] Rename implementation agent to code agent across the repo MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Team consensus from the April 6 sync: the "implementation agent" is now the "code agent" for conciseness and consistency with the other short agent names (triage, review). ADRs are left untouched as point-in-time records. - Workflow: implementation.yml → code.yml, labels ready-to-implement → ready-to-code, slash command /implement → /code, secrets and env vars renamed accordingly - Living docs: architecture, glossary, landscape, and all problem docs updated (agent-architecture, code-review, production-feedback, security-threat-model, agent-infrastructure) Made-with: Cursor --- dispatch/github/actions/fullsend/action.yml | 2 +- .../{implementation.yml => code.yml} | 24 ++++++------- docs/architecture.md | 16 ++++----- docs/glossary.md | 20 +++++------ docs/landscape.md | 4 +-- docs/problems/agent-architecture.md | 34 +++++++++---------- docs/problems/agent-infrastructure.md | 4 +-- docs/problems/code-review.md | 8 ++--- docs/problems/production-feedback.md | 8 ++--- docs/problems/security-threat-model.md | 12 +++---- 10 files changed, 66 insertions(+), 66 deletions(-) rename dispatch/github/workflows/{implementation.yml => code.yml} (51%) diff --git a/dispatch/github/actions/fullsend/action.yml b/dispatch/github/actions/fullsend/action.yml index 111b0eb795..49ae65dfea 100644 --- a/dispatch/github/actions/fullsend/action.yml +++ b/dispatch/github/actions/fullsend/action.yml @@ -9,7 +9,7 @@ branding: inputs: stage: - description: Pipeline stage passed to fullsend (e.g. triage, implementation, review). + description: Pipeline stage passed to fullsend (e.g. triage, code, review). required: true scm: description: Value for fullsend --scm (optional; defaults to github). diff --git a/dispatch/github/workflows/implementation.yml b/dispatch/github/workflows/code.yml similarity index 51% rename from dispatch/github/workflows/implementation.yml rename to dispatch/github/workflows/code.yml index 2507dfe902..0f54f3173c 100644 --- a/dispatch/github/workflows/implementation.yml +++ b/dispatch/github/workflows/code.yml @@ -1,4 +1,4 @@ -name: Implementation +name: Code on: issues: @@ -7,17 +7,17 @@ on: types: [created] concurrency: - group: fullsend-implementation-${{ github.event.issue.number }} + group: fullsend-code-${{ github.event.issue.number }} cancel-in-progress: true jobs: - implementation: - name: Implementation + code: + name: Code if: | - (github.event_name == 'issues' && github.event.action == 'labeled' && github.event.label.name == 'ready-to-implement') || + (github.event_name == 'issues' && github.event.action == 'labeled' && github.event.label.name == 'ready-to-code') || (github.event_name == 'issue_comment' && ( - startsWith(github.event.comment.body || '', '/implement ') || - (github.event.comment.body || '') == '/implement' + startsWith(github.event.comment.body || '', '/code ') || + (github.event.comment.body || '') == '/code' )) runs-on: ubuntu-latest permissions: @@ -29,16 +29,16 @@ jobs: - name: Checkout repository uses: actions/checkout@v4 with: - token: ${{ secrets.FULLSEND_IMPLEMENTATION_BOT_TOKEN }} + token: ${{ secrets.FULLSEND_CODE_BOT_TOKEN }} fetch-depth: 1 - name: Setup agent environment env: - STAGE_PREFIX: IMPLEMENTATION_ - IMPLEMENTATION_FULLSEND_IMPLEMENTATION_BOT_TOKEN: ${{ secrets.FULLSEND_IMPLEMENTATION_BOT_TOKEN }} + STAGE_PREFIX: CODE_ + CODE_FULLSEND_CODE_BOT_TOKEN: ${{ secrets.FULLSEND_CODE_BOT_TOKEN }} run: bash .github/scripts/setup-agent-env.sh - - name: Run implementation agent(s) + - name: Run code agent(s) uses: ./.github/actions/fullsend with: - stage: implementation + stage: code diff --git a/docs/architecture.md b/docs/architecture.md index 71e0958d67..2253f41e3e 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -31,7 +31,7 @@ Infrastructure platform choice and configuration are specified in the adopting o **Open questions:** - Do we adopt a 3rd party platform, use existing internal infrastructure, or build our own? (See [agent-infrastructure.md](problems/agent-infrastructure.md) for the three directions.) -- Can different agent types (short-lived review vs. long-running implementation) run on different infrastructure? +- Can different agent types (short-lived review vs. long-running code) run on different infrastructure? - Who in the org owns and operates this, and how does it relate to existing platform or CI ownership? - Should model and MCP (or other tool-protocol) traffic from agent runtimes go through a **shared gateway** for authentication, spend limits, allowlists, and telemetry? (See [landscape.md](landscape.md#agent-gateway).) @@ -104,7 +104,7 @@ The existing design principle is that [the repo is the coordinator](problems/age **Open questions:** -- Is GitHub's event system sufficient, or do we need additional coordination logic (e.g. to prevent two implementation agents from picking up the same issue)? +- Is GitHub's event system sufficient, or do we need additional coordination logic (e.g. to prevent two code agents from picking up the same issue)? - How does work assignment interact with the backlog/priority agent described in [agent-architecture.md](problems/agent-architecture.md)? - What happens when work needs to be cancelled, retried, or reassigned? - Does the coordinator need state (a queue, a lock, a claim system), or can it be stateless and event-driven? @@ -177,7 +177,7 @@ ADR 0002: [Building block 1](ADRs/0002-initial-fullsend-design.md#1-webhook--dis ### 2. Slash-command parser + ACL -Parses `/triage`, `/implement`, `/review`, and related commands and enforces who is allowed to invoke each. +Parses `/triage`, `/code`, `/review`, and related commands and enforces who is allowed to invoke each. ADR 0002: [Building block 2](ADRs/0002-initial-fullsend-design.md#2-slash-command-parser--acl). ### 3. Label state machine guard @@ -205,19 +205,19 @@ ADR 0002: [Building block 6](ADRs/0002-initial-fullsend-design.md#6-repro-sandbo Formats triage test artifacts in repo-native conventions for PR handoff. ADR 0002: [Building block 7](ADRs/0002-initial-fullsend-design.md#7-test-artifact-formatter). -### 8. implementation agent runtime +### 8. code agent runtime Implements changes, runs local/CI-equivalent tests, handles check failures, and advances handoff to **Review** (`ready-for-review`). ADR 0002: [Building block 8](ADRs/0002-initial-fullsend-design.md#8-implementation-agent-runtime). ### 9. PR sandbox / CI mirror -Execution environment for **Implementation** and test loops, aligned to contributor/CI toolchains. +Execution environment for **Code** and test loops, aligned to contributor/CI toolchains. ADR 0002: [Building block 9](ADRs/0002-initial-fullsend-design.md#9-pr-sandbox--ci-mirror). ### 10. Check failure triage -Fetches and classifies failing check logs to guide **implementation agent** remediation loops. +Fetches and classifies failing check logs to guide **code agent** remediation loops. ADR 0002: [Building block 10](ADRs/0002-initial-fullsend-design.md#10-check-failure-triage). ### 11. review agent runtime @@ -230,12 +230,12 @@ ADR 0002: [Building block 11](ADRs/0002-initial-fullsend-design.md#11-review-age Aggregates review verdicts and applies labels: - unanimous approve-merge → `ready-for-merge` (for the **current** PR head at the end of that round only) -- unanimous rework → `ready-to-implement` +- unanimous rework → `ready-to-code` - split/conflicting (including conflicting security severities) → `requires-manual-review` - each **review run start** (including push-triggered re-review) clears **`ready-for-merge`** together with **`ready-for-review`** so merge approval is never stale after new commits ADR 0002: [Building block 12](ADRs/0002-initial-fullsend-design.md#12-coordinator-merge-algorithm). ### 13. Observability -Traceability layer across issue, **Triage**, **Implementation**, **Review**, checks, and merge for incident response and correlation across automation runs. +Traceability layer across issue, **Triage**, **Code**, **Review**, checks, and merge for incident response and correlation across automation runs. ADR 0002: [Building block 13](ADRs/0002-initial-fullsend-design.md#13-observability). diff --git a/docs/glossary.md b/docs/glossary.md index f1295fa4b3..c31eb33a9b 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -15,7 +15,7 @@ See [architecture.md](architecture.md) and [agent-infrastructure.md](problems/ag ### Agent Registry -The catalog of available agent roles and their configurations. It bridges the abstract roles defined in the agent architecture (triage, implementation, review) and the concrete runtime configurations the harness uses to instantiate each agent. Fullsend provides a base set; adopting organizations extend it via their `.fullsend` repository. +The catalog of available agent roles and their configurations. It bridges the abstract roles defined in the agent architecture (triage, code, review) and the concrete runtime configurations the harness uses to instantiate each agent. Fullsend provides a base set; adopting organizations extend it via their `.fullsend` repository. See [architecture.md](architecture.md). ### Agent Runtime @@ -62,7 +62,7 @@ A workflow concept where a repository automatically stays up-to-date with depend ### Flapping -When agents enter a cycle of conflicting feedback that prevents convergence. Example: the security review agent rejects what the implementation agent produces to satisfy the correctness review agent, and vice versa, creating an oscillating loop. Flapping is a primary trigger for [escalation](#escalation) — after a configurable number of cycles, the system stops and routes to humans. +When agents enter a cycle of conflicting feedback that prevents convergence. Example: the security review agent rejects what the code agent produces to satisfy the correctness review agent, and vice versa, creating an oscillating loop. Flapping is a primary trigger for [escalation](#escalation) — after a configurable number of cycles, the system stops and routes to humans. See [autonomy-spectrum.md](problems/autonomy-spectrum.md). ## H @@ -83,7 +83,7 @@ See [architecture.md](architecture.md) and [agent-architecture.md](problems/agen ### Label State Machine -The set of valid label transitions on issues and PRs that encode workflow state. Labels like `ready-to-implement`, `ready-for-review`, `ready-for-merge`, and `requires-manual-review` are control markers that drive agent dispatch and enforce ordering. The label state machine guard validates that transitions are legal and enforces mutual exclusion — for example, starting a triage run clears downstream labels so stale state does not carry forward. +The set of valid label transitions on issues and PRs that encode workflow state. Labels like `ready-to-code`, `ready-for-review`, `ready-for-merge`, and `requires-manual-review` are control markers that drive agent dispatch and enforce ordering. The label state machine guard validates that transitions are legal and enforces mutual exclusion — for example, starting a triage run clears downstream labels so stale state does not carry forward. See [ADR 0002](ADRs/0002-initial-fullsend-design.md) building block 3. ## M @@ -114,9 +114,9 @@ See [architecture.md](architecture.md) and [governance.md](problems/governance.m ## R -### Ready to Implement +### Ready to Code -A label indicating an issue has passed triage and is cleared for the implementation agent to begin work. It is a key transition point in the [label state machine](#label-state-machine) — the triage agent sets it after confirming the issue is not a duplicate, is reproducible (if applicable), is a bug (not a feature, unless features are in scope), and has sufficient detail for implementation. The implementation agent watches for this label as its trigger to begin work. +A label indicating an issue has passed triage and is cleared for the code agent to begin work. It is a key transition point in the [label state machine](#label-state-machine) — the triage agent sets it after confirming the issue is not a duplicate, is reproducible (if applicable), is a bug (not a feature, unless features are in scope), and has sufficient detail for the code agent. The code agent watches for this label as its trigger to begin work. See [ADR 0002](ADRs/0002-initial-fullsend-design.md). ### Rework Rate @@ -142,7 +142,7 @@ See [architecture.md](architecture.md) and [codebase-context.md](problems/codeba ### Stage -A higher-level workflow component in the fullsend pipeline (e.g., triage, implementation, review). The team formally chose "stage" over "phase" to avoid overloading the general SDLC use of "phase" and to maintain distinct vocabulary from Tekton's pipeline/task/step hierarchy, since fullsend may run on Tekton infrastructure. Each stage contains one or more [steps](#step). +A higher-level workflow component in the fullsend pipeline (e.g., triage, code, review). The team formally chose "stage" over "phase" to avoid overloading the general SDLC use of "phase" and to maintain distinct vocabulary from Tekton's pipeline/task/step hierarchy, since fullsend may run on Tekton infrastructure. Each stage contains one or more [steps](#step). See [ADR 0002](ADRs/0002-initial-fullsend-design.md). ### Step @@ -152,7 +152,7 @@ See [ADR 0002](ADRs/0002-initial-fullsend-design.md). ### Slash Command -A GitHub comment in the form `/triage`, `/implement`, `/review`, etc., that manually triggers an agent workflow. Slash commands are parsed by the entry point and gated by an ACL — not every user can invoke every command. They provide an explicit human-initiated trigger alongside the automatic label-based triggers. +A GitHub comment in the form `/triage`, `/code`, `/review`, etc., that manually triggers an agent workflow. Slash commands are parsed by the entry point and gated by an ACL — not every user can invoke every command. They provide an explicit human-initiated trigger alongside the automatic label-based triggers. See [ADR 0002](ADRs/0002-initial-fullsend-design.md) building block 2. ## T @@ -164,7 +164,7 @@ See [architecture.md](architecture.md) (building block 1). ### Triage -In fullsend, triage means routing, deduplicating, assessing completeness, and checking reproducibility — **not** fixing. The triage agent reads the issue, determines if it is a duplicate, assesses whether it is a bug or a feature (and denies if features are not in scope), checks if the issue has enough detail for implementation, and optionally attempts reproduction. The scope of triage has been a recurring discussion point, particularly around whether reproducibility and test generation belong in triage or implementation. +In fullsend, triage means routing, deduplicating, assessing completeness, and checking reproducibility — **not** fixing. The triage agent reads the issue, determines if it is a duplicate, assesses whether it is a bug or a feature (and denies if features are not in scope), checks if the issue has enough detail for the code agent, and optionally attempts reproduction. The scope of triage has been a recurring discussion point, particularly around whether reproducibility and test generation belong in triage or implementation. See [ADR 0002](ADRs/0002-initial-fullsend-design.md) building block 4 and [#86](https://github.com/fullsend-ai/fullsend/issues/86). ### Trust @@ -182,12 +182,12 @@ See [security-threat-model.md](problems/security-threat-model.md) and [agent-arc ### Work Coordinator -The mechanism that assigns work to agents and prevents conflicts. The existing design principle is that the **repo is the coordinator** — branch protection, CODEOWNERS, status checks, and GitHub events provide coordination without a central orchestrator. The work coordinator may be just the glue connecting GitHub webhooks to agent infrastructure, or it may need to be more (e.g., a claim/lock system to prevent two implementation agents from picking up the same issue). +The mechanism that assigns work to agents and prevents conflicts. The existing design principle is that the **repo is the coordinator** — branch protection, CODEOWNERS, status checks, and GitHub events provide coordination without a central orchestrator. The work coordinator may be just the glue connecting GitHub webhooks to agent infrastructure, or it may need to be more (e.g., a claim/lock system to prevent two code agents from picking up the same issue). See [architecture.md](architecture.md) and [#77](https://github.com/fullsend-ai/fullsend/issues/77). ## Z ### Zero Trust -In fullsend's agent-to-agent model, zero trust means **nothing is trusted implicitly based on identity alone**. It does **not** mean "accept zero inputs" or "block everything." Every agent assumes every other agent — and every external input — could be compromised. The implementation agent assumes the triage output may contain prompt injection. The review agent assumes the submitted PR is designed to trick it. Defense is layered: input sanitization, scoped permissions, sandbox containment, and output validation all work together. +In fullsend's agent-to-agent model, zero trust means **nothing is trusted implicitly based on identity alone**. It does **not** mean "accept zero inputs" or "block everything." Every agent assumes every other agent — and every external input — could be compromised. The code agent assumes the triage output may contain prompt injection. The review agent assumes the submitted PR is designed to trick it. Defense is layered: input sanitization, scoped permissions, sandbox containment, and output validation all work together. See [security-threat-model.md](problems/security-threat-model.md) (Threat 5) and [#102](https://github.com/fullsend-ai/fullsend/issues/102). diff --git a/docs/landscape.md b/docs/landscape.md index 1e062b25d7..2989c98cb6 100644 --- a/docs/landscape.md +++ b/docs/landscape.md @@ -51,7 +51,7 @@ Takes a fundamentally different angle: stacked PRs. Instead of reviewing one mas **Merge queue:** Stack-aware merge queue batches and tests multiple PRs in parallel. "Merge when ready" auto-pilots stack merges once approved — but approval is still human. -**Relevance to fullsend:** The stacked PR insight is important for our implementation agents. Smaller, focused changes are easier for review sub-agents to evaluate with confidence. If implementation agents produce stacked PRs rather than monolithic ones, the review problem becomes more tractable. The merge queue concept is also relevant — our system needs something similar for sequencing autonomous merges. +**Relevance to fullsend:** The stacked PR insight is important for our code agents. Smaller, focused changes are easier for review sub-agents to evaluate with confidence. If code agents produce stacked PRs rather than monolithic ones, the review problem becomes more tractable. The merge queue concept is also relevant — our system needs something similar for sequencing autonomous merges. ### Qodo (formerly PR-Agent) @@ -103,7 +103,7 @@ Launched November 2025. The closest thing in the industry to autonomous merging. ## Production agent orchestration systems -While the tools above focus on code review, a separate category of systems addresses end-to-end agent orchestration — from task intake through implementation and merge. These are closer to the fullsend vision than review-only tools. +While the tools above focus on code review, a separate category of systems addresses end-to-end agent orchestration — from task intake through coding and merge. These are closer to the fullsend vision than review-only tools. ### Stripe Minions diff --git a/docs/problems/agent-architecture.md b/docs/problems/agent-architecture.md index c269966f5f..f90dd08c9e 100644 --- a/docs/problems/agent-architecture.md +++ b/docs/problems/agent-architecture.md @@ -16,10 +16,10 @@ These may require different agent specializations with different domain knowledg No agent trusts another agent's output because of who (or what) produced it. Trust is derived from the repository's permission model: - A reviewer's authority to **block a merge** comes from CODEOWNERS and GitHub approval rights — not from being "the security review agent" -- An implementation agent treats feedback from all reviewers the same way — it doesn't give special weight to a comment because it appears to come from a system agent +- A code agent treats feedback from all reviewers the same way — it doesn't give special weight to a comment because it appears to come from a system agent - Every agent treats every input as potentially adversarial, regardless of apparent source -**The one exception:** if a reviewer has approval rights in the repo (via CODEOWNERS or branch protection), the implementation agent can recognize that reviewer's authority to raise *blocking* concerns. It still must take defensive measures when processing that reviewer's comments — authorized identity doesn't mean safe content. +**The one exception:** if a reviewer has approval rights in the repo (via CODEOWNERS or branch protection), the code agent can recognize that reviewer's authority to raise *blocking* concerns. It still must take defensive measures when processing that reviewer's comments — authorized identity doesn't mean safe content. This mirrors how humans work today. You don't trust a code reviewer because they claim to be senior. You trust their authority to block because GitHub shows they have approval rights on that path. @@ -29,23 +29,23 @@ Code review happens twice: before a PR is submitted and after. Both phases run t ### Phase 1: Pre-PR review (shift left) -Before the implementation agent commits or opens a PR, it invokes the same review sub-agents locally. This catches problems before they consume attention at the PR level. +Before the code agent commits or opens a PR, it invokes the same review sub-agents locally. This catches problems before they consume attention at the PR level. -- Higher quality output — the implementation agent iterates on its own work before exposing it -- Faster cycle time — fewer round-trips between implementation and review +- Higher quality output — the code agent iterates on its own work before exposing it +- Faster cycle time — fewer round-trips between coding and review - Lower resource waste — bad changes never become PRs This is a normal pattern for humans using coding agents today. The agent writes code, reviews it, fixes issues, and only then submits. ### Phase 2: PR-level review -The PR is open. Review sub-agents evaluate it with no special trust granted because the code came from an implementation agent. The review process is identical whether the PR author is an agent or a human. The review agents don't know or care. +The PR is open. Review sub-agents evaluate it with no special trust granted because the code came from a code agent. The review process is identical whether the PR author is an agent or a human. The review agents don't know or care. -This is important: **the PR-level review is not a rubber stamp of the pre-PR review.** It's a fully independent evaluation. The pre-PR review helps the implementation agent produce better output; the PR-level review is the actual gate. +This is important: **the PR-level review is not a rubber stamp of the pre-PR review.** It's a fully independent evaluation. The pre-PR review helps the code agent produce better output; the PR-level review is the actual gate. ## Agent roles -### Implementation agent +### Code agent Writes code to address an issue. This is the most mature capability of current AI coding tools. @@ -85,7 +85,7 @@ This is a consequential choice. A narrow fix is safer and faster but leaves iden A useful decision boundary: **can the pattern be expressed as a static analysis rule?** -- **If yes** (e.g., unchecked nil dereference, missing error return check, deprecated API usage, format string mismatch): the triage agent should recommend broad remediation. The pattern is mechanical, the fix is deterministic, and a linter or codemod can validate completeness. In this case, the triage agent creates a single issue scoped to the pattern, not the instance. The implementation agent applies the fix codebase-wide and ideally adds a linter rule or CI check to prevent recurrence. +- **If yes** (e.g., unchecked nil dereference, missing error return check, deprecated API usage, format string mismatch): the triage agent should recommend broad remediation. The pattern is mechanical, the fix is deterministic, and a linter or codemod can validate completeness. In this case, the triage agent creates a single issue scoped to the pattern, not the instance. The code agent applies the fix codebase-wide and ideally adds a linter rule or CI check to prevent recurrence. - **If no** (e.g., a race condition in a specific interaction, a logic error in business rules, an incorrect algorithm for a particular domain case): the fix requires contextual judgment at each call site. Applying it broadly risks introducing incorrect behavior where the pattern superficially matches but the semantics differ. In this case, the triage agent should fix the reported instance and then **scan for similar occurrences to create derivative issues** — one per location — so each gets individual analysis and review. @@ -114,9 +114,9 @@ Conversely, if the triage agent creates 30 derivative issues for the same patter Some bugs sit between the two categories — the pattern is somewhat mechanical but has enough contextual variation that a blanket fix is risky. In these cases, a hybrid approach is possible: -- The implementation agent fixes the reported instance +- The code agent fixes the reported instance - The triage agent creates derivative issues for candidate locations -- The derivative issues include a suggested fix but flag it as "needs verification" — the implementation agent working each derivative must confirm the fix is appropriate for that specific context before applying it +- The derivative issues include a suggested fix but flag it as "needs verification" — the code agent working each derivative must confirm the fix is appropriate for that specific context before applying it This is more expensive than either pure approach but avoids both the blast radius of incorrect broad fixes and the amnesia of purely narrow fixes. @@ -124,7 +124,7 @@ This is more expensive than either pure approach but avoids both the blast radiu Determines what should be worked on next based on priority, urgency, and available capacity. -- **Authority:** Assign work to implementation agents, reorder priority +- **Authority:** Assign work to code agents, reorder priority - **Considerations:** Needs access to strategic intent to make good decisions ### Quality/drift detection agent @@ -158,7 +158,7 @@ No agent orchestrates other agents. Each agent independently observes the state 1. A PR is opened → review sub-agents are triggered (by webhook/GitHub event) 2. Each review sub-agent independently evaluates the PR and posts its findings (as status checks or structured comments) -3. If a review sub-agent requests changes → the implementation agent sees the comment and responds (treating it as untrusted input, but recognizing blocking authority if the reviewer has approval rights) +3. If a review sub-agent requests changes → the code agent sees the comment and responds (treating it as untrusted input, but recognizing blocking authority if the reviewer has approval rights) 4. The merge decision is a **deterministic function of state**: all required status checks pass, all required CODEOWNERS approvals present, no blocking reviews outstanding The "coordination logic" is the repository's branch protection configuration — not an LLM making judgment calls about when to proceed. @@ -184,8 +184,8 @@ There is no side channel. No agent-to-agent API. No shared state outside the rep Without a coordinator, what happens when agents disagree? (e.g., correctness agent approves, security agent blocks) - **Security and intent sub-agents have veto power** via required status checks. If they block, the PR doesn't merge. This is configured in branch protection, not in agent logic. -- **The implementation agent can iterate** — push new commits to address blocking concerns, which re-triggers the review sub-agents -- **Persistent disagreement escalates to humans** — if an implementation agent can't satisfy a blocking reviewer after N iterations, the PR is flagged for human intervention. This is a safeguard against infinite loops, not a normal path. The escalation can use [dual-interpretation escalation](code-review.md#dual-interpretation-escalation) to present the human with the approving and blocking agents' readings — while making clear the human can reject both framings or the PR itself — so the human resolves the disagreement quickly rather than re-reviewing the entire PR. +- **The code agent can iterate** — push new commits to address blocking concerns, which re-triggers the review sub-agents +- **Persistent disagreement escalates to humans** — if a code agent can't satisfy a blocking reviewer after N iterations, the PR is flagged for human intervention. This is a safeguard against infinite loops, not a normal path. The escalation can use [dual-interpretation escalation](code-review.md#dual-interpretation-escalation) to present the human with the approving and blocking agents' readings — while making clear the human can reject both framings or the PR itself — so the human resolves the disagreement quickly rather than re-reviewing the entire PR. - **Humans can always override** — a human with approval rights can approve despite agent objections. The system assists; humans retain ultimate authority. ## Relationship to multi-agent frameworks @@ -216,9 +216,9 @@ The multi-agent framework space is expanding rapidly, with new entries appearing - Should there be one instance of each agent type per repo, per org, or shared? Per-repo is simpler but more expensive. Shared agents need careful isolation. (Infrastructure constrains this — see [agent-infrastructure.md](agent-infrastructure.md).) - ~~What's the right model for agent identity? Agents need GitHub accounts to post comments and status checks. Separate bot accounts per agent role? A single bot account with role indicated in the comment? GitHub App installations?~~ Decided in [ADR 0007](../ADRs/0007-per-role-github-apps.md): per-role GitHub Apps with manifest-based creation. - How do we test the interaction model? Can we simulate adversarial scenarios (injection attempts, unauthorized changes, agent disagreements) in a sandbox repo? -- How does the two-phase review model work in practice? Does the implementation agent run all six sub-agents locally, or a subset? Is the pre-PR review a lighter version? (Depends on [agent-infrastructure.md](agent-infrastructure.md) — what compute is available where.) +- How does the two-phase review model work in practice? Does the code agent run all six sub-agents locally, or a subset? Is the pre-PR review a lighter version? (Depends on [agent-infrastructure.md](agent-infrastructure.md) — what compute is available where.) - What's the iteration limit before human escalation? Too low and humans get pulled in constantly. Too high and the system wastes resources on unresolvable conflicts. -- How do we handle agent-generated PR content that is itself an injection vector? An implementation agent's code, commit messages, and PR description are all consumed by review agents. The injection defense agent needs to evaluate this content, but how do we prevent the injection defense agent itself from being influenced by it? +- How do we handle agent-generated PR content that is itself an injection vector? A code agent's code, commit messages, and PR description are all consumed by review agents. The injection defense agent needs to evaluate this content, but how do we prevent the injection defense agent itself from being influenced by it? - How does the triage agent determine whether a bug pattern is mechanical enough for broad remediation vs. context-dependent enough to require per-instance analysis? Can this classification itself be expressed as a heuristic, or does it always require LLM judgment? - What's the threshold for derivative issue accumulation before the quality/drift detection agent should recommend consolidation into a pattern-level fix? Is it a count (e.g., 10+ derivatives for the same pattern), a density (e.g., more than N% of files in a package), or a cost measure? - When the triage agent creates derivative issues, how should the priority agent handle them — batch all at the same priority as the original, deprioritize as "known technical debt," or evaluate each independently? diff --git a/docs/problems/agent-infrastructure.md b/docs/problems/agent-infrastructure.md index d6a17520aa..dfe5adee49 100644 --- a/docs/problems/agent-infrastructure.md +++ b/docs/problems/agent-infrastructure.md @@ -63,7 +63,7 @@ Design and operate dedicated agent infrastructure: runner pool, sandboxing, tool Agents are often discussed as if they run on a developer workstation: fast local builds, an interactive shell, and a stable working tree. In practice, many organizations will run them on **shared CI runners, Kubernetes, or other ephemeral, network-only environments**. That shift surfaces tensions that are easy to underestimate when prototyping locally. -- **Privilege versus validation** — Implementation agents may need to build container images, run integration tests, or reproduce fixtures that mirror CI. That pressure leads toward Docker-in-Docker, nested builders, or highly capable pods. Granting **`privileged`-equivalent or host-level access** to a workload whose behavior is driven by an LLM greatly expands blast radius; the overlap with [security-threat-model.md](security-threat-model.md) is direct. The design problem is how to validate changes **without** making the agent runtime a root-equivalent attack surface. +- **Privilege versus validation** — Code agents may need to build container images, run integration tests, or reproduce fixtures that mirror CI. That pressure leads toward Docker-in-Docker, nested builders, or highly capable pods. Granting **`privileged`-equivalent or host-level access** to a workload whose behavior is driven by an LLM greatly expands blast radius; the overlap with [security-threat-model.md](security-threat-model.md) is direct. The design problem is how to validate changes **without** making the agent runtime a root-equivalent attack surface. - **Monolithic runner images** — Putting every compiler, SDK, and linter into a single “agent runner” image minimizes per-job setup, but it produces **large images, slow provisioning, a wide dependency footprint, and painful upgrade cycles**. It also fights stack heterogeneity: real orgs use many languages and build systems (see [applied/konflux-ci](applied/konflux-ci/README.md) for one example). Finer-grained patterns — dedicated tool or task images, hermetic layers, or on-demand tooling — trade pull and scheduling latency against maintainability and security review surface. @@ -91,7 +91,7 @@ ACP may still be useful for **narrow experiments** where those constraints are a - **Thin orchestration layer** — We build a small layer that triggers agents, gathers results, and posts status checks; the actual compute is 3rd party or internal. This keeps coordination logic in our control while deferring platform choice. - **Phase by phase** — Start with a 3rd party or internal option for early experiments (e.g. review agents only); decide later whether to replace or extend with custom infrastructure as autonomy expands. -- **By agent type** — Triage and review agents might run on one platform (e.g. event-driven, short-lived); implementation agents that need more tooling and longer runs might need a different environment. +- **By agent type** — Triage and review agents might run on one platform (e.g. event-driven, short-lived); code agents that need more tooling and longer runs might need a different environment. ## Kubernetes SIG Agent Sandbox diff --git a/docs/problems/code-review.md b/docs/problems/code-review.md index 4acfa963d3..c6d1e57515 100644 --- a/docs/problems/code-review.md +++ b/docs/problems/code-review.md @@ -18,13 +18,13 @@ Code review happens twice — before and after PR submission. Both phases run th ### Phase 1: Pre-PR review (shift left) -Before the implementation agent commits or opens a PR, it invokes the review sub-agents locally. This catches problems before they consume attention at the PR level. The implementation agent iterates on its own work — fixing issues, improving test coverage, addressing security concerns — before exposing the change to the broader system. +Before the code agent commits or opens a PR, it invokes the review sub-agents locally. This catches problems before they consume attention at the PR level. The code agent iterates on its own work — fixing issues, improving test coverage, addressing security concerns — before exposing the change to the broader system. This is a normal pattern for humans using coding agents today. It produces higher quality output faster and wastes fewer resources. ### Phase 2: PR-level review (the actual gate) -The PR is open. Review sub-agents evaluate it with no special trust granted because the code came from an implementation agent that already ran pre-PR review. The PR-level review is a fully independent evaluation — not a rubber stamp of Phase 1. +The PR is open. Review sub-agents evaluate it with no special trust granted because the code came from a code agent that already ran pre-PR review. The PR-level review is a fully independent evaluation — not a rubber stamp of Phase 1. The review process is identical whether the PR author is an agent or a human. The review agents don't know or care about authorship. They evaluate: @@ -113,7 +113,7 @@ Evaluates adherence to repo-specific patterns and conventions. - Naming conventions, API patterns, error handling idioms - Documentation adequacy -**Context needed:** The diff, repo style guides, examples of existing patterns. This is the lowest-stakes review concern and could potentially be handled by the implementation agent's pre-PR self-review rather than a separate sub-agent. +**Context needed:** The diff, repo style guides, examples of existing patterns. This is the lowest-stakes review concern and could potentially be handled by the code agent's pre-PR self-review rather than a separate sub-agent. ## How sub-agents compose a decision @@ -135,7 +135,7 @@ Each sub-agent produces a score. A weighted aggregate determines the outcome. Se ### Veto-based with tiers -Security and intent agents have veto power (any rejection blocks). Correctness and style agents can flag concerns but not block — their concerns are surfaced for human review or implementation agent iteration. +Security and intent agents have veto power (any rejection blocks). Correctness and style agents can flag concerns but not block — their concerns are surfaced for human review or code agent iteration. **Pros:** Balances safety with throughput. Security is non-negotiable, style is advisory. **Cons:** Still need to define what counts as a "security" concern vs. a "correctness" concern. Boundary is fuzzy. diff --git a/docs/problems/production-feedback.md b/docs/problems/production-feedback.md index bbd4f7072a..ad03c38e0d 100644 --- a/docs/problems/production-feedback.md +++ b/docs/problems/production-feedback.md @@ -53,7 +53,7 @@ Correlating user-reported problems with platform signals serves two purposes: **Review agent** uses platform reliability history to calibrate scrutiny on PRs. A code path responsible for a high fraction of recent scheduling timeouts or failure spikes warrants deeper edge-case analysis than a low-traffic utility. This also feeds tier classification — a "bug fix" touching a historically high-blast-radius path may warrant Tier 2 treatment regardless of how the issue was filed. -**Implementation agent** uses failure logs, error distributions, and timing correlation as starting context — richer than a human-written issue. The agent can correlate log patterns to code paths and generate a root cause hypothesis before writing any code. +**Code agent** uses failure logs, error distributions, and timing correlation as starting context — richer than a human-written issue. The agent can correlate log patterns to code paths and generate a root cause hypothesis before writing any code. **Post-merge validation** is where the loop closes. A fix that passes CI but doesn't move the platform failure rate has not solved the problem. If the signal doesn't return to baseline after deploy, the issue re-opens and the agent flags for human review. @@ -68,7 +68,7 @@ Triage agent detects pattern, creates issue with signal data ↓ Priority agent weights issue by impact breadth ↓ -Implementation agent uses failure logs as context, implements fix +Code agent uses failure logs as context, implements fix ↓ Review agents evaluate change, with execution-informed risk context ↓ @@ -101,11 +101,11 @@ Without reliable attribution, the triage agent files issues for problems the pla **Alert fatigue at agent scale**: Filing an issue for every anomalous signal generates more work than can be absorbed. The triage agent needs minimum thresholds on duration, breadth, and statistical significance before acting. -**False-positive remediation loops**: When attribution misclassifies a user error or supply chain change as a platform bug, the implementation agent proposes a fix. The fix merges, deploys, and the signal is unchanged — because the root cause was never in the platform's code. The issue re-opens and the cycle repeats. Each iteration adds real codebase changes that increase complexity without improving reliability. If reviewers see repeated small patches to the same area with no visible effect, they may begin rubber-stamping — eroding the oversight that would otherwise catch the loop. +**False-positive remediation loops**: When attribution misclassifies a user error or supply chain change as a platform bug, the code agent proposes a fix. The fix merges, deploys, and the signal is unchanged — because the root cause was never in the platform's code. The issue re-opens and the cycle repeats. Each iteration adds real codebase changes that increase complexity without improving reliability. If reviewers see repeated small patches to the same area with no visible effect, they may begin rubber-stamping — eroding the oversight that would otherwise catch the loop. Detection requires two complementary stopping conditions. First, iteration count: if post-merge validation shows no improvement across N consecutive iterations on the same signal, the loop must halt and escalate to human investigation. Second, cost budget: each iteration burns compute resources and token budget — CI runs, agent context processing, code review cycles. A runaway loop is not just an oversight risk; it is a measurable resource cost. Budget exhaustion per signal (e.g., cumulative agent cost above a threshold for a single issue lineage) should be an independent stopping condition, not a consequence of hitting the iteration cap. Both limits must be built in explicitly and both must trigger escalation, not silent abandonment. -Prevention: the triage agent should not generate an implementation-ready issue without sufficient corroborating evidence — a correlated deploy event, minimum cross-tenant breadth, signal-to-noise ratio above threshold, and failure log content consistent with a platform origin. Below that threshold, the output is a flagged observation for human triage, not an actionable issue. +Prevention: the triage agent should not generate a code-ready issue without sufficient corroborating evidence — a correlated deploy event, minimum cross-tenant breadth, signal-to-noise ratio above threshold, and failure log content consistent with a platform origin. Below that threshold, the output is a flagged observation for human triage, not an actionable issue. ## Product discovery signals diff --git a/docs/problems/security-threat-model.md b/docs/problems/security-threat-model.md index 7eb6d4be42..3e910ab256 100644 --- a/docs/problems/security-threat-model.md +++ b/docs/problems/security-threat-model.md @@ -205,9 +205,9 @@ This cross-cutting nature is why the model-as-toolchain risk deserves explicit t A compromised or manipulated agent crafts its output (PR content, review comments, commit messages) specifically to influence other agents downstream. Unlike external prompt injection, this exploits the implicit trust between agents in the system. Examples: -- An implementation agent embeds instructions in code comments that manipulate the review agent into approving +- A code agent embeds instructions in code comments that manipulate the review agent into approving - A triage agent crafts issue labels or descriptions that bias the prioritization agent -- A review agent's feedback is crafted to make the implementation agent introduce a vulnerability in its "fix" +- A review agent's feedback is crafted to make the code agent introduce a vulnerability in its "fix" ### Why it's dangerous @@ -305,13 +305,13 @@ An attacker triggers excessive consumption of compute, API tokens, or event-proc **Event flooding:** - Rapidly filing issues, posting comments, toggling labels, or creating PRs to trigger agent invocations at scale -- Abusing slash commands (`/triage`, `/implement`) to queue expensive operations +- Abusing slash commands (`/triage`, `/code`) to queue expensive operations - Creating issues in bulk across multiple repos in an organization to saturate shared infrastructure **Cost amplification:** - Crafting issues that cause maximum LLM token consumption (extremely long descriptions, requests for exhaustive analysis) -- Triggering implementation agents on problems designed to maximize iteration loops (ambiguous requirements that never converge) -- Exploiting the implement-review feedback loop to cause unbounded cycles between agents +- Triggering code agents on problems designed to maximize iteration loops (ambiguous requirements that never converge) +- Exploiting the code-review feedback loop to cause unbounded cycles between agents - Filing issues that reference enormous external documents (via URLs) that the agent will attempt to fetch and process **Cascade amplification:** @@ -338,7 +338,7 @@ Agentic systems are uniquely vulnerable to DOS because: Agentic DOS requires defenses beyond standard infrastructure hardening (sandbox resource quotas via cgroups/Kata Containers, compute limits). The following focus on what is unique to this context: - **Cost budgets** — set per-repo and per-org budgets for LLM API token consumption. When a budget threshold is reached, require human approval before further agent invocations. -- **Loop circuit breakers** — enforce hard limits on implement-review cycles. The entry point script should enforce these limits deterministically, not rely on the agent's self-restraint. +- **Loop circuit breakers** — enforce hard limits on code-review cycles. The entry point script should enforce these limits deterministically, not rely on the agent's self-restraint. - **Event debouncing and deduplication** — collapse rapid-fire events on the same issue/PR into a single agent invocation rather than spawning one per event. - **Tiered response based on actor trust** — events from non-org-members or new contributors could be subject to stricter rate limits or require human approval before triggering agents. - **Input size limits** — cap the size of issue descriptions, comments, and referenced content that agents will process. Truncate or reject inputs above a threshold.