Skip to content

docs: add CLI architecture deep-dive guides - #1087

Merged
waynesun09 merged 10 commits into
mainfrom
docs-cli-architecture
May 18, 2026
Merged

docs: add CLI architecture deep-dive guides#1087
waynesun09 merged 10 commits into
mainfrom
docs-cli-architecture

Conversation

@waynesun09

Copy link
Copy Markdown
Member

Summary

Adds comprehensive CLI architecture documentation organized by audience:

  • Developer reference (docs/guides/dev/cli-internals.md) - CLI command structure, unified installation pipeline, sandbox lifecycle, workflow deployment, and source code reference
  • Admin reference (docs/guides/admin/infrastructure-reference.md) - Token mint (OIDC), WIF provisioning, secrets/variables deployment, and GCF provisioner flow
  • User guide (docs/guides/user/customizing-agents.md) - Harness configuration, layered config resolution, agent roles, and customization examples

These guides provide implementation details that complement the existing high-level installation and workflow documentation.

Content Organization

Developer Guide - CLI internals for contributors:

  • CLI command tree and token resolution chain
  • Unified installation flow (per-org vs per-repo modes)
  • OpenShell sandbox runtime lifecycle
  • Workflow deployment & scaffold system
  • End-to-end flow diagrams
  • Source file reference table

Admin Guide - Infrastructure details for operators:

  • OIDC token mint architecture and security controls
  • Role permissions matrix
  • Vertex AI with WIF authentication
  • Secrets & variables deployment by mode
  • GCF provisioner end-to-end flow
  • Multi-org support

User Guide - Customization for end users:

  • Harness YAML structure and configuration
  • Layered configuration resolution (upstream → org → repo)
  • Agent roles and GitHub App mapping
  • Practical customization examples
  • Per-repo override instructions

Changes

  • New files:

    • docs/guides/dev/cli-internals.md (460 lines)
    • docs/guides/admin/infrastructure-reference.md (291 lines)
    • docs/guides/user/customizing-agents.md (220 lines)
  • Cross-references added between related guides

  • All content verified against current codebase implementation

Test Plan

  • Verified documentation matches current implementation
  • Checked all internal cross-references
  • Removed any internal/research-specific content
  • Organized content by appropriate audience (dev/admin/user)

🤖 Generated with Claude Code

Port and reorganize the CLI architecture deep-dive documentation from
internal research into three audience-specific guides:

- docs/guides/dev/cli-internals.md - Developer reference covering CLI
  command structure, unified installation pipeline, sandbox lifecycle,
  workflow deployment, and source code reference

- docs/guides/admin/infrastructure-reference.md - Admin reference
  covering token mint (OIDC), WIF provisioning, secrets/variables
  deployment, and GCF provisioner flow

- docs/guides/user/customizing-agents.md - User guide covering harness
  configuration, layered config resolution, agent roles, and
  customization examples

These guides provide implementation details that complement the existing
high-level installation and workflow documentation.

Signed-off-by: Wayne Sun <gsun@redhat.com>
@github-actions

github-actions Bot commented May 18, 2026

Copy link
Copy Markdown

Site preview

Preview: https://bc962b21-site.fullsend-ai.workers.dev

Commit: 1cb77aba9f53198998e20fa55c0fc235eb92daaa

@fullsend-ai-review

fullsend-ai-review Bot commented May 18, 2026

Copy link
Copy Markdown

Review

Findings

Low

  • [correctness] docs/guides/dev/cli-internals.md:440 — Variable name FULLSEND_PER_REPO_GUARD is incorrect. The actual environment variable is FULLSEND_PER_REPO_INSTALL (the Go constant PerRepoGuardVar has value "FULLSEND_PER_REPO_INSTALL"). The admin guide infrastructure-reference.md correctly uses the right name.
    Remediation: Change FULLSEND_PER_REPO_GUARD=true to FULLSEND_PER_REPO_INSTALL=true on line 440.

  • [documentation-currency] docs/guides/README.md:1 — The guides index (docs/guides/README.md) was not updated to include links to the three new files: admin/infrastructure-reference.md, dev/cli-internals.md, and user/customizing-agents.md. After merge, the index will be incomplete.
    Remediation: Add entries under the appropriate audience sections in docs/guides/README.md.

Previous run

Review

Findings

No findings.

All three documentation files were verified against the current codebase implementation:

  • infrastructure-reference.md — Role permissions matrix, mint architecture, GCF provisioner flow, secret/variable names, and multi-org support all match the source code in internal/mint/main.go, internal/dispatch/gcf/provisioner.go, and related files.
  • cli-internals.md — CLI command tree, token resolution chain, unified install pipeline, sandbox lifecycle, scaffold system, and source file reference table are all accurate. Line counts are within expected drift margins.
  • customizing-agents.md — Harness YAML structure, layered configuration resolution, agent roles, and customization examples match internal/harness/harness.go and internal/scaffold/scaffold.go.

No sensitive data exposure detected — all identifiers use template patterns ({org}, {project}, ${VAR}), environment variable names (not values), and public endpoints. No real GCP project names, service account identifiers, or credentials are present.

All internal cross-references resolve to existing files (installation.md, local-dev.md, bugfix-workflow.md, ADR 0035).

Previous run (2)

Review

Findings

Medium

  • [docs-currency] docs/guides/README.md — The guides index lists existing guides under Administration, User guides, and Development sections but does not include the three new documents added by this PR. Readers browsing docs/guides/README.md will not discover the new CLI internals, infrastructure reference, or customizing agents guides.
    Remediation: Add entries for admin/infrastructure-reference.md, dev/cli-internals.md, and user/customizing-agents.md to the appropriate sections of docs/guides/README.md.

Low

  • [correctness] docs/guides/dev/cli-internals.md:441 — Phase 6 diagram uses FULLSEND_PER_REPO_GUARD=true but the actual environment variable is FULLSEND_PER_REPO_INSTALL (defined as PerRepoGuardVar in internal/forge/forge.go). The admin infrastructure-reference.md correctly uses FULLSEND_PER_REPO_INSTALL.
    Remediation: Replace FULLSEND_PER_REPO_GUARD with FULLSEND_PER_REPO_INSTALL in the Phase 6 diagram.

Info

  • [completeness] docs/guides/user/customizing-agents.md — The harness YAML reference omits three fields present in the Harness struct (internal/harness/harness.go): description, api_servers, and agent_input. The example does not claim to be exhaustive, so this is informational — consider adding them if they are user-facing.
Previous run (3)

Review

Findings

No findings.

The prior medium-severity finding (incorrect security block types in customizing-agents.md) has been resolved — the YAML example now correctly matches the Go struct shapes in internal/harness/harness.go. All documented claims verified against the codebase: role permissions matrix, secret naming pattern, token resolution chain, sandbox constants, CLI command tree, harness YAML types, and cross-references.

Previous run (4)

Review

Findings

No findings.

The prior medium-severity finding (incorrect security block types in customizing-agents.md) has been resolved — the YAML example now correctly matches the Go struct shapes in internal/harness/harness.go. All documented claims verified against the codebase: role permissions matrix, secret naming pattern, token resolution chain, sandbox constants, CLI command tree, harness YAML types, and cross-references.

Previous run (5)

Review

Findings

Medium

  • [correctness] docs/guides/user/customizing-agents.md:826-838 — The security block in the harness YAML example uses incorrect types for four fields. host_scanners is shown as a list of strings (- llm-guard, - tirith), but the actual Go struct (HostScanners in internal/harness/harness.go:95) has boolean fields (unicode_normalizer, context_injection, ssrf_validator, secret_redactor) and a nested llm_guard config object. sandbox_hooks is similarly shown as a list of strings, but the actual struct (SandboxHooks, line 114) has boolean and nested-struct fields (tirith, ssrf_pretool, secret_redact_posttool, unicode_posttool, etc.). escalation is shown as a bare string block, but it is actually *EscalationConfig (line 139) with on_critical and review_label subfields. trace is shown as a bare boolean, but it is actually *TraceConfig (line 145) with an enabled subfield. Users following this example would produce invalid YAML that fails to parse.
    Remediation: Update the security block to match the actual struct shapes — host_scanners and sandbox_hooks as objects with boolean/nested fields, escalation as an object with on_critical/review_label, and trace as an object with enabled.
Previous run (6)

Review

Findings

Medium

  • [correctness] docs/guides/dev/cli-internals.md — The CLI command tree is incomplete. scan has subcommands (input, output, context, url) and both enable and disable have a repos subcommand. Developers relying on this tree will not discover available commands.
    Remediation: Update the command tree to include subcommands for scan, enable, and disable.

  • [correctness] docs/guides/user/customizing-agents.md — The harness YAML examples document three fields with incorrect types. plugins is shown as a list of objects with name/config subfields, but the actual Go struct (internal/harness/harness.go:196) defines it as []string. providers is similarly shown as structured objects but is actually []string (line 197). runner_env is shown as a list of strings but is actually map[string]string (line 205). Users following these examples would produce invalid YAML that fails to parse.
    Remediation: Update the harness YAML example to match the actual struct types — plugins and providers as string lists, runner_env as a key-value map.

Low

  • [correctness] docs/guides/dev/cli-internals.md — Four entries in the "Key Source Files Reference" table have line counts that are significantly off: root.go (34 actual vs ~50 claimed), layers.go (159 vs ~100), dispatch.go (364 vs ~250), config.go (264 vs ~200). The table includes a disclaimer about approximate line counts, which partially mitigates this.
    Remediation: Update the line counts to current values.
Previous run (7)

Review

Findings

High

  • [correctness] docs/guides/admin/infrastructure-reference.md:83-91 — Role permissions matrix contains multiple inaccuracies compared to internal/mint/main.go rolePermissions map. Discrepancies include: triage lists pull_requests: read (not in code); coder lists actions: read, checks: write (actual: checks: read, no actions); review lists checks: write (actual: checks: read); fix lists actions: read, checks: write (neither exists in code); retro lists issues: read (actual: issues: write); prioritize lists pull_requests: read (not in code, missing organization_projects: write); fullsend lists issues: write, checks: write, members: read (none present in code, missing actions_variables: read, workflows: write).
    Remediation: Regenerate the permissions table directly from the rolePermissions map in internal/mint/main.go (lines 750-757).

  • [correctness] docs/guides/admin/infrastructure-reference.md:55 — Secret naming pattern documented as {org}-{role}-github-app-pem but the actual pattern in code is fullsend-{org}--{role}-app-pem (with fullsend- prefix and double-hyphen separator). See internal/mint/main.go line 144.
    Remediation: Update to fullsend-{org}--{role}-app-pem to match the smPEMAccessor implementation.

Low

  • [correctness] docs/guides/dev/cli-internals.md:744-755 — Source file line counts will drift as the codebase evolves, creating a maintenance burden. Currently accurate (within ~5% verified), but these will become stale with any non-trivial code change.
    Remediation: Consider removing exact line counts or adding a note that they are approximate and may drift.

Address review feedback:

- Fix role permissions matrix to match actual code in
  internal/mint/main.go rolePermissions map:
  - fullsend: add workflows, actions_variables; remove issues, checks,
    members
  - triage: remove pull_requests
  - coder: remove actions, change checks to read
  - review: change checks to read
  - fix: remove actions, checks
  - retro: add actions (read); change pull_requests to read
  - prioritize: remove pull_requests; add organization_projects

- Fix secret naming pattern from {org}-{role}-github-app-pem to
  fullsend-{org}--{role}-app-pem (matches smPEMAccessor implementation)

- Add note to source file reference table that line counts are
  approximate and may drift

Signed-off-by: Wayne Sun <gsun@redhat.com>
Address review feedback:

- Expand CLI command tree to show subcommands:
  - scan: input, output, context, url
  - enable: repos
  - disable: repos

- Fix harness YAML example types to match internal/harness/harness.go:
  - plugins: changed from object list to string list
  - providers: changed from object list to string list
  - runner_env: changed from string list to key-value map

- Update source file line counts to current values:
  - root.go: 50 → 34
  - layers.go: 100 → 159
  - dispatch.go: 250 → 364
  - config.go: 200 → 264

Signed-off-by: Wayne Sun <gsun@redhat.com>
@fullsend-ai-review fullsend-ai-review Bot added requires-manual-review Review requires human judgment and removed requires-manual-review Review requires human judgment labels May 18, 2026
Address review feedback:

- Fix security block to match actual struct types from
  internal/harness/harness.go:

  - host_scanners: changed from string list to object with boolean
    fields (unicode_normalizer, context_injection, ssrf_validator,
    secret_redactor) and nested llm_guard config object

  - sandbox_hooks: changed from string list to object with boolean
    and nested config fields (tirith object with enabled/fail_on,
    plus boolean flags for ssrf_pretool, secret_redact_posttool,
    unicode_posttool, context_suppress_posttool, canary_pretool,
    canary_posttool)

  - escalation: changed from string to object with on_critical and
    review_label fields

  - trace: changed from boolean to object with enabled field

Users can now copy this example and produce valid harness YAML.

Signed-off-by: Wayne Sun <gsun@redhat.com>
@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge and removed requires-manual-review Review requires human judgment labels May 18, 2026
With OIDC/WIF as the standard deployment mode, PEM keys are stored in
GCP Secret Manager, not as GitHub repo secrets. The FULLSEND_{ROLE}_APP_PRIVATE_KEY
entry under '.fullsend repo secrets' was confusing because it claimed to be
a repo secret while simultaneously noting it's NOT a repo secret in OIDC mode.

The Secrets Layer Behavior section already correctly documents that in OIDC mode,
Install is a no-op and PEMs are stored in Secret Manager. Removing the misleading
entry from the active secrets list.

Signed-off-by: Wayne Sun <gsun@redhat.com>
@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge and removed ready-for-merge All reviewers approved — ready to merge labels May 18, 2026
- Fix FULLSEND_PER_REPO_GUARD → FULLSEND_PER_REPO_INSTALL per forge.go
- Improve runner_env examples to show variable references instead of empty strings
- All Go code blocks already have language tags

Signed-off-by: Wayne Sun <gsun@redhat.com>
@fullsend-ai-review fullsend-ai-review Bot added requires-manual-review Review requires human judgment and removed ready-for-merge All reviewers approved — ready to merge labels May 18, 2026
Replace invalid plugin names (github, sourcebot) with realistic example
(gopls-lsp). These are Claude Code plugins from anthropics/claude-plugins-official,
not MCP servers. MCP servers are configured separately in Claude Code settings,
not in harness plugin lists.

Actual available plugins include:
- plugins/gopls-lsp (Go language server)
- plugins/clangd-lsp (C/C++ language server)
- plugins/code-review (code review enhancements)
- plugins/feature-dev (feature development workflows)

MCP servers like github, context7, etc. are not harness plugins.

Signed-off-by: Wayne Sun <gsun@redhat.com>
Split harness YAML structure into two sections:
1. Minimal realistic example based on actual code.yaml harness
2. Optional fields section showing security/providers configs

Key changes:
- Use actual agent name 'code' (not 'coder')
- Use actual skill name 'code-implementation' (not generic examples)
- Show model: opus (actual scaffold value) in minimal example
- Show model: claude-opus-4-6 in customization example (clearer)
- Clarify that security/providers fields are optional with secure defaults
- Note that all scaffold harnesses omit security block (rely on defaults)
- Match field structure to internal/harness/harness.go
- Match examples to internal/scaffold/fullsend-repo/harness/*.yaml

Signed-off-by: Wayne Sun <gsun@redhat.com>
@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge and removed requires-manual-review Review requires human judgment labels May 18, 2026
Added 'How Override Resolution Works' section explaining:
- Overrides are file-level replacement, not YAML field merging
- Complete workflow example: copy upstream → modify → add custom files
- Runtime behavior: upstream defaults + customizations overlay
- Important caveat: must maintain full harness structure

Updated 'Customizing Harness Configuration' example to:
- Show full harness YAML (not partial snippet)
- Highlight changed fields with inline comments
- Emphasize this is a complete replacement
- Include step to create custom skill file

This addresses confusion about whether adding a skill requires
copying the entire harness (answer: yes, file-level replacement).

Signed-off-by: Wayne Sun <gsun@redhat.com>
└─────────────────────────────────────────────────────────────────┘
```

---

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] correctness

Variable name FULLSEND_PER_REPO_GUARD is incorrect. The actual environment variable is FULLSEND_PER_REPO_INSTALL (the Go constant PerRepoGuardVar has value "FULLSEND_PER_REPO_INSTALL"). The admin guide infrastructure-reference.md correctly uses the right name.

Suggested fix: Change FULLSEND_PER_REPO_GUARD=true to FULLSEND_PER_REPO_INSTALL=true.

@fullsend-ai-review

Copy link
Copy Markdown

Review follow-ups

Created follow-up issues for actionable non-blocking review findings:

  • #1122 — Variable name FULLSEND_PER_REPO_GUARD is incorrect. The actual environment variable is FULLSEND_PER_REPO_INSTALL (the Go constant PerRepoGuardVar has value "FULLSEND_PER_REPO_INSTALL"). The admin guide infrastructure-reference.md correctly uses the right name.
  • #1123 — The guides index (docs/guides/README.md) was not updated to include links to the three new files: admin/infrastructure-reference.md, dev/cli-internals.md, and user/customizing-agents.md. After merge, the index will be incomplete.

Pre-commit hook detected trailing whitespace on line 168
(blank line in heredoc example). Fixed by stripping all trailing
whitespace from the file.

Signed-off-by: Wayne Sun <gsun@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-merge All reviewers approved — ready to merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant