Skip to content

docs: per-agent reference with icons, commands, and labels - #1017

Merged
ralphbean merged 11 commits into
mainfrom
docs/agents-reference
May 26, 2026
Merged

docs: per-agent reference with icons, commands, and labels#1017
ralphbean merged 11 commits into
mainfrom
docs/agents-reference

Conversation

@ralphbean

Copy link
Copy Markdown
Member

Summary

  • Add docs/agents/ with a README index and per-agent docs (triage, prioritize, code, review, fix, retro) covering how each works, commands, control labels, customization guidance, and source links
  • Add docs/guides/user/customizing-with-skills.md explaining CLAUDE.md, custom commands, and skill shadowing with concrete examples (issue-labels, customer-research)
  • Copy agent icon PNGs to docs/agents/icons/
  • Add doc: field to Harness struct and all OOTB harness YAMLs
  • Add hack/lint-agent-docs pre-commit hook enforcing doc coverage, README linkage, and section structure
  • Add reminder comment in scaffold.go to update docs when adding agents

Relates to #605

Test plan

  • make lint-all passes (all 23 hooks green)
  • go test ./internal/harness/ passes
  • lint-agent-docs catches missing doc: fields, broken doc paths, missing README links, and missing/extra sections

…cement

Add docs/agents/ with a README index and one markdown file per OOTB agent
(triage, prioritize, code, review, fix, retro). Each doc covers how the
agent works, how it helps, commands, control labels, configuration and
extension guidance, and a source link to the harness YAML.

Add docs/guides/user/customizing-with-skills.md explaining how users can
influence agent behavior via CLAUDE.md, custom commands, and skill
shadowing. Includes concrete examples for the triage `issue-labels` skill
and the prioritize `customer-research` skill.

Copy agent icon PNGs and COLORS.md to docs/agents/icons/.

Add a `doc` field to the Harness struct and populate it in all OOTB harness
YAMLs. Add hack/lint-agent-docs pre-commit hook enforcing that every OOTB
harness YAML has a doc: field pointing to an existing file, that the README
links to every agent doc, and that every agent doc has the required section
structure. Add a reminder comment in scaffold.go to update docs when adding
new agents.

Relates to #605

Signed-off-by: Ralph Bean <rbean@redhat.com>
Assisted-by: Claude claude-opus-4-6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
@github-actions

Copy link
Copy Markdown

fullsend review is working on this — view logs

@github-actions

github-actions Bot commented May 15, 2026

Copy link
Copy Markdown

Site preview

Preview: https://f622bdf3-site.fullsend-ai.workers.dev

Commit: 0ac78865ce41bf193dce3ba46c23c860a027fc4d

@fullsend-ai-review

fullsend-ai-review Bot commented May 15, 2026

Copy link
Copy Markdown

Review

Findings

Medium

Low

  • [docs-staleness] docs/ADRs/0024-harness-definitions.md — The "Harness YAML schema" section documents every harness field but does not include the new doc: field added by this PR. While doc: is source-repo-only metadata for the lint hook, the ADR is the authoritative schema reference and should list it with a note that it is not resolved at runtime.

  • [docs-staleness] README.md — The "What's here" section serves as the document index (per CLAUDE.md: "See README.md for the full document index"). The new docs/agents/ directory and docs/guides/user/customizing-with-agents-md.md / docs/guides/user/customizing-with-skills.md guides are not linked from the index.

  • [docs-staleness] docs/guides/README.md — The user guides index lists bugfix-workflow.md and running-agents-locally.md but does not link to the two new guides (customizing-with-agents-md.md and customizing-with-skills.md) added by this PR. This is the canonical entry point for user guides per ADR 0023.

  • [style] docs/agents/code.md:3 — Uses markdown image syntax ![Code agent icon](icons/coder.png) while all other agent docs (fix.md, prioritize.md, retro.md, review.md, triage.md) use HTML <img src="..." width="80">. The HTML variant controls rendered size; code.md should match for consistency.
    Remediation: Change to <img src="icons/coder.png" alt="Code agent icon" width="80">.

Previous run

Review

Findings

Medium

Low

  • [docs-staleness] docs/ADRs/0024-harness-definitions.md — The "Harness YAML schema" section documents every harness field but does not include the new doc: field added by this PR. While doc: is source-repo-only metadata for the lint hook, the ADR is the authoritative schema reference and should list it with a note that it is not resolved at runtime.

  • [docs-staleness] README.md — The "What's here" section serves as the document index (per CLAUDE.md: "See README.md for the full document index"). The new docs/agents/ directory and docs/guides/user/customizing-with-agents-md.md / docs/guides/user/customizing-with-skills.md guides are not linked from the index.

  • [docs-staleness] docs/guides/README.md — The user guides index lists bugfix-workflow.md and running-agents-locally.md but does not link to the two new guides (customizing-with-agents-md.md and customizing-with-skills.md) added by this PR. This is the canonical entry point for user guides per ADR 0023.

  • [style] docs/agents/code.md:3 — Uses markdown image syntax ![Code agent icon](icons/coder.png) while all other agent docs (fix.md, prioritize.md, retro.md, review.md, triage.md) use HTML <img src="..." width="80">. The HTML variant controls rendered size; code.md should match for consistency.
    Remediation: Change to <img src="icons/coder.png" alt="Code agent icon" width="80">.

Previous run (2)

Review

Findings

Medium

Low

  • [docs-staleness] docs/ADRs/0024-harness-definitions.md — The "Harness YAML schema" section (line ~307) documents every harness field but does not include the new doc: field added by this PR. While doc: is source-repo-only metadata for the lint hook, the ADR is the authoritative schema reference and should list it with a note that it is not resolved at runtime.

  • [docs-staleness] README.md — The "What's here" section serves as the document index (per CLAUDE.md: "See README.md for the full document index"). The new docs/agents/ directory and docs/guides/user/customizing-with-agents-md.md / docs/guides/user/customizing-with-skills.md guides are not linked from the index.

  • [style] docs/agents/code.md:3 — Uses markdown image syntax ![Code agent icon](icons/coder.png) while all other agent docs (fix.md, prioritize.md, retro.md, review.md, triage.md) use HTML <img src="..." width="80">. The HTML variant controls rendered size; code.md should match for consistency.

Previous run (3)

Review

Findings

Medium

  • [protected-path] .pre-commit-config.yaml — This PR modifies a protected infrastructure file (adds the lint-agent-docs pre-commit hook). The linked issue (docs: create docs/agents/ reference with per-agent documentation and CI enforcement #605) and the PR description explain the rationale (CI enforcement of agent doc coverage). Human approval is always required for protected-path changes, regardless of context.

  • [docs-staleness] docs/guides/README.md — This PR adds two new user guides (customizing-with-agents-md.md and customizing-with-skills.md) but does not update the guides index at docs/guides/README.md to link to them. The index currently lists only the bugfix-workflow guide under "User guides". The new guides are discoverable from docs/agents/README.md but not from the guides index, which is the canonical entry point per ADR 0023.
    Remediation: Add links to both new guides in the "User guides" section of docs/guides/README.md.

Low

  • [style] docs/agents/code.md:3 — Uses ![Code agent icon](icons/coder.png) markdown image syntax, while all other agent docs (fix.md, prioritize.md, retro.md, review.md, triage.md) use <img src="icons/..." alt="..." width="80"> HTML tags with a width constraint. This means the code agent icon renders at full resolution while all others render at 80px.
    Remediation: Change to <img src="icons/coder.png" alt="Code agent icon" width="80"> for consistency.
Previous run (4)

Review: #1017

Head SHA: 7c740fd
Timestamp: 2026-05-15T00:00:00Z
Outcome: approve

Summary

This PR delivers well-structured per-agent reference documentation, user-facing customization guides, a lint script enforcing doc coverage and structure, and a minor Go struct addition — all consistent with the intent of issue #605. The change is documentation-only with a small, backward-compatible Go struct field addition (Doc string with omitempty). No security concerns, no secrets, no injection patterns. The lint enforcement is a strong addition that will prevent doc drift as new agents are added.

Findings

Medium

  • [Correctness] hack/lint-agent-docs:line ~1030-1035 — The "unexpected section" check increments the error counter and fails the lint, but the message says "adding new sections is fine." This is contradictory — if extra ## sections are truly allowed, they shouldn't be errors. If they should be errors, the message should not say they're fine. Currently all agent docs happen to have exactly the required sections (extras are at ### level), so this doesn't bite today, but it will surprise a future contributor who adds a ## Troubleshooting section and gets a lint failure with a message saying it's fine.
    Remediation: Either downgrade unexpected sections to a warning (don't increment errors) or change the message to clearly say extra sections must be added to the required list before use.

Low

  • [Intent alignment] Issue docs: create docs/agents/ reference with per-agent documentation and CI enforcement #605 specifies "Diagram" and "Icon" as required sections in per-agent docs. The PR uses a different section structure (Commands, Control labels) and embeds icons via <img> tags rather than a dedicated section. This appears to be a deliberate evolution of the original spec rather than an omission — the chosen structure is more practical — but the issue should be updated to reflect the agreed-upon structure to avoid confusion for future contributors.
    Remediation: Consider updating docs: create docs/agents/ reference with per-agent documentation and CI enforcement #605 or closing it with a note that the delivered structure intentionally diverges from the original proposal.

  • [Style] docs/agents/fix.md — The fix agent uses the same icon as the code agent (icons/coder.png). If this is intentional (they share an identity), it's fine, but it may confuse users scanning the docs visually. The icons directory includes refinement.png which might be a better fit for the fix agent.
    Remediation: Confirm whether coder.png is intentional for the fix agent or if a dedicated icon should be used.

Info

  • [Correctness] internal/harness/harness.go — The Doc field is correctly tagged omitempty and documented as source-repo-only. Existing tests don't reference it, so no test updates are needed. The field position after Agent is logical.

  • [Correctness] docs/guides/user/customizing-with-skills.md — References to ADR 0035 (../../ADRs/0035-layered-content-resolution.md) are valid; the ADR exists on main.

Footer

Outcome: approve
This review applies to SHA 7c740fd18948b140a430ac3e51c98017e5416b0e. Any push to the PR head clears this review and requires a new evaluation.


Protected paths detected — this PR modifies files under one or more
protected paths. The review agent cannot approve PRs that touch these paths.
A human reviewer must approve this PR.

Protected files in this PR:

  • .pre-commit-config.yaml
Previous run (5)

Review: #1017

Head SHA: f5fc78c
Timestamp: 2026-05-15T00:00:00Z
Outcome: comment-only

Summary

This is a well-structured documentation PR that adds per-agent reference docs, two user customization guides, icon assets, a doc: field to the Harness struct and YAML files, and a lint script to enforce documentation parity. The code changes are minimal and correct — the Doc field uses omitempty and is purely metadata for the lint hook, with no runtime impact. The lint script is thorough and well-designed. A few minor items are worth noting but nothing blocks merge.

Findings

Medium

Low

  • [Correctness] docs/guides/user/customizing-with-skills.md:1 — The link to https://agentskills.org/ could not be verified due to network restrictions. If this domain doesn't exist or isn't controlled by the project, the link will be a dead reference or could be domain-squatted.
    Remediation: Verify the link resolves correctly before merge.

  • [Style/conventions] docs/agents/fix.md:3 — The fix agent uses icons/coder.png (the same icon as the code agent) rather than having its own icon. This may be intentional since fix is closely related to code, but it could cause confusion in the README table if icons are displayed there later.
    Remediation: Confirm this is intentional or assign a distinct icon.

Info

  • [Correctness] hack/lint-agent-docs — The lint script treats any ## section not in the REQUIRED_SECTIONS list as an error ("unexpected"). This means adding new sections to any agent doc requires updating the lint script first. The error message helpfully explains this, but it does create coupling between docs and the lint script. This is a deliberate design choice that enforces consistency, which is reasonable.

  • [Injection defense] PR body and commit messages — No prompt injection patterns, non-rendering Unicode, or suspicious directives found in the PR description or diff content.

  • [Intent alignment] The PR adds two user guides (customizing-with-claude-md.md, customizing-with-skills.md) that were not part of issue docs: create docs/agents/ reference with per-agent documentation and CI enforcement #605's scope. These are valuable additions that complement the agent reference docs and don't represent scope creep — they provide the customization guidance that every agent doc links to.

Footer

Outcome: comment-only
This review applies to SHA f5fc78cca0dcc4fc0e2e5a1f9fcc74157be0aab5. Any push to the PR head clears this review and requires a new evaluation.

Previous run (6)

Review: #1017

Head SHA: 29a67e7
Timestamp: 2026-05-15T00:00:00Z
Outcome: comment-only

Summary

This PR adds per-agent reference documentation under docs/agents/, two user customization guides, icon assets, a doc: field on the Harness Go struct and all OOTB harness YAMLs, and a hack/lint-agent-docs pre-commit hook enforcing doc coverage, README linkage, and section structure. The change is well-scoped to issue #605 and implements the three CI enforcement checks requested there. No critical or high findings. A few medium/low observations are noted below.

Findings

Medium

  • [Correctness] hack/lint-agent-docs — The structure-check section flags ## headers that are not in the required list as "unexpected" and increments the error count, which means any agent doc that adds a subsection beyond the six required ones (e.g., ## Examples in triage.md and prioritize.md, or ## What not to do in the guides) would fail the lint. However, reviewing the actual agent docs, the extra ## sections are all under ### (h3), so they won't match the ^## pattern. This is fine for now, but the strict "no extra h2 sections" policy is fragile — if a future doc legitimately needs an additional h2, the lint will reject it without an obvious explanation in the error message. Consider adding a comment in the script or a more descriptive error.

Low

  • [Correctness] internal/harness/harness_test.go — The existing harness test file does not include the new Doc field in any test fixtures. Since Doc is omitempty and lint-only (not used at runtime), this is low risk, but a test confirming round-trip YAML parsing of doc: would prevent silent regression if the field name or tag changes.

  • [Style/conventions] docs/agents/fix.md — The fix agent uses icons/coder.png as its icon (same as the code agent). The icons directory includes a refinement.png which may be a better semantic fit, though this could be intentional if fix and code are considered variants of the same role.

  • [Intent alignment] Issue docs: create docs/agents/ reference with per-agent documentation and CI enforcement #605 specifies a "Diagram" section in each agent doc showing how the agent fits into the workflow. The PR omits diagrams from all agent docs. The lint script's required sections list also omits "Diagram", so the lint won't catch this gap. This may be a deliberate deferral — worth confirming.

  • [Style/conventions] docs/guides/user/customizing-with-skills.md:82 — The ADR reference ADR 0035 links to ../../ADRs/0035-layered-content-resolution.md. The relative path is correct from the file's location. No issue, just noting the cross-reference is valid.

Info

  • [Injection defense] PR body and commit messages inspected for prompt injection patterns and non-rendering Unicode. None found.
  • [Content security] The prioritize.md example skill scripts (query-salesforce.sh, search-drive.sh) correctly use environment variables ($SFDC_TOKEN, $GDRIVE_TOKEN) for credentials rather than hardcoded values. The scripts are documentation examples, not executable code shipped in the product.
  • [Scope authorization] The PR body says "Relates to docs: create docs/agents/ reference with per-agent documentation and CI enforcement #605" (not "Closes"), which is accurate — diagrams and the custom-agent guide are deferred. Scope is appropriate.

Footer

Outcome: comment-only
This review applies to SHA 29a67e77846067ca7d4181e636883f27d7a820d9. Any push to the PR head clears this review and requires a new evaluation.

Previous run (7)

Review: #1017

Head SHA: a171783
Timestamp: 2026-05-15T00:00:00Z
Outcome: approve

Summary

Clean documentation PR that delivers per-agent reference docs, a README index, a doc: field on the harness struct, and a pre-commit lint hook enforcing doc coverage, README linkage, and section structure. All six OOTB agents have corresponding docs. The Harness.Doc field is correctly excluded from ResolveRelativeTo and ValidateFilesExist since it's repo-level metadata, not a sandbox runtime path. No correctness, security, or injection concerns.

Findings

Info

  • [style] docs/agents/fix.md:3 — Fix agent reuses icons/coder.png (the Code agent's icon) rather than having a distinct icon. Both agents share the same container image, so this may be intentional. Consider icons/refinement.png if a visual distinction is desired.

Footer

Outcome: approve
This review applies to SHA a1717837364acb0782b739530add3e8a08f214bf. Any push to the PR head clears this review and requires a new evaluation.


Protected paths detected — this PR modifies files under one or more
protected paths. The review agent cannot approve PRs that touch these paths.
A human reviewer must approve this PR.

Protected files in this PR:

  • .pre-commit-config.yaml

…lint fixes

- Use HTML img tags with width=80 for agent icons so GitHub renders
  them at a reasonable size instead of full resolution
- Split customizing-with-skills.md into two docs: one focused on
  skills (with link to agentskills.org) and a separate
  customizing-with-claude-md.md for project-wide instructions
- Separate built-in skills from extension points in the well-known
  skills table, add missing finding-agent-runs for retro
- Fix triage example control labels list (not-ready -> blocked)
- Add Doc field comment explaining it's not resolved at runtime
- Fix lint-agent-docs: use #!/bin/bash, add nullglob, anchor README
  link check with grep -F to prevent substring false positives

Signed-off-by: Ralph Bean <rbean@redhat.com>
Assisted-by: Claude claude-opus-4-6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
@github-actions

Copy link
Copy Markdown

fullsend review is working on this — view logs

ralphbean added 4 commits May 15, 2026 13:20
The review agent can write files locally in its sandbox but cannot push
them. "Cannot write files" was inaccurate.

Assisted-by: Claude claude-opus-4-6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
Prioritization currently only helps humans see ranking on their RICE
board. The code agent does not consume priority scores today.

Assisted-by: Claude claude-opus-4-6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
Consistency: "# Triage" → "# Triage Agent", etc. across all agent
reference docs.

Assisted-by: Claude claude-opus-4-6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
Add a lint check requiring agent doc top-level headings to end with
" Agent". Improve the unexpected-section error message to clarify that
new sections are welcome — they just need to be added to the required
list for cross-file consistency.

Assisted-by: Claude claude-opus-4-6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
@github-actions

Copy link
Copy Markdown

fullsend review is working on this — view logs

Comment thread docs/guides/user/customizing-with-claude-md.md Outdated
Comment thread docs/agents/prioritize.md Outdated
Comment thread docs/agents/prioritize.md Outdated
Comment thread docs/agents/prioritize.md Outdated
Comment thread docs/agents/triage.md Outdated
Comment thread docs/guides/user/customizing-with-agents-md.md
Comment thread docs/guides/user/customizing-with-claude-md.md Outdated
Comment thread docs/guides/user/customizing-with-agents-md.md
Comment thread docs/guides/user/customizing-with-skills.md Outdated
Comment thread docs/guides/user/customizing-with-skills.md Outdated
- Recommend AGENTS.md with CLAUDE.md pointing at it for portability
- Recommend .agents/skills/ with symlink from .claude/skills/ everywhere
- Move skill helper scripts into scripts/ subdirectory convention
- Add allowed_tools to customer-research skill frontmatter example
- Drop obvious "don't put secrets" advice from CLAUDE.md guide
- Add progressive disclosure guidance for AGENTS.md sizing
- Recommend skill creator skills (Anthropic /skill, superpowers)
- Add repo-level skill shadowing path to triage agent doc

Assisted-by: Claude claude-opus-4-6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
@github-actions

Copy link
Copy Markdown

fullsend review is working on this — view logs

@ralphbean

Copy link
Copy Markdown
Member Author

/fs-fix fix the "it's fine" message to encourage the developer to add the new section only if they're comfortable adding it to all agent doc files and updating the linter to require the new section. The goal of the linter is to help us realize we're about to introduce entropy into the codebase.

@ralphbean

Copy link
Copy Markdown
Member Author

/fs-retro why did fix agent fail here? this PR isn't from a fork.

Comment thread docs/guides/user/customizing-with-skills.md

@rh-hemartin rh-hemartin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More stuff. If you think we need to move forward feel free to compile my feedback into an issue and merge this.

@ralphbean

ralphbean commented May 19, 2026

Copy link
Copy Markdown
Member Author

If you think we need to move forward feel free to compile my feedback into an issue and merge this.

No rush, we can improve it here. The only ~major risk of taking more time here is that the codebase changes while this PR stays static. What is written here may drift out of sync before merge. That said - I intend to process the latest feedback here and revise. Thank you for taking the time.

- Rename "Bring Your Own Agent (BYOA)" to "Custom Agents"
- Scrub "OOTB" references to "default" in lint script, scaffold comment
- Add trigger line to code agent intro
- Add ready-for-review to code agent control labels
- Simplify fix agent sandbox description

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
@fullsend-ai-review fullsend-ai-review Bot added requires-manual-review Review requires human judgment and removed requires-manual-review Review requires human judgment labels May 21, 2026
The prioritize agent is not triggered after triage — it runs on a
schedule (polling for unscored/stale issues) or on-demand via
/fs-prioritize.

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
@fullsend-ai-review fullsend-ai-review Bot added requires-manual-review Review requires human judgment and removed requires-manual-review Review requires human judgment labels May 21, 2026
- prioritize: remove read-only paragraph, rename to Skill section,
  apply reworded intro, drop salesforce/gdrive script examples
- retro: apply sandbox description suggestion, trim control labels
- review: add schema validation step, remove token mention, add
  ready-for-review label
- triage: add clarification to intro, simplify /fs-triage, apply
  label management text, rename to Skill section
- agents-md: fix pronoun (your → its)
- skills: add prioritize agent to built-in skills table

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
@ralphbean ralphbean changed the title docs: per-agent reference with icons, commands, labels, and lint enforcement docs: per-agent reference with icons, commands, and labels May 21, 2026
@fullsend-ai-review fullsend-ai-review Bot added requires-manual-review Review requires human judgment and removed requires-manual-review Review requires human judgment labels May 21, 2026
@ralphbean
ralphbean requested a review from rh-hemartin May 21, 2026 19:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

requires-manual-review Review requires human judgment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants