feat: add self-improvement principle and agent routing to pulse system#2295
Conversation
Add universal self-improvement section to AGENTS.md — every agent session should observe outcomes from existing state (GitHub, TODO.md) and file issues for systemic problems rather than patching around them. Add agent routing to AGENTS.md, pulse.md, and runners.md — not every task is code. The pulse supervisor now routes workers to the appropriate primary agent (SEO, Content, Marketing, etc.) via --agent flag based on task domain. Also: - pulse.md: add Step 2a outcome observation, allow merges when slots full - runners.md: align /pulse description with actual behaviour, add --agent - full-loop.md: replace stale supervisor DB lookup with gh issue search - AGENTS.md: remove stale supervisor DB reference
WalkthroughThese changes transition the agent orchestration system from a single-worker, supervisor-database model to a multi-agent, GitHub-centric architecture featuring concurrent worker dispatch, self-improvement observation mechanisms, and intelligent task routing with outcome-driven iteration. Changes
Sequence DiagramsequenceDiagram
participant Pulse as Pulse<br/>(Orchestrator)
participant GitHub as GitHub<br/>(State DB)
participant PulseAgent as Pulse<br/>(Step 2a)
participant Router as Router<br/>(Step 3-4)
participant Agents as Agent Pool<br/>(Dispatch)
rect rgba(100, 150, 255, 0.5)
Note over Pulse: Circuit Breaker Check<br/>(Step 0)
Pulse->>Pulse: Check available slots<br/>(< 6 workers)
end
rect rgba(100, 200, 150, 0.5)
Note over PulseAgent: Observe Outcomes<br/>(Step 2a - Self-Improvement)
PulseAgent->>GitHub: Query PRs, Issues, Patterns
GitHub-->>PulseAgent: Stale PRs, Repeated Failures,<br/>Duplicate Work
PulseAgent->>GitHub: File improvement issues
end
rect rgba(200, 150, 100, 0.5)
Note over Router: Route & Prioritize<br/>(Step 3-4)
Router->>GitHub: Fetch available tasks
GitHub-->>Router: PRs needing work,<br/>Issues needing implementation
Router->>Router: Apply priority framework<br/>+ tie-breaking rules
end
rect rgba(150, 200, 100, 0.5)
Note over Router: Dispatch with Routing<br/>(Step 4)
Router->>Agents: dispatch [--agent <agent>]<br/>(Background)
Agents->>Agents: Execute task
Agents->>GitHub: Update PR/Issue state
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~22 minutes Possibly related PRs
Suggested labels
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary of ChangesHello @marcusquinn, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the autonomous agent system by integrating a self-improvement mechanism and a flexible agent routing system. The changes enable agents to proactively identify and address systemic issues by observing task outcomes on GitHub, and to dispatch tasks to specialized agents based on their domain. This refactoring streamlines the system by centralizing state management within GitHub, making the overall process more robust and adaptable. Highlights
Changelog
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
🔍 Code Quality Report�[0;35m[MONITOR]�[0m Code Review Monitoring Report �[0;34m[INFO]�[0m Latest Quality Status: �[0;34m[INFO]�[0m Recent monitoring activity: 📈 Current Quality Metrics
Generated on: Wed Feb 25 16:57:55 UTC 2026 Generated by AI DevOps Framework Code Review Monitoring |
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces significant new concepts: a universal self-improvement principle for agents and a task-based agent routing system. The changes are well-documented across AGENTS.md, pulse.md, and runners.md, providing clear guidance on the new features. The move away from a local supervisor database to using GitHub as the single source of truth is a good simplification. I have one suggestion to improve error handling in a shell command to make debugging easier.
| TASK_DESC=$(gh issue list --repo "$(gh repo view --json nameWithOwner -q .nameWithOwner 2>/dev/null)" \ | ||
| --search "$TASK_ID" --json title -q '.[0].title' 2>/dev/null || echo "") |
There was a problem hiding this comment.
Using 2>/dev/null to suppress errors can hide important issues like authentication failures or syntax errors, making debugging difficult. It's better to allow errors to be visible on stderr and rely on || echo "" to handle command failures gracefully.
This change aligns with the repository's general rule to avoid blanket error suppression.
| TASK_DESC=$(gh issue list --repo "$(gh repo view --json nameWithOwner -q .nameWithOwner 2>/dev/null)" \ | |
| --search "$TASK_ID" --json title -q '.[0].title' 2>/dev/null || echo "") | |
| TASK_DESC=$(gh issue list --repo "$(gh repo view --json nameWithOwner -q .nameWithOwner)" \ | |
| --search "$TASK_ID" --json title -q '.[0].title' || echo "") |
References
- Avoid using '2>/dev/null' for blanket suppression of command errors in shell scripts to ensure that authentication, syntax, or system issues remain visible for debugging.
🤖 Augment PR SummarySummary: This PR updates the supervisor/worker playbooks to treat GitHub as the single source of truth for state and to improve how work is routed and learned from over time. Changes:
Technical Notes: Outcome observation focuses on lightweight heuristics (stale PRs, closed-without-merge failures, duplicate work) and feeds improvements back into the existing GitHub issue queue. 🤖 Was this summary useful? React with 👍 or 👎 |
![augmentcode[bot]](https://avatars.githubusercontent.com/in/1027498?s=60&v=4){kind=small}
| TASK_DESC=$(~/.aidevops/agents/scripts/supervisor-helper.sh db \ | ||
| "SELECT description FROM tasks WHERE id = '$TASK_ID';" 2>/dev/null || echo "") | ||
| TASK_DESC=$(gh issue list --repo "$(gh repo view --json nameWithOwner -q .nameWithOwner 2>/dev/null)" \ | ||
| --search "$TASK_ID" --json title -q '.[0].title' 2>/dev/null || echo "") |
There was a problem hiding this comment.
gh issue list ... -q '.[0].title' will often print the literal null when no issues match, which makes $TASK_DESC non-empty and can prevent the later “(task not found)” fallback from triggering. Consider ensuring “no match” results in an empty string rather than null so the control flow behaves as intended.
Severity: medium
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
| ``` | ||
|
|
||
| - If `WORKER_COUNT >= 6`: output `Pulse: all 6 slots full. Skipping.` and **exit immediately**. | ||
| - If `WORKER_COUNT >= 6`: set `AVAILABLE=0` — no new workers, but continue to Step 2 (merges and outcome observation don't need slots). |
There was a problem hiding this comment.
When WORKER_COUNT >= 6 sets AVAILABLE=0, Step 3’s “pick up to AVAILABLE items” can be read as “do nothing,” even though Step 4 says merges don’t count against concurrency. Consider clarifying that merge-ready PRs should still be processed even when AVAILABLE=0 so busy periods don’t stall merges.
Severity: medium
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
.agents/AGENTS.md (1)
64-108: Optional: trim dispatch examples to stay within AGENTS.md's progressive-disclosure budget.The 12-row routing table and the routing rules (lines 85–90) are genuinely universal and belong here. However, the three
opencode runbash examples at lines 94–106 duplicate content already present inpulse.mdandrunners.md. Keeping them here inflates the instruction budget without adding unique information.Consider replacing the example block with a pointer:
♻️ Proposed trim
-**Dispatch example:** - -```bash -# Code task (default — Build+ implied) -opencode run --dir ~/Git/awardsapp --title "Issue `#42`: Fix auth" \ - "/full-loop Implement issue `#42` -- Fix authentication bug" & - -# SEO task -opencode run --dir ~/Git/awardsapp --agent SEO --title "Issue `#55`: SEO audit" \ - "/full-loop Implement issue `#55` -- Run SEO audit on landing pages" & - -# Content task -opencode run --dir ~/Git/awardsapp --agent Content --title "Issue `#60`: Blog post" \ - "/full-loop Implement issue `#60` -- Write launch announcement blog post" & -``` +**Dispatch examples**: see `scripts/commands/pulse.md` "Agent routing" and `scripts/commands/runners.md` "Step 2: Dispatch Workers".Based on learnings: "Use progressive disclosure in AGENTS.md with pointers to subagents rather than inline content" and "Limit root AGENTS.md to ~50-100 max instructions with universal applicability to >80% of tasks."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.agents/AGENTS.md around lines 64 - 108, Remove the three duplicate opencode run bash examples in the "Dispatch example" block and replace them with a single pointer line referencing the existing detailed docs (e.g., "Dispatch examples: see scripts/commands/pulse.md 'Agent routing' and scripts/commands/runners.md 'Step 2: Dispatch Workers'"); keep the "Available primary agents" table and the "Routing rules" section intact, and ensure the new pointer uses the proposed concise wording so AGENTS.md stays within its progressive-disclosure budget.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.agents/scripts/commands/full-loop.md:
- Around line 33-37: The lookup using gh issue list with --search "$TASK_ID" can
return a wrong match; update the TASK_DESC fetch to restrict matches to issues
whose title starts with the task ID and add a short comment documenting the
convention (supervisor-created issues must embed the task ID at the start of the
title). Replace the current -q '.[0].title' selection with a jq filter that
selects only entries where .title startswith the TASK_ID (e.g., use a query that
selects .[] | select(.title | startswith(TASK_ID)) and then takes the first
.title) so unrelated body matches are ignored, keeping the existing empty-string
fallback behavior. Ensure you reference the TASK_DESC variable and the gh issue
list --json title invocation when making the change.
In @.agents/scripts/commands/pulse.md:
- Around line 66-72: The GH issue command in pulse.md uses a nonstandard
combined label "priority:high" which will cause gh issue create to fail if that
label doesn't exist; update the example in pulse.md to either (A) remove the
custom label and use only default labels (e.g., keep "bug") or (B) pass each
label with separate --label flags (e.g., --label "bug" --label "priority:high")
and add a one-time label-creation step in setup.sh to create "priority:high" in
the managed repos; ensure the example shows the chosen approach and keep the
reference to the gh issue create invocation for discoverability.
---
Nitpick comments:
In @.agents/AGENTS.md:
- Around line 64-108: Remove the three duplicate opencode run bash examples in
the "Dispatch example" block and replace them with a single pointer line
referencing the existing detailed docs (e.g., "Dispatch examples: see
scripts/commands/pulse.md 'Agent routing' and scripts/commands/runners.md 'Step
2: Dispatch Workers'"); keep the "Available primary agents" table and the
"Routing rules" section intact, and ensure the new pointer uses the proposed
concise wording so AGENTS.md stays within its progressive-disclosure budget.
ℹ️ Review info
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
.agents/AGENTS.md.agents/scripts/commands/full-loop.md.agents/scripts/commands/pulse.md.agents/scripts/commands/runners.md
| # Priority 3: Query GitHub issues (for dynamically-created tasks not yet in TODO.md) | ||
| if [[ -z "$TASK_DESC" ]]; then | ||
| TASK_DESC=$(~/.aidevops/agents/scripts/supervisor-helper.sh db \ | ||
| "SELECT description FROM tasks WHERE id = '$TASK_ID';" 2>/dev/null || echo "") | ||
| TASK_DESC=$(gh issue list --repo "$(gh repo view --json nameWithOwner -q .nameWithOwner 2>/dev/null)" \ | ||
| --search "$TASK_ID" --json title -q '.[0].title' 2>/dev/null || echo "") | ||
| fi |
There was a problem hiding this comment.
--search "$TASK_ID" relies on an undocumented convention — silent wrong match is possible.
gh issue list --search performs a full-text search across titles and bodies. $TASK_ID values like t061 are TODO.md-internal identifiers with no guaranteed presence in GitHub issue content. Two failure modes:
- Silent miss (gracefully handled) — no issue mentions
t061, returns empty, falls through to"(task not found)". - Silent wrong match — an unrelated issue whose body happens to mention
t061floats to.[0], and its title becomes the session title for the wrong task.
Add a comment documenting the convention required for this lookup to work (e.g., "supervisor-created issues must embed the task ID in the title"), or tighten the jq filter to only match when the title starts with the task ID:
🛡️ Proposed safer filter
- TASK_DESC=$(gh issue list --repo "$(gh repo view --json nameWithOwner -q .nameWithOwner 2>/dev/null)" \
- --search "$TASK_ID" --json title -q '.[0].title' 2>/dev/null || echo "")
+ # NOTE: Relies on supervisor-created issues embedding the task ID at the
+ # start of their title (e.g., "t061: Fix login bug"). Matches first result
+ # whose title begins with the task ID.
+ TASK_DESC=$(gh issue list --repo "$(gh repo view --json nameWithOwner -q .nameWithOwner 2>/dev/null)" \
+ --search "$TASK_ID" --json title \
+ -q "[.[] | select(.title | startswith(\"$TASK_ID:\")) | .title] | .[0] // \"\"" \
+ 2>/dev/null || echo "")🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.agents/scripts/commands/full-loop.md around lines 33 - 37, The lookup using
gh issue list with --search "$TASK_ID" can return a wrong match; update the
TASK_DESC fetch to restrict matches to issues whose title starts with the task
ID and add a short comment documenting the convention (supervisor-created issues
must embed the task ID at the start of the title). Replace the current -q
'.[0].title' selection with a jq filter that selects only entries where .title
startswith the TASK_ID (e.g., use a query that selects .[] | select(.title |
startswith(TASK_ID)) and then takes the first .title) so unrelated body matches
are ignored, keeping the existing empty-string fallback behavior. Ensure you
reference the TASK_DESC variable and the gh issue list --json title invocation
when making the change.
| **Stale PRs:** If any open PR was last updated more than 6 hours ago, something is stuck. Check if it has a worker branch with no recent commits. If so, create a GitHub issue: | ||
|
|
||
| ```bash | ||
| gh issue create --repo <owner/repo> --title "Stuck PR #<number>: <title>" \ | ||
| --body "PR #<number> has been open for 6+ hours with no progress. Last updated: <timestamp>. Likely cause: <hypothesis>. Suggested fix: <action>." \ | ||
| --label "bug,priority:high" | ||
| ``` |
There was a problem hiding this comment.
priority:high is a custom label — gh issue create will fail if it doesn't exist in the target repo.
bug is a GitHub default label, but priority:high is not. gh issue create fails with label errors when label names don't match exactly. If priority:high hasn't been pre-created in marcusquinn/aidevops or awardsapp/awardsapp, every self-improvement issue creation will error out silently.
Two options:
🛡️ Option A — drop the custom label, use only defaults
-gh issue create --repo <owner/repo> --title "Stuck PR #<number>: <title>" \
- --body "PR #<number> has been open for 6+ hours with no progress. Last updated: <timestamp>. Likely cause: <hypothesis>. Suggested fix: <action>." \
- --label "bug,priority:high"
+gh issue create --repo <owner/repo> --title "Stuck PR #<number>: <title>" \
+ --body "PR #<number> has been open for 6+ hours with no progress. Last updated: <timestamp>. Likely cause: <hypothesis>. Suggested fix: <action>." \
+ --label "bug"🛡️ Option B — use separate --label flags (more readable)
-gh issue create --repo <owner/repo> --title "Stuck PR #<number>: <title>" \
- --body "..." \
- --label "bug,priority:high"
+gh issue create --repo <owner/repo> --title "Stuck PR #<number>: <title>" \
+ --body "..." \
+ --label "bug" --label "priority:high"(Pair this with a one-time label pre-creation step in setup.sh for each managed repo.)
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| **Stale PRs:** If any open PR was last updated more than 6 hours ago, something is stuck. Check if it has a worker branch with no recent commits. If so, create a GitHub issue: | |
| ```bash | |
| gh issue create --repo <owner/repo> --title "Stuck PR #<number>: <title>" \ | |
| --body "PR #<number> has been open for 6+ hours with no progress. Last updated: <timestamp>. Likely cause: <hypothesis>. Suggested fix: <action>." \ | |
| --label "bug,priority:high" | |
| ``` | |
| **Stale PRs:** If any open PR was last updated more than 6 hours ago, something is stuck. Check if it has a worker branch with no recent commits. If so, create a GitHub issue: | |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.agents/scripts/commands/pulse.md around lines 66 - 72, The GH issue command
in pulse.md uses a nonstandard combined label "priority:high" which will cause
gh issue create to fail if that label doesn't exist; update the example in
pulse.md to either (A) remove the custom label and use only default labels
(e.g., keep "bug") or (B) pass each label with separate --label flags (e.g.,
--label "bug" --label "priority:high") and add a one-time label-creation step in
setup.sh to create "priority:high" in the managed repos; ensure the example
shows the chosen approach and keep the reference to the gh issue create
invocation for discoverability.
1 participant
Summary
--agentflagChanges
.agents/AGENTS.md.agents/scripts/commands/pulse.md.agents/scripts/commands/runners.md/pulsedescription; add--agentto dispatch examples; self-improvement ref.agents/scripts/commands/full-loop.mdgh issuesearchDesign Decisions
--agentfor code tasks. Only specify when the domain clearly matches another agent.Summary by CodeRabbit
Release Notes