-
Notifications
You must be signed in to change notification settings - Fork 488
feat(skills): add reflect skill -- in-session learning capture and mistake prevention #621
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
6a656b0
Merge pull request #570 from bradygaster/dev
bradygaster fe5a68a
feat(skills): add reflect skill — in-session learning capture and mis…
d7b1b0d
fix: correct changeset package names to @bradygaster scoped names per…
e163e2b
feat: add proposal document for reflect skill per review policy
7161216
Merge branch 'dev' into feat/reflect-skill
bradygaster 17fa9a8
fix: address copilot PR reviewer feedback on reflect skill
tamirdresher File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@bradygaster/squad-cli": minor | ||
| "@bradygaster/squad-sdk": minor | ||
| --- | ||
| feat: add reflect skill for in-session learning capture and mistake prevention |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| # Proposal: Reflect Skill | ||
|
|
||
| **Issue:** bradygaster/squad#621 | ||
| **Author:** tamirdresher | ||
| **Date:** 2026-03-26 | ||
| **Status:** Proposal | ||
|
|
||
| --- | ||
|
|
||
| ## Problem Statement | ||
|
|
||
| Agents repeat mistakes that were already identified in the same session. When an approach fails | ||
| or a user provides corrective feedback, agents lack a standardized mechanism to: | ||
| 1. Capture the learning before continuing | ||
| 2. Check captured learnings before attempting similar actions | ||
| 3. Prevent recurrence without human intervention | ||
|
|
||
| This is different from the history-hygiene skill (which covers cross-session history writing). | ||
| Reflect targets in-session learning capture during active work. | ||
|
|
||
| --- | ||
|
|
||
| ## Proposed Approach | ||
|
|
||
| A structured reflection protocol triggered by failure or feedback events: | ||
|
|
||
| **Trigger classification:** | ||
| - HIGH: Agent makes same error twice, user explicitly says "that's wrong again" | ||
| - MED: Tool error after a previous warning, misunderstood requirement after clarification | ||
| - LOW: Minor adjustment requested, first-time error with clear cause | ||
|
|
||
| **Reflect protocol (HIGH/MED triggers):** | ||
| 1. Write a ## Reflection entry to current session context | ||
| 2. State: what failed, why, what changes for next attempt | ||
| 3. Check reflection notes before any action of the same category | ||
| 4. If same failure occurs after reflection: escalate (do not retry silently) | ||
|
|
||
| **Confidence threshold table:** Maps trigger severity to required escalation path | ||
|
|
||
| --- | ||
|
|
||
| ## Fit with Existing Architecture | ||
|
|
||
| - **Complements** history-hygiene skill (history.md captures final outcomes; reflect captures | ||
| in-flight corrections before the outcome is known) | ||
| - **Complements** error-recovery skill (recover handles the failure response; reflect captures | ||
| the learning to prevent recurrence) | ||
| - **Integrates** with .squad/decisions.md for persistent pattern learnings | ||
| - **No code changes** — template-only | ||
|
|
||
| --- | ||
|
|
||
| ## What Changes | ||
|
|
||
| - New skill: packages/squad-cli/templates/skills/reflect/SKILL.md | ||
| - New skill: packages/squad-sdk/templates/skills/reflect/SKILL.md | ||
| - New changeset: .changeset/reflect-skill.md | ||
|
|
||
| ## What Stays the Same | ||
|
|
||
| - history-hygiene skill behavior unchanged | ||
| - error-recovery skill behavior unchanged | ||
| - No CLI or SDK runtime code changed | ||
|
|
||
| --- | ||
|
|
||
| ## Risks and Mitigations | ||
|
|
||
| | Risk | Likelihood | Impact | Mitigation | | ||
| |------|-----------|--------|------------| | ||
| | Reflect overhead slows agents | Low | Low | Only triggered by HIGH/MED events, not routine actions | | ||
| | Overlaps with history-hygiene | Low | Low | Different scope (in-session corrections vs final outcomes) | | ||
|
|
||
| --- | ||
|
|
||
| ## References | ||
|
|
||
| - Issue: bradygaster/squad#621 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,229 @@ | ||
| --- | ||
| name: reflect | ||
| description: Learning capture system that extracts HIGH/MED/LOW confidence patterns from conversations to prevent repeating mistakes. Use after user corrections ("no", "wrong"), praise ("perfect", "exactly"), or when discovering edge cases. Complements .squad/agents/{agent}/history.md and .squad/decisions.md. | ||
| license: MIT | ||
| version: 1.0.0-squad | ||
| domain: team-memory, learning | ||
| confidence: high | ||
| --- | ||
|
|
||
| # Reflect Skill | ||
|
|
||
| **Critical learning capture system** for Squad. Prevents repeating mistakes and preserves successful patterns across sessions. | ||
|
|
||
| Analyze conversations and propose improvements to squad knowledge based on what worked, what didn't, and edge cases discovered. **Every correction is a learning opportunity.** | ||
|
|
||
| --- | ||
|
|
||
| ## Integration with Squad Architecture | ||
|
|
||
| **Reflect complements existing Squad knowledge systems:** | ||
|
|
||
| 1. **`.squad/agents/{agent}/history.md`** — Permanent learnings from completed work (append-only; each agent updates their own file; Scribe propagates cross-agent updates) | ||
| 2. **`.squad/decisions.md`** — Team-wide decisions that all agents respect | ||
| 3. **`reflect` skill** — Captures in-flight learnings from conversations that may graduate to history.md or decisions.md | ||
|
|
||
| **Workflow:** | ||
| - Use `reflect` during work to capture learnings | ||
| - At session end, review captured learnings | ||
| - Promote HIGH confidence patterns → lead agent for decision.md review | ||
| - Promote agent-specific patterns → `{agent}/history.md` updates | ||
|
|
||
| --- | ||
|
|
||
| ## Triggers | ||
|
|
||
| ### 🔴 HIGH Priority (Invoke Immediately) | ||
|
|
||
| | Trigger | Example | Why Critical | | ||
| |---------|---------|--------------| | ||
| | User correction | "no", "wrong", "not like that", "never do" | Captures mistakes to prevent repetition | | ||
| | Architectural insight | "you removed that without understanding why" | Documents design decisions (Chesterton's Fence) | | ||
| | Immediate fixes | "debug", "root cause", "fix all" | Learns from errors in real-time | | ||
|
|
||
| ### 🟡 MEDIUM Priority (Invoke After Multiple) | ||
|
|
||
| | Trigger | Example | Why Important | | ||
| |---------|---------|---------------| | ||
| | User praise | "perfect", "exactly", "great" | Reinforces successful patterns | | ||
| | Tool preferences | "use X instead of Y", "prefer" | Builds workflow preferences | | ||
| | Edge cases | "what if X happens?", "don't forget", "ensure" | Captures scenarios to handle | | ||
|
|
||
| ### 🟢 LOW Priority (Invoke at Session End) | ||
|
|
||
| | Trigger | Example | Why Useful | | ||
| |---------|---------|------------| | ||
| | Repeated patterns | Frequent use of specific commands/tools | Identifies workflow preferences | | ||
| | Session end | After complex work | Consolidates all session learnings | | ||
|
|
||
| --- | ||
|
|
||
| ## Process | ||
|
|
||
| ### Phase 1: Identify Learning Target | ||
|
|
||
| Determine what knowledge system should be updated: | ||
|
|
||
| 1. **Agent-specific learning** → `.squad/agents/{agent}/history.md` | ||
| 2. **Team-wide decision** → `.squad/decisions/inbox/{agent}-{topic}.md` | ||
| 3. **Skill-specific improvement** → Document in session, recommend to skill owner | ||
|
|
||
| ### Phase 2: Analyze Conversation | ||
|
|
||
| Scan for learning signals with confidence levels: | ||
|
|
||
| #### HIGH Confidence: Corrections | ||
|
|
||
| User actively steered or corrected output. | ||
|
|
||
| **Detection patterns:** | ||
| - Explicit rejection: "no", "not like that", "that's wrong" | ||
| - Strong directives: "never do", "always do", "don't ever" | ||
| - User provided alternative implementation | ||
|
|
||
| **Example:** | ||
| ```text | ||
| User: "No, use the azure-devops MCP tool instead of raw API calls" | ||
| → [HIGH] + Add constraint: "Prefer azure-devops MCP tools over REST API" | ||
| ``` | ||
|
|
||
| #### MEDIUM Confidence: Success Patterns | ||
|
|
||
| Output was accepted or praised. | ||
|
|
||
| **Detection patterns:** | ||
| - Explicit praise: "perfect", "great", "yes", "exactly" | ||
| - User built on output without modification | ||
| - Output was committed without changes | ||
|
|
||
| **Example:** | ||
| ```text | ||
| User: "Perfect, that's exactly what I needed" | ||
| → [MED] + Add preference: "Include usage examples in documentation" | ||
| ``` | ||
|
|
||
| #### MEDIUM Confidence: Edge Cases | ||
|
|
||
| Scenarios not anticipated. | ||
|
|
||
| **Detection patterns:** | ||
| - Questions not answered | ||
| - Workarounds user had to apply | ||
| - Error handling gaps discovered | ||
|
|
||
| #### LOW Confidence: Preferences | ||
|
|
||
| Accumulated patterns over time. | ||
|
|
||
| --- | ||
|
|
||
| ### Phase 3: Propose Learnings | ||
|
|
||
| Present findings: | ||
|
|
||
| ```text | ||
| ┌─────────────────────────────────────────────────────────────┐ | ||
| │ REFLECTION: {target (agent/decision/skill)} │ | ||
| ├─────────────────────────────────────────────────────────────┤ | ||
| │ │ | ||
| │ [HIGH] + Add constraint: "{specific constraint}" │ | ||
| │ Source: "{quoted user correction}" │ | ||
| │ Target: .squad/decisions/inbox/{agent}-{topic}.md │ | ||
| │ │ | ||
| │ [MED] + Add preference: "{specific preference}" │ | ||
| │ Source: "{evidence from conversation}" │ | ||
| │ Target: .squad/agents/{agent}/history.md │ | ||
| │ │ | ||
| │ [LOW] ~ Note for review: "{observation}" │ | ||
| │ Source: "{pattern observed}" │ | ||
| │ Target: Session notes only │ | ||
| │ │ | ||
| ├─────────────────────────────────────────────────────────────┤ | ||
| │ Apply changes? [Y/n/edit] │ | ||
| └─────────────────────────────────────────────────────────────┘ | ||
| ``` | ||
|
|
||
| **Confidence Threshold:** | ||
|
|
||
| | Threshold | Action | | ||
| |-----------|--------| | ||
| | ≥1 HIGH signal | Always propose (user explicitly corrected) | | ||
| | ≥2 MED signals | Propose (sufficient pattern) | | ||
| | ≥3 LOW signals | Propose (accumulated evidence) | | ||
| | 1-2 LOW only | Skip (insufficient evidence) | | ||
|
|
||
| ### Phase 4: Persist Learnings | ||
|
|
||
| **ALWAYS show changes before applying.** | ||
|
|
||
| After user approval: | ||
|
|
||
| 1. **For Agent History:** | ||
| - Append to `.squad/agents/{agent}/history.md` under `## Learnings` section | ||
| - Format: Date, assignment context, key learning | ||
|
|
||
| 2. **For Team Decisions:** | ||
| - Create `.squad/decisions/inbox/{agent}-{topic}.md` | ||
| - Lead agent reviews and merges to `decisions.md` if appropriate | ||
|
|
||
| 3. **For Skills:** | ||
| - Document recommendation in session notes | ||
| - Squad lead reviews and routes to skill owner | ||
|
|
||
| --- | ||
|
|
||
| ## Usage Examples | ||
|
|
||
| ### Example 1: User Correction | ||
|
|
||
| **Conversation:** | ||
| ``` | ||
| Agent: "I'll use grep to search the repository" | ||
| User: "No, use the code search tools first, grep is too slow" | ||
| ``` | ||
|
|
||
| **Reflection Output:** | ||
| ``` | ||
| [HIGH] + Add constraint: "Use code intelligence tools before grep" | ||
| Source: "No, use the code search tools first, grep is too slow" | ||
| Target: .squad/agents/{agent}/history.md | ||
| ``` | ||
|
|
||
| ### Example 2: Success Pattern | ||
|
|
||
| **Conversation:** | ||
| ``` | ||
| Agent: [Creates PR with detailed description and test plan] | ||
| User: "Perfect! This is exactly the format I want for all PRs" | ||
| ``` | ||
|
|
||
| **Reflection Output:** | ||
| ``` | ||
| [MED] + Add preference: "Include test plan in PR descriptions" | ||
| Source: User praised detailed PR format | ||
| Target: .squad/decisions/inbox/pr-format.md (for team adoption) | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## When to Use | ||
|
|
||
| ✅ **Use reflect when:** | ||
| - User says "no", "wrong", "not like that" (HIGH priority) | ||
| - User says "perfect", "exactly", "great" (MED priority) | ||
| - You discover edge cases or gaps | ||
| - Complex work session with multiple learnings | ||
| - At end of sprint/milestone to consolidate patterns | ||
|
|
||
| ❌ **Don't use reflect when:** | ||
| - Simple one-off questions with no pattern | ||
| - User is just exploring ideas (no concrete decisions) | ||
| - Learning is already captured in history.md/decisions.md | ||
|
|
||
| --- | ||
|
|
||
| ## See Also | ||
|
|
||
| - `.squad/decisions.md` — Team-wide decisions | ||
| - `.squad/agents/*/history.md` — Agent-specific learnings | ||
| - `.squad/routing.md` — Work assignment patterns | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The markdown tables are written with a leading
||(double pipe), which renders as an empty first column in most markdown renderers. Use a single leading|for standard tables so they render correctly.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Verified the raw file content — all tables use single leading
|pipes. This appears to be a false positive from the reviewer. The tables render correctly in standard markdown.