feat: add continuous-learning skill for knowledge extraction - #51
Conversation
|
Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. 📝 WalkthroughSummary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings. WalkthroughAdds a new documentation file describing a protocol for extracting debugging knowledge into standalone skill files: format, metadata, extraction criteria, workflow, examples, anti-patterns, storage scope, and maintenance guidance (no code or functional changes). Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~5 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@skills/continuous-learning/SKILL.md`:
- Around line 72-91: The Quick Extraction Flow snippet in SKILL.md currently
provides only a Bash example; update the code examples so Fish shell is shown
first (primary) and Bash included as an alternative, replacing the single
```bash block with a Fish block (e.g., start with ```fish showing mkdir -p
.claude/skills and cat > .claude/skills/my-discovery.md << 'EOF' ... EOF) and
then add a separate ````bash``` block labeled "Bash (alternative)" with the
original content; ensure headings and fencing remain valid in the "## Quick
Extraction Flow" section and keep the same file content and metadata inside the
heredoc.
| ## Quick Extraction Flow | ||
|
|
||
| ```bash | ||
| # 1. Create skill directory if needed | ||
| mkdir -p .claude/skills | ||
|
|
||
| # 2. Write skill file | ||
| cat > .claude/skills/my-discovery.md << 'EOF' | ||
| --- | ||
| name: my-discovery | ||
| description: Fix for XYZ error when ABC happens | ||
| triggers: | ||
| - "exact error message here" | ||
| verified: 2025-01-18 | ||
| --- | ||
|
|
||
| # Problem | ||
| ... | ||
| EOF | ||
| ``` |
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major
Add Fish shell example per coding guidelines.
The coding guidelines specify: "Provide Fish shell snippets for automation examples with Bash as optional fallback." Currently, only Bash is provided. Based on coding guidelines, add a Fish shell example as the primary automation snippet.
🐚 Proposed refactor with Fish shell primary
## Quick Extraction Flow
+### Fish Shell
+
+```fish
+# 1. Create skill directory if needed
+mkdir -p .claude/skills
+
+# 2. Write skill file
+cat > .claude/skills/my-discovery.md << 'EOF'
+---
+name: my-discovery
+description: Fix for XYZ error when ABC happens
+triggers:
+ - "exact error message here"
+verified: 2025-01-18
+---
+
+# Problem
+...
+EOF
+```
+
+### Bash (alternative)
+
```bash
# 1. Create skill directory if needed
mkdir -p .claude/skills
# 2. Write skill file
cat > .claude/skills/my-discovery.md << 'EOF'
---
name: my-discovery
description: Fix for XYZ error when ABC happens
triggers:
- "exact error message here"
verified: 2025-01-18
---
# Problem
...
EOF
</details>
<details>
<summary>🤖 Prompt for AI Agents</summary>
In @skills/continuous-learning/SKILL.md around lines 72 - 91, The Quick
Extraction Flow snippet in SKILL.md currently provides only a Bash example;
update the code examples so Fish shell is shown first (primary) and Bash
included as an alternative, replacing the single bash block with a Fish block (e.g., start with fish showing mkdir -p .claude/skills and cat >
.claude/skills/my-discovery.md << 'EOF' ... EOF) and then add a separate
````bash``` block labeled "Bash (alternative)" with the original content; ensure
headings and fencing remain valid in the "## Quick Extraction Flow" section and
keep the same file content and metadata inside the heredoc.
</details>
<!-- fingerprinting:phantom:triton:puma -->
<!-- This is an auto-generated comment by CodeRabbit -->
There was a problem hiding this comment.
Pull request overview
This PR adds a new continuous-learning skill that enables Claude Code to extract and persist reusable knowledge from debugging sessions into skill files. The skill provides a framework for deciding when to extract knowledge (non-obvious solutions, misleading errors, project-specific patterns), a structured format for skill files, and guidance on storage locations.
Changes:
- Add continuous-learning skill definition with extraction triggers and decision framework
- Document skill file format with YAML frontmatter including triggers and verification fields
- Provide storage location guidance for project-specific vs. cross-project knowledge
- Include practical Prisma serverless example and anti-patterns to avoid
| @@ -0,0 +1,169 @@ | |||
| --- | |||
| name: continuous-learning | |||
| allowed-tools: Read, Write, Glob, Grep, Bash, TodoWrite | |||
There was a problem hiding this comment.
The allowed-tools list includes "Bash" but according to the repository's shell usage guidelines, Fish shell should be the default. Consider updating this to reflect that Fish is the primary shell, or clarify that "Bash" here refers to shell command execution in general (which could include Fish).
| - **Discovery**: <How this was found> | ||
| - **Caveats**: <Edge cases or limitations> | ||
| ``` | ||
|
|
There was a problem hiding this comment.
The skill file format example (lines 36-63) includes "triggers" and "verified" fields in the YAML frontmatter, but the actual skill file for continuous-learning doesn't include these fields. This creates an inconsistency - either the example should match the structure used by actual skills in the repository, or there should be an explanation of why continuous-learning uses a different format than the skills it will create.
| Note: The frontmatter at the very top of this file describes the **agent skill definition** | |
| (`name`, `allowed-tools`, `description`) used by the automation framework. By contrast, the | |
| example above shows the format for **debugging knowledge skill files** that | |
| `/continuous-learning` will create. These generated skills add: | |
| - `triggers`: concrete error messages or symptoms that should cause the skill to be considered. | |
| - `verified`: the date the solution was last confirmed to work. | |
| This intentional difference lets the agent skill remain minimal while the reusable knowledge | |
| skills carry richer metadata for matching and maintenance. |
| --- | ||
| name: continuous-learning | ||
| allowed-tools: Read, Write, Glob, Grep, Bash, TodoWrite | ||
| description: Extract and persist reusable knowledge from debugging sessions into skill files | ||
| --- | ||
|
|
||
| # /continuous-learning — Extract debugging knowledge into skills | ||
|
|
||
| Autonomously identify non-obvious solutions discovered during tasks and persist them as reusable skill files for future sessions. | ||
|
|
||
| ## When to Extract | ||
|
|
||
| Extract knowledge when you encounter: | ||
|
|
||
| | Trigger | Example | | ||
| |---------|---------| | ||
| | Non-obvious solution | Debugging required investigation beyond docs | | ||
| | Misleading error | Error message didn't point to root cause | | ||
| | Project-specific pattern | Workaround unique to this codebase | | ||
| | Tool integration quirk | Behavior not documented in official docs | | ||
| | Performance discovery | Optimization found through profiling | | ||
|
|
||
| ## Extraction Decision | ||
|
|
||
| Ask yourself: | ||
|
|
||
| 1. **Did I discover this?** — Not just read it from documentation | ||
| 2. **Is it reusable?** — Will this help in future similar situations | ||
| 3. **Is it non-trivial?** — Would take time to rediscover | ||
| 4. **Can I verify it?** — Solution has clear verification steps | ||
|
|
||
| If **all four** are yes, extract the knowledge. | ||
|
|
||
| ## Skill File Format | ||
|
|
||
| ```markdown | ||
| --- | ||
| name: <kebab-case-name> | ||
| description: <one-line description with specific trigger words> | ||
| triggers: | ||
| - "<exact error message or symptom>" | ||
| - "<another trigger condition>" | ||
| verified: <date> | ||
| --- | ||
|
|
||
| # Problem | ||
|
|
||
| <What went wrong and why it's non-obvious> | ||
|
|
||
| # Solution | ||
|
|
||
| <Step-by-step fix with code examples> | ||
|
|
||
| # Verification | ||
|
|
||
| <How to confirm the fix worked> | ||
|
|
||
| # Context | ||
|
|
||
| - **Environment**: <OS, runtime, versions> | ||
| - **Discovery**: <How this was found> | ||
| - **Caveats**: <Edge cases or limitations> | ||
| ``` | ||
|
|
||
| ## Storage Locations | ||
|
|
||
| | Scope | Path | Use Case | | ||
| |-------|------|----------| | ||
| | Project | `.claude/skills/<name>.md` | Project-specific patterns | | ||
| | User | `~/.claude/skills/<name>.md` | Cross-project knowledge | | ||
|
|
||
| ## Quick Extraction Flow | ||
|
|
||
| ```bash | ||
| # 1. Create skill directory if needed | ||
| mkdir -p .claude/skills | ||
|
|
||
| # 2. Write skill file | ||
| cat > .claude/skills/my-discovery.md << 'EOF' | ||
| --- | ||
| name: my-discovery | ||
| description: Fix for XYZ error when ABC happens | ||
| triggers: | ||
| - "exact error message here" | ||
| verified: 2025-01-18 | ||
| --- | ||
|
|
||
| # Problem | ||
| ... | ||
| EOF | ||
| ``` | ||
|
|
||
| ## Description Best Practices | ||
|
|
||
| Write descriptions for semantic matching: | ||
|
|
||
| | Bad | Good | | ||
| |-----|------| | ||
| | "Database issues" | "Fix PrismaClientKnownRequestError in serverless" | | ||
| | "Build problems" | "Resolve vite circular dependency with barrel files" | | ||
| | "API errors" | "Handle 429 rate limit with exponential backoff" | | ||
|
|
||
| ## Example: Extracted Skill | ||
|
|
||
| ```markdown | ||
| --- | ||
| name: prisma-serverless-pool | ||
| description: Fix connection pool exhaustion in serverless with Prisma | ||
| triggers: | ||
| - "P2024: Timed out fetching a new connection" | ||
| - "too many connections for role" | ||
| verified: 2025-01-15 | ||
| --- | ||
|
|
||
| # Problem | ||
|
|
||
| Serverless functions spawn multiple Prisma clients, each opening 5 connections by default. Under load, this exhausts database connection limits. | ||
|
|
||
| # Solution | ||
|
|
||
| 1. Use connection pooling service (PgBouncer, Prisma Accelerate) | ||
| 2. Configure connection limit in URL: | ||
| ``` | ||
| DATABASE_URL="postgresql://...?connection_limit=1" | ||
| ``` | ||
| 3. Implement singleton pattern: | ||
| ```typescript | ||
| const globalForPrisma = globalThis as unknown as { prisma: PrismaClient } | ||
| export const prisma = globalForPrisma.prisma ?? new PrismaClient() | ||
| if (process.env.NODE_ENV !== 'production') globalForPrisma.prisma = prisma | ||
| ``` | ||
|
|
||
| # Verification | ||
|
|
||
| 1. Load test deployment with concurrent requests | ||
| 2. Monitor connection count stays within limits | ||
| 3. No P2024 errors under sustained load | ||
|
|
||
| # Context | ||
|
|
||
| - **Environment**: Node.js 18+, Prisma 5.x, Vercel/AWS Lambda | ||
| - **Discovery**: Production incident with connection timeouts | ||
| - **Caveats**: Singleton pattern only helps in dev; pooling required for prod | ||
| ``` | ||
|
|
||
| ## Retrospective Trigger | ||
|
|
||
| Run `/continuous-learning` at session end or after solving complex problems. Ask: | ||
|
|
||
| > "Did I solve anything non-obvious that I'd want to remember?" | ||
|
|
||
| If yes, extract it before the session ends. | ||
|
|
||
| ## Anti-Patterns | ||
|
|
||
| Avoid extracting: | ||
|
|
||
| - **Documentation lookups** — Just reading official docs | ||
| - **Trivial fixes** — Typos, missing imports | ||
| - **Highly specific** — Only applies to one exact scenario | ||
| - **Unverified** — Hasn't been confirmed to work | ||
|
|
||
| ## Guidelines | ||
|
|
||
| - One skill per file, focused on single problem | ||
| - Include exact error messages in triggers | ||
| - Date your verification | ||
| - Update skills when they become stale | ||
| - Delete skills that no longer apply |
There was a problem hiding this comment.
The skill file is added to skills/continuous-learning/ but not to .ruler/skills/continuous-learning/. According to the repository's ruler-apply guidelines, skills should be present in both locations. After adding this skill, pnpm run ruler:apply should be run to ensure consistency, or the skill should be added to .ruler/skills/ as well to match the pattern of other skills in the repository.
| | Bad | Good | | ||
| |-----|------| | ||
| | "Database issues" | "Fix PrismaClientKnownRequestError in serverless" | | ||
| | "Build problems" | "Resolve vite circular dependency with barrel files" | |
There was a problem hiding this comment.
The build tool name should be capitalized: "Vite" not "vite".
| | "Build problems" | "Resolve vite circular dependency with barrel files" | | |
| | "Build problems" | "Resolve Vite circular dependency with barrel files" | |
| ```bash | ||
| # 1. Create skill directory if needed | ||
| mkdir -p .claude/skills | ||
|
|
||
| # 2. Write skill file | ||
| cat > .claude/skills/my-discovery.md << 'EOF' | ||
| --- | ||
| name: my-discovery | ||
| description: Fix for XYZ error when ABC happens | ||
| triggers: | ||
| - "exact error message here" | ||
| verified: 2025-01-18 | ||
| --- | ||
|
|
||
| # Problem | ||
| ... | ||
| EOF | ||
| ``` |
There was a problem hiding this comment.
According to the repository's shell usage guidelines, Bash heredoc syntax should be replaced with Fish shell syntax. The repository standard is to use Fish shell (#!/usr/bin/env fish) for scripts and examples. The heredoc syntax shown here (cat > file << 'EOF') is Bash-specific and should be converted to Fish syntax or provided as a Fish-compatible alternative.
5501ab6 to
b200f2d
Compare
Add skill that enables Claude Code to autonomously extract and persist reusable knowledge from debugging sessions into skill files. - Define extraction triggers (non-obvious solutions, misleading errors) - Provide skill file format with triggers and verification sections - Include storage location guidance (project vs user level) - Add example for Prisma connection pool exhaustion - Document anti-patterns to avoid over-extraction
b200f2d to
0828df1
Compare
Changes
continuous-learningskill that enables Claude Code to extract and persist reusable knowledge from debugging sessions.claude/skills/vs user~/.claude/skills/)Technical Details
Testing
Generated with Claude Code by claude-opus-4-5-20251101
Summary by cubic
Adds a continuous-learning skill to extract and save reusable debugging knowledge as skill files. It sets clear triggers, a decision framework, and a simple workflow so verified fixes are easy to reuse across projects.
Written for commit 0828df1. Summary will update on new commits.