diff --git a/.claude/commands/create-or-update-issue.md b/.claude/commands/create-or-update-issue.md new file mode 100644 index 000000000..e6a90293d --- /dev/null +++ b/.claude/commands/create-or-update-issue.md @@ -0,0 +1,25 @@ +# Create or Update Issue + +> Create or update GitHub issue specifications + +## Instructions + +Create a new GitHub issue or update an existing one with detailed specifications for implementation. + +Follow these steps: + +- Determine repository owner and name from git remote: extract from `git remote get-url origin` (format: `https://github.com/owner/repo.git` or `git@github.com:owner/repo.git`) and export variables: + - `OWNER=` + - `REPO=` +- Accept optional issue ID from ${ARGUMENTS}; if no argument provided, treat this as creating a new issue. +- If issue ID is provided, fetch the existing issue using `gh api repos/${OWNER}/${REPO}/issues/"${ARGUMENTS}"` and also fetch related items using `gh api repos/${OWNER}/${REPO}/issues/"${ARGUMENTS}"/timeline` to understand context. +- Read `.github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md` to understand the template structure and commented instructions, regardless of whether creating new or updating existing issue. +- If updating existing issue and sections have been removed, reference the template file for the commented instructions on what should go in each section. +- Use TaskCreate to create tasks for each specification section that needs to be filled out: Overview, Context (bug/feature/task description), and Changes (solutions/recommendations as bullets, action items as checkboxes). +- Work interactively with the user to collect information for each section. Use TaskUpdate to set tasks to in_progress while gathering info, then completed once the user approves that section. +- As sections are filled out, pause and ask "Is there a clearer, more precise way to specify this?" especially for complex requirements or decisions. +- Once all sections are complete, present the full issue content for user review and approval. +- After approval, either create a new issue using `gh api repos/${OWNER}/${REPO}/issues -f title="..." -f body="..."` or update existing using `gh api repos/${OWNER}/${REPO}/issues/"${ARGUMENTS}" -X PATCH -f body="..."`. +- When updating existing issues, preserve any existing labels by fetching them first and including them in the update. +- Read `.github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md` to extract the project from frontmatter `projects:` field (format: `["org/number"]`), parse the organization name and project number, then add the issue using GraphQL API: first get the project node ID with `query { organization(login: "") { projectV2(number: ) { id } } }` and the issue node ID, then use the addProjectV2ItemById mutation. +- Provide the final issue URL to the user and summarize what was created/updated. diff --git a/.claude/commands/fix-pull-request-feedback.md b/.claude/commands/fix-pull-request-feedback.md deleted file mode 100644 index bba57510f..000000000 --- a/.claude/commands/fix-pull-request-feedback.md +++ /dev/null @@ -1,19 +0,0 @@ -# Fix Pull Request Feedback - -> Programmatic feedback resolution - -## Instructions - -Analyze and address comments left on GitHub pull request: $ARGUMENTS. - -Follow these steps: - -- Accept the pull request ID from $ARGUMENTS and fetch outstanding feedback from GitHub using `gh pr view $ARGUMENTS --json comments,reviews` and `gh api repos/:owner/:repo/pulls/$ARGUMENTS/comments` to get comprehensive review data. -- Check the current branch state using git commands to understand what changes have already been made locally, helping avoid duplicate work. -- Parse the fetched feedback to identify unresolved, actionable comments that require code changes (filter out resolved comments, general approvals, or comments already addressed in current branch). -- Use TodoWrite to create a comprehensive checklist of all feedback items that need addressing, including the comment author, file location, and specific change requested. -- For each todo item in the checklist, read the relevant files, implement the requested changes, and mark the todo as in_progress then completed as you work through it. -- After implementing changes for each feedback item, run the appropriate validation command based on the language of the modified files: `mask development python all` for Python changes or `mask development rust all` for Rust changes. -- If Claude is in "accept edits" mode, continue through all changes automatically; if in "approve edits" mode, stop at each proposed change for user review and approval before proceeding. -- After all feedback has been addressed and validation commands have passed, generate a concise bullet point summary of all changes made, organized by file or by feedback topic. -- If validation commands fail, fix the issues and re-run the validation before moving to the next feedback item or generating the final summary. diff --git a/.claude/commands/update-notes.md b/.claude/commands/update-notes.md new file mode 100644 index 000000000..0f0a3a46e --- /dev/null +++ b/.claude/commands/update-notes.md @@ -0,0 +1,20 @@ +# Update Notes + +> Consolidate lessons into CLAUDE.md + +## Instructions + +Review and consolidate lesson patterns from `.claude/tasks/lessons.md` into the `CLAUDE.md` Notes section. + +Follow these steps: + +- Check if `.claude/tasks/lessons.md` exists; if not, create it with a header and explanation that it tracks mistake patterns with timestamps for consolidation. +- Read the entire contents of `.claude/tasks/lessons.md` to identify all rules and their timestamps. +- Calculate the date that was 7 days ago from the current date when this command is run. +- Identify rules with timestamps older than that 7-day threshold. +- For each rule older than 7 days, draft a consolidated bullet point suitable for the `CLAUDE.md` "## Notes" section following the existing style and conventions in that file. +- Present the consolidation plan to the user showing: (1) rules to be consolidated with their timestamps, (2) proposed consolidated text for CLAUDE.md, (3) confirmation that these will be removed from lessons.md. +- Wait for user approval before proceeding. +- After approval, append the consolidated bullet points to the `CLAUDE.md` "## Notes" section maintaining logical grouping with existing notes. +- Remove the consolidated rules from `.claude/tasks/lessons.md`, preserving any rules newer than 7 days. +- Provide a summary of how many rules were consolidated and how many remain in lessons.md for future consolidation. diff --git a/.claude/commands/update-pull-request.md b/.claude/commands/update-pull-request.md new file mode 100644 index 000000000..8fa64a532 --- /dev/null +++ b/.claude/commands/update-pull-request.md @@ -0,0 +1,300 @@ +# Update Pull Request + +> Address PR feedback and fix failing checks + +## Instructions + +Analyze and address all feedback and failing checks on a GitHub pull request, then respond to and resolve all comments. + +Follow these steps: + +### 1. Fetch PR Data + +- Accept the pull request ID from ${ARGUMENTS}; error if no argument is provided with a clear message that a PR number is required. +- Determine the scratchpad directory path from the system reminder message (shown at session start, format: `/private/tmp/claude-*/scratchpad`). Use this for all temporary file storage instead of `/tmp/` to ensure session isolation and automatic cleanup. +- Determine repository owner and name from git remote: extract from `git remote get-url origin` (format: `https://github.com/owner/repo.git` or `git@github.com:owner/repo.git`) and export variables: + - `OWNER=` + - `REPO=` +- Fetch comprehensive PR data using a single GraphQL query, saving to a file to avoid token limit issues: + + ```bash + + gh api graphql -f query=' + query($owner: String!, $repo: String!, $number: Int!) { + repository(owner: $owner, name: $repo) { + pullRequest(number: $number) { + id + title + headRefName + headRefOid + baseRefName + + reviewThreads(first: 100) { + nodes { + id + isResolved + isOutdated + comments(first: 100) { + nodes { + id + databaseId + body + author { login } + path + position + createdAt + } + } + } + } + + comments(first: 100) { + nodes { + id + databaseId + body + author { login } + createdAt + } + } + + reviews(first: 50) { + nodes { + id + state + body + author { login } + submittedAt + } + } + + commits(last: 1) { + nodes { + commit { + checkSuites(first: 50) { + nodes { + workflowRun { + id + databaseId + } + checkRuns(first: 50) { + nodes { + name + status + conclusion + detailsUrl + } + } + } + } + } + } + } + } + } + } + ' -f owner="${OWNER}" -f repo="${REPO}" -F number=${ARGUMENTS} > ${SCRATCHPAD}/pr_data.json + ``` + +- This single query replaces multiple REST API calls and includes thread IDs needed for later resolution. +- **Important**: Save output to a file (`${SCRATCHPAD}/pr_data.json`) to avoid token limit errors when reading large responses. Parse this file using `jq` for subsequent processing. +- **Critical**: The PR data file will be too large to read directly with the Read tool. Always use `jq` to parse and extract specific fields. Never attempt to read the entire file. + +### 2. Analyze Check Failures + +- Identify failing checks (Python or Rust checks specifically). Note that check-runs and workflow runs are distinct; to fetch logs, first obtain the workflow run ID from the check-run's check_suite, then use `gh api repos/${OWNER}/${REPO}/actions/runs/{run_id}/logs` (replacing `{run_id}` with the actual run ID). +- If logs are inaccessible via API, run `mask development python all` or `mask development rust all` locally to replicate the errors and capture the failure details. +- Add check failures to the list of items that need fixes. + +### 3. Group and Analyze Feedback + +- Parse the saved PR data from `${SCRATCHPAD}/pr_data.json` using these extraction rules: + - From `data.repository.pullRequest.reviewThreads.nodes[]`: + - **First, identify outdated threads**: Filter for `isResolved: false` AND `isOutdated: true` + - Auto-resolve these immediately (see step 3a below) since they're no longer relevant to current code + - **Then, extract unresolved threads**: Filter for `isResolved: false` AND `isOutdated: false` + - For each unresolved thread, extract: + - Thread ID: `.id` (format: `PRRT_*`) for later resolution + - For each comment in `.comments.nodes[]`: + - Comment database ID: `.databaseId` (integer) + - Comment node ID: `.id` (format: `PRRC_*`) for GraphQL replies + - Comment body: `.body` + - Author: `.author.login` + - File location: `.path` and `.position` + - From `data.repository.pullRequest.comments.nodes[]`: + - Extract issue-level comments (PR conversation): + - Comment database ID: `.databaseId` + - Comment body: `.body` + - Author: `.author.login` + - From check runs in `commits.nodes[].commit.checkSuites.nodes[].checkRuns.nodes[]`: + - Filter where `conclusion: "FAILURE"` or `conclusion: "TIMED_OUT"` +- Store complete metadata for each feedback item to use in later steps. +- Group related feedback using judgement: by file, by theme, by type of change, or whatever makes most sense for the specific PR; ensure each group maintains the full metadata for all comments it contains. +- Analyze dependencies between feedback groups to determine which are independent (can be worked in parallel) and which are interdependent (must be handled sequentially). +- For each piece of feedback, evaluate whether to address it (make code changes) or reject it (explain why the feedback doesn't apply); provide clear reasoning for each decision. + +### 3a. Resolve Outdated Threads + +- Before processing feedback, auto-resolve all outdated threads that are still unresolved: + + ```bash + # Log outdated threads before resolution for verification + echo "=== Auto-resolving outdated threads ===" + jq -r '.data.repository.pullRequest.reviewThreads.nodes[] | select(.isResolved == false and .isOutdated == true) | "\(.id) | \(.comments.nodes[0].path) | \(.comments.nodes[0].author.login) | \(.comments.nodes[0].body[:80])"' ${SCRATCHPAD}/pr_data.json + + # Extract outdated thread IDs and resolve + jq -r '.data.repository.pullRequest.reviewThreads.nodes[] | select(.isResolved == false and .isOutdated == true) | .id' ${SCRATCHPAD}/pr_data.json | while read thread_id; do + gh api graphql -f query=" + mutation { + resolveReviewThread(input: {threadId: \"$thread_id\"}) { + thread { id isResolved } + } + } + " + done + ``` + +- Log which threads were auto-resolved as outdated for the final summary. + +### 4. Enter Plan Mode + +- Enter plan mode to organize the work. +- Present a high-level overview showing: (1) total number of feedback groups identified, (2) brief description of each group, (3) which groups are independent vs interdependent, (4) check failures that need fixes. +- Wait for user acknowledgment of the overview before proceeding to detailed planning. + +### 5. Present Plans Consecutively + +**Default to consecutive presentation to provide clear, reviewable chunks** - even for superficial fixes. Present each logical group separately and wait for approval before proceeding. + +- For each feedback group: + - Present a detailed plan including: grouped feedback items with metadata (comment IDs, commenters, file/line), recommended actions (address vs reject with reasoning), and implementation approach for fixes. + - For non-trivial changes, pause and ask "Is there a more elegant solution?" before implementing. + - Wait for user approval of this group's plan before proceeding. + +- Only when explicitly grouping truly independent work that benefits from simultaneous execution: + - Note: "The following groups are independent and will be implemented in parallel using subagents to keep main context clean." + - Wait for user approval before spawning parallel subagents. + +### 6. Implement Fixes + +- After plan approval for a group (or parallel groups): + - For independent groups, spawn parallel subagents to implement solutions simultaneously, keeping main context clean. + - For interdependent groups, implement sequentially. + - Implement all fixes for feedback being addressed and for check failures in this group. + +### 7. Verify Changes + +- After implementing each group's fixes, run verification checks locally: + - Run `mask development python all` if any Python files were modified. + - Run `mask development rust all` if any Rust files were modified. + - Skip redundant checks if the next group will touch the same language files (batch them), but always run comprehensive checks at the end. + - **Note**: Local verification confirms fixes work in the development environment, but remote CI on the PR will not re-run or reflect these results until changes are pushed in Section 10. +- If checks fail, resolve issues and re-run until passing before moving to the next group. +- Do not proceed to the next feedback group until current group's changes pass verification. + +### 8. Iterate Through All Groups + +- Repeat steps 5-7 for each feedback group until all have been addressed. +- Always run final comprehensive verification using both `mask development python all` and `mask development rust all` regardless of which files were changed. + +### 9. Respond to and Resolve Comments + +- For each piece of feedback (both addressed and rejected), draft a response comment explaining what was done or why it was rejected, using the commenter name for personalization. +- Post all response comments to their respective threads: + - For review comments (code-level), use GraphQL `addPullRequestReviewComment` mutation: + + ```bash + # IMPORTANT: Keep response text simple - avoid newlines, code blocks, and special characters + # GraphQL string literals cannot contain raw newlines; use spaces or simple sentences + # If complex formatting is needed, save response to a variable first and ensure proper escaping + + gh api graphql -f query=' + mutation { + addPullRequestReviewComment(input: { + pullRequestId: "", + body: "", + inReplyTo: "" + }) { + comment { id } + } + } + ' + ``` + + Use the PR's node ID from step 1's query (`data.repository.pullRequest.id`) for `pullRequestId`. + Use the comment's node ID (format: `PRRC_*`) for `inReplyTo` parameter. + + **Response formatting guidelines**: + - Keep responses concise and single-line when possible + - Avoid embedding code blocks or complex markdown in mutation strings + - Use simple sentences: "Fixed in step X" or "Updated to use GraphQL approach" + - For longer responses, reference line numbers or file paths instead of quoting code + + - For issue comments (PR-level), use REST API: + + ```bash + gh api repos/${OWNER}/${REPO}/issues/"${ARGUMENTS}"/comments -f body="" + ``` + +- For each response posted, capture the returned comment ID for verification. +- Auto-resolve all comment threads after posting responses: + - For review comment threads: + - Use the thread ID (format: `PRRT_*`) captured during parsing from GraphQL response's `reviewThreads.nodes[].id` field. + - Resolve thread using GraphQL mutation: + + ```bash + gh api graphql -f query=' + mutation { + resolveReviewThread(input: {threadId: ""}) { + thread { + id + isResolved + } + } + } + ' + ``` + + - Map each comment back to its parent thread using the data structure from step 3 parsing. + - Resolve both addressed and rejected feedback threads (explanation provided in response). + - **Note**: Threads are resolved based on local verification. The fixes will not appear on the remote PR branch until Section 10's commit and push. Remote CI will not re-run until changes are pushed. + + - For issue comments (PR-level): + - No resolution mechanism (issue comments don't have thread states). + - Only post response; no resolution step needed. + +- **Do not create any git commits yet.** All changes remain unstaged. + +### 10. Commit Changes (After User Confirmation) + +- After user confirms that responses and resolutions are correct, create a git commit: + - Stage all modified files: `git add ` + - Create commit with descriptive message following CLAUDE.md conventions: + - Include detailed summary of what was fixed and why + - Reference PR number + - Add co-author line: extract model name from system context (format: "You are powered by the model named X") and use `Co-Authored-By: Claude X ` + - Example: + + ```bash + git add file1.py file2.md + git commit -m "$(cat <<'EOF' + Address PR # feedback: + + + + Co-Authored-By: Claude + EOF + )" + ``` + +- Ask user: "Ready to push these changes to the remote branch to update the PR? This will trigger remote CI to re-run and make the fixes visible to other reviewers." + +### 11. Final Summary + +- Provide a comprehensive summary showing: + - Outdated threads auto-resolved (count and thread IDs). + - Total feedback items processed (with count of addressed vs rejected). + - Which checks were fixed. + - Confirmation that all comments have been responded to and resolved. + - Final verification status (all local checks passing; note that remote CI status will update after pushing changes). +- For check failures that were fixed, note that no comments were posted - the fixes will be reflected in re-run checks after pushing to the remote branch. diff --git a/.github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md index 2c18c4767..5f640a1b3 100644 --- a/.github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md +++ b/.github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md @@ -1,43 +1,21 @@ --- -name: Spec -about: Create a specification for Ralph autonomous implementation +name: Issue +about: Create an issue title: '' -labels: ["feature", "in-refinement"] projects: ["oscmcompany/1"] - --- +# Overview -# Description - - - -**Goal:** - -## Requirements - -### Category 1 - -- [ ] Requirement (testable/verifiable) -- [ ] Another requirement - -### Category 2 - -- [ ] Requirement -- [ ] Requirement - -## Open Questions - -- [ ] Question that needs resolution before ready? -- [ ] Another question? - -## Decisions - -- [ ] **Decision name:** Choice made and rationale +## Context -## Specification + - +## Changes -## Implementation Notes + - diff --git a/.gitignore b/.gitignore index 26f8fd3e8..fa601bce8 100644 --- a/.gitignore +++ b/.gitignore @@ -22,3 +22,4 @@ data/ **/*.json notes.md etc/ +.claude/tasks/ diff --git a/CLAUDE.md b/CLAUDE.md index f0dff9323..078cf0322 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,6 +4,8 @@ ## Notes +This is a collection of guidelines and references. + - Rust and Python are the primary project languages - [Flox](https://flox.dev/) manages project environment and packages - [Mask](https://github.com/jacobdeichert/mask) is used for command management @@ -19,6 +21,8 @@ - Use full word variables in code whenever possible - Follow Rust and Python recommended casing conventions - Strictly use Python version 3.12.10 +- Folder names under the `applications/` directory should end with `model` for machine learning services + and end with `manager` for all others - Scan and remove unused dependencies from `pyproject.toml` files - Move duplicate dependencies into root workspace `pyproject.toml` - Introduce new dependencies only after approval @@ -45,71 +49,11 @@ - `libraries/` folder contains shared code resources - `infrastructure/` folder contains Pulumi infrastructure as code - See `README.md` "Principles" section for developer philosophy - -## Ralph Workflow - -Ralph is an autonomous development loop for implementing GitHub issue specs. - -### Commands - -- `mask ralph setup` - Create required labels (run once before first use) -- `mask ralph spec [issue_number]` - Interactive spec refinement (creates new issue if no number provided) -- `mask ralph ready ` - Mark a spec as ready for implementation -- `mask ralph loop ` - Run autonomous loop on a ready spec -- `mask ralph backlog` - Review open issues for duplicates, overlaps, and implementation status -- `mask ralph pull-request [pull_request_number]` - Process pull request review feedback interactively - -### Labels - -**Status labels:** - -- `in-refinement` - Spec being built or discussed -- `ready` - Spec complete, ready for implementation -- `in-progress` - Work actively in progress -- `attention-needed` - Blocked or needs human intervention -- `backlog-review` - Backlog review tracking issue - -**Actor label:** - -- `ralph` - Ralph is actively working on this (remove to hand off to human) - -### Workflow - -1. Create or refine spec: `mask ralph spec` or `mask ralph spec ` -2. When spec is complete, mark as ready: `mask ralph ready ` -3. Run autonomous loop: `mask ralph loop ` -4. Loop assigns the issue and resulting pull request to the current GitHub user -5. Loop creates pull request with `Closes #` on completion -6. Pull request merge auto-closes issue - -### Context Rotation - -- Complete logically related requirements together (same files, same concepts) -- Exit after meaningful progress to allow fresh context on next iteration -- Judgment factors: relatedness, complexity, context size, dependencies - -### Completion Signal - -Output `COMPLETE` when all requirement checkboxes are checked to signal task completion. - -### Commit as Verification - -After implementing requirements, ALWAYS attempt `git commit`. The commit triggers pre-commit hooks which -run all tests/linting. This IS the verification step: - -- If commit fails → fix issues and retry -- If commit succeeds → requirement is verified, check it off in issue -- Do not skip this step or run tests separately - -### Ralph Learnings - -Document failure patterns here after Ralph loops to prevent recurrence. Periodically compact this section -by merging similar learnings and removing entries that have been incorporated into the workflow or specs above. - -#### 2026-01-26: #723 (spec: commit-as-verification not explicit) - -**Issue:** Loop implemented requirements but didn't attempt git commit to verify. - -**Root cause:** Spec said "commit is the verification gate" but didn't explicitly say to always attempt commit after implementing. - -**Fix:** Added explicit "Commit-as-Verification" section requiring commit attempt after every implementation. +- If something goes wrong during a task, stop immediately and re-plan rather than continuing +- Use subagents to keep main context window clean and offload research, exploration, and analysis work +- After user corrections, update `.claude/tasks/lessons.md` with timestamp to prevent repeating mistakes +- Prove changes work before marking tasks complete - run `mask` checks, compare behavior, demonstrate correctness +- For non-trivial changes, pause and ask "Is there a more elegant way?" before implementing +- Make every change as simple as possible and impact minimal code +- Find root causes and avoid temporary fixes - maintain high standards +- Changes should only touch what's necessary to avoid introducing bugs diff --git a/maskfile.md b/maskfile.md index 25a025d8a..3fc7f17cf 100644 --- a/maskfile.md +++ b/maskfile.md @@ -732,7 +732,7 @@ remove_if_present() { claude mcp remove "$name" >/dev/null 2>&1 || true } -echo "Removing existing MCP entries (ignore errors if not present)..." +echo "Removing existing MCP entries (ignore errors if not present)" remove_if_present awslabs-ecs-mcp-server remove_if_present awslabs-cloudwatch-logs-mcp-server remove_if_present awslabs-cloudwatch-mcp-server @@ -748,14 +748,14 @@ echo "Detecting uvx executable for CloudWatch package: $CW_PKG" CW_EXE="$(detect_uvx_exe "$CW_PKG")" echo "-> CloudWatch executable: $CW_EXE" -echo "Adding ECS MCP server..." +echo "Adding ECS MCP server" claude mcp add awslabs-ecs-mcp-server -s project \ -e FASTMCP_LOG_LEVEL="${FASTMCP_LOG_LEVEL:-info}" \ -e AWS_PROFILE="$AWS_PROFILE" \ -e AWS_REGION="$AWS_REGION" \ -- "$UVX_PATH" --from "$ECS_PKG" "$ECS_EXE" -echo "Adding CloudWatch MCP server..." +echo "Adding CloudWatch MCP server" claude mcp add awslabs-cloudwatch-mcp-server -s project \ -e FASTMCP_LOG_LEVEL="${FASTMCP_LOG_LEVEL:-info}" \ -e AWS_PROFILE="$AWS_PROFILE" \ @@ -766,800 +766,3 @@ echo echo "Done. Current MCP status:" claude mcp list ``` - -## ralph - -> Ralph autonomous development workflow - -### setup - -> Create required labels for Ralph workflow - -```bash -set -euo pipefail - -source "${MASKFILE_DIR}/tools/ralph_preflight.sh" -ralph_preflight - -echo "Setting up Ralph labels" - -labels='[ - {"name": "ralph", "color": "5319e7", "description": "Ralph is actively working on this"}, - {"name": "in-refinement", "color": "d93f0b", "description": "Spec being built or discussed"}, - {"name": "ready", "color": "0e8a16", "description": "Spec complete, ready for implementation"}, - {"name": "in-progress", "color": "fbca04", "description": "Work actively in progress"}, - {"name": "attention-needed", "color": "b60205", "description": "Blocked or needs human intervention"}, - {"name": "backlog-review", "color": "0052cc", "description": "Backlog review tracking"} -]' - -existing=$(gh label list --json name --jq '.[].name') - -echo "$labels" | jq -c '.[]' | while read -r label; do - name=$(echo "$label" | jq -r '.name') - color=$(echo "$label" | jq -r '.color') - desc=$(echo "$label" | jq -r '.description') - - if echo "$existing" | grep -qx "$name"; then - echo "Label '$name' already exists" - else - gh label create "$name" --color "$color" --description "$desc" - echo "Created label '$name'" - fi -done - -echo "Setup complete" -``` - -### spec [issue_number] - -> Build or refine a spec through interactive conversation - -```bash -set -euo pipefail - -source "${MASKFILE_DIR}/tools/ralph_preflight.sh" -ralph_preflight --claude --jq - -echo "Starting Ralph spec refinement" - -if [ -z "${issue_number:-}" ]; then - echo "Creating new spec issue" - - issue_json=$(gh issue create \ - --template "ISSUE_TEMPLATE.md" \ - --title "New Spec: [TITLE]" \ - --label "in-refinement" \ - --label "feature" \ - --json number) - - issue_number=$(echo "$issue_json" | jq -r '.number') - - echo "Created issue #${issue_number}" - echo "Opening issue in browser" - gh issue view "${issue_number}" --web & -fi - -echo "Refining issue #${issue_number}" - -issue_title=$(gh issue view "${issue_number}" --json title --jq '.title') -issue_body=$(gh issue view "${issue_number}" --json body --jq '.body') - -system_prompt="You are helping refine a technical specification in GitHub issue #${issue_number}. - -CURRENT SPEC: -Title: ${issue_title} - -${issue_body} - -YOUR ROLE: -1. Probe the user with questions to refine this spec -2. Ask about: problem clarity, requirements completeness, performance, security, testing, edge cases, dependencies -3. When decisions are made, update the issue incrementally using: gh issue edit ${issue_number} --body \"...\" -4. Keep Open Questions section updated as questions are resolved -5. Move resolved questions to Decisions section with rationale - -IMPORTANT: -- You do NOT add the 'ready' label - the human decides when the spec is complete -- Update the issue body incrementally as decisions are made -- Use the spec template format (Problem, Requirements, Open Questions, Decisions, Specification) -- Be thorough but conversational - -Start by reviewing the current spec and asking clarifying questions." - -claude --system-prompt "$system_prompt" - -echo "" -echo "Spec refinement session ended" -echo "When ready, mark the spec as ready:" -echo "mask ralph ready ${issue_number}" -``` - -### ready (issue_number) - -> Mark a spec as ready for implementation - -```bash -set -euo pipefail - -source "${MASKFILE_DIR}/tools/ralph_preflight.sh" -ralph_preflight - -echo "Marking issue #${issue_number} as ready" - -if ! gh issue view "${issue_number}" &> /dev/null; then - echo "Error: Issue #${issue_number} not found" - exit 1 -fi - -gh issue edit "${issue_number}" --add-label "ready" --remove-label "in-refinement" - -echo "Issue #${issue_number} marked as ready" -echo "Run the loop with: mask ralph loop ${issue_number}" -``` - -### loop (issue_number) - -> Run autonomous loop on a ready spec - -```bash -set -euo pipefail - -source "${MASKFILE_DIR}/tools/ralph_preflight.sh" -ralph_preflight --claude --jq - -max_iterations="${RALPH_MAX_ITERATIONS:-10}" - -echo "Starting Ralph loop for issue #${issue_number}" - -echo "Running pre-flight checks" - -if [ -n "$(git status --porcelain)" ]; then - echo "Error: Working directory has uncommitted changes" - echo "Commit or stash changes before running ralph loop" - exit 1 -fi -echo "Working directory is clean" - -default_branch=$(git remote show origin | grep 'HEAD branch' | cut -d' ' -f5) -current_branch=$(git rev-parse --abbrev-ref HEAD) -if [ "$current_branch" != "$default_branch" ]; then - echo "Error: Not on default branch ${default_branch} (currently on: ${current_branch})" - echo "Run: git checkout ${default_branch}" - exit 1 -fi -echo "On default branch (${default_branch})" - -echo "Pulling latest ${default_branch}" -if ! git pull --ff-only origin "$default_branch"; then - echo "Error: Could not pull latest ${default_branch}" - echo "Resolve conflicts or check network/auth" - exit 1 -fi -echo "${default_branch} is up to date" - -if ! labels=$(gh issue view "${issue_number}" --json labels --jq '.labels[].name'); then - echo "Error: Could not fetch issue #${issue_number}" - echo "Check network connectivity and issue existence" - exit 1 -fi -if ! echo "$labels" | grep -q "^ready$"; then - echo "Error: Issue #${issue_number} does not have 'ready' label" - echo "Current labels: ${labels:-none}" - exit 1 -fi -echo "Issue has 'ready' label" - -issue_title=$(gh issue view "${issue_number}" --json title --jq '.title') -short_desc=$(echo "$issue_title" | tr '[:upper:]' '[:lower:]' | tr ' ' '-' | tr -cd 'a-z0-9-' | cut -c1-30) -branch_name="ralph/${issue_number}-${short_desc}" - -if git show-ref --verify --quiet "refs/heads/${branch_name}" 2>/dev/null; then - echo "Error: Local branch '${branch_name}' already exists" - echo "Delete with: git branch -d ${branch_name}" - exit 1 -fi - -if git ls-remote --heads origin "${branch_name}" 2>/dev/null | grep -q .; then - echo "Error: Remote branch '${branch_name}' already exists" - echo "Delete with: git push origin --delete ${branch_name}" - exit 1 -fi -echo "Branch '${branch_name}' does not exist" - -echo "Pre-flight checks passed" - -echo "Creating branch: ${branch_name}" -git checkout -b "${branch_name}" - -echo "Updating labels: removing 'ready', adding 'in-progress' and 'ralph'" -gh issue edit "${issue_number}" --remove-label "ready" --add-label "in-progress" --add-label "ralph" - -current_user=$(gh api user --jq '.login' 2>/dev/null || echo "") -if [ -n "$current_user" ]; then - echo "Assigning issue to ${current_user}" - gh issue edit "${issue_number}" --add-assignee "${current_user}" 2>/dev/null || echo "Warning: Could not assign issue" -fi - -cleanup_on_error() { - local exit_code=$? - if [ $exit_code -ne 0 ]; then - echo "" - echo "Error: Script failed unexpectedly (exit code: $exit_code)" - - local git_status=$(git status --short 2>/dev/null | head -20 || echo "unavailable") - local recent_commits=$(git log --oneline -5 2>/dev/null || echo "unavailable") - - gh issue edit "${issue_number}" --remove-label "in-progress" --remove-label "ralph" --add-label "attention-needed" 2>/dev/null || true - gh issue comment "${issue_number}" --body "## Ralph Loop Error - -The loop exited unexpectedly with code $exit_code. - -**Branch:** \`${branch_name}\` - -### Git Status -\`\`\` -${git_status} -\`\`\` - -### Recent Commits -\`\`\` -${recent_commits} -\`\`\` - -Check the terminal output for full details." 2>/dev/null || true - fi -} -trap cleanup_on_error EXIT - -system_prompt="You are executing an autonomous development loop for GitHub issue #${issue_number}. - -WORKFLOW: -1. Read the spec: gh issue view ${issue_number} -2. PLAN: Identify unchecked requirements, group logically related ones -3. EXECUTE: Implement the grouped requirements -4. TEST: Run pre-commit hooks (they run mask development python/rust all) -5. UPDATE: Check off completed requirements in the issue -6. DECIDE: If more unchecked requirements remain AND you've completed a logical group, exit to rotate context - -COMPLETION: -- When ALL requirement checkboxes are checked, output COMPLETE -- This signals the loop is done and triggers pull request creation - -CONTEXT ROTATION: -- Complete logically related requirements together (same files, same concepts) -- Exit after meaningful progress to allow fresh context on next iteration -- Don't try to do everything in one pass - -CHECKBOX UPDATE: -- Use gh issue edit to update the issue body with checked boxes -- Update checkboxes BEFORE exiting to preserve progress - -GIT: -- ALWAYS attempt git commit after implementing a requirement -- The commit triggers pre-commit hooks which verify the change -- If commit fails, fix the issues and retry -- If commit succeeds, the requirement is verified - check it off - -IMPORTANT: -- Start with planning before any code changes -- Be thorough but exit after completing related requirements -- The commit gate is the verification (pre-commit = mask development all)" - -stream_text='select(.type == "assistant").message.content[]? | select(.type == "text").text // empty | gsub("\n"; "\r\n") | . + "\r\n\n"' -final_result='select(.type == "result").result // empty' - -tmpfile=$(mktemp) - -cleanup_and_fail() { - rm -f "$tmpfile" - echo "" - echo "Unexpected error - cleaning up" - - local git_status=$(git status --short 2>/dev/null | head -20 || echo "unavailable") - local recent_commits=$(git log --oneline -5 2>/dev/null || echo "unavailable") - - gh issue edit "${issue_number}" --remove-label "in-progress" --remove-label "ralph" --add-label "attention-needed" 2>/dev/null || true - gh issue comment "${issue_number}" --body "## Ralph Loop Error - -The loop exited unexpectedly. Branch: \`${branch_name}\` - -### Git Status -\`\`\` -${git_status} -\`\`\` - -### Recent Commits -\`\`\` -${recent_commits} -\`\`\` - -Retry with: \`mask ralph loop ${issue_number}\`" 2>/dev/null || true - exit 1 -} - -trap "rm -f $tmpfile" EXIT -trap cleanup_and_fail ERR - -iteration=1 -while [ $iteration -le $max_iterations ]; do - echo "" - echo "Iteration ${iteration}/${max_iterations}" - - spec=$(gh issue view "${issue_number}" --json body --jq '.body') - - claude \ - --print \ - --output-format stream-json \ - --system-prompt "${system_prompt}" \ - --dangerously-skip-permissions \ - "Current spec state:\n\n${spec}\n\nBegin iteration ${iteration}. Start with planning." \ - | grep --line-buffered '^{' \ - | tee "$tmpfile" \ - | jq --unbuffered -rj "$stream_text" - - result=$(jq -r "$final_result" "$tmpfile") - - if [[ "$result" == *"COMPLETE"* ]]; then - echo "" - echo "Ralph complete after ${iteration} iterations" - - echo "Updating labels: removing 'in-progress' and 'ralph'" - gh issue edit "${issue_number}" --remove-label "in-progress" --remove-label "ralph" - - echo "Pushing branch" - git push -u origin "${branch_name}" - - echo "Creating pull request" - pr_body=$(cat <COMPLETE -EOF -) - pr_url=$(gh pr create \ - --title "${issue_title}" \ - --body "$pr_body" \ - --assignee "${current_user:-}") - - echo "Pull request created: ${pr_url}" - echo "Issue will auto-close on merge" - trap - EXIT - exit 0 - fi - - iteration=$((iteration + 1)) -done - -echo "" -echo "Max iterations reached (${max_iterations})" - -echo "Pushing branch for review" -branch_pushed="false" -if git push -u origin "${branch_name}" 2>/dev/null; then - echo "Branch pushed successfully" - branch_pushed="true" -else - echo "Warning: Could not push branch (progress is local only)" -fi - -gh issue edit "${issue_number}" --remove-label "in-progress" --remove-label "ralph" --add-label "attention-needed" - -modified_files=$(git diff --name-only "origin/${default_branch}" 2>/dev/null || echo "none") - -if [ "$branch_pushed" = "true" ]; then - branch_info="**Branch:** [\`${branch_name}\`](../../tree/${branch_name}) (pushed to remote)" -else - branch_info="**Branch:** \`${branch_name}\` (local only - push failed)" -fi - -failure_comment=$(cat < Review open issues for duplicates, overlaps, and implementation status - -```bash -set -euo pipefail - -source "${MASKFILE_DIR}/tools/ralph_preflight.sh" -ralph_preflight --claude --jq - -echo "Starting Ralph backlog review" - -tracking_issue_title="Backlog Review" - -echo "Checking for existing tracking issue" -existing_issue=$(gh issue list --search "\"${tracking_issue_title}\" in:title" --state open --json number --jq '.[0].number // empty') - -if [ -z "$existing_issue" ]; then - echo "Creating tracking issue: ${tracking_issue_title}" - - tracking_body=$(cat <<'TRACKING_TEMPLATE' -# Backlog Review - -This issue tracks periodic backlog review reports generated by `mask ralph backlog`. - -Each comment contains an analysis of open issues looking for: -- Potential duplicates or overlapping issues -- Issues that may already be implemented -- Consolidation opportunities - -Staleness is handled separately by the stale workflow action. - -Run `mask ralph backlog` to generate a new report. -TRACKING_TEMPLATE -) - - existing_issue=$(gh issue create \ - --title "${tracking_issue_title}" \ - --body "$tracking_body" \ - --label "backlog-review" \ - --json number -q '.number') - - echo "Created tracking issue #${existing_issue}" -else - echo "Found existing tracking issue #${existing_issue}" -fi - -echo "Fetching open issues" -issues_json=$(gh issue list --state open --limit 500 --json number,title,body,labels,updatedAt,createdAt) -issue_count=$(echo "$issues_json" | jq 'length') -echo "Found ${issue_count} open issues" - -echo "Analyzing backlog with Claude" - -today=$(date +%Y-%m-%d) - -system_prompt="You are analyzing a GitHub issue backlog for consolidation opportunities. - -TODAY'S DATE: ${today} -TRACKING ISSUE: #${existing_issue} (do NOT include this in analysis) - -ANALYSIS TASKS: -1. DUPLICATES: Find issues with similar titles/descriptions that might be duplicates -2. OVERLAPS: Find issues that cover related functionality and could be consolidated -3. IMPLEMENTED: Search the codebase for keywords that suggest an issue might already be done - -OUTPUT FORMAT: -Generate a markdown report following this exact structure: - -## Backlog Review - ${today} - -### Potential Duplicates - - -### Potentially Implemented - - -### Consolidation Suggestions - - -### Summary - - -IMPORTANT: -- Be conservative with duplicate detection - only flag clear matches -- For 'potentially implemented', actually search the codebase using Grep/Glob -- Exclude the tracking issue #${existing_issue} from all analysis -- Use high/medium/low confidence levels -- Keep the report concise and actionable" - -report=$(claude \ - --print \ - --dangerously-skip-permissions \ - --system-prompt "${system_prompt}" \ - "Analyze this issue backlog and generate a report: - -${issues_json} - -Search the codebase as needed to check if issues might already be implemented.") - -echo "Posting report to tracking issue #${existing_issue}" -gh issue comment "${existing_issue}" --body "${report}" - -echo "" -echo "Backlog review complete" -echo "Report posted to: https://github.com/$(gh repo view --json nameWithOwner -q .nameWithOwner)/issues/${existing_issue}" -``` - -### pull-request [pull_request_number] - -> Process pull request review feedback interactively - -```bash -set -euo pipefail - -source "${MASKFILE_DIR}/tools/ralph_preflight.sh" -ralph_preflight --claude --jq - -echo "Starting Ralph pull request review" - -if [ -n "${pull_request_number:-}" ]; then - pr_num="$pull_request_number" - echo "Using pull request #${pr_num}" -else - echo "Auto-detecting pull request from current branch" - pr_num=$(gh pr view --json number --jq '.number' 2>/dev/null || echo "") - if [ -z "$pr_num" ]; then - echo "Error: No pull request found for current branch" - echo "Use: mask ralph pull-request " - exit 1 - fi - echo "Found pull request #${pr_num}" -fi - -repo_info=$(gh repo view --json nameWithOwner --jq '.nameWithOwner') -owner=$(echo "$repo_info" | cut -d'/' -f1) -repo=$(echo "$repo_info" | cut -d'/' -f2) - -echo "Fetching review comments" - -review_threads=$(gh api graphql -f query=' -query($owner: String!, $repo: String!, $pr: Int!) { - repository(owner: $owner, name: $repo) { - pullRequest(number: $pr) { - reviewThreads(first: 100) { - nodes { - id - isResolved - path - line - comments(first: 10) { - nodes { - id - body - author { login } - createdAt - } - } - } - } - } - } -}' -f owner="$owner" -f repo="$repo" -F pr="$pr_num") - -unresolved_threads=$(echo "$review_threads" | jq '[.data.repository.pullRequest.reviewThreads.nodes[] | select(.isResolved == false)]') -thread_count=$(echo "$unresolved_threads" | jq 'length') - -if [ "$thread_count" -eq 0 ]; then - echo "No unresolved review conversations found" - exit 0 -fi - -echo "Found ${thread_count} unresolved conversation(s)" -echo "" - -plan_file=$(mktemp) -trap "rm -f $plan_file" EXIT -echo "[]" > "$plan_file" - -index=0 -while [ $index -lt "$thread_count" ]; do - thread=$(echo "$unresolved_threads" | jq ".[$index]") - thread_id=$(echo "$thread" | jq -r '.id') - path=$(echo "$thread" | jq -r '.path // "unknown"') - line=$(echo "$thread" | jq -r '.line // "?"') - first_comment=$(echo "$thread" | jq '.comments.nodes[0]') - author=$(echo "$first_comment" | jq -r '.author.login // "unknown"') - body=$(echo "$first_comment" | jq -r '.body') - - display_num=$((index + 1)) - echo "" - echo "[${display_num}/${thread_count}] @${author} on ${path}:${line}" - echo "" - - # Truncate for display: show text before code blocks, summarize code blocks - display_body=$(echo "$body" | awk ' - BEGIN { in_code=0; code_lines=0; shown_code=0 } - /^```/ { - if (in_code) { - if (code_lines > 3) printf " ... (%d lines)\n", code_lines - print "```" - in_code=0; code_lines=0; shown_code=0 - } else { - print - in_code=1 - } - next - } - in_code { - code_lines++ - if (shown_code < 3) { print; shown_code++ } - next - } - { print } - ' | head -25) - - echo "$display_body" - - total_lines=$(echo "$body" | wc -l | tr -d ' ') - if [ "$total_lines" -gt 25 ]; then - echo "... (${total_lines} lines total)" - fi - echo "" - - while true; do - printf "[A]ccept / [S]kip / [M]odify / [Q]uit: " - read -r choice < /dev/tty - - case "$choice" in - [Aa]) - jq --arg tid "$thread_id" --arg path "$path" --arg line "$line" --arg body "$body" --arg author "$author" \ - '. += [{"thread_id": $tid, "path": $path, "line": $line, "suggestion": $body, "author": $author, "action": "accept", "modification": null}]' \ - "$plan_file" > "${plan_file}.tmp" && mv "${plan_file}.tmp" "$plan_file" - echo "Added to plan" - break - ;; - [Ss]) - echo "Skipped" - break - ;; - [Mm]) - echo "Enter your modification (end with empty line):" - modification="" - while IFS= read -r mod_line < /dev/tty; do - [ -z "$mod_line" ] && break - modification="${modification}${mod_line}\n" - done - jq --arg tid "$thread_id" --arg path "$path" --arg line "$line" --arg body "$body" --arg author "$author" --arg mod "$modification" \ - '. += [{"thread_id": $tid, "path": $path, "line": $line, "suggestion": $body, "author": $author, "action": "modify", "modification": $mod}]' \ - "$plan_file" > "${plan_file}.tmp" && mv "${plan_file}.tmp" "$plan_file" - echo "Added to plan with modification" - break - ;; - [Qq]) - echo "Quitting" - exit 0 - ;; - *) - echo "Invalid choice. Use A/S/M/Q" - ;; - esac - done - - echo "" - index=$((index + 1)) -done - -plan_count=$(jq 'length' "$plan_file") - -if [ "$plan_count" -eq 0 ]; then - echo "No suggestions accepted. Nothing to do." - exit 0 -fi - -echo "" -echo "Plan summary: ${plan_count} suggestion(s) accepted" -echo "" -jq -r '.[] | "- \(.path):\(.line) (\(.action))"' "$plan_file" -echo "" - -printf "Proceed with execution? [Y/n]: " -read -r confirm < /dev/tty - -if [ "$confirm" = "n" ] || [ "$confirm" = "N" ]; then - echo "Aborted" - exit 0 -fi - -echo "" -echo "Starting execution phase" - -plan_json=$(cat "$plan_file") - -system_prompt="You are implementing approved pull request review suggestions. - -TASK: -For each suggestion in the plan, implement the requested change. - -PLAN: -${plan_json} - -WORKFLOW: -1. For each item in the plan: - - Read the file at the specified path - - Implement the suggestion (or the modification if provided) - - The 'suggestion' field contains the reviewer's comment - - The 'modification' field (if present) contains the user's adjusted approach -2. After implementing, commit with a message like: 'Address review: ' -3. Output a JSON array of results for posting replies, format: - [{\"thread_id\": \"...\", \"reply\": \"Fixed in . \"}] - -IMPORTANT: -- Make minimal, focused changes -- Don't refactor beyond what's requested -- If a suggestion doesn't make sense, skip it and note why in the reply -- Output the JSON results array at the end, wrapped in ... tags" - -result=$(claude \ - --print \ - --dangerously-skip-permissions \ - --system-prompt "${system_prompt}" \ - "Implement the suggestions in the plan. Output results JSON when done.") - -results_json=$(echo "$result" | awk '//,/<\/results>/' | sed '1s/.*//; $s/<\/results>.*//' | tr -d '\n') - -if [ -z "$results_json" ]; then - echo "Warning: Could not parse results from Claude output" - echo "Changes may have been made but replies were not posted" - exit 1 -fi - -echo "Pushing changes" -if ! git push; then - echo "Error: Failed to push changes" - echo "Replies will not be posted until push succeeds" - exit 1 -fi - -echo "Posting replies and resolving conversations" - -echo "$results_json" | jq -c '.[]' | while read -r item; do - thread_id=$(echo "$item" | jq -r '.thread_id') - reply=$(echo "$item" | jq -r '.reply') - - echo "Replying to thread ${thread_id}" - - gh api graphql -f query=' - mutation($threadId: ID!, $body: String!) { - addPullRequestReviewThreadReply(input: {pullRequestReviewThreadId: $threadId, body: $body}) { - comment { id } - } - }' -f threadId="$thread_id" -f body="$reply" > /dev/null 2>&1 || echo "Warning: Failed to post reply" - - gh api graphql -f query=' - mutation($threadId: ID!) { - resolveReviewThread(input: {threadId: $threadId}) { - thread { isResolved } - } - }' -f threadId="$thread_id" > /dev/null 2>&1 || echo "Warning: Failed to resolve thread" -done - -echo "" -echo "Pull request review complete" -echo "View pull request: https://github.com/${repo_info}/pull/${pr_num}" -``` diff --git a/tools/ralph_preflight.sh b/tools/ralph_preflight.sh deleted file mode 100755 index f2662af2f..000000000 --- a/tools/ralph_preflight.sh +++ /dev/null @@ -1,36 +0,0 @@ -#!/usr/bin/env bash -# Ralph preflight checks - shared dependency validation -# Source this script and call ralph_preflight with required tools - -ralph_preflight() { - local require_claude=false - local require_jq=false - - for arg in "$@"; do - case "$arg" in - --claude) require_claude=true ;; - --jq) require_jq=true ;; - esac - done - - if ! command -v gh &> /dev/null; then - echo "GitHub CLI (gh) is required" - exit 1 - fi - - if [ "$require_claude" = true ] && ! command -v claude &> /dev/null; then - echo "Claude CLI is required" - exit 1 - fi - - if [ "$require_jq" = true ] && ! command -v jq &> /dev/null; then - echo "jq is required" - exit 1 - fi - - if ! gh auth status &> /dev/null; then - echo "GitHub CLI not authenticated" - echo "Run: gh auth login" - exit 1 - fi -}