docs: agent operations roadmap + telemetry plan - #26
Merged
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughAdds agent-ops roadmap and Codex SDLC orchestration docs and examples; introduces automation tooling (issue selection, branch creation, PR creation/verification, branch validation, CodeRabbit polling and feedback collection), a GitHub Actions orchestrator, and supporting Bash/Python scripts to run SDLC workflows. Changes
Sequence Diagram(s)sequenceDiagram
participant User as "User"
participant GHAction as "GitHub Actions\nOrchestrator"
participant Runner as "Runner\n(scripts)"
participant Agents as "Codex Agents"
participant GitHub as "GitHub API / PRs / Checks"
User->>GHAction: trigger workflow (mode, issue_label)
GHAction->>Runner: run `select_issue.sh`
Runner->>GitHub: list/edit issues (gh)
GitHub-->>Runner: selected issue JSON
Runner->>GHAction: selected_issue.json
GHAction->>Runner: run `create_issue_branch.sh`
Runner->>GitHub: fetch/create branch
GitHub-->>Runner: branch created
Runner->>GHAction: branch.json
GHAction->>Agents: run SDLC workflow (sdlc-auto/sdlc-smoke)
Agents-->>Runner: produce changes on branch
Runner->>GHAction: files/commit results
GHAction->>Runner: run `open_or_create_pr.py`
Runner->>GitHub: create/list PR
GitHub-->>Runner: PR number
GHAction->>Runner: run `wait_coderabbit.sh`
Runner->>GitHub: poll status checks for CodeRabbit
GitHub-->>Runner: check results (pass/fail/timeout)
Runner->>GHAction: coderabbit_status.json
alt CodeRabbit feedback present
GHAction->>Runner: run `collect_coderabbit_feedback.sh`
Runner->>GitHub: fetch comments & reviews
Runner-->>GHAction: coderabbit-feedback.md
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches
🧪 Generate unit tests (beta)
📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment Tip Rust Clippy can be used to improve the quality of Rust code reviews.Clippy is the official Rust linter. It provides lints to catch common mistakes and improve your Rust code. To configure Clippy, add a See Clippy Documentation for more details. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (2)
docs/AGENT_OPS_ROADMAP.md (2)
50-50: Consider markdown-safe formatting for time duration.The
<30 minnotation may render inconsistently in some markdown parsers due to the less-than symbol being interpreted as an HTML tag start.📝 Suggested formatting alternatives
- - <30 min time-to-first-value path + - Under 30 min time-to-first-value pathor
- - <30 min time-to-first-value path + - < 30 min time-to-first-value path🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/AGENT_OPS_ROADMAP.md` at line 50, The markdown line containing "<30 min time-to-first-value path" can be misinterpreted as an HTML tag; update that token to a markdown-safe form such as "Under 30 min time-to-first-value path", "`<30 min` time-to-first-value path" (wrap in backticks), or replace "<" with the HTML entity "<" so the text renders consistently across parsers; modify the literal "<30 min time-to-first-value path" accordingly.
87-91: Consider adding quantifiable success metrics.The success criteria are primarily qualitative. While appropriate for a strategic roadmap, adding specific measurable targets would make it easier to track progress and declare success objectively.
Examples of quantifiable metrics:
- Activation: "≥ 60% of installs reach first successful workflow within 24h"
- Reliability: "Run failure rate < 5% and MTTR < 2 hours"
- Adoption: "≥ 3 production-ready templates with documented use cases"
- Retention: "D7 retention ≥ 40%"
These can supplement the existing qualitative criteria.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/AGENT_OPS_ROADMAP.md` around lines 87 - 91, Update the "Success criteria" section to include specific, measurable targets alongside the qualitative bullets: under the "Success criteria" heading add numeric goals for Activation (e.g., "≥ 60% of installs reach first successful workflow within 24h"), Reliability (e.g., "Run failure rate < 5% and MTTR < 2 hours"), Adoption (e.g., "≥ 3 production-ready templates with documented use cases"), and Retention (e.g., "D7 retention ≥ 40%"); ensure each existing qualitative item is supplemented with a corresponding quantifiable metric so progress can be objectively tracked.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@docs/AGENT_OPS_ROADMAP.md`:
- Line 88: The success criterion line "Activation >= target baseline and
trending up" lacks a concrete baseline; update that line to either include a
specific numeric target (for example "Activation >= 60% within 24h") or
explicitly reference where/when the baseline will be defined (for example
"Activation >= target baseline [to be defined after initial 2-week data
collection] and trending up"); make the chosen approach clear in the document so
the activation metric in AGENT_OPS_ROADMAP.md is measurable and tied to a stated
baseline or process for establishing it.
- Around line 67-72: The event instrumentation section lacks explicit PII
boundaries; update the Event instrumentation block to list for each event (e.g.,
tutti_install_started, tutti_install_completed, first_workspace_created,
first_up_success, first_send, first_workflow_run) the allowed data fields,
explicit exclusions (e.g., no workspace names, user emails, full message
content), and the required hashing/anonymization scheme for identifiers (salted
hash or one-way pseudonymization) plus retention rules and examples of safe
fields (counts, booleans, timestamps). Ensure the note references the
outcome/failure categories (routing/tool/model/policy) and DAW/cohort metrics so
implementers know how to record those without PII, and add a short line pointing
to the "Redaction/privacy defaults" deliverable for enforcement and verification
steps.
---
Nitpick comments:
In `@docs/AGENT_OPS_ROADMAP.md`:
- Line 50: The markdown line containing "<30 min time-to-first-value path" can
be misinterpreted as an HTML tag; update that token to a markdown-safe form such
as "Under 30 min time-to-first-value path", "`<30 min` time-to-first-value path"
(wrap in backticks), or replace "<" with the HTML entity "<" so the text
renders consistently across parsers; modify the literal "<30 min
time-to-first-value path" accordingly.
- Around line 87-91: Update the "Success criteria" section to include specific,
measurable targets alongside the qualitative bullets: under the "Success
criteria" heading add numeric goals for Activation (e.g., "≥ 60% of installs
reach first successful workflow within 24h"), Reliability (e.g., "Run failure
rate < 5% and MTTR < 2 hours"), Adoption (e.g., "≥ 3 production-ready templates
with documented use cases"), and Retention (e.g., "D7 retention ≥ 40%"); ensure
each existing qualitative item is supplemented with a corresponding quantifiable
metric so progress can be objectively tracked.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 314fc895-3489-4f26-a55a-7b4df5ed69db
📒 Files selected for processing (1)
docs/AGENT_OPS_ROADMAP.md
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@docs/examples/tutti-codex-sdlc.toml`:
- Around line 87-100: The current wait_coderabbit step's Python run block uses
subprocess.check_call to invoke wait_coderabbit.sh which raises
CalledProcessError on 'fail' or 'timeout' and prevents
collect_coderabbit_feedback.sh from running; change the invocation in the Python
here-doc so wait_coderabbit.sh is executed without raising on non-zero exit
(e.g., use subprocess.run([...], check=False) or wrap
subprocess.check_call([...]) in try/except and continue), ensuring
collect_coderabbit_feedback.sh always runs afterward; update the references in
the same block (the calls to wait_coderabbit.sh and
collect_coderabbit_feedback.sh) so the feedback markdown is collected regardless
of wait_coderabbit.sh exit status.
- Around line 49-74: The implement prompt will commit to the wrong branch
because ensure_conductor initializes the conductor worktree on tutti/conductor
while create_branch runs in the project root and does not switch the conductor's
worktree; fix by either scoping the create_branch step to the conductor agent
(add agent = "conductor" to the [[workflow.step]] with id = "create_branch" so
the branch is created in the conductor worktree), or modify the implement prompt
(id = "implement") to explicitly run a git checkout to the branch from
.tutti/state/auto/branch.json before making changes, committing, and pushing.
In `@scripts/automation/collect_coderabbit_feedback.sh`:
- Around line 13-15: The script currently assigns DATA from gh pr view (variable
DATA) and pipes it into the Python heredoc (python3 <<'PY' "$OUT_FILE" "$DATA"),
but gh pr view --json comments,reviews omits inline diff comments; call gh api
repos/{owner}/{repo}/pulls/{pull_number}/comments (store in a new variable like
INLINE_COMMENTS) and merge or package both RESULTS (DATA and INLINE_COMMENTS)
into a single JSON/markdown payload passed to the Python block so the heredoc
receives inline review comments as well as conversation comments; update
references to DATA in the heredoc invocation (or pass a second arg) and adjust
the Python input handling so it consumes the combined data.
In `@scripts/automation/select_issue.sh`:
- Around line 13-33: The script currently picks the first matching open issue
but never reserves it, allowing duplicates; modify the Python selection to
filter out issues already labeled e.g. "automation-claimed" (change the items
parsing to exclude any issue where "automation-claimed" is in issue["labels"])
and after selecting issue (after building payload) call the GH CLI to reserve it
(run gh issue edit <issue_number> --add-label "automation-claimed") so
concurrent or re-run workflows skip claimed issues; update references in this
script (the JSON variable, the python block that defines items/issue/payload) to
implement both the filter and the post-selection label-add.
In `@scripts/automation/wait_coderabbit.sh`:
- Around line 44-49: The current logic computes states and then treats anything
not in the `bad` list as PASS; change it to whitelist accepted conclusions
instead: keep the existing `states` and the pending check (`any(s != "COMPLETED"
for s,_ in states)`), but replace the `bad` computation with a check for any
completed conclusion not in an explicit allowlist (e.g., define `allowed =
("SUCCESS",)` and compute `bad = [x for x in states if x[1] not in allowed]`),
then print "FAIL" if `bad` else "PASS" so only whitelisted conclusions produce
PASS; update references to `states` and `bad` accordingly.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 9b4f876b-42ad-4c99-b239-9c41378b7ead
📒 Files selected for processing (6)
docs/CODEX_SDLC_ORCHESTRATION.mddocs/examples/tutti-codex-sdlc.tomlscripts/automation/collect_coderabbit_feedback.shscripts/automation/create_issue_branch.shscripts/automation/select_issue.shscripts/automation/wait_coderabbit.sh
✅ Files skipped from review due to trivial changes (1)
- docs/CODEX_SDLC_ORCHESTRATION.md
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
.github/workflows/sdlc-orchestrator.yml (1)
56-59: Prefer environment variables for input interpolation.Passing
${{ inputs.issue_label }}directly in therun:block can be fragile with special characters. Using an environment variable is a safer pattern.♻️ Suggested improvement
env: GH_TOKEN: ${{ github.token }} + ISSUE_LABEL: ${{ inputs.issue_label }} run: | - scripts/automation/select_issue.sh .tutti/state/auto/selected_issue.json "${{ inputs.issue_label }}" + mkdir -p .tutti/state/auto + scripts/automation/select_issue.sh .tutti/state/auto/selected_issue.json "$ISSUE_LABEL" cargo run --quiet -- run sdlc-smoke --strict🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/sdlc-orchestrator.yml around lines 56 - 59, The run block is interpolating ${{ inputs.issue_label }} directly which can break with special characters; set an environment variable (e.g., ISSUE_LABEL) from inputs and reference that variable inside the run step instead of inline interpolation—update the step to export ISSUE_LABEL from inputs.issue_label and call scripts/automation/select_issue.sh .tutti/state/auto/selected_issue.json "$ISSUE_LABEL" (and keep GH_TOKEN as-is); this change affects the run invocation that calls select_issue.sh and uses inputs.issue_label.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/sdlc-orchestrator.yml:
- Around line 44-48: The workflow step name "Install gh auth" is misleading
because the step only runs the command `gh auth status`; rename the step (change
the value of the name key) to something accurate like "Verify gh auth" or "Check
gh authentication" and keep the existing env GH_TOKEN and run command `gh auth
status` intact so the intent matches the step name.
- Around line 66-72: The Run full auto workflow step currently calls
scripts/automation/select_issue.sh with a path
.tutti/state/auto/selected_issue.json and an inputs.issue_label directly; update
the step to first create the target directory (mkdir -p .tutti/state/auto)
before invoking the script, and change the label argument to use an environment
variable (e.g., ISSUE_LABEL) by adding env: ISSUE_LABEL: ${{ inputs.issue_label
}} and passing "$ISSUE_LABEL" to scripts/automation/select_issue.sh so the
script receives the label via env variable and the directory always exists.
- Around line 57-59: The workflow fails when select_issue.sh writes to
.tutti/state/auto/selected_issue.json because the parent directory may not
exist; update the writer to ensure the parent directory is created before
opening the file (in the shell-based select_issue.sh create the parent directory
for the target path, or in the Python code that calls open(out, "w") call
os.makedirs on dirname(out) with exist_ok=True) so the open(out, "w") no longer
raises FileNotFoundError.
---
Nitpick comments:
In @.github/workflows/sdlc-orchestrator.yml:
- Around line 56-59: The run block is interpolating ${{ inputs.issue_label }}
directly which can break with special characters; set an environment variable
(e.g., ISSUE_LABEL) from inputs and reference that variable inside the run step
instead of inline interpolation—update the step to export ISSUE_LABEL from
inputs.issue_label and call scripts/automation/select_issue.sh
.tutti/state/auto/selected_issue.json "$ISSUE_LABEL" (and keep GH_TOKEN as-is);
this change affects the run invocation that calls select_issue.sh and uses
inputs.issue_label.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 88cb46cd-1070-441f-acea-8084c400592e
📒 Files selected for processing (1)
.github/workflows/sdlc-orchestrator.yml
There was a problem hiding this comment.
♻️ Duplicate comments (1)
docs/examples/tutti-codex-sdlc.toml (1)
181-185:⚠️ Potential issue | 🟠 MajorFeedback collection still bypassed on CodeRabbit failure.
The Python script at line 184 uses
subprocess.check_callforwait_coderabbit.sh, which raisesCalledProcessErroron non-zero exit (fail/timeout). This preventscollect_coderabbit_feedback.shfrom running, leaving.tutti/state/auto/coderabbit-feedback.mdmissing when theapply_coderabbit_feedbackstep tries to inject it.Use
subprocess.run(..., check=False)or wrap in try/except to ensure feedback is always collected:🛠️ Proposed fix
-run = "python3 - <<'PY'\nimport subprocess,json\nbranch=json.load(open('.tutti/state/auto/branch.json'))['branch']\nout=subprocess.check_output(['gh','pr','list','--state','open','--head',branch,'--json','number'], text=True)\npr=json.loads(out)[0]['number']\nsubprocess.check_call(['scripts/automation/wait_coderabbit.sh',str(pr),'45','.tutti/state/auto/coderabbit_status.json'])\nsubprocess.check_call(['scripts/automation/collect_coderabbit_feedback.sh',str(pr),'.tutti/state/auto/coderabbit-feedback.md'])\nPY" +run = "python3 - <<'PY'\nimport subprocess,json,sys\nbranch=json.load(open('.tutti/state/auto/branch.json'))['branch']\nout=subprocess.check_output(['gh','pr','list','--state','open','--head',branch,'--json','number'], text=True)\npr=json.loads(out)[0]['number']\nwait_result=subprocess.run(['scripts/automation/wait_coderabbit.sh',str(pr),'45','.tutti/state/auto/coderabbit_status.json'])\nsubprocess.check_call(['scripts/automation/collect_coderabbit_feedback.sh',str(pr),'.tutti/state/auto/coderabbit-feedback.md'])\nsys.exit(wait_result.returncode)\nPY",
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/examples/tutti-codex-sdlc.toml` around lines 181 - 185, The inline Python step "wait_coderabbit" currently uses subprocess.check_call for 'scripts/automation/wait_coderabbit.sh' which raises CalledProcessError and aborts before 'scripts/automation/collect_coderabbit_feedback.sh' runs; update the inline script to either (a) call subprocess.run([...], check=False) for wait_coderabbit.sh so the script continues regardless of non-zero exit, or (b) wrap the check_call in try/except (catch subprocess.CalledProcessError) and ensure the except block still invokes subprocess.check_call(['scripts/automation/collect_coderabbit_feedback.sh', str(pr), '.tutti/state/auto/coderabbit-feedback.md']) and optionally write a failure marker to '.tutti/state/auto/coderabbit_status.json' so downstream apply_coderabbit_feedback has a deterministic file to read.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In `@docs/examples/tutti-codex-sdlc.toml`:
- Around line 181-185: The inline Python step "wait_coderabbit" currently uses
subprocess.check_call for 'scripts/automation/wait_coderabbit.sh' which raises
CalledProcessError and aborts before
'scripts/automation/collect_coderabbit_feedback.sh' runs; update the inline
script to either (a) call subprocess.run([...], check=False) for
wait_coderabbit.sh so the script continues regardless of non-zero exit, or (b)
wrap the check_call in try/except (catch subprocess.CalledProcessError) and
ensure the except block still invokes
subprocess.check_call(['scripts/automation/collect_coderabbit_feedback.sh',
str(pr), '.tutti/state/auto/coderabbit-feedback.md']) and optionally write a
failure marker to '.tutti/state/auto/coderabbit_status.json' so downstream
apply_coderabbit_feedback has a deterministic file to read.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 9438a22a-f458-4b68-8e0d-ebcf7fcb58e1
📒 Files selected for processing (2)
docs/CODEX_SDLC_ORCHESTRATION.mddocs/examples/tutti-codex-sdlc.toml
🚧 Files skipped from review as they are similar to previous changes (1)
- docs/CODEX_SDLC_ORCHESTRATION.md
Contributor
Author
|
Follow-up with clean formatting (previous comment had shell-escaping artifacts). Addressed review feedback in Fixed
Will merge once checks are green. |
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (2)
scripts/automation/select_issue.sh (1)
36-42:⚠️ Potential issue | 🟡 MinorConsider atomic write or cleanup on claim failure.
If
gh issue editfails after the payload is written (line 36-37), the script exits but leavesselected_issue.jsonon disk with an unclaimed issue. A subsequent manual or partial re-run could pick up stale state.Consider either:
- Writing to a temp file and renaming atomically after successful claim
- Removing the output file on claim failure
🛡️ Suggested fix using atomic rename
-with open(out, "w", encoding="utf-8") as f: - json.dump(payload, f, indent=2) +import tempfile, os +tmp = out + ".tmp" +with open(tmp, "w", encoding="utf-8") as f: + json.dump(payload, f, indent=2) print(issue["number"]) PY ) gh issue edit "$ISSUE_NUM" --repo "$REPO" --add-label "automation-claimed" >/dev/null + +# Finalize output only after successful claim +mv "${OUT_FILE}.tmp" "$OUT_FILE"🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@scripts/automation/select_issue.sh` around lines 36 - 42, The script writes the claim payload to the output file (variable out / selected_issue.json) before running gh issue edit, leaving a stale file on failures; change the flow in select_issue.sh to write to a temporary file (e.g., out.tmp) and only atomically rename it to the final out after gh issue edit succeeds, or alternatively trap failures and remove the created out file if gh issue edit("$ISSUE_NUM") exits non‑zero; update the block around json.dump/write (the lines that open out and print issue["number"]) and the gh issue edit invocation to implement the temp-and-rename or cleanup on error so stale selected_issue.json is never left behind.docs/examples/tutti-codex-sdlc.toml (1)
115-119:⚠️ Potential issue | 🟠 MajorBranch creation is not bound to the implementer worktree, so commits can land on the wrong branch.
Line 116-119 creates the issue branch without an
agent, while Line 133-140 asks theimplementeragent to commit/push. This can desync branch state between workspace and implementer worktree.Proposed fix
[[workflow.step]] id = "create_branch" type = "command" +agent = "implementer" run = "scripts/automation/create_issue_branch.sh .tutti/state/auto/selected_issue.json .tutti/state/auto/branch.json" fail_mode = "closed"Also applies to: 133-140
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/examples/tutti-codex-sdlc.toml` around lines 115 - 119, The create_branch workflow step (id "create_branch", run "scripts/automation/create_issue_branch.sh ...") runs outside the implementer agent, causing branch creation to occur in a different worktree than the subsequent implementer commit/push steps and risking desynced branches; fix by binding branch creation to the same agent/worktree as the implementer (add agent = "implementer" to the "create_branch" step or move the script invocation into the implementer step) and apply the same change to any other branch-creation steps referenced near the commit/push steps so branch creation and commits occur in the same worktree.
🧹 Nitpick comments (1)
docs/examples/tutti-codex-sdlc.toml (1)
175-178: Make PR creation idempotent for reruns.
open_pralways callsgh pr create; rerunning the workflow for the same branch will fail hard when a PR already exists.Proposed fix
run = "python3 - <<'PY'\nimport json,subprocess\ni=json.load(open('.tutti/state/auto/selected_issue.json'))\nb=json.load(open('.tutti/state/auto/branch.json'))\nbranch=b['branch']\nissue=i['issue_number']\ntitle=f\"[auto] #{issue} {i['title']}\"\nbody=f\"Automated SDLC cycle for #{issue}.\\n\\n- planner: completed\\n- implementation: completed\\n- tests: updated\\n- docs/changelog: updated\\n- version: bumped if required\"\n-subprocess.check_call(['gh','pr','create','--title',title,'--body',body,'--head',branch,'--base','main'])\n+existing=subprocess.check_output(['gh','pr','list','--state','open','--head',branch,'--json','number'], text=True)\n+prs=json.loads(existing)\n+if prs:\n+ print(f\"PR already exists: #{prs[0]['number']}\")\n+else:\n+ subprocess.check_call(['gh','pr','create','--title',title,'--body',body,'--head',branch,'--base','main'])\n PY"🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/examples/tutti-codex-sdlc.toml` around lines 175 - 178, The open_pr task always runs gh pr create and fails on reruns; update the run script in the id = "open_pr" block to first check for an existing PR for the branch (e.g., call gh pr view --head branch or attempt gh pr list with head) and, if a PR exists, exit successfully (or optionally update the PR body/title) instead of calling subprocess.check_call(['gh','pr','create',...]); only call subprocess.check_call to create the PR when the branch has no existing PR. Ensure you reference the branch variable (branch) and the subprocess.check_call invocation when making the change.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@docs/examples/tutti-codex-sdlc.toml`:
- Around line 46-47: The scope "docs/**" contradicts the agent's prompt which
instructs it to update files outside that scope (README, CHANGELOG, Cargo.toml);
update the config so the docs-release agent can modify those files by expanding
the scope (e.g., include "README", "CHANGELOG", "Cargo.toml", or use a broader
pattern like "." or "{.,docs/**}") or change the prompt to limit changes to
docs/** only; edit the scope value and/or the prompt in the docs-release config
so they match (referencing the scope and prompt entries in the toml).
In `@scripts/automation/select_issue.sh`:
- Around line 13-42: Add a workflow-level concurrency guard to prevent TOCTOU
races between runs selecting issues in scripts/automation/select_issue.sh:
update the orchestrator job (job name orchestrate in
.github/workflows/sdlc-orchestrator.yml) to include a concurrency block (group
name including sdlc-orchestrator and the repository, e.g., sdlc-orchestrator-${{
github.repository }}) with cancel-in-progress: false so simultaneous runs cannot
both fetch JSON and race to label an issue; alternatively, if you must change
the script instead, modify select_issue.sh to attempt the label addition (gh
issue edit --add-label "automation-claimed") immediately after selecting an
issue number and then re-fetch the issue labels (via gh issue view or a
subsequent gh issue list JSON check) to confirm this run holds the
"automation-claimed" label before proceeding with the chosen ISSUE_NUM.
---
Duplicate comments:
In `@docs/examples/tutti-codex-sdlc.toml`:
- Around line 115-119: The create_branch workflow step (id "create_branch", run
"scripts/automation/create_issue_branch.sh ...") runs outside the implementer
agent, causing branch creation to occur in a different worktree than the
subsequent implementer commit/push steps and risking desynced branches; fix by
binding branch creation to the same agent/worktree as the implementer (add agent
= "implementer" to the "create_branch" step or move the script invocation into
the implementer step) and apply the same change to any other branch-creation
steps referenced near the commit/push steps so branch creation and commits occur
in the same worktree.
In `@scripts/automation/select_issue.sh`:
- Around line 36-42: The script writes the claim payload to the output file
(variable out / selected_issue.json) before running gh issue edit, leaving a
stale file on failures; change the flow in select_issue.sh to write to a
temporary file (e.g., out.tmp) and only atomically rename it to the final out
after gh issue edit succeeds, or alternatively trap failures and remove the
created out file if gh issue edit("$ISSUE_NUM") exits non‑zero; update the block
around json.dump/write (the lines that open out and print issue["number"]) and
the gh issue edit invocation to implement the temp-and-rename or cleanup on
error so stale selected_issue.json is never left behind.
---
Nitpick comments:
In `@docs/examples/tutti-codex-sdlc.toml`:
- Around line 175-178: The open_pr task always runs gh pr create and fails on
reruns; update the run script in the id = "open_pr" block to first check for an
existing PR for the branch (e.g., call gh pr view --head branch or attempt gh pr
list with head) and, if a PR exists, exit successfully (or optionally update the
PR body/title) instead of calling
subprocess.check_call(['gh','pr','create',...]); only call subprocess.check_call
to create the PR when the branch has no existing PR. Ensure you reference the
branch variable (branch) and the subprocess.check_call invocation when making
the change.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 134188fd-48f4-4750-a368-8a301364c08d
📒 Files selected for processing (2)
docs/examples/tutti-codex-sdlc.tomlscripts/automation/select_issue.sh
Contributor
Author
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
Contributor
Author
|
Pushed another hardening pass in
If checks/review are green after this, I’ll merge immediately. |
Contributor
Author
|
@coderabbitai resume |
✅ Actions performedReviews resumed. |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/sdlc-orchestrator.yml:
- Around line 62-64: Remove the duplicate pre-selection step that calls
scripts/automation/select_issue.sh and writes
.tutti/state/auto/selected_issue.json before invoking the sdlc jobs;
specifically, delete the mkdir -p .tutti/state/auto and
scripts/automation/select_issue.sh .tutti/state/auto/selected_issue.json
"$ISSUE_LABEL" (and the associated cargo run invocation if it was only there to
exercise that pre-selection) in the orchestrator so that sdlc-smoke and
sdlc-auto rely on their own internal select_issue logic; ensure no other steps
still depend on .tutti/state/auto/selected_issue.json being created earlier.
In `@docs/examples/tutti-codex-sdlc.toml`:
- Line 63: The run command entries currently pass the hard-coded label
"agent-ops" to the selector script (the run value
"scripts/automation/select_issue.sh .tutti/state/auto/selected_issue.json
agent-ops"); replace that literal with the runtime workflow input variable
(e.g., use the workflow dispatch input like "${{ inputs.issue_label }}" or an
env var such as "$ISSUE_LABEL") in both places where the run key calls
scripts/automation/select_issue.sh so the workflow-dispatch issue_label is
honored; ensure the selector invocation and any related environment/export in
the workflow propagate that variable to the script.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: d9a85685-edc8-4868-96fa-5a61ce39e917
📒 Files selected for processing (4)
.github/workflows/sdlc-orchestrator.ymldocs/AGENT_OPS_ROADMAP.mddocs/examples/tutti-codex-sdlc.tomlscripts/automation/select_issue.sh
🚧 Files skipped from review as they are similar to previous changes (1)
- scripts/automation/select_issue.sh
Contributor
Author
|
Yes — choir is addressing it. I just pushed
This removes double-claim risk and ensures workflow-dispatch |
Contributor
Author
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
🧹 Nitpick comments (1)
docs/examples/tutti-codex-sdlc.toml (1)
172-186: Consider extracting inline Python scripts for maintainability.The inline Python heredocs (lines 172, 179, 186) contain non-trivial logic for branch verification, PR creation, and CodeRabbit polling. While functional, extracting these to separate scripts in
scripts/automation/(alongside the existingselect_issue.sh,create_issue_branch.sh, etc.) would improve:
- Testability (scripts can be unit tested independently)
- Debuggability (clearer stack traces, easier local testing)
- Consistency with existing script organization
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/examples/tutti-codex-sdlc.toml` around lines 172 - 186, The workflow embeds non-trivial inline Python heredocs in the "run" of steps that perform branch verification, PR creation, and CodeRabbit polling (the blocks under the steps with ids implied by their comments like the branch-check, "open_pr", and "wait_coderabbit" logic); extract each heredoc into a dedicated executable script under scripts/automation (e.g., branch_check.py, create_pr.py, wait_coderabbit.py or similar), preserve behavior (stdout messages, exit codes, JSON file reads/writes, and gh/git calls), update the workflow steps' run to invoke python3 scripts/automation/<script>.py with the same arguments or env expectations, and ensure scripts are covered by tests or have logging/error handling consistent with existing scripts/automation/*.sh so maintainability and debuggability improve.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@docs/examples/tutti-codex-sdlc.toml`:
- Around line 172-186: The workflow embeds non-trivial inline Python heredocs in
the "run" of steps that perform branch verification, PR creation, and CodeRabbit
polling (the blocks under the steps with ids implied by their comments like the
branch-check, "open_pr", and "wait_coderabbit" logic); extract each heredoc into
a dedicated executable script under scripts/automation (e.g., branch_check.py,
create_pr.py, wait_coderabbit.py or similar), preserve behavior (stdout
messages, exit codes, JSON file reads/writes, and gh/git calls), update the
workflow steps' run to invoke python3 scripts/automation/<script>.py with the
same arguments or env expectations, and ensure scripts are covered by tests or
have logging/error handling consistent with existing scripts/automation/*.sh so
maintainability and debuggability improve.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 94fabcd1-17b0-4740-bea1-5a1147766655
📒 Files selected for processing (2)
.github/workflows/sdlc-orchestrator.ymldocs/examples/tutti-codex-sdlc.toml
Contributor
Author
|
Pushed What changed
Requesting fresh review to clear the block. |
|
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@scripts/automation/wait_and_collect_coderabbit.py`:
- Around line 25-31: The call to subprocess.check_call(...) for collecting
feedback can raise CalledProcessError and bypass
sys.exit(wait_result.returncode), so wrap the feedback collection (the
subprocess.check_call call that runs
"scripts/automation/collect_coderabbit_feedback.sh" with str(pr) and
feedback_file) in a try-except that catches subprocess.CalledProcessError (and
optionally Exception) and logs or warns but does not re-raise, ensuring
execution falls through to sys.exit(wait_result.returncode) so the original
wait_result.returncode is always used as the process exit status.
- Around line 15-20: The code assumes json.loads(out)[0] exists when calling
subprocess.check_output to list PRs; change the logic after running
subprocess.check_output and json.loads(out) to verify the resulting list is
non-empty before accessing index 0 (the PR number), and if empty print or log a
clear error referencing the branch and exit cleanly (e.g., sys.exit(1)); update
references to pr, branch, subprocess.check_output and json.loads accordingly so
you only set pr when the list contains at least one item.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 84617435-bb9d-4ccc-8699-a94ec96e6725
📒 Files selected for processing (4)
docs/examples/tutti-codex-sdlc.tomlscripts/automation/open_or_create_pr.pyscripts/automation/verify_branch_has_changes.pyscripts/automation/wait_and_collect_coderabbit.py
🚧 Files skipped from review as they are similar to previous changes (1)
- docs/examples/tutti-codex-sdlc.toml
Contributor
Author
|
Addressed the new actionable items in Changes in
@coderabbitai review |
|
Triggering a fresh incremental review to cover the updated ✅ Actions performedReview triggered.
|
nutt-adam
dismissed stale reviews from coderabbitai[bot], coderabbitai[bot], coderabbitai[bot], coderabbitai[bot], coderabbitai[bot], and coderabbitai[bot]
March 15, 2026 13:02
Dismissing stale CodeRabbit block per maintainer force-merge request after fixes applied.
Contributor
Author
|
Addressed the remaining unresolved review items in c53d7db:\n\n- collect_coderabbit_feedback.sh now includes inline PR review comments via GitHub API () and appends them to feedback markdown with path/line context.\n- wait_coderabbit.sh now uses a strict allowlist ( only) for PASS classification; any other completed conclusion is treated as FAIL.\n\nRequesting fresh review. |
Contributor
Author
|
Follow-up (clean): addressed remaining unresolved review items in commit c53d7db.
Requesting fresh review. |
This was referenced Mar 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds docs/AGENT_OPS_ROADMAP.md with a prioritized 10-item roadmap plus telemetry/activation instrumentation plan.
Summary by CodeRabbit
Documentation
New Features