feat(jira): add ready-to-solve eval harness with 4 test cases - #618
Merged
openshift-merge-bot[bot] merged 1 commit intoJul 14, 2026
Merged
openshift-merge-bot[bot] merged 1 commit into
openshift-merge-bot[bot] merged 1 commit into
Conversation
openshift-ci
Bot
added
the
approved
Contributor
ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (15)
💤 Files with no reviewable changes (1)
🚧 Files skipped from review as they are similar to previous changes (12)
WalkthroughThe Jira plugin version is bumped to 0.8.3. The ready-to-solve skill gains a section-validation CLI, revised PASS/FAIL workflow guidance, evaluation configuration, and four Jira fixtures covering invalid structure, ambiguous criteria, and fix mode. ChangesJira ready-to-solve workflow
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant ReadyToSolve
participant CheckSections
participant AIAssessment
participant EvalResult
ReadyToSolve->>CheckSections: validate issue description
CheckSections-->>ReadyToSolve: deterministic checks
ReadyToSolve->>AIAssessment: assess qualitative readiness
AIAssessment-->>ReadyToSolve: PASS/FAIL dimensions
ReadyToSolve->>EvalResult: write structured result and fix output
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 9 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (9 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
enxebre
force-pushed
the
eval/jira-ready-to-solve
branch
from
July 14, 2026 08:22
067e43b to
7fd28a2
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
plugins/jira/evals/eval-ready-to-solve.yaml (1)
201-236: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
fix_output_validjudge hardcodes fixture-specific content and uses loose keyword matching.Two concerns in this shared judge:
- Lines 231-233 hardcode
"IsDeploymentReady"/"rollout signal"— literals lifted from case-004's specific description — to verify "preserves original content." Any future--fixcase with different original text will silently pass/fail incorrectly since this check isn't parameterized per-case.- The
context_re/ac_re/tech_repatterns just search for bare words likeWhyorContextanywhere in the output text, which can match incidental prose (e.g., "...explains why the rollout...") rather than an actual added section heading, producing false positives that mask a missing section.Consider driving the "preserves original" check off an annotation field (e.g. a list of snippets from
input.yaml) instead of literals baked into the judge, and anchoring the heading regexes to line starts (e.g.^h2\.\s*(Context|...)or^#+\s*(Context|...)).♻️ Example refactor for original-content check
- if ann.get("expected_fix_preserves_original"): - if "IsDeploymentReady" not in fix and "rollout signal" not in fix: - errors.append("Fix does not preserve original content") + preserved_snippets = ann.get("expected_fix_preserved_snippets", []) + if preserved_snippets and not any(s in fix for s in preserved_snippets): + errors.append("Fix does not preserve original content")🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/jira/evals/eval-ready-to-solve.yaml` around lines 201 - 236, Update the fix_output_valid judge to remove the hardcoded IsDeploymentReady and rollout signal checks, instead validating preservation using per-case annotation snippets derived from the original input. Tighten context_re, ac_re, and tech_re so they match section headings anchored at line starts, supporting the established heading formats rather than incidental prose.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@plugins/jira/skills/ready-to-solve/scripts/check_sections.py`:
- Around line 212-252: Update main’s parsed-input validation after json.loads to
require data to be a dictionary and description to be a string before calling
get or strip. For malformed-but-valid JSON, emit the existing JSON error
response and exit with status 1, preserving the current missing-description
handling for valid objects with an empty or absent description.
- Around line 25-41: Update find_section() to use a heading-boundary matcher
consistent with build_heading_regex(): include colon and plain heading formats,
support case-insensitive matching, and recognize wiki headings regardless of
capitalization. Ensure section extraction stops at any supported heading so
later sections are not included.
---
Nitpick comments:
In `@plugins/jira/evals/eval-ready-to-solve.yaml`:
- Around line 201-236: Update the fix_output_valid judge to remove the hardcoded
IsDeploymentReady and rollout signal checks, instead validating preservation
using per-case annotation snippets derived from the original input. Tighten
context_re, ac_re, and tech_re so they match section headings anchored at line
starts, supporting the established heading formats rather than incidental prose.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 397b0d7e-1b3a-459b-8cc4-8808e59c6b80
📒 Files selected for processing (15)
.claude-plugin/marketplace.jsondocs/index.htmlplugins/jira/.claude-plugin/plugin.jsonplugins/jira/commands/ready-to-solve.mdplugins/jira/evals/cases/ready-to-solve/case-001-missing-standard-sections/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-001-missing-standard-sections/input.yamlplugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/input.yamlplugins/jira/evals/cases/ready-to-solve/case-003-partial-pass-has-context/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-003-partial-pass-has-context/input.yamlplugins/jira/evals/cases/ready-to-solve/case-004-fix-mode/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-004-fix-mode/input.yamlplugins/jira/evals/eval-ready-to-solve.yamlplugins/jira/skills/ready-to-solve/SKILL.mdplugins/jira/skills/ready-to-solve/scripts/check_sections.py
💤 Files with no reviewable changes (1)
- plugins/jira/commands/ready-to-solve.md
Contributor
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 2
🧹 Nitpick comments (1)
plugins/jira/evals/eval-ready-to-solve.yaml (1)
201-236: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
fix_output_validjudge hardcodes fixture-specific content and uses loose keyword matching.Two concerns in this shared judge:
- Lines 231-233 hardcode
"IsDeploymentReady"/"rollout signal"— literals lifted from case-004's specific description — to verify "preserves original content." Any future--fixcase with different original text will silently pass/fail incorrectly since this check isn't parameterized per-case.- The
context_re/ac_re/tech_repatterns just search for bare words likeWhyorContextanywhere in the output text, which can match incidental prose (e.g., "...explains why the rollout...") rather than an actual added section heading, producing false positives that mask a missing section.Consider driving the "preserves original" check off an annotation field (e.g. a list of snippets from
input.yaml) instead of literals baked into the judge, and anchoring the heading regexes to line starts (e.g.^h2\.\s*(Context|...)or^#+\s*(Context|...)).♻️ Example refactor for original-content check
- if ann.get("expected_fix_preserves_original"): - if "IsDeploymentReady" not in fix and "rollout signal" not in fix: - errors.append("Fix does not preserve original content") + preserved_snippets = ann.get("expected_fix_preserved_snippets", []) + if preserved_snippets and not any(s in fix for s in preserved_snippets): + errors.append("Fix does not preserve original content")🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/jira/evals/eval-ready-to-solve.yaml` around lines 201 - 236, Update the fix_output_valid judge to remove the hardcoded IsDeploymentReady and rollout signal checks, instead validating preservation using per-case annotation snippets derived from the original input. Tighten context_re, ac_re, and tech_re so they match section headings anchored at line starts, supporting the established heading formats rather than incidental prose.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@plugins/jira/skills/ready-to-solve/scripts/check_sections.py`:
- Around line 212-252: Update main’s parsed-input validation after json.loads to
require data to be a dictionary and description to be a string before calling
get or strip. For malformed-but-valid JSON, emit the existing JSON error
response and exit with status 1, preserving the current missing-description
handling for valid objects with an empty or absent description.
- Around line 25-41: Update find_section() to use a heading-boundary matcher
consistent with build_heading_regex(): include colon and plain heading formats,
support case-insensitive matching, and recognize wiki headings regardless of
capitalization. Ensure section extraction stops at any supported heading so
later sections are not included.
---
Nitpick comments:
In `@plugins/jira/evals/eval-ready-to-solve.yaml`:
- Around line 201-236: Update the fix_output_valid judge to remove the hardcoded
IsDeploymentReady and rollout signal checks, instead validating preservation
using per-case annotation snippets derived from the original input. Tighten
context_re, ac_re, and tech_re so they match section headings anchored at line
starts, supporting the established heading formats rather than incidental prose.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 397b0d7e-1b3a-459b-8cc4-8808e59c6b80
📒 Files selected for processing (15)
.claude-plugin/marketplace.jsondocs/index.htmlplugins/jira/.claude-plugin/plugin.jsonplugins/jira/commands/ready-to-solve.mdplugins/jira/evals/cases/ready-to-solve/case-001-missing-standard-sections/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-001-missing-standard-sections/input.yamlplugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/input.yamlplugins/jira/evals/cases/ready-to-solve/case-003-partial-pass-has-context/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-003-partial-pass-has-context/input.yamlplugins/jira/evals/cases/ready-to-solve/case-004-fix-mode/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-004-fix-mode/input.yamlplugins/jira/evals/eval-ready-to-solve.yamlplugins/jira/skills/ready-to-solve/SKILL.mdplugins/jira/skills/ready-to-solve/scripts/check_sections.py
💤 Files with no reviewable changes (1)
- plugins/jira/commands/ready-to-solve.md
🛑 Comments failed to post (2)
plugins/jira/skills/ready-to-solve/scripts/check_sections.py (2)
25-41: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash # Check whether the eval fixtures rely on non-wiki heading styles that would # be affected by this boundary-detection gap. fd input.yaml plugins/jira/evals/cases/ready-to-solve --exec grep -nE '^h[1-6]\.|^#{1,6}\s|^\*\*.+\*\*|^[A-Za-z ]+:\s*$' {} \;Repository: openshift-eng/ai-helpers
Length of output: 184
🏁 Script executed:
#!/bin/bash set -euo pipefail printf '\n## File outline\n' ast-grep outline plugins/jira/skills/ready-to-solve/scripts/check_sections.py --view expanded || true printf '\n## Relevant lines in check_sections.py\n' sed -n '1,220p' plugins/jira/skills/ready-to-solve/scripts/check_sections.py printf '\n## Search for heading regex usage\n' rg -n "ANY_HEADING_RE|build_heading_regex|find_section|HEADING_PATTERNS|CONTEXT_HEADINGS|AC_HEADINGS|TECH_HEADINGS" plugins/jira/skills/ready-to-solve/scripts/check_sections.pyRepository: openshift-eng/ai-helpers
Length of output: 9437
🏁 Script executed:
#!/bin/bash set -euo pipefail printf '\n## Locate ready-to-solve fixtures\n' fd -a input.yaml plugins/jira/evals/cases/ready-to-solve || true printf '\n## Search heading styles in fixtures\n' rg -n "^(h[1-6]\\.|#{1,6}\\s|\\*\\*.+\\*\\*|[A-Za-z ]+:\\s*$)" plugins/jira/evals/cases/ready-to-solve -g 'input.yaml' || trueRepository: openshift-eng/ai-helpers
Length of output: 731
🏁 Script executed:
#!/bin/bash set -euo pipefail for f in plugins/jira/evals/cases/ready-to-solve/*/input.yaml; do echo "### $f" sed -n '1,140p' "$f" | nl -ba | sed -n '1,140p' echo doneRepository: openshift-eng/ai-helpers
Length of output: 292
🏁 Script executed:
#!/bin/bash set -euo pipefail python3 - <<'PY' from pathlib import Path import re files = sorted(Path("plugins/jira/evals/cases/ready-to-solve").glob("*/input.yaml")) for path in files: text = path.read_text() print(f"\n### {path}") # Print any lines inside the YAML description block that look like headings in_desc = False indent = None for i, line in enumerate(text.splitlines(), 1): if line.startswith("description:"): in_desc = True indent = None continue if not in_desc: continue if indent is None and line.strip(): indent = len(line) - len(line.lstrip(" ")) if line.strip() == "": continue cur_indent = len(line) - len(line.lstrip(" ")) if indent is not None and cur_indent < indent: break stripped = line.strip() if re.match(r'^(h[1-6]\.|#{1,6}\s|\*\*.+\*\*|[A-Za-z][A-Za-z ]*:\s*$)', stripped): print(f"{i}: {stripped}") PYRepository: openshift-eng/ai-helpers
Length of output: 501
🏁 Script executed:
#!/bin/bash set -euo pipefail sed -n '1,120p' plugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/input.yaml | nl -baRepository: openshift-eng/ai-helpers
Length of output: 202
🏁 Script executed:
#!/bin/bash set -euo pipefail python3 - <<'PY' from pathlib import Path import re def extract_description(text): lines = text.splitlines() desc = [] in_desc = False desc_indent = None for i, line in enumerate(lines): if not in_desc: if line.startswith("description:"): in_desc = True rest = line[len("description:"):] if rest.strip(): # inline scalar, keep only the inline content desc.append(rest.lstrip()) return "\n".join(desc) continue if desc_indent is None: if not line.strip(): continue desc_indent = len(line) - len(line.lstrip(" ")) cur_indent = len(line) - len(line.lstrip(" ")) if line.strip() and cur_indent < desc_indent: break desc.append(line[desc_indent:] if len(line) >= desc_indent else line) return "\n".join(desc).rstrip("\n") heading_re = re.compile(r'^(?:h[1-6]\.\s+.+|#{1,6}\s+.+|\*\*.+\*\*|.+:\s*|.+)$', re.I) known_heading_re = re.compile( r'^(?:' r'(?:h[1-6]\.\s+.+|#{1,6}\s+.+|\*\*.+\*\*)' r'|(?:Context|Description|Background|Overview|Why|Acceptance Criteria|AC|Definition of Done|Technical Details|Technical Context|Implementation Details|Implementation Notes|Technical Notes)\s*:\s*' r'|(?:Context|Description|Background|Overview|Why|Acceptance Criteria|AC|Definition of Done|Technical Details|Technical Context|Implementation Details|Implementation Notes|Technical Notes)\s*$' r')$', re.I ) for path in sorted(Path("plugins/jira/evals/cases/ready-to-solve").glob("*/input.yaml")): text = path.read_text() desc = extract_description(text) print(f"\n### {path}") print(desc[:1500]) print("\n-- headings present --") for n, line in enumerate(desc.splitlines(), 1): s = line.strip() if re.match(r'^(h[1-6]\.|#{1,6}\s|\*\*.+\*\*|[A-Za-z][A-Za-z ]*:\s*$)', s): print(f"{n}: {s}") PYRepository: openshift-eng/ai-helpers
Length of output: 601
🏁 Script executed:
#!/bin/bash set -euo pipefail python3 - <<'PY' from pathlib import Path import re def extract_description(text): lines = text.splitlines() desc = [] in_desc = False desc_indent = None for i, line in enumerate(lines): if not in_desc: if line.startswith("description:"): in_desc = True rest = line[len("description:"):] if rest.strip(): desc.append(rest.lstrip()) return "\n".join(desc) continue if desc_indent is None: if not line.strip(): continue desc_indent = len(line) - len(line.lstrip(" ")) cur_indent = len(line) - len(line.lstrip(" ")) if line.strip() and cur_indent < desc_indent: break desc.append(line[desc_indent:] if len(line) >= desc_indent else line) return "\n".join(desc).rstrip("\n") for path in sorted(Path("plugins/jira/evals/cases/ready-to-solve").glob("*/input.yaml")): text = path.read_text() desc = extract_description(text) print(f"\n### {path}") print(desc[:1500]) print("\n-- headings present --") for n, line in enumerate(desc.splitlines(), 1): s = line.strip() if re.match(r'^(h[1-6]\.|#{1,6}\s|\*\*.+\*\*|[A-Za-z][A-Za-z ]*:\s*$)', s): print(f"{n}: {s}") PYRepository: openshift-eng/ai-helpers
Length of output: 601
find_section()misses some supported heading boundaries
build_heading_regex()recognizes colon/plain headings and matches case-insensitively, butfind_section()stops onANY_HEADING_RE, which only covers wiki/markdown/bold styles and is case-sensitive. AContext:/ bare-heading layout (orH2. Acceptance Criteria) can pull later sections into the first match, skewing length/item checks and hiding short or empty sections.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/jira/skills/ready-to-solve/scripts/check_sections.py` around lines 25 - 41, Update find_section() to use a heading-boundary matcher consistent with build_heading_regex(): include colon and plain heading formats, support case-insensitive matching, and recognize wiki headings regardless of capitalization. Ensure section extraction stops at any supported heading so later sections are not included.
212-252: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
No type validation on parsed JSON input; malformed-but-valid JSON crashes instead of erroring gracefully.
If
rawparses to something other than a dict (e.g.null, a JSON array, or a bare string),data.get("description")at Line 234 raises an unhandledAttributeError. Similarly, ifdescriptionis present but not a string (e.g. a number),description.strip()at Line 235 raisesAttributeErrortoo. Both bypass the documented JSON-error/exit-code contract (exit 1 for bad input) and instead produce an uncaught traceback.🛡️ Proposed fix
data = json.loads(raw) except json.JSONDecodeError as e: print(json.dumps({"error": f"Invalid JSON input: {e}"}), file=sys.stdout) sys.exit(1) - description = data.get("description") - if not description or not description.strip(): + if not isinstance(data, dict): + print(json.dumps({"error": "Input must be a JSON object"})) + sys.exit(1) + + description = data.get("description") + if not isinstance(description, str) or not description.strip():The "use jsonify" static-analysis hints on this segment are false positives — this is a standalone CLI script, not a Flask app.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.def main(): parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("--verbose", action="store_true", help="Include matched section content in output") args = parser.parse_args() try: raw = sys.stdin.read() if not raw.strip(): print(json.dumps({ "overall_pass": False, "checks": [], "stats": {"total": 0, "passed": 0, "failed": 0, "warnings": 0}, "error": "No input provided on stdin", })) sys.exit(2) data = json.loads(raw) except json.JSONDecodeError as e: print(json.dumps({"error": f"Invalid JSON input: {e}"}), file=sys.stdout) sys.exit(1) if not isinstance(data, dict): print(json.dumps({"error": "Input must be a JSON object"})) sys.exit(1) description = data.get("description") if not isinstance(description, str) or not description.strip(): print(json.dumps({ "overall_pass": False, "checks": [ make_check("has_description", "Issue has a description", "REQUIRED", False, "Issue description is empty or null"), ], "stats": {"total": 1, "passed": 0, "failed": 1, "warnings": 0}, })) sys.exit(2) result = run_checks(description, verbose=args.verbose) print(json.dumps(result, indent=2)) if __name__ == "__main__": main()🧰 Tools
🪛 ast-grep (0.44.1)
[info] 220-225: use jsonify instead of json.dumps for JSON output
Context: json.dumps({
"overall_pass": False,
"checks": [],
"stats": {"total": 0, "passed": 0, "failed": 0, "warnings": 0},
"error": "No input provided on stdin",
})
Note: [CWE-116] Improper Encoding or Escaping of Output.(use-jsonify)
[info] 230-230: use jsonify instead of json.dumps for JSON output
Context: json.dumps({"error": f"Invalid JSON input: {e}"})
Note: [CWE-116] Improper Encoding or Escaping of Output.(use-jsonify)
[info] 235-243: use jsonify instead of json.dumps for JSON output
Context: json.dumps({
"overall_pass": False,
"checks": [
make_check("has_description", "Issue has a description",
"REQUIRED", False,
"Issue description is empty or null"),
],
"stats": {"total": 1, "passed": 0, "failed": 1, "warnings": 0},
})
Note: [CWE-116] Improper Encoding or Escaping of Output.(use-jsonify)
[info] 247-247: use jsonify instead of json.dumps for JSON output
Context: json.dumps(result, indent=2)
Note: [CWE-116] Improper Encoding or Escaping of Output.(use-jsonify)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/jira/skills/ready-to-solve/scripts/check_sections.py` around lines 212 - 252, Update main’s parsed-input validation after json.loads to require data to be a dictionary and description to be a string before calling get or strip. For malformed-but-valid JSON, emit the existing JSON error response and exit with status 1, preserving the current missing-description handling for valid objects with an empty or absent description.
enxebre
force-pushed
the
eval/jira-ready-to-solve
branch
3 times, most recently
from
July 14, 2026 08:34
8034b46 to
a024a42
Compare
- Add eval config with 6 judges (5 deterministic + 1 LLM) testing readiness validation: verdict correctness, deterministic check accuracy, AI assessment alignment, and --fix output validity - 4 fixture-based test cases using static descriptions (no live Jira): case-001: non-standard headings (all REQUIRED checks fail) case-002: detailed bug report (wrong structure, good content) case-003: partial pass (has Description, missing AC/Tech) case-004: --fix mode (generates revised description) - Consolidate skill: delete command file, move implementation to SKILL.md with canonical sections (Name/Synopsis/Description/ Implementation/Return Value/Examples/Arguments) - Move check_sections.py to scripts/ directory - Bump jira plugin 0.8.2 -> 0.8.3 Validated: 3 consecutive runs, 6/6 judges, 0 regressions, ~$1.70/run. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
enxebre
force-pushed
the
eval/jira-ready-to-solve
branch
from
July 14, 2026 08:36
a024a42 to
0a02956
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@plugins/jira/skills/ready-to-solve/scripts/check_sections.py`:
- Around line 229-235: Update the JSON handling before the description logic in
the script’s main validation flow to require that the parsed root is an object
and that data.get("description") is a string before calling strip or mapping
methods. For valid JSON with an array, null root, or non-string description,
return the documented structured error through the existing JSON stdout and exit
path instead of raising a runtime exception.
- Around line 59-62: Update ANY_HEADING_RE used by the section-boundary logic to
recognize accepted colon headings, including Context: and Acceptance Criteria:,
plus the supported plain-heading format. Ensure next_heading in the section
parsing flow stops content at any accepted heading so subsequent sections are
evaluated independently.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 3d5c55a3-9797-430b-b008-9361bbcfb3ff
📒 Files selected for processing (15)
.claude-plugin/marketplace.jsondocs/index.htmlplugins/jira/.claude-plugin/plugin.jsonplugins/jira/commands/ready-to-solve.mdplugins/jira/evals/cases/ready-to-solve/case-001-missing-standard-sections/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-001-missing-standard-sections/input.yamlplugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/input.yamlplugins/jira/evals/cases/ready-to-solve/case-003-has-standard-sections-but-ambiguous/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-003-has-standard-sections-but-ambiguous/input.yamlplugins/jira/evals/cases/ready-to-solve/case-004-fix-mode/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-004-fix-mode/input.yamlplugins/jira/evals/eval-ready-to-solve.yamlplugins/jira/skills/ready-to-solve/SKILL.mdplugins/jira/skills/ready-to-solve/scripts/check_sections.py
💤 Files with no reviewable changes (1)
- plugins/jira/commands/ready-to-solve.md
🚧 Files skipped from review as they are similar to previous changes (9)
- plugins/jira/evals/cases/ready-to-solve/case-004-fix-mode/annotations.yaml
- plugins/jira/evals/cases/ready-to-solve/case-001-missing-standard-sections/annotations.yaml
- plugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/annotations.yaml
- docs/index.html
- plugins/jira/evals/cases/ready-to-solve/case-004-fix-mode/input.yaml
- plugins/jira/evals/cases/ready-to-solve/case-001-missing-standard-sections/input.yaml
- plugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/input.yaml
- plugins/jira/evals/eval-ready-to-solve.yaml
- plugins/jira/skills/ready-to-solve/SKILL.md
Contributor
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@plugins/jira/skills/ready-to-solve/scripts/check_sections.py`:
- Around line 229-235: Update the JSON handling before the description logic in
the script’s main validation flow to require that the parsed root is an object
and that data.get("description") is a string before calling strip or mapping
methods. For valid JSON with an array, null root, or non-string description,
return the documented structured error through the existing JSON stdout and exit
path instead of raising a runtime exception.
- Around line 59-62: Update ANY_HEADING_RE used by the section-boundary logic to
recognize accepted colon headings, including Context: and Acceptance Criteria:,
plus the supported plain-heading format. Ensure next_heading in the section
parsing flow stops content at any accepted heading so subsequent sections are
evaluated independently.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 3d5c55a3-9797-430b-b008-9361bbcfb3ff
📒 Files selected for processing (15)
.claude-plugin/marketplace.jsondocs/index.htmlplugins/jira/.claude-plugin/plugin.jsonplugins/jira/commands/ready-to-solve.mdplugins/jira/evals/cases/ready-to-solve/case-001-missing-standard-sections/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-001-missing-standard-sections/input.yamlplugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/input.yamlplugins/jira/evals/cases/ready-to-solve/case-003-has-standard-sections-but-ambiguous/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-003-has-standard-sections-but-ambiguous/input.yamlplugins/jira/evals/cases/ready-to-solve/case-004-fix-mode/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-004-fix-mode/input.yamlplugins/jira/evals/eval-ready-to-solve.yamlplugins/jira/skills/ready-to-solve/SKILL.mdplugins/jira/skills/ready-to-solve/scripts/check_sections.py
💤 Files with no reviewable changes (1)
- plugins/jira/commands/ready-to-solve.md
🚧 Files skipped from review as they are similar to previous changes (9)
- plugins/jira/evals/cases/ready-to-solve/case-004-fix-mode/annotations.yaml
- plugins/jira/evals/cases/ready-to-solve/case-001-missing-standard-sections/annotations.yaml
- plugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/annotations.yaml
- docs/index.html
- plugins/jira/evals/cases/ready-to-solve/case-004-fix-mode/input.yaml
- plugins/jira/evals/cases/ready-to-solve/case-001-missing-standard-sections/input.yaml
- plugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/input.yaml
- plugins/jira/evals/eval-ready-to-solve.yaml
- plugins/jira/skills/ready-to-solve/SKILL.md
🛑 Comments failed to post (2)
plugins/jira/skills/ready-to-solve/scripts/check_sections.py (2)
59-62: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Stop sections at accepted colon and plain headings.
ANY_HEADING_REomits the acceptedContext:,Acceptance Criteria:, and plain-heading formats. Their preceding section therefore absorbs subsequent sections, potentially producing falseoverall_passresults.Proposed fix
start = match.end() - next_heading = ANY_HEADING_RE.search(text, start) + known_heading_re = build_heading_regex( + CONTEXT_HEADINGS + AC_HEADINGS + TECH_HEADINGS + ) + candidates = [ + candidate + for candidate in ( + ANY_HEADING_RE.search(text, start), + known_heading_re.search(text, start), + ) + if candidate + ] + next_heading = min(candidates, key=lambda candidate: candidate.start()) if candidates else None end = next_heading.start() if next_heading else len(text)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.start = match.end() known_heading_re = build_heading_regex( CONTEXT_HEADINGS + AC_HEADINGS + TECH_HEADINGS ) candidates = [ candidate for candidate in ( ANY_HEADING_RE.search(text, start), known_heading_re.search(text, start), ) if candidate ] next_heading = min(candidates, key=lambda candidate: candidate.start()) if candidates else None end = next_heading.start() if next_heading else len(text) content = text[start:end].strip()🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/jira/skills/ready-to-solve/scripts/check_sections.py` around lines 59 - 62, Update ANY_HEADING_RE used by the section-boundary logic to recognize accepted colon headings, including Context: and Acceptance Criteria:, plus the supported plain-heading format. Ensure next_heading in the section parsing flow stops content at any accepted heading so subsequent sections are evaluated independently.
229-235: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Validate JSON and description types before calling mapping/string methods.
Valid JSON such as
[],null, or{"description": 123}currently crashes instead of returning the documented structured script error.Proposed fix
data = json.loads(raw) except json.JSONDecodeError as e: print(json.dumps({"error": f"Invalid JSON input: {e}"}), file=sys.stdout) sys.exit(1) + if not isinstance(data, dict): + print(json.dumps({"error": "Input must be a JSON object"})) + sys.exit(1) + description = data.get("description") + if description is not None and not isinstance(description, str): + print(json.dumps({"error": "description must be a string"})) + sys.exit(1) + if not description or not description.strip():📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.data = json.loads(raw) except json.JSONDecodeError as e: print(json.dumps({"error": f"Invalid JSON input: {e}"}), file=sys.stdout) sys.exit(1) if not isinstance(data, dict): print(json.dumps({"error": "Input must be a JSON object"})) sys.exit(1) description = data.get("description") if description is not None and not isinstance(description, str): print(json.dumps({"error": "description must be a string"})) sys.exit(1) if not description or not description.strip():🧰 Tools
🪛 ast-grep (0.44.1)
[info] 230-230: use jsonify instead of json.dumps for JSON output
Context: json.dumps({"error": f"Invalid JSON input: {e}"})
Note: [CWE-116] Improper Encoding or Escaping of Output.(use-jsonify)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/jira/skills/ready-to-solve/scripts/check_sections.py` around lines 229 - 235, Update the JSON handling before the description logic in the script’s main validation flow to require that the parsed root is an object and that data.get("description") is a string before calling strip or mapping methods. For valid JSON with an array, null root, or non-string description, return the documented structured error through the existing JSON stdout and exit path instead of raising a runtime exception.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@plugins/jira/skills/ready-to-solve/scripts/check_sections.py`:
- Around line 25-79: The section boundary logic in find_section currently uses
ANY_HEADING_RE, which misses plain and colon-form headings and incorrectly
treats Jira # list items as headings. Replace that boundary matching with a
regex built from the combined CONTEXT_HEADINGS, AC_HEADINGS, and TECH_HEADINGS
via build_heading_regex, preserving section extraction while recognizing only
supported headings.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 32c0b392-44a3-4d11-a961-76b9c1aac37b
📒 Files selected for processing (15)
.claude-plugin/marketplace.jsondocs/index.htmlplugins/jira/.claude-plugin/plugin.jsonplugins/jira/commands/ready-to-solve.mdplugins/jira/evals/cases/ready-to-solve/case-001-missing-standard-sections/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-001-missing-standard-sections/input.yamlplugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/input.yamlplugins/jira/evals/cases/ready-to-solve/case-003-has-standard-sections-but-ambiguous/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-003-has-standard-sections-but-ambiguous/input.yamlplugins/jira/evals/cases/ready-to-solve/case-004-fix-mode/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-004-fix-mode/input.yamlplugins/jira/evals/eval-ready-to-solve.yamlplugins/jira/skills/ready-to-solve/SKILL.mdplugins/jira/skills/ready-to-solve/scripts/check_sections.py
💤 Files with no reviewable changes (1)
- plugins/jira/commands/ready-to-solve.md
🚧 Files skipped from review as they are similar to previous changes (12)
- plugins/jira/evals/cases/ready-to-solve/case-004-fix-mode/annotations.yaml
- plugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/input.yaml
- plugins/jira/evals/cases/ready-to-solve/case-003-has-standard-sections-but-ambiguous/annotations.yaml
- plugins/jira/evals/cases/ready-to-solve/case-001-missing-standard-sections/annotations.yaml
- plugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/annotations.yaml
- .claude-plugin/marketplace.json
- plugins/jira/.claude-plugin/plugin.json
- plugins/jira/evals/cases/ready-to-solve/case-004-fix-mode/input.yaml
- plugins/jira/evals/cases/ready-to-solve/case-001-missing-standard-sections/input.yaml
- plugins/jira/evals/cases/ready-to-solve/case-003-has-standard-sections-but-ambiguous/input.yaml
- plugins/jira/evals/eval-ready-to-solve.yaml
- plugins/jira/skills/ready-to-solve/SKILL.md
Contributor
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@plugins/jira/skills/ready-to-solve/scripts/check_sections.py`:
- Around line 25-79: The section boundary logic in find_section currently uses
ANY_HEADING_RE, which misses plain and colon-form headings and incorrectly
treats Jira # list items as headings. Replace that boundary matching with a
regex built from the combined CONTEXT_HEADINGS, AC_HEADINGS, and TECH_HEADINGS
via build_heading_regex, preserving section extraction while recognizing only
supported headings.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 32c0b392-44a3-4d11-a961-76b9c1aac37b
📒 Files selected for processing (15)
.claude-plugin/marketplace.jsondocs/index.htmlplugins/jira/.claude-plugin/plugin.jsonplugins/jira/commands/ready-to-solve.mdplugins/jira/evals/cases/ready-to-solve/case-001-missing-standard-sections/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-001-missing-standard-sections/input.yamlplugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/input.yamlplugins/jira/evals/cases/ready-to-solve/case-003-has-standard-sections-but-ambiguous/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-003-has-standard-sections-but-ambiguous/input.yamlplugins/jira/evals/cases/ready-to-solve/case-004-fix-mode/annotations.yamlplugins/jira/evals/cases/ready-to-solve/case-004-fix-mode/input.yamlplugins/jira/evals/eval-ready-to-solve.yamlplugins/jira/skills/ready-to-solve/SKILL.mdplugins/jira/skills/ready-to-solve/scripts/check_sections.py
💤 Files with no reviewable changes (1)
- plugins/jira/commands/ready-to-solve.md
🚧 Files skipped from review as they are similar to previous changes (12)
- plugins/jira/evals/cases/ready-to-solve/case-004-fix-mode/annotations.yaml
- plugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/input.yaml
- plugins/jira/evals/cases/ready-to-solve/case-003-has-standard-sections-but-ambiguous/annotations.yaml
- plugins/jira/evals/cases/ready-to-solve/case-001-missing-standard-sections/annotations.yaml
- plugins/jira/evals/cases/ready-to-solve/case-002-detailed-bug-no-ac/annotations.yaml
- .claude-plugin/marketplace.json
- plugins/jira/.claude-plugin/plugin.json
- plugins/jira/evals/cases/ready-to-solve/case-004-fix-mode/input.yaml
- plugins/jira/evals/cases/ready-to-solve/case-001-missing-standard-sections/input.yaml
- plugins/jira/evals/cases/ready-to-solve/case-003-has-standard-sections-but-ambiguous/input.yaml
- plugins/jira/evals/eval-ready-to-solve.yaml
- plugins/jira/skills/ready-to-solve/SKILL.md
🛑 Comments failed to post (1)
plugins/jira/skills/ready-to-solve/scripts/check_sections.py (1)
25-79: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash python3 - <<'PY' import sys sys.path.insert(0, "plugins/jira/skills/ready-to-solve/scripts") import check_sections as cs # Case 1: Jira-native ordered list inside AC section text1 = """Acceptance Criteria # First criterion is met and should count as list content # Second criterion also present here for the section body """ found1, content1 = cs.find_section(text1, cs.AC_HEADINGS) print("case1 found:", found1, "content:", repr(content1)) # Case 2: plain-style headings bleeding into later sections text2 = """Context Short context. Acceptance Criteria - item one - item two """ found2, content2 = cs.find_section(text2, cs.CONTEXT_HEADINGS) print("case2 found:", found2, "content:", repr(content2)) PYRepository: openshift-eng/ai-helpers
Length of output: 283
🏁 Script executed:
#!/bin/bash set -euo pipefail sed -n '1,220p' plugins/jira/skills/ready-to-solve/scripts/check_sections.py printf '\n--- tests/search ---\n' rg -n "find_section|count_list_items|ANY_HEADING_RE|Acceptance Criteria|Definition of Done|Technical Details" -S plugins/jira -g '!**/__pycache__/**' || trueRepository: openshift-eng/ai-helpers
Length of output: 13976
🏁 Script executed:
#!/bin/bash python3 - <<'PY' import sys sys.path.insert(0, "plugins/jira/skills/ready-to-solve/scripts") import check_sections as cs cases = { "plain_headings": """Context Short context. Acceptance Criteria - item one - item two Technical Details more stuff """, "colon_headings": """Context: Short context. Acceptance Criteria: - item one - item two Technical Details: more stuff """, "jira_list_in_ac": """Acceptance Criteria # First criterion is met and should count as list content # Second criterion also present here for the section body Technical Details more stuff """, } for name, text in cases.items(): found, content = cs.find_section(text, cs.AC_HEADINGS) print(name, "found=", found, "content=", repr(content), "count=", cs.count_list_items(content)) PYRepository: openshift-eng/ai-helpers
Length of output: 418
🏁 Script executed:
#!/bin/bash set -euo pipefail python3 - <<'PY' from pathlib import Path p = Path("plugins/jira/skills/ready-to-solve/scripts/check_sections.py") for i, line in enumerate(p.read_text().splitlines(), 1): if 1 <= i <= 220: print(f"{i:4}: {line}") PY printf '\n--- rg ---\n' rg -n "find_section|count_list_items|ANY_HEADING_RE|build_heading_regex|ctx_len|ac_len|technical" plugins/jira/skills/ready-to-solve/scripts -SRepository: openshift-eng/ai-helpers
Length of output: 11458
Use the same heading matcher for section boundaries.
ANY_HEADING_REonly recognizes Jira wiki/Markdown/bold headings, so plain and colon headings never end a section and later content bleeds into the current one. It also treats Jira# itemlist lines as Markdown headings, which truncates AC sections that start with ordered lists.Build the boundary regex from
CONTEXT_HEADINGS + AC_HEADINGS + TECH_HEADINGS(or reusebuild_heading_regex) instead of matching arbitrary heading-like lines.🧰 Tools
🪛 ast-grep (0.44.1)
[warning] 49-49: Regex pattern passed to re is built from a non-literal (variable, call, concatenation, or f-string) value. If that value is attacker-controlled it can introduce a malicious pattern with catastrophic backtracking (ReDoS). Use a hardcoded literal pattern, or validate/escape untrusted input with re.escape() and bound the regex complexity before compiling.
Context: re.compile("|".join(parts), re.MULTILINE | re.IGNORECASE)
Note: [CWE-1333] Inefficient Regular Expression Complexity.(redos-non-literal-regex-python)
[warning] 74-74: Regex pattern passed to re is built from a non-literal (variable, call, concatenation, or f-string) value. If that value is attacker-controlled it can introduce a malicious pattern with catastrophic backtracking (ReDoS). Use a hardcoded literal pattern, or validate/escape untrusted input with re.escape() and bound the regex complexity before compiling.
Context: re.match(pattern, line)
Note: [CWE-1333] Inefficient Regular Expression Complexity.(redos-non-literal-regex-python)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/jira/skills/ready-to-solve/scripts/check_sections.py` around lines 25 - 79, The section boundary logic in find_section currently uses ANY_HEADING_RE, which misses plain and colon-form headings and incorrectly treats Jira # list items as headings. Replace that boundary matching with a regex built from the combined CONTEXT_HEADINGS, AC_HEADINGS, and TECH_HEADINGS via build_heading_regex, preserving section extraction while recognizing only supported headings.
bryan-cox
reviewed
Jul 14, 2026
| @@ -130,8 +144,6 @@ Post or update a Jira comment reflecting the validation result so the ticket aut | |||
|
|
|||
| Fetch the issue with comments included via `getJiraIssue` and search for one whose body starts with `**Automated Readiness Check`. Save its `comment_id` if found. | |||
|
|
|||
| Iterate through the comments in the returned issue payload. If any comment body starts with `**Automated Readiness Check`, store its `comment_id` for editing in Step 3. | |||
Contributor
There was a problem hiding this comment.
Are you wanting to purposefully drop this? Was something using this before?
Contributor
Author
There was a problem hiding this comment.
yeah, it seems pretty much duplicating the same sentence above.
I'll want to follow up to make the comment steps a deterministic script anyways.
Contributor
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: bryan-cox, enxebre The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
jira:ready-to-solveskill with 6 judges (5 deterministic + 1 LLM)check_sections.pytoscripts/directoryCounterpart release PR: openshift/release#81820
Test plan
make lintpasses (0 errors)🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
--dry-run,--verbose, and--fixworkflows and the structured reporting format.Documentation
/jira:ready-to-solveworkflow and messaging, including revised skill descriptions and updated guidance for automated comment handling.Tests
Chores