Deep-dive into failure patterns for install regressions

[dgoodwin]

This is a step closer to deep diving into any test failure regression.

Summary by CodeRabbit

• New Features

  • Added Deep Install Failure Analysis to the regression analysis command, which automatically detects install failures and performs root-cause analysis across failed job runs, classifying results by single or multiple root causes across stages.

• Chores

  • Updated CI plugin version from 0.0.35 to 0.0.36.

[coderabbitai]

Walkthrough

Version bump of the CI plugin from 0.0.35 to 0.0.36 across marketplace, documentation, and plugin manifests. Documentation updated to describe a new optional install failure analysis step triggered by specific test name patterns.

Changes

Cohort / File(s)	Summary
Plugin Version Updates .claude-plugin/marketplace.json, docs/data.json, plugins/ci/.claude-plugin/plugin.json	Incremented CI plugin version from 0.0.35 to 0.0.36 across all configuration and manifest files.
Command Documentation plugins/ci/commands/analyze-regression.md	Added new optional "Deep Install Failure Analysis" step that triggers when test names contain "install should succeed". Analyzes up to 5 high-failure job runs using the prow-job-analyze-install-failure skill, classifies results by root-cause consistency, and includes findings in the generated report as a new "Install Failure Analysis" section.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 8

✅ Passed checks (8 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title directly reflects the main change: adding deep-dive analysis for install failure patterns in the analyze-regression command.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
No Real People Names In Style References	✅ Passed	PR summary shows only version bumps and install failure analysis documentation with no real person names used as style references or examples.
No Assumed Git Remote Names	✅ Passed	No hardcoded git remote names found in any changed files. All modifications consist of version bumps and documentation without git commands or remote assumptions.
Git Push Safety Rules	✅ Passed	No git push commands, force push operations, or unsafe git operations found in the modified files.
No Untrusted Mcp Servers	✅ Passed	The pull request contains only version bumps and documentation updates referencing an existing internal skill, introducing no new MCP server installations.
Ai-Helpers Overlap Detection	✅ Passed	PR enhances existing command by adding conditional step invoking pre-existing skill; no new duplicate functionality or naming conflicts detected.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@plugins/ci/commands/analyze-regression.md`:
- Line 450: The bullet under the heading "Consistent stage, multiple root
causes" is a sentence fragment starting with "May be a single unstable area with
varied surface failures"; change it to a full sentence by adding a subject,
e.g., "This may be a single unstable area with varied surface failures," so the
bullet reads: "All runs fail at the same stage but with different error messages
(e.g., all `infrastructure` failures but different cloud API errors). This may
be a single unstable area with varied surface failures." Use the existing bullet
text ("Consistent stage, multiple root causes" / the line that begins "May be")
to locate and replace the fragment.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 836a93f8-262e-49dd-b448-31e68eabdf57

📥 Commits

Reviewing files that changed from the base of the PR and between 543a2fc and eaf80d8.

📒 Files selected for processing (4)

• .claude-plugin/marketplace.json
• docs/data.json
• plugins/ci/.claude-plugin/plugin.json
• plugins/ci/commands/analyze-regression.md

[coderabbitai]

⚠️ Potential issue | 🟡 Minor

Fix sentence fragment in consistency classification bullet.

Line 450 starts with “May be…” without a subject, which reads as a fragment. Consider making it a full sentence for clarity.

✏️ Suggested wording

-   - **Consistent stage, multiple root causes**: All runs fail at the same stage but with different error messages (e.g., all `infrastructure` failures but different cloud API errors). May be a single unstable area with varied surface failures.
+   - **Consistent stage, multiple root causes**: All runs fail at the same stage but with different error messages (e.g., all `infrastructure` failures but different cloud API errors). This may indicate a single unstable area with varied surface failures.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	- **Consistent stage, multiple root causes**: All runs fail at the same stage but with different error messages (e.g., all `infrastructure` failures but different cloud API errors). May be a single unstable area with varied surface failures.
	- **Consistent stage, multiple root causes**: All runs fail at the same stage but with different error messages (e.g., all `infrastructure` failures but different cloud API errors). This may indicate a single unstable area with varied surface failures.

🧰 Tools

🪛 LanguageTool

[style] ~450-~450: To form a complete sentence, be sure to include a subject or ‘there’.
Context: ...ilures but different cloud API errors). May be a single unstable area with varied s...

(MISSING_IT_THERE)

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@plugins/ci/commands/analyze-regression.md` at line 450, The bullet under the
heading "Consistent stage, multiple root causes" is a sentence fragment starting
with "May be a single unstable area with varied surface failures"; change it to
a full sentence by adding a subject, e.g., "This may be a single unstable area
with varied surface failures," so the bullet reads: "All runs fail at the same
stage but with different error messages (e.g., all `infrastructure` failures but
different cloud API errors). This may be a single unstable area with varied
surface failures." Use the existing bullet text ("Consistent stage, multiple
root causes" / the line that begins "May be") to locate and replace the
fragment.

[cblecker]

/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cblecker, dgoodwin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [cblecker,dgoodwin]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment