feat(prow-job): add JIRA export feature for test failure analysis - #321
openshift-merge-bot[bot] merged 2 commits into
Conversation
openshift-ci
Bot
added
the
needs-ok-to-test
|
Hi @wangke19. Thanks for your PR. I'm waiting for a openshift-eng member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughBumps prow-job plugin to 0.0.5 and adds an optional Changes
Sequence Diagram(s)sequenceDiagram
participant User as User
participant Plugin as Prow-Job Plugin
participant Store as Artifact Store
participant Analyzer as Must-Gather Analyzer
participant FS as Filesystem
User->>Plugin: /prow-job:analyze-test-failure <url> <test> [--fast] [--export-jira]
Plugin->>Store: locate/download logs, intervals, must-gather
Store-->>Plugin: artifacts (or not found)
alt must-gather available
Plugin->>Analyzer: run cluster/must-gather analysis
Analyzer-->>Plugin: analysis outputs
end
Plugin->>Plugin: correlate logs, intervals, and cluster analysis
alt EXPORT_JIRA=true
Plugin->>FS: write .work/.../analysis-jira.txt (JIRA wiki markup)
else
Plugin->>FS: write .work/.../analysis.md (Markdown)
end
Plugin->>User: display summary and artifact locations
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes 🚥 Pre-merge checks | ✅ 8✅ Passed checks (8 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches🧪 Generate unit tests (beta)
Comment |
1eaa76a to
796e5b7
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In `@plugins/prow-job/skills/prow-job-analyze-test-failure/SKILL.md`:
- Around line 983-996: Update the summary in SKILL.md that currently lists only
".work/prow-job-analyze-test-failure/{build_id}/must-gather/logs/" so it
correctly shows HyperShift must-gather paths for unified/dual layouts: detect
HyperShift context (or document both options) and display mgmt and hosted
must-gather paths conditionally (e.g.,
".work/.../{build_id}/must-gather/mgmt/logs/" and
".work/.../{build_id}/must-gather/hosted/logs/") or list both when unsure;
adjust the user-facing text around the Reports/Artifacts block and preserve
existing placeholders like {build_id} and the --export-jira note to ensure paths
are accurate for HyperShift setups.
- Around line 372-398: When OUTPUT_DIR is missing, short-circuit immediately
instead of letting later code reference unified-path variables: after the
existing rm -rf "$TMP_EXTRACT" add a short-circuit that clears relevant
variables and exits the current flow (e.g. set HAS_HOSTED_CLUSTER="false" and
unset HOSTED_DIR and OUTPUT_DIR, then either return 0 if inside a function or
exit 0 if in the top-level script) so subsequent uses of OUTPUT_DIR, HOSTED_DIR
or HAS_HOSTED_CLUSTER (and moves into
.work/prow-job-analyze-test-failure/{build_id}/...) cannot run.
f990f97 to
9270011
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@plugins/prow-job/skills/prow-job-analyze-test-failure/SKILL.md`:
- Around line 895-920: In SKILL.md under "Step 5.5: Generate JIRA-Formatted
Output" update the fenced code block that echoes the JIRA file path so it
includes a language specifier (e.g., change the triple backtick fence before the
echo to ```bash or ```text) to match other examples; locate the echo snippet
(the line containing echo "JIRA-formatted analysis saved to:
.work/prow-job-analyze-test-failure/{build_id}/analysis-jira.txt") and add the
language tag on the opening fence while leaving the closing fence intact.
9c193de to
a514577
Compare
📊 Test Analysis Results - Both Output FormatsI've generated analysis for the test failure in both formats to demonstrate the JIRA export feature. Test analyzed: 📝 Format 1: Markdown Output (analysis.md)Click to view Markdown format# Prow Job Test Failure Analysis
## Job Information
- **Job Name**: pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-ipv6
- **Build ID**: 2021677521346498560
- **Job URL**: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/30585/pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-ipv6/2021677521346498560
- **PR**: openshift/origin#30585 (OCPBUGS-60853: cert tests: mark TLS registry test as informing)
- **Status**: FAILED
- **Duration**: ~3h 59m
## Failed Test
**Test Name**: `[Monitor:legacy-test-framework-invariants-pathological][sig-arch] events should not repeat pathologically`
## Failure Summary
The test failed because **6 pathological event patterns** were detected, meaning these events repeated more than the allowed threshold (typically 20 times).
## Pathological Events Detected
### 1-4. Container Probe Test Failures (4 instances)
- Liveness probe redirects (23 occurrences each)
- Startup probe failures (23 occurrences each)
- **Analysis**: E2E container probe tests intentionally creating unhealthy pods
### 5-6. CNV Must-Gather Image Pull Failures (2 instances)
- Failed to pull `registry.redhat.io/container-native-virtualization/cnv-must-gather-rhel9:v4.15.0`
- 18 occurrences each on node `master-1.ostest.test.metalkube.org`
- **Time**: 2026-02-11 23:28:46Z to 23:41:02Z
## Verdict
**Assessment**: This is likely a **flaky test failure** unrelated to the PR changes.
**Rationale**:
1. Container probe events are expected test behaviors that exceeded threshold
2. Must-gather image pull failures are environmental issues during cleanup
3. PR only modifies certificate test annotations, unrelated to event monitoring
## Recommendations
- **Action**: `/retest` - Safe to rerun
- **Reason**: Failure unrelated to PR changes, appears to be environmental flake🎫 Format 2: JIRA OCPBUGS Output (analysis-jira.txt)Click to view JIRA OCPBUGS format (ready for copy-paste)✅ Comparison: Key Differences
💡 Usage TipsMarkdown format (
JIRA OCPBUGS format (
Both formats contain the same analysis, just formatted for different platforms! 🎯 |
mgencur
left a comment
mgencur
left a comment
There was a problem hiding this comment.
I love it. Thanks!
That's exactly how I imagined this. One tip for the future (not now!): there is a claude command in this repository under plugins/jira/commands/create.md . We could add this as another option, something like: Label: "Yes - File a JIRA using the jira:create command"
It would then feed the information into the jira command and file the bug directly. But that's an enhancement for the future. I didn't suggest it earlier because I just got this idea now.
a514577 to
6dceb60
Compare
openshift-merge-robot
added
the
needs-rebase
This commit enhances the /prow-job:analyze-test-failure command to support
JIRA export using the OCPBUGS issue template format.
Changes:
- Remove --export-jira command-line flag (UX improvement)
- Ask user at the end if they want JIRA export (better workflow)
- Use OCPBUGS format template for JIRA output:
- Description of problem
- Version-Release number of selected component
- How reproducible
- Steps to Reproduce
- Actual results
- Expected results
- Additional info
- Add bug fix for OUTPUT_DIR error handling (prevent undefined variable usage)
- Display both file locations (Markdown and JIRA) for easy review
Benefits:
- More natural workflow: analyze first, then decide on export format
- OCPBUGS format allows direct copy-paste into JIRA issues
- Follows Red Hat/OpenShift bug reporting conventions
- Users don't need to know about JIRA export upfront
Example workflow:
1. Run: /prow-job:analyze-test-failure <url> <test-name>
2. Review analysis
3. Prompted: "Would you like to export to JIRA format?"
4. Choose "Yes - Export to JIRA (OCPBUGS format)" or "No"
5. Get appropriately formatted output
Output files:
- .work/prow-job-analyze-test-failure/{build_id}/analysis.md (always)
- .work/prow-job-analyze-test-failure/{build_id}/analysis-jira.txt (if requested)
6dceb60 to
e512817
Compare
openshift-merge-robot
removed
the
needs-rebase
|
/lgtm |
|
@mgencur: changing LGTM is restricted to collaborators DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
/ok-to-test |
openshift-ci
Bot
added
ok-to-test
|
/lgtm |
| **JIRA export (prompted at end)**: | ||
| - After analysis completes, you'll be asked if you want to export to JIRA format | ||
| - If yes, generates JIRA-formatted output using OCPBUGS template | ||
| - Perfect for copy-pasting directly into JIRA OCPBUGS issues |
There was a problem hiding this comment.
Won't hold up this PR, but you could then just turn around and use the jira plugin/skills to make the OCPBUGS instead of copying into Jira manually
There was a problem hiding this comment.
Great idea! We actually discussed this exact enhancement — adding a third option like "Yes - File a JIRA using the jira:create command" that would feed the analysis data directly into /jira:create to file the bug automatically. Noted as a follow-up enhancement.
| done | ||
|
|
||
| # Move hosted cluster data (hostedcluster-* subdirectory) | ||
| if [ "$HAS_HOSTED_CLUSTER" = "true" ]; then |
There was a problem hiding this comment.
Where is this getting set to true prior to this? I'm missing it.
There was a problem hiding this comment.
HAS_HOSTED_CLUSTER is set earlier in Step 3 (pattern detection), around lines 193-216 of SKILL.md. When the code identifies the archive pattern:
- Line 193 (Pattern 1 - unified):
HAS_HOSTED_CLUSTER=$(tar -tf "$TMP_CHECK" 2>/dev/null | grep -q "hostedcluster-" && echo "true" || echo "false") - Line 205 (Pattern 2 - dual): Same tar check on the hypershift-dump archive
- Lines 211/216 (Pattern 3/none):
HAS_HOSTED_CLUSTER=false
So by the time execution reaches this block (Pattern 1 extraction at line 376), the variable was already determined during archive detection. This code block at line 376 is pre-existing from PR #294 — this PR only restructured it into an if/else to properly short-circuit when OUTPUT_DIR is missing.
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: bryan-cox, mgencur, wangke19 The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
Bot
added
the
approved
|
The new feature has been tested with two real Prow jobs and posted the results as a PR comment. Both Markdown and JIRA (OCPBUGS format) outputs were generated and verified:
Both produced |
5 participants
Summary
This PR adds JIRA export capability to
/prow-job:analyze-test-failureusing the OCPBUGS issue template format.Changes
1. JIRA Export with User Prompt
After analysis completes, the user is prompted:
If the user chooses "Yes", a JIRA-formatted file is generated using the OCPBUGS template.
2. OCPBUGS Format Template
The JIRA export uses the standard Red Hat OCPBUGS bug template:
Benefits:
3. Bug Fix: OUTPUT_DIR Error Handling
Issue: When must-gather extraction fails to find OUTPUT_DIR, code continued to reference undefined variables.
Fix:
rm -rf "$TMP_EXTRACT"HAS_HOSTED_CLUSTER="false",unset HOSTED_DIR,unset OUTPUT_DIRelseblock4. Output Display
When JIRA export is chosen:
When JIRA export is skipped:
Files Changed
plugins/prow-job/commands/analyze-test-failure.mdplugins/prow-job/skills/prow-job-analyze-test-failure/SKILL.mdplugins/prow-job/.claude-plugin/plugin.json.claude-plugin/marketplace.jsonPLUGINS.mddocs/data.jsonTotal: 6 files, 116 insertions, 25 deletions
Commit Structure
Following repository guidelines:
Commit 1 (35a4a7f): Infrastructure code
Commit 2 (a514577): Generated files
Testing
make lintpassesmake updateexecutedReady for review! 🎉