Skip to content
This repository was archived by the owner on Sep 9, 2026. It is now read-only.

[AI] OSAC-1827: Operator treats transient KubeVirt PrintableStatus as fatal provisioning failure - #338

Merged
openshift-merge-bot[bot] merged 2 commits into
mainfrom
osac-jira-ai-issue-solver/OSAC-1827
Jul 14, 2026
Merged

openshift-merge-bot[bot] merged 2 commits into
mainfrom
osac-jira-ai-issue-solver/OSAC-1827

Conversation

@osac-jira-ai-issue-solver

@osac-jira-ai-issue-solver osac-jira-ai-issue-solver Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Resolves OSAC-1827

Summary

  • Fix transient KubeVirt PrintableStatus values (empty string and Unschedulable) being treated as fatal provisioning failures in the ComputeInstance controller
  • Empty PrintableStatus now preserves the current Provisioned condition instead of setting ProvisioningFailed
  • Unschedulable now maps to Starting phase with a Scheduling condition reason instead of Failed/ProvisioningFailed
  • Add new ReasonScheduling condition constant and schedulingWaitMessage() helper

Test plan

  • Updated existing tests for determinePhaseFromPrintableStatus, provisioningErrorMessage, and handleKubeVirtVM to reflect new transient status handling
  • Added new tests for empty PrintableStatus condition preservation, Unschedulable with condition detail extraction, and schedulingWaitMessage helper
  • All 584 controller tests pass with no regressions
  • golangci-lint run --fix reports 0 issues
  • E2E validation: stress test to confirm VMs no longer transiently enter Failed phase during boot

Fixes: OSAC-1827

…l provisioning failure

Co-authored-by: osac-dev-bot <osac-automation@redhat.com>
@osac-jira-ai-issue-solver

osac-jira-ai-issue-solver Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

AI Session Costs

Session Cost
New ticket $3.99
Feedback #1 $0.56
Feedback #2 $0.48
Feedback #3 $4.14
Feedback #4 $2.86
Feedback #5 $1.83
Feedback #6 $2.34
Feedback #7 $2.79
Feedback #8 $2.06
Feedback #9 $2.07
Feedback #10 $2.07
Feedback #11 $2.27
Feedback #12 $2.04
Feedback #13 $2.55
Feedback #14 $2.15
Feedback (15) (error) $2.21
Feedback (14) retry 1 (error) $2.10
Feedback (14) retry 2 (error) $2.67
Feedback (14) retry 3 (error) $1.95
Feedback (15) (error) $1.26
Feedback (14) retry 1 (error) $1.05
Feedback (14) retry 2 (error) $1.00
Feedback (14) retry 3 (error) $0.72
Feedback (15) (error) $0.99
Feedback (14) retry 1 (error) $0.94
Feedback (14) retry 2 (error) $0.87
Feedback (14) retry 3 (error) $0.92
Feedback (15) (error) $1.00
Feedback (14) retry 1 (error) $1.04
Feedback (14) retry 2 (error) $0.83
Feedback (14) retry 3 (error) $0.94
Feedback (15) (error) $1.13
Feedback (14) retry 1 (error) $0.84
Feedback (14) retry 2 (error) $0.91
Feedback (14) retry 3 (error) $0.70
Feedback (20) (error) $1.13
Feedback (20) retry 1 (error) $0.80
Feedback (20) retry 2 (error) $0.83
Feedback (20) retry 3 (error) $0.75
Total $61.78

@openshift-ci-robot

openshift-ci-robot commented Jul 6, 2026

Copy link
Copy Markdown

@osac-jira-ai-issue-solver[bot]: This pull request references OSAC-1827 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Resolves OSAC-1827

Summary

  • Fix transient KubeVirt PrintableStatus values (empty string and Unschedulable) being treated as fatal provisioning failures in the ComputeInstance controller
  • Empty PrintableStatus now preserves the current Provisioned condition instead of setting ProvisioningFailed
  • Unschedulable now maps to Starting phase with a Scheduling condition reason instead of Failed/ProvisioningFailed
  • Add new ReasonScheduling condition constant and schedulingWaitMessage() helper

Test plan

  • Updated existing tests for determinePhaseFromPrintableStatus, provisioningErrorMessage, and handleKubeVirtVM to reflect new transient status handling
  • Added new tests for empty PrintableStatus condition preservation, Unschedulable with condition detail extraction, and schedulingWaitMessage helper
  • All 584 controller tests pass with no regressions
  • golangci-lint run --fix reports 0 issues
  • E2E validation: stress test to confirm VMs no longer transiently enter Failed phase during boot

Fixes: OSAC-1827

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci

openshift-ci Bot commented Jul 6, 2026

Copy link
Copy Markdown

Hi @osac-jira-ai-issue-solver[bot]. Thanks for your PR.

I'm waiting for a osac-project member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@osac-jira-ai-issue-solver

Copy link
Copy Markdown
Contributor Author

In comment, @openshift-ci-robot said:

@osac-jira-ai-issue-solver[bot]: This pull request references OSAC-1827 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Resolves OSAC-1827

Summary

  • Fix transient KubeVirt PrintableStatus values (empty string and Unschedulable) being treated as fatal provisioning failures in the ComputeInstance controller
  • Empty PrintableStatus now preserves the current Provisioned condition instead of setting ProvisioningFailed
  • Unschedulable now maps to Starting phase with a Scheduling condition reason instead of Failed/ProvisioningFailed
  • Add new ReasonScheduling condition constant and schedulingWaitMessage() helper

Test plan

  • Updated existing tests for determinePhaseFromPrintableStatus, provisioningErrorMessage, and handleKubeVirtVM to reflect new transient status handling
  • Added new tests for empty PrintableStatus condition preservation, Unschedulable with condition detail extraction, and schedulingWaitMessage helper
  • All 584 controller tests pass with no regressions
  • golangci-lint run --fix reports 0 issues
  • E2E validation: stress test to confirm VMs no longer transiently enter Failed phase during boot

Fixes: OSAC-1827

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Acknowledged — the Jira target version warning is a ticket configuration issue, not a code change. No code modifications needed.

@osac-jira-ai-issue-solver

Copy link
Copy Markdown
Contributor Author

In comment, @openshift-ci[bot] said:

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: osac-jira-ai-issue-solver[bot]
Once this PR has been reviewed and has the lgtm label, please assign eliorerz for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Acknowledged — approval status notification. Waiting for human reviewer.

@osac-jira-ai-issue-solver

Copy link
Copy Markdown
Contributor Author

In comment, @openshift-ci[bot] said:

Hi @osac-jira-ai-issue-solver[bot]. Thanks for your PR.

I'm waiting for a osac-project member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Acknowledged — ok-to-test gate. Waiting for org member to verify the patch.

@eranco74
eranco74 marked this pull request as ready for review July 6, 2026 19:23
@openshift-ci
openshift-ci Bot requested review from larsks and tzvatot July 6, 2026 19:23
@eranco74

eranco74 commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

/ok-to-test

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown

Labeled ok-to-test. No pending workflow runs found to approve — tests will be authorized on next trigger.

@eranco74 eranco74 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

1. Permanently unschedulable VMs will stay in Starting forever (critical)

The fix correctly handles the transient case (VM booting, scheduler needs a moment), but removes the safety net for the permanent case (VM requests impossible resources like 1TB RAM or a nonexistent GPU type). There's no timeout or escalation — handleKubeVirtVM will keep setting ReasonScheduling on every reconcile, the feedback controller will keep syncing COMPUTE_INSTANCE_STATE_STARTING to fulfillment-service, and the user will see a VM stuck in "Starting" indefinitely with no indication it will never succeed.

Consider adding a timeout or retry counter: after N minutes of continuous Unschedulable, escalate to Failed/ProvisioningFailed so the user knows intervention is needed.

2. Stale doc comments (minor)

Two doc comments still reference ErrorUnschedulable as mapping to Failed:

  • provisioningErrorMessage doc (line ~898): says "ErrorUnschedulable, etc." but the case was removed from the switch
  • determinePhaseFromPrintableStatus function-level doc (line ~974): says "All remaining values (…ErrorUnschedulable…) map to Failed" — the inline comment at the default case was correctly updated, but this one was missed

@eranco74

eranco74 commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Follow-up task created for the timeout concern: OSAC-2309 — Add timeout for permanently unschedulable ComputeInstances.

/lgtm

@openshift-ci openshift-ci Bot added the lgtm label Jul 9, 2026
adalton added a commit to flightctl/jira-ai-issue-solver that referenced this pull request Jul 10, 2026
## Summary

- Fix cost label oscillation observed in production on [osac-operator PR
#338](osac-project/osac-operator#338 (comment)):
labels cycled between "Feedback (15)" and "Feedback (14) retry 1"
because `countFeedbackRounds` excluded error entries from the count
- Remove `!strings.Contains(e.Label, "error")` filter — error entries
with round numbers (e.g., "Feedback (15) (error)") should count as
rounds since `feedbackLabel` always includes the round number
- Follow-up to #44 which introduced the error exclusion based on a code
review concern about bare "Feedback (error)" labels, which can't occur
in practice

## Test plan

- [x] Updated `TestCountFeedbackRounds` — "error entries count as
rounds" replaces "error entries do not count as rounds"
- [x] All executor tests pass with `-race`
- [x] Lint clean

Assisted-by: Claude <noreply@anthropic.com>
@eranco74

Copy link
Copy Markdown
Contributor

/approve

@openshift-ci

openshift-ci Bot commented Jul 12, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: eranco74, osac-jira-ai-issue-solver[bot]

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@eranco74

Copy link
Copy Markdown
Contributor

/lgtm

@openshift-ci openshift-ci Bot added the lgtm label Jul 14, 2026
@openshift-merge-bot
openshift-merge-bot Bot merged commit b39c249 into main Jul 14, 2026
11 checks passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants