Skip to content

OCPNODE-3877: add normal grace period allow non-drain updates to complete #30480

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
openshift-merge-bot merged 1 commit into from
Nov 15, 2025
Merged

OCPNODE-3877: add normal grace period allow non-drain updates to complete #30480

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions test/e2e/upgrade/upgrade.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -619,6 +619,9 @@ func clusterUpgrade(f *framework.Framework, c configv1client.Interface, dc dynam
}

var errMasterUpdating error
var violationStartTime *time.Time
const gracePeriod = 2 * time.Minute

if err := disruption.RecordJUnit(
f,
"[sig-mco] Machine config pools complete upgrade",
Expand Down Expand Up @@ -657,6 +660,15 @@ func clusterUpgrade(f *framework.Framework, c configv1client.Interface, dc dynam

// Invariant: when CVO reaches level, MCO is required to have rolled out control plane updates
if p.GetName() == "master" && requiresUpdate && !allNodesReady && errMasterUpdating == nil {
if violationStartTime == nil {
now := time.Now()
violationStartTime = &now
framework.Logf("Invariant violation detected: master pool requires update but nodes not ready. Waiting up to %v for non-draining updates to complete", gracePeriod)
return false, nil
}
if time.Since(*violationStartTime) <= gracePeriod {
return false, nil
}
errMasterUpdating = fmt.Errorf("the %q pool should be updated before the CVO reports available at the new version", p.GetName())
framework.Logf("Invariant violation detected: %s", errMasterUpdating)
}
Expand Down