CNTRLPLANE-3306: add ExternalOIDCWithUpstreamParity e2e tests - #8287
Conversation
|
Pipeline controller notification For optional jobs, comment This repository is configured in: LGTM mode |
openshift-ci-robot
added
the
jira/valid-reference
|
@ehearne-redhat: This pull request references CNTRLPLANE-3306 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdded unit and e2e tests and updated a test utility to exercise the ExternalOIDCWithUpstreamParity feature gate. Tests and the utility now emit and verify CEL-based claim mappings for username and groups, add CEL claim validation rules (email presence/non-empty and email_verified == true), and add a CEL user validation rule preventing usernames with the Sequence Diagram(s)sequenceDiagram
actor TestRunner
participant HostedClusterSpec as "HostedCluster Spec"
participant Controller as "Control Plane Operator"
participant KAS as "Kube API Server (KAS)"
participant SSR as "SelfSubjectReview"
participant OIDC as "External OIDC Provider"
TestRunner->>HostedClusterSpec: create spec with ExternalOIDCWithUpstreamParity enabled
HostedClusterSpec->>Controller: reconcile spec
Controller->>KAS: produce AuthenticationConfiguration (DiscoveryURL, CEL claim mappings, claim/user validation rules)
KAS->>OIDC: use DiscoveryURL to fetch metadata / validate tokens
OIDC-->>KAS: return token and user claims
KAS->>SSR: evaluate ClaimMappings and UserValidationRules (CEL)
SSR-->>TestRunner: return UserInfo (username, groups) and validation result
🚥 Pre-merge checks | ✅ 10 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (10 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
openshift-ci
Bot
added
do-not-merge/needs-area
do-not-merge/work-in-progress
openshift-ci
Bot
added
area/control-plane-operator
423d933 to
67f1e85
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
control-plane-operator/controllers/hostedcontrolplane/v2/kas/auth_test.go (1)
1964-1996: Tighten this negative case by asserting error reason.This case currently checks only that an error occurs. Please assert an expected error substring (e.g., empty CEL expression) so unrelated failures can’t satisfy the test.
🔧 Suggested pattern
type testCase struct { name string client crclient.Reader expectedAuthenticationConfiguration *AuthenticationConfiguration hcpAuthenticationSpec *configv1.AuthenticationSpec shouldError bool + expectedErrContains string featureGates []featuregate.Feature } // in negative test case: shouldError: true, +expectedErrContains: "expression is not set", // in assertion block: if tc.shouldError { if err == nil { t.Fatal("expected an error to have occurred but got none") } + if tc.expectedErrContains != "" && !strings.Contains(err.Error(), tc.expectedErrContains) { + t.Fatalf("expected error containing %q, got: %v", tc.expectedErrContains, err) + } return }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@control-plane-operator/controllers/hostedcontrolplane/v2/kas/auth_test.go` around lines 1964 - 1996, The test case "claimValidationRule with CEL - empty expression, error" currently only sets shouldError = true; tighten it by adding an expected error substring (e.g., expectedErrorSubstring: "empty" or "CEL expression") and changing the test assertions in the test loop to assert the returned error string contains that substring rather than merely checking shouldError; update the test harness code that reads shouldError (the test loop in auth_test.go) to look for expectedErrorSubstring when an error is returned (and fail if no substring match) so unrelated failures cannot satisfy the test.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@control-plane-operator/controllers/hostedcontrolplane/v2/kas/auth_test.go`:
- Around line 1964-1996: The test case "claimValidationRule with CEL - empty
expression, error" currently only sets shouldError = true; tighten it by adding
an expected error substring (e.g., expectedErrorSubstring: "empty" or "CEL
expression") and changing the test assertions in the test loop to assert the
returned error string contains that substring rather than merely checking
shouldError; update the test harness code that reads shouldError (the test loop
in auth_test.go) to look for expectedErrorSubstring when an error is returned
(and fail if no substring match) so unrelated failures cannot satisfy the test.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: 829cd042-7de5-4a8b-9700-9504081ff9a9
📒 Files selected for processing (3)
control-plane-operator/controllers/hostedcontrolplane/v2/kas/auth_test.gotest/e2e/external_oidc_test.gotest/e2e/util/external_oidc.go
67f1e85 to
92c4197
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #8287 +/- ##
==========================================
+ Coverage 41.66% 42.34% +0.68%
==========================================
Files 758 774 +16
Lines 93929 97612 +3683
==========================================
+ Hits 39135 41335 +2200
- Misses 52046 53393 +1347
- Partials 2748 2884 +136
... and 87 files with indirect coverage changes
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
ehearne-redhat
changed the title
openshift-ci
Bot
removed
the
do-not-merge/work-in-progress
92c4197 to
cfa62cb
Compare
| g.Expect(selfSubjectReview.Status.UserInfo.Username).NotTo(ContainSubstring("@")) | ||
| // equals the actual email prefix | ||
| // e2eutil.ExternalOIDCExtraKeyFoo --> claims.email expression | ||
| emailValues := selfSubjectReview.Status.UserInfo.Extra[e2eutil.ExternalOIDCExtraKeyFoo] |
There was a problem hiding this comment.
@everettraven do we know if this field is feature gate dependent?
So if ExternalOIDCWithUIDAndExtraClaimMappings feature gate was disabled in the future what impact does it have on accessing this field for email verification?
There was a problem hiding this comment.
Since I have ensured the field can accessible on either feature gate enablement this shouldn't be an issue. :)
There was a problem hiding this comment.
That feature gate should already be enabled by default for a couple releases and therefore should never be disabled in the future. We actually need to remove it in the near future.
I don't think we need to check the UserInfo.Extra field as part of these tests though because that is functionality from an entirely different feature.
There was a problem hiding this comment.
Ah OK - thanks for clarifying! In that case we would need to use another mechanism as you rightfully pointed out in #8287 (comment) .
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
control-plane-operator/controllers/hostedcontrolplane/v2/kas/auth_test.go (1)
2220-2221:⚠️ Potential issue | 🟠 MajorUse optional claim access before calling
orValuehere.These two cases still use
claims.groups.orValue([])..., butorValue([])only works on an optional value. The earlier parity case already uses the correct form:claims.?groups.orValue([]). As written, these"shouldError: false"cases are asserting invalid CEL as a supported mapping.Suggested fix
- Expression: "claims.groups.orValue([]).filter(g, g.startsWith('ocp-'))", + Expression: "claims.?groups.orValue([]).filter(g, g.startsWith('ocp-'))",- Expression: "claims.groups.orValue([]).filter(g, g.startsWith('ocp-'))", + Expression: "claims.?groups.orValue([]).filter(g, g.startsWith('ocp-'))",Also applies to: 2258-2260, 2311-2312, 2356-2358
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@control-plane-operator/controllers/hostedcontrolplane/v2/kas/auth_test.go` around lines 2220 - 2221, The test cases are using non-optional access `claims.groups.orValue([])...`, which is invalid because `orValue` requires an optional; update each offending expression to use optional access `claims.?groups.orValue([]).filter(g, g.startsWith('ocp-'))` (i.e., change `claims.groups.orValue([])` to `claims.?groups.orValue([])` in the test vectors). Locate and fix the same pattern occurrences referenced in the file (the similar expressions around the other test cases that currently use `claims.groups.orValue([])`).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@test/e2e/external_oidc_test.go`:
- Around line 124-128: The username assertion currently reads the expected value
from the extra claim key e2eutil.ExternalOIDCExtraKeyFoo which only exists when
the ExternalOIDCWithUIDAndExtraClaimMappings feature is enabled; update the test
in external_oidc_test.go so it first checks whether
ExternalOIDCWithUIDAndExtraClaimMappings is enabled and only then asserts the
username from
selfSubjectReview.Status.UserInfo.Extra[e2eutil.ExternalOIDCExtraKeyFoo],
otherwise derive the expectedUserName from the test’s chosen upstream user (use
the variable that holds the selected test user/email used to create the upstream
identity) and assert that selfSubjectReview.Status.UserInfo.Username equals that
derived username.
---
Duplicate comments:
In `@control-plane-operator/controllers/hostedcontrolplane/v2/kas/auth_test.go`:
- Around line 2220-2221: The test cases are using non-optional access
`claims.groups.orValue([])...`, which is invalid because `orValue` requires an
optional; update each offending expression to use optional access
`claims.?groups.orValue([]).filter(g, g.startsWith('ocp-'))` (i.e., change
`claims.groups.orValue([])` to `claims.?groups.orValue([])` in the test
vectors). Locate and fix the same pattern occurrences referenced in the file
(the similar expressions around the other test cases that currently use
`claims.groups.orValue([])`).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: 0971fdb6-6682-4fe9-b2fd-40b3d5169afd
📒 Files selected for processing (3)
control-plane-operator/controllers/hostedcontrolplane/v2/kas/auth_test.gotest/e2e/external_oidc_test.gotest/e2e/util/external_oidc.go
cfa62cb to
ac11bf0
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
test/e2e/external_oidc_test.go (1)
132-139: Consider verifying actual groups mapping, not just configuration.This subtest only validates that the groups expression is configured but doesn't verify the actual
selfSubjectReview.Status.UserInfo.Groupscontains expected values. If the Keycloak test users have groups configured, asserting their presence would strengthen this test.💡 Suggested enhancement
t.Run("[OCPFeatureGate:ExternalOIDCWithUpstreamParity] Test CEL groups expression mapping", func(t *testing.T) { g := NewWithT(t) t.Logf("begin to test CEL groups expression mapping") // Groups expression uses: has(claims.groups) && type(claims.groups) == list ? claims.groups : [] // If the token has groups, they should be present without prefix (no prefix in CEL expression) g.Expect(hostedCluster.Spec.Configuration.Authentication.OIDCProviders[0].ClaimMappings.Groups.Expression).NotTo(BeEmpty()) + // Verify groups are actually present in the response if user has groups configured + g.Expect(selfSubjectReview.Status.UserInfo.Groups).NotTo(BeEmpty()) t.Logf("CEL groups expression configured: %s", hostedCluster.Spec.Configuration.Authentication.OIDCProviders[0].ClaimMappings.Groups.Expression) })🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/e2e/external_oidc_test.go` around lines 132 - 139, The test currently only asserts the CEL groups expression is configured; instead, after confirming hostedCluster.Spec.Configuration.Authentication.OIDCProviders[0].ClaimMappings.Groups.Expression is not empty, call the same code path that performs a self-subject review (inspect the SelfSubjectReview object used in this test) and assert that selfSubjectReview.Status.UserInfo.Groups contains the expected group names for the Keycloak test user(s). Locate the subtest block (the t.Run with label "Test CEL groups expression mapping") and add an assertion using the existing test helper that retrieves the SelfSubjectReview (or create one via the API client used elsewhere in the test suite) to verify the groups slice is non-empty and includes the known group(s) for the test account.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@control-plane-operator/controllers/hostedcontrolplane/v2/kas/auth_test.go`:
- Around line 2219-2221: The CEL expressions in test fixtures using
PrefixedClaimOrExpression.Expression wrongly call orValue() on possibly-missing
fields (e.g. "claims.groups.orValue([])") which requires optional access; update
each Expression string to use the optional operator before the field (e.g.
"claims.?groups.orValue([])") and do the same for roles (use
"claims.?roles.orValue([])"); apply this change to all occurrences of
PrefixedClaimOrExpression where Expression references claims.groups or
claims.roles (the instances noted around the test cases at the four groups lines
and the two roles lines).
---
Nitpick comments:
In `@test/e2e/external_oidc_test.go`:
- Around line 132-139: The test currently only asserts the CEL groups expression
is configured; instead, after confirming
hostedCluster.Spec.Configuration.Authentication.OIDCProviders[0].ClaimMappings.Groups.Expression
is not empty, call the same code path that performs a self-subject review
(inspect the SelfSubjectReview object used in this test) and assert that
selfSubjectReview.Status.UserInfo.Groups contains the expected group names for
the Keycloak test user(s). Locate the subtest block (the t.Run with label "Test
CEL groups expression mapping") and add an assertion using the existing test
helper that retrieves the SelfSubjectReview (or create one via the API client
used elsewhere in the test suite) to verify the groups slice is non-empty and
includes the known group(s) for the test account.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: 66e94f4c-86ef-48ad-9645-5acece62f594
📒 Files selected for processing (3)
control-plane-operator/controllers/hostedcontrolplane/v2/kas/auth_test.gotest/e2e/external_oidc_test.gotest/e2e/util/external_oidc.go
🚧 Files skipped from review as they are similar to previous changes (1)
- test/e2e/util/external_oidc.go
ac11bf0 to
842e4e3
Compare
There was a problem hiding this comment.
🧹 Nitpick comments (2)
test/e2e/external_oidc_test.go (2)
132-139: Outdated comment contradicts actual implementation.The comment on line 135 states:
// Groups expression uses: has(claims.groups) && type(claims.groups) == list ? claims.groups : []However, the actual expression set in
test/e2e/util/external_oidc.goline 172 isclaims.?groups.orValue([]). Please update the comment to reflect the correct expression.Additionally, this test only verifies configuration was applied (checking
Expressionis non-empty), not actual behavior. Consider strengthening the test by verifyingselfSubjectReview.Status.UserInfo.Groupsmatches expected values, similar to how username is verified.📝 Proposed fix for the comment
- // Groups expression uses: has(claims.groups) && type(claims.groups) == list ? claims.groups : [] - // If the token has groups, they should be present without prefix (no prefix in CEL expression) + // Groups expression uses: claims.?groups.orValue([]) + // Groups are returned as-is when present, or empty list when absent🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/e2e/external_oidc_test.go` around lines 132 - 139, The comment inside the Test CEL groups expression mapping test is stale and should be updated to match the actual CEL expression used (claims.?groups.orValue([])) — locate the test (t.Run "[OCPFeatureGate:ExternalOIDCWithUpstreamParity] Test CEL groups expression mapping") and replace the old comment with the correct expression string referencing hostedCluster.Spec.Configuration.Authentication.OIDCProviders[0].ClaimMappings.Groups.Expression; additionally, strengthen the test by fetching the SelfSubjectReview and asserting selfSubjectReview.Status.UserInfo.Groups contains the expected group values (similar to the existing username assertion) to verify behavior, not just configuration presence.
141-156: Test assertions are tightly coupled to implementation details.The tests hard-code the exact CEL expressions being verified. If the implementation changes these expressions, the tests will fail even if the new expressions are functionally equivalent.
Consider extracting these to constants shared between
external_oidc.goand this test file, or using more behavioral assertions where possible.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/e2e/external_oidc_test.go` around lines 141 - 156, The test hard-codes exact CEL strings from ClaimValidationRules (hostedCluster.Spec.Configuration.Authentication.OIDCProviders[0].ClaimValidationRules) which couples the test to implementation; extract the two CEL expressions into exported constants (e.g., ClaimExprEmailExists and ClaimExprEmailVerified) in the package that defines external_oidc.go and reference those constants in this test, or replace the exact-string assertions with behavioral checks (e.g., assert the rule Type is TokenValidationRuleTypeCEL and the CEL expression contains/semantically matches the expected predicate) so the test verifies intent not exact syntax.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@test/e2e/external_oidc_test.go`:
- Around line 132-139: The comment inside the Test CEL groups expression mapping
test is stale and should be updated to match the actual CEL expression used
(claims.?groups.orValue([])) — locate the test (t.Run
"[OCPFeatureGate:ExternalOIDCWithUpstreamParity] Test CEL groups expression
mapping") and replace the old comment with the correct expression string
referencing
hostedCluster.Spec.Configuration.Authentication.OIDCProviders[0].ClaimMappings.Groups.Expression;
additionally, strengthen the test by fetching the SelfSubjectReview and
asserting selfSubjectReview.Status.UserInfo.Groups contains the expected group
values (similar to the existing username assertion) to verify behavior, not just
configuration presence.
- Around line 141-156: The test hard-codes exact CEL strings from
ClaimValidationRules
(hostedCluster.Spec.Configuration.Authentication.OIDCProviders[0].ClaimValidationRules)
which couples the test to implementation; extract the two CEL expressions into
exported constants (e.g., ClaimExprEmailExists and ClaimExprEmailVerified) in
the package that defines external_oidc.go and reference those constants in this
test, or replace the exact-string assertions with behavioral checks (e.g.,
assert the rule Type is TokenValidationRuleTypeCEL and the CEL expression
contains/semantically matches the expected predicate) so the test verifies
intent not exact syntax.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: 98f452a1-8817-4b7c-898f-55d646f9551c
📒 Files selected for processing (3)
control-plane-operator/controllers/hostedcontrolplane/v2/kas/auth_test.gotest/e2e/external_oidc_test.gotest/e2e/util/external_oidc.go
842e4e3 to
7870823
Compare
| g.Expect(selfSubjectReview.Status.UserInfo.Username).NotTo(ContainSubstring("@")) | ||
| // equals the actual email prefix | ||
| // e2eutil.ExternalOIDCExtraKeyFoo --> claims.email expression | ||
| emailValues := selfSubjectReview.Status.UserInfo.Extra[e2eutil.ExternalOIDCExtraKeyFoo] |
There was a problem hiding this comment.
That feature gate should already be enabled by default for a couple releases and therefore should never be disabled in the future. We actually need to remove it in the near future.
I don't think we need to check the UserInfo.Extra field as part of these tests though because that is functionality from an entirely different feature.
|
/uncc |
|
/retest |
|
@ehearne-redhat This looks like it is ready for review from a hypershift approver to try and grab the |
enxebre
added
the
tide/merge-method-squash
…eded This change patches auth config on the fly so we can improve costs and run time by using one cluster. It follows conventions from `test/e2e/v2/lifecycle/azure.gotest/e2e/v2/lifecycle/azure.go` from `postCreateExternalOIDC()` .
This change corrects the test names of introducted tests in `control-plane-operator/controllers/hostedcontrolplane/v2/ kas/auth_test.go` by renaming the tests following the `When... it should... ` convention. Example: When CEL expression for username and groups with filtering omits prefix and prefixPolicy, it should generate valid authentication configuration.
|
/test e2e-azure-aks-external-oidc-techpreview |
This change fixes Eventually() panic error, by adding gomega to it --> g.Eventually() .
|
/test e2e-azure-aks-external-oidc-techpreview |
|
/test e2e-azure-aks-external-oidc-techpreview |
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: ehearne-redhat, enxebre, everettraven The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
Bot
added
the
approved
|
/verified by e2e-aws-external-oidc-techpreview, e2e-azure-aks-external-oidc-techpreview |
openshift-ci-robot
added
the
verified
|
@ehearne-redhat: This PR has been marked as verified by DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/lgtm |
|
Scheduling tests matching the |
|
/test e2e-aks-4-22 |
|
/test e2e-aws-4-22 |
|
@ehearne-redhat: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
Now I have the complete picture. Here is the final report: Test Failure Analysis CompleteJob Information
Test Failure AnalysisErrorSummaryThe Root CauseThe PR adds 5 files but 4 are excluded from Codecov instrumentation:
The only tracked file is import (
cpofeaturegate "github.com/openshift/hypershift/control-plane-operator/featuregates"
)
// In NewStartCommand():
// Configure feature set from CPO (needed to propagate feature gates like TechPreviewNoUpgrade)
cpofeaturegate.ConfigureFeatureSet(featureSet)These 3 lines live inside Since these 3 lines are the only tracked lines in the entire diff, the patch coverage computes to 0.00%, failing the default Codecov patch threshold of 41.66%. Recommendations
Evidence
|
6 participants
What this PR does / why we need it:
This change:
Adds extensive testing for the ExternalOIDCWithUpstreamParity feature gate in
control-plane-operator/controllers/hostedcontrolplane/v2/kas/auth_test.go.Tests the feature gate in a HyperShift cluster in
test/e2e/external_oidc_test.go.Provides basic auth config testing of the feature gate in
test/e2e/util/external_oidc.go.Configures the control plane operator feature sets in hypershift-operator.
This change is required as hypershift-operator re-uses control-plane-operator
auth config validation code, which checks specifically for control-plane-operator
feature set enablement.
Without this change, if we try to enable feature set such as TechPreviewNoUpgrade,
validation will fail as hypershift-operator does not configure control-plane-operator
feature sets.
Refactors existing tests to use a specified AuthConfig so that when the feature graduates it is easier to add the appropriate fields.
new users/groups as needed for better testing coverage.
This change will allow us to test feature gates behind TechPreviewNoUpgrade and
others that are present in control-plane-operator, throughout HyperShift.
This should allow us to progress in promotion of the feature from TechPreview to GA.
Which issue(s) this PR fixes:
https://redhat.atlassian.net/browse/CNTRLPLANE-3306.
Allows us to progress in promotion of the ExternalOIDCWithUpstreamParity feature from TechPreview to GA.
Special notes for your reviewer:
Checklist:
Summary by CodeRabbit
New Features
Tests