Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 48 additions & 2 deletions docs/content/how-to/ci/v2-testing/writing-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ func DeploymentGenerationTest(getTestCtx internal.TestContextGetter) {
})
}

var _ = Describe("Control Plane Workloads", Label("control-plane-workloads"), func() {
var _ = Describe("[sig-hypershift][Jira:Hypershift][Feature:ControlPlaneWorkloads] Control Plane Workloads", Label("control-plane-workloads"), func() {
var testCtx *internal.TestContext
BeforeEach(func() {
testCtx = internal.GetTestContext()
Expand Down Expand Up @@ -162,6 +162,52 @@ Sequential: []SequentialGroup{
!!! tip "Adding a test with an existing label"
If your test uses a label already in a filter expression (e.g., `hosted-cluster-health`), it runs automatically in the appropriate CI jobs. If you introduce a new label, you must add it to existing filter expressions in the TestMatrix configuration in the hypershift repository (not the release repository).

## Sippy/CR test name annotations

All test name strings must include annotations for [Sippy Component Readiness](https://sippy.dptools.openshift.org) (CR) mapping. These are parsed from the full Ginkgo test path and enable automatic Jira component assignment and per-feature regression tracking.

**Required annotations:**

1. **`[sig-hypershift][Jira:Hypershift]`** — on every top-level `Describe` block. Maps all tests to the Hypershift Jira component.
2. **`[Feature:XYZ]`** — maps to a specific feature/capability. Placement depends on the file:
- **On the `Describe`** when the entire file tests one cohesive feature (most files).
- **On individual `Context`/`When` blocks** when a file covers multiple distinct capabilities.

Feature names use PascalCase with no spaces (e.g., `BackupRestore`, `AzurePrivateLink`, `NodePoolLifecycle`).

**Single-feature file:**

```go
var _ = Describe("[sig-hypershift][Jira:Hypershift][Feature:Health] Hosted Cluster Health", Label("hosted-cluster-health"), func() {
// all tests in this file map to Feature:Health
})
```

**Multi-feature file** (e.g., platform-specific files covering multiple capabilities):

```go
var _ = Describe("[sig-hypershift][Jira:Hypershift] Hosted Cluster Azure", Label("hosted-cluster-azure"), func() {
// Jira component set at Describe level, Feature set per Context
})

Context("[Feature:AzureWorkloadIdentity] Azure Public Cluster", Label("Azure"), func() {
It("should mutate pods with workload identity federated credentials", func() { ... })
})

Context("[Feature:AzurePrivateLink] Azure Private Topology", Label("Azure"), func() {
It("should create AzurePrivateLinkService CR with PLS alias", func() { ... })
})
```

!!! warning "Do not rename tests after Sippy import"
Renaming tests after Sippy imports them loses historical data and requires manual re-mapping. Add annotations before tests are imported.

Check existing Feature names before adding new ones:

```bash
grep -r '\[Feature:' test/e2e/v2/tests/
```

## Platform guards

Use `Skip` in `BeforeEach` to skip tests when platform preconditions are not met:
Expand Down Expand Up @@ -266,7 +312,7 @@ Lifecycle tests may modify cluster state but must:
- Check `IsNotFound()` in cleanup to handle missing resources gracefully

```go
var _ = Describe("NodePool Lifecycle", Label("lifecycle", "nodepool-lifecycle"), func() {
var _ = Describe("[sig-hypershift][Jira:Hypershift][Feature:NodePoolLifecycle] NodePool Lifecycle", Label("lifecycle", "nodepool-lifecycle"), func() {
var originalReplicas int32
BeforeEach(func() {
// Capture original state
Expand Down
52 changes: 49 additions & 3 deletions docs/content/reference/aggregated-docs.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

43 changes: 42 additions & 1 deletion test/e2e/v2/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ func RegisterControlPlaneWorkloadsTests(getTestCtx internal.TestContextGetter) {
// ...
}

var _ = Describe("Control Plane Workloads", Label("control-plane-workloads"), func() {
var _ = Describe("[sig-hypershift][Jira:Hypershift][Feature:ControlPlaneWorkloads] Control Plane Workloads", Label("control-plane-workloads"), func() {
// BeforeEach gets TestContext, then calls RegisterControlPlaneWorkloadsTests
})
```
Expand Down Expand Up @@ -203,6 +203,47 @@ g.Expect(ic.Spec.EndpointPublishingStrategy.Type).To(Equal(operatorv1.LoadBalanc
g.Expect(ic.Spec.EndpointPublishingStrategy.Type).To(Equal(expectedStrategy.Type))
```

### 19. Sippy/CR Test Name Annotations

All test name strings must include annotations for Sippy Component Readiness (CR) mapping. These annotations are parsed from the full Ginkgo test path and enable automatic Jira component assignment and per-feature regression tracking.

**Required annotations:**

1. **`[sig-hypershift][Jira:Hypershift]`** — on every top-level `Describe` block. Maps all tests to the Hypershift Jira component.

2. **`[Feature:XYZ]`** — maps to a specific feature/capability. Placement depends on the file:
- **On the `Describe`** when the entire file tests one cohesive feature (most files).
- **On individual `Context`/`When` blocks** when a file covers multiple distinct capabilities.

Feature names use PascalCase with no spaces (e.g., `BackupRestore`, `AzurePrivateLink`, `NodePoolLifecycle`).

**Single-feature file example:**

```go
var _ = Describe("[sig-hypershift][Jira:Hypershift][Feature:Health] Hosted Cluster Health", Label("hosted-cluster-health"), func() {
// all tests in this file map to Feature:Health
})
```

**Multi-feature file example:**

```go
var _ = Describe("[sig-hypershift][Jira:Hypershift] Hosted Cluster Azure", Label("hosted-cluster-azure"), func() {
// Jira component set at Describe level, Feature set per Context
})

// In the registration function:
Context("[Feature:AzureWorkloadIdentity] Azure Public Cluster", Label("Azure", "self-managed-azure-public"), func() {
It("should mutate pods with workload identity federated credentials", func() { ... })
})

Context("[Feature:AzurePrivateLink] Azure Private Topology", Label("Azure", "self-managed-azure-private"), func() {
It("should create AzurePrivateLinkService CR with PLS alias", func() { ... })
})
```

**When adding new test files:** Choose a Feature name that maps to a distinct capability. Check existing Feature names in the codebase (`grep -r '\[Feature:' test/e2e/v2/tests/`) to avoid duplicates.

## Expanding v2

When adding new test areas:
Expand Down
4 changes: 2 additions & 2 deletions test/e2e/v2/tests/backup_restore_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ var backupRestorePlatforms = map[hyperv1.PlatformType]backupRestorePlatformConfi
},
}

var _ = Describe("BackupRestore", Label("backup-restore"), Ordered, Serial, func() {
var _ = Describe("[sig-hypershift][Jira:Hypershift][Feature:BackupRestore] BackupRestore", Label("backup-restore"), Ordered, Serial, func() {

var (
platformCfg backupRestorePlatformConfig
Expand Down Expand Up @@ -380,7 +380,7 @@ func validatePostRestoreControlPlane(testCtx *internal.TestContext, excludeWorkl
}
}

var _ = Describe("BackupRestoreEtcdSnapshot", Label("backup-restore", "etcd-snapshot"), Ordered, Serial, func() {
var _ = Describe("[sig-hypershift][Jira:Hypershift][Feature:EtcdSnapshot] BackupRestoreEtcdSnapshot", Label("backup-restore", "etcd-snapshot"), Ordered, Serial, func() {

var (
platformCfg backupRestorePlatformConfig
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/v2/tests/control_plane_infrastructure_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ func RegisterControlPlaneInfrastructureTests(getTestCtx internal.TestContextGett
InfrastructureResourceRequestsTest(getTestCtx)
}

var _ = Describe("Control Plane Infrastructure Workloads", Label("control-plane-workloads"), func() {
var _ = Describe("[sig-hypershift][Jira:Hypershift][Feature:ControlPlaneInfrastructure] Control Plane Infrastructure Workloads", Label("control-plane-workloads"), func() {
var testCtx *internal.TestContext
BeforeEach(func() {
testCtx = internal.GetTestContext()
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/v2/tests/control_plane_upgrade_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ func RegisterControlPlaneUpgradeTests(getTestCtx internal.TestContextGetter) {
ControlPlaneUpgradeTest(getTestCtx)
}

var _ = Describe("Control Plane Upgrade", Label("lifecycle", "control-plane-upgrade"), func() {
var _ = Describe("[sig-hypershift][Jira:Hypershift][Feature:ControlPlaneUpgrade] Control Plane Upgrade", Label("lifecycle", "control-plane-upgrade"), func() {
var testCtx *internal.TestContext

BeforeEach(func() {
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/v2/tests/control_plane_workloads_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1065,7 +1065,7 @@ func RegisterControlPlaneWorkloadsTests(getTestCtx internal.TestContextGetter) {
SecurityContextUIDTest(getTestCtx)
}

var _ = Describe("Control Plane Workloads", Label("control-plane-workloads"), func() {
var _ = Describe("[sig-hypershift][Jira:Hypershift][Feature:ControlPlaneWorkloads] Control Plane Workloads", Label("control-plane-workloads"), func() {
var (
testCtx *internal.TestContext
)
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/v2/tests/etcd_chaos_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ func RegisterEtcdChaosTests(getTestCtx internal.TestContextGetter) {
EtcdMissingMemberRecoveryTest(getTestCtx)
}

var _ = Describe("Etcd Chaos", Label("lifecycle", "etcd-chaos"), Ordered, func() {
var _ = Describe("[sig-hypershift][Jira:Hypershift][Feature:EtcdResilience] Etcd Chaos", Label("lifecycle", "etcd-chaos"), Ordered, func() {
var testCtx *internal.TestContext

BeforeAll(func() {
Expand Down
6 changes: 3 additions & 3 deletions test/e2e/v2/tests/hosted_cluster_aws_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ func RegisterHostedClusterAWSTests(getTestCtx internal.TestContextGetter) {
}

func EnsureDefaultSecurityGroupTagsTest(getTestCtx internal.TestContextGetter) {
When("a day-2 resource tag is added to the HostedCluster spec", func() {
When("[Feature:AWSSecurityGroups] a day-2 resource tag is added to the HostedCluster spec", func() {
It("should apply the tag to the default worker security group via AWS API", Label("AWS"), func() {
tc := getTestCtx()
if e2eutil.IsLessThan(e2eutil.Version420) {
Expand Down Expand Up @@ -127,7 +127,7 @@ func EnsureDefaultSecurityGroupTagsTest(getTestCtx internal.TestContextGetter) {
}

func AWSCCMWithCustomizationsTest(getTestCtx internal.TestContextGetter) {
Context("AWS CCM NLB Security Group", Label("AWS", "CCM"), func() {
Context("[Feature:AWSNLB] AWS CCM NLB Security Group", Label("AWS", "CCM"), func() {
BeforeEach(func() {
tc := getTestCtx()
if e2eutil.IsLessThan(e2eutil.Version423) {
Expand Down Expand Up @@ -275,7 +275,7 @@ func extractLBNameFromHostname(hostname string) string {
return firstLabel[:lastHyphen]
}

var _ = Describe("Hosted Cluster AWS", Label("lifecycle", "hosted-cluster-aws"), func() {
var _ = Describe("[sig-hypershift][Jira:Hypershift] Hosted Cluster AWS", Label("lifecycle", "hosted-cluster-aws"), func() {
var testCtx *internal.TestContext

BeforeEach(func() {
Expand Down
8 changes: 4 additions & 4 deletions test/e2e/v2/tests/hosted_cluster_azure_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ import (
// These tests verify workload identity, KAS allowed CIDRs, and ingress operator configuration
// on Azure platform clusters.
func AzurePublicClusterTest(getTestCtx internal.TestContextGetter) {
Context("Azure Public Cluster", Label("Azure", "self-managed-azure-public"), func() {
Context("[Feature:AzureWorkloadIdentity] Azure Public Cluster", Label("Azure", "self-managed-azure-public"), func() {
BeforeEach(func() {
testCtx := getTestCtx()
hc := testCtx.GetHostedCluster()
Expand Down Expand Up @@ -88,7 +88,7 @@ func AzurePublicClusterTest(getTestCtx internal.TestContextGetter) {
// These tests verify private-router Service annotation, PrivateLinkService CRs, and DNS zone configuration
// on Azure clusters with Private topology.
func AzurePrivateTopologyTest(getTestCtx internal.TestContextGetter) {
Context("Azure Private Topology", Label("Azure", "self-managed-azure-private"), Ordered, func() {
Context("[Feature:AzurePrivateLink] Azure Private Topology", Label("Azure", "self-managed-azure-private"), Ordered, func() {
var testCtx *internal.TestContext
var controlPlaneNamespace string

Expand Down Expand Up @@ -227,7 +227,7 @@ func listPLS(ctx context.Context, client crclient.Client, namespace string) ([]*
// These tests verify that OAuth is properly exposed via a LoadBalancer Service and that the
// OAuth token flow works through that endpoint.
func AzureOAuthLoadBalancerTest(getTestCtx internal.TestContextGetter) {
Context("Azure OAuth LoadBalancer", Label("Azure", "self-managed-azure-oauth-lb"), func() {
Context("[Feature:AzureOAuth] Azure OAuth LoadBalancer", Label("Azure", "self-managed-azure-oauth-lb"), func() {
BeforeEach(func() {
testCtx := getTestCtx()
hc := testCtx.GetHostedCluster()
Expand Down Expand Up @@ -293,7 +293,7 @@ func RegisterHostedClusterAzureTests(getTestCtx internal.TestContextGetter) {
AzureOAuthLoadBalancerTest(getTestCtx)
}

var _ = Describe("Hosted Cluster Azure", Label("hosted-cluster-azure"), func() {
var _ = Describe("[sig-hypershift][Jira:Hypershift] Hosted Cluster Azure", Label("hosted-cluster-azure"), func() {
var testCtx *internal.TestContext

BeforeEach(func() {
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/v2/tests/hosted_cluster_ccm_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ func RegisterHostedClusterCCMTests(getTestCtx internal.TestContextGetter) {
GCPCloudControllerManagerTest(getTestCtx)
}

var _ = Describe("Hosted Cluster CCM", Label("hosted-cluster-ccm"), func() {
var _ = Describe("[sig-hypershift][Jira:Hypershift][Feature:CloudControllerManager] Hosted Cluster CCM", Label("hosted-cluster-ccm"), func() {
var testCtx *internal.TestContext

BeforeEach(func() {
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/v2/tests/hosted_cluster_compliance_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ func EnsureAllRoutesUseHCPRouterTest(getTestCtx internal.TestContextGetter) {
})
}

var _ = Describe("Hosted Cluster Compliance", Label("hosted-cluster-compliance"), func() {
var _ = Describe("[sig-hypershift][Jira:Hypershift][Feature:Compliance] Hosted Cluster Compliance", Label("hosted-cluster-compliance"), func() {
var testCtx *internal.TestContext

BeforeEach(func() {
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/v2/tests/hosted_cluster_cpo_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ func VerifyCPOOverrideImageTest(getTestCtx internal.TestContextGetter) {
})
}

var _ = Describe("Hosted Cluster CPO", Label("hosted-cluster-cpo"), func() {
var _ = Describe("[sig-hypershift][Jira:Hypershift][Feature:ControlPlaneOperator] Hosted Cluster CPO", Label("hosted-cluster-cpo"), func() {
var testCtx *internal.TestContext

BeforeEach(func() {
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/v2/tests/hosted_cluster_dns_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ func EnsureKubeAPIDNSNameCustomCertTest(getTestCtx internal.TestContextGetter) {
})
}

var _ = Describe("Hosted Cluster DNS", Label("hosted-cluster-dns"), func() {
var _ = Describe("[sig-hypershift][Jira:Hypershift][Feature:DNS] Hosted Cluster DNS", Label("hosted-cluster-dns"), func() {
var testCtx *internal.TestContext

BeforeEach(func() {
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/v2/tests/hosted_cluster_external_oidc_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ func RegisterExternalOIDCTests(getTestCtx internal.TestContextGetter) {
ExternalOIDCKeycloakAuthTest(getTestCtx)
}

var _ = Describe("External OIDC", Label("external-oidc"), func() {
var _ = Describe("[sig-hypershift][Jira:Hypershift][Feature:ExternalOIDC] External OIDC", Label("external-oidc"), func() {
var testCtx *internal.TestContext

BeforeEach(func() {
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/v2/tests/hosted_cluster_health_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ func ValidateConfigurationStatusTest(getTestCtx internal.TestContextGetter) {
})
}

var _ = Describe("Hosted Cluster Health", Label("hosted-cluster-health"), func() {
var _ = Describe("[sig-hypershift][Jira:Hypershift][Feature:Health] Hosted Cluster Health", Label("hosted-cluster-health"), func() {
var testCtx *internal.TestContext

BeforeEach(func() {
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/v2/tests/hosted_cluster_image_registry_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -279,7 +279,7 @@ func verifyExternalAccountCred(credData []byte, key, expectedEmail string) {
"service_account_impersonation_url should reference the imageRegistry GSA email %s", expectedEmail)
}

var _ = Describe("Hosted Cluster Image Registry", Label("hosted-cluster-image-registry"), func() {
var _ = Describe("[sig-hypershift][Jira:Hypershift][Feature:ImageRegistry] Hosted Cluster Image Registry", Label("hosted-cluster-image-registry"), func() {
var testCtx *internal.TestContext

BeforeEach(func() {
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/v2/tests/hosted_cluster_ingress_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ func ValidateIngressOperatorConfigurationTest(getTestCtx internal.TestContextGet
})
}

var _ = Describe("Hosted Cluster Ingress", Label("hosted-cluster-ingress"), func() {
var _ = Describe("[sig-hypershift][Jira:Hypershift][Feature:Ingress] Hosted Cluster Ingress", Label("hosted-cluster-ingress"), func() {
var testCtx *internal.TestContext

BeforeEach(func() {
Expand Down
Loading