OCPBUGS-84327: Product-cli missing controller-runtime logger initialization causes noisy warning

[amogh-redhat]

What this PR does / why we need it:

product-cli/main.go does not initialize the controller-runtime logger, while the hypershift CLI (main.go) properly initializes it at line 43 with ctrl.SetLogger(zap.New(...)). This causes a noisy warning to be emitted during operations like create infra when using the product-cli:

[controller-runtime] log.SetLogger(...) was never called; logs will not be displayed.
The hypershift CLI does not produce this warning because the logger is properly initialized.

Added "ctrl.SetLogger(zap.New(...))." to the product-cli/main.go to fix the above mentioned issue.

Which issue(s) this PR fixes:

Fixes : https://redhat.atlassian.net/browse/OCPBUGS-84327

Special notes for your reviewer:

Checklist:

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

Summary by CodeRabbit

• Chores
  • CLI now initializes structured JSON logging at startup, improving machine-readability and consistency of log output.
  • Timestamps in logs use RFC3339 formatting for better interoperability and easier correlation across systems.
  • Result: clearer, more parseable logs to aid debugging and monitoring of CLI behavior.

[openshift-merge-bot]

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[coderabbitai]

📝 Walkthrough

Walkthrough

The pull request updates product-cli/main.go to configure controller-runtime logging at startup. It adds imports for sigs.k8s.io/controller-runtime and Zap logging (including zapcore) and, in main(), initializes a Zap JSON logger with RFC3339 time encoding and sets it as the controller-runtime logger via ctrl.SetLogger().

🚥 Pre-merge checks | ✅ 11 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (11 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	Custom check is not applicable. PR modifies only product-cli/main.go for logger initialization. No Ginkgo tests are involved—tests in product-cli use standard Go testing package, not Ginkgo.
Test Structure And Quality	✅ Passed	Custom check for Ginkgo test code quality is not applicable. This PR only modifies product-cli/main.go to add controller-runtime logger initialization with no test code changes.
Microshift Test Compatibility	✅ Passed	PR modifies product-cli/main.go to initialize controller-runtime logger. No Ginkgo e2e tests are added. Custom check for MicroShift test compatibility is not applicable.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	This PR only modifies product-cli/main.go to initialize controller-runtime logging. No Ginkgo e2e tests are added, so the SNO test compatibility check is not applicable.
Topology-Aware Scheduling Compatibility	✅ Passed	PR modifies only product-cli/main.go to initialize controller-runtime logger. No deployment manifests, operators, or controllers introducing scheduling constraints are added/modified.
Ote Binary Stdout Contract	✅ Passed	The PR initializes zap logging with default stderr output. No WriteTo() option or stdout configuration exists. Initialization produces no output. Matches reference implementation in main.go.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	Custom check does not apply. This PR does not add any Ginkgo e2e tests—it only modifies product-cli/main.go to initialize controller-runtime logger.
Title check	✅ Passed	The title clearly and specifically describes the main change: adding controller-runtime logger initialization to product-cli to eliminate a warning message.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

• Generate code and open pull requests
• Plan features and break down work
• Investigate incidents and troubleshoot customer tickets together
• Automate recurring tasks and respond to alerts with triggers
• Summarize progress and report instantly

Built for teams:

• Shared memory across your entire org—no repeating context
• Per-thread sandboxes to safely plan and execute work
• Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codecov]

Codecov Report

❌ Patch coverage is 80.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 42.84%. Comparing base (ac1a1c2) to head (f68481e).
⚠️ Report is 369 commits behind head on main.

Files with missing lines	Patch %	Lines
product-cli/main.go	80.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8491      +/-   ##
==========================================
+ Coverage   41.50%   42.84%   +1.34%     
==========================================
  Files         758      775      +17     
  Lines       93689    98111    +4422     
==========================================
+ Hits        38882    42037    +3155     
- Misses      52070    53162    +1092     
- Partials     2737     2912     +175     

Files with missing lines	Coverage Δ
product-cli/main.go	11.76% <80.00%> (+11.76%)	⬆️

... and 121 files with indirect coverage changes

Flag	Coverage Δ
cmd-support	36.63% <ø> (+1.77%)	⬆️
cpo-hostedcontrolplane	45.91% <ø> (+2.31%)	⬆️
cpo-other	45.11% <ø> (+1.94%)	⬆️
hypershift-operator	51.15% <ø> (-0.42%)	⬇️
other	32.11% <80.00%> (+0.47%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[muraee]

/approve

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: amogh-redhat, muraee

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [muraee]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[vsolanki12]

/retitle OCPBUGS-84327: Product-cli missing controller-runtime logger initialization causes noisy warning

[openshift-ci-robot]

@amogh-redhat: This pull request references Jira Issue OCPBUGS-84327, which is invalid:

• expected the bug to target the "5.0.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

What this PR does / why we need it:

product-cli/main.go does not initialize the controller-runtime logger, while the hypershift CLI (main.go) properly initializes it at line 43 with ctrl.SetLogger(zap.New(...)). This causes a noisy warning to be emitted during operations like create infra when using the product-cli:

[controller-runtime] log.SetLogger(...) was never called; logs will not be displayed.
The hypershift CLI does not produce this warning because the logger is properly initialized.

Added "ctrl.SetLogger(zap.New(...))." to the product-cli/main.go to fix the above mentioned issue.

Which issue(s) this PR fixes:

Fixes : https://redhat.atlassian.net/browse/OCPBUGS-84327

Special notes for your reviewer:

Checklist:

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

Summary by CodeRabbit

• Chores
• CLI now initializes structured JSON logging at startup, improving machine-readability and consistency of log output.
• Timestamps in logs use RFC3339 formatting for better interoperability and easier correlation across systems.
• Result: clearer, more parseable logs to aid debugging and monitoring of CLI behavior.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[vsolanki12]

/jira refresh

[openshift-ci-robot]

@vsolanki12: This pull request references Jira Issue OCPBUGS-84327, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (5.0.0) matches configured target version for branch (5.0.0)
• bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[csrwng]

/lgtm
/override "codecov/patch"

[openshift-ci]

@csrwng: Overrode contexts on behalf of csrwng: codecov/patch

Details

In response to this:

/lgtm
/override "codecov/patch"

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[openshift-merge-bot]

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-azure-self-managed
/test e2e-azure-v2-self-managed
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-gke

[csrwng]

/area cli

[amogh-redhat]

/retest

[amogh-redhat]

/retest

[sdminonne]

/lgtm

[openshift-merge-bot]

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-azure-self-managed
/test e2e-azure-v2-self-managed
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-gke

[amogh-redhat]

/retest

[amogh-redhat]

/test e2e-aws

[amogh-redhat]

Created a unit test "main_test.go" using claude to test the fix. This is similar to the unit test in https://github.com/openshift/hypershift/pull/8955/changes.

Test passed :
ambewad@aambewad-mac hypershift-amogh % go test -v -race ./product-cli/ -run TestNewLogger
=== RUN TestNewLogger
=== RUN TestNewLogger/When_creating_a_new_logger_it_should_produce_JSON_output_with_RFC3339_timestamps
--- PASS: TestNewLogger (0.00s)
--- PASS: TestNewLogger/When_creating_a_new_logger_it_should_produce_JSON_output_with_RFC3339_timestamps (0.00s)
PASS
ok github.com/openshift/hypershift/product-cli 2.659s

[sdminonne]

/lgtm

[openshift-merge-bot]

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-azure-v2-self-managed
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-gke

[amogh-redhat]

/verified by me

Created a unit test "main_test.go" using claude to test the fix. This is similar to the unit test in https://github.com/openshift/hypershift/pull/8955/changes.

Test passed :
ambewad@aambewad-mac hypershift-amogh % go test -v -race ./product-cli/ -run TestNewLogger
=== RUN TestNewLogger
=== RUN TestNewLogger/When_creating_a_new_logger_it_should_produce_JSON_output_with_RFC3339_timestamps
--- PASS: TestNewLogger (0.00s)
--- PASS: TestNewLogger/When_creating_a_new_logger_it_should_produce_JSON_output_with_RFC3339_timestamps (0.00s)
PASS
ok github.com/openshift/hypershift/product-cli 2.659s

[openshift-ci-robot]

@amogh-redhat: This PR has been marked as verified by me.

Details

In response to this:

/verified by me

Created a unit test "main_test.go" using claude to test the fix. This is similar to the unit test in https://github.com/openshift/hypershift/pull/8955/changes.

Test passed :
ambewad@aambewad-mac hypershift-amogh % go test -v -race ./product-cli/ -run TestNewLogger
=== RUN TestNewLogger
=== RUN TestNewLogger/When_creating_a_new_logger_it_should_produce_JSON_output_with_RFC3339_timestamps
--- PASS: TestNewLogger (0.00s)
--- PASS: TestNewLogger/When_creating_a_new_logger_it_should_produce_JSON_output_with_RFC3339_timestamps (0.00s)
PASS
ok github.com/openshift/hypershift/product-cli 2.659s

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[openshift-ci]

@amogh-redhat: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-aws	f68481e	link	true	/test e2e-aws
ci/prow/e2e-v2-aws	f68481e	link	true	/test e2e-v2-aws
ci/prow/e2e-aws-upgrade-hypershift-operator	f68481e	link	true	/test e2e-aws-upgrade-hypershift-operator
ci/prow/e2e-aks	f68481e	link	true	/test e2e-aks

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[hypershift-jira-solve-ci]

Now I have all the evidence needed. Let me produce the final mandatory markdown report:

Test Failure Analysis: ci/prow/e2e-aws

Job: pull-ci-openshift-hypershift-main-e2e-aws
Build ID: 2076650588229079040
Target: e2e-aws
Result: 41 test failures across 13+ hosted clusters

Verdict: ❌ NOT caused by PR #8491

PR #8491 only adds ctrl.SetLogger(newLogger()) to product-cli/main.go — a logger initialization for the hcp CLI binary. The crashing components (control-plane-operator, capi-provider, hosted-cluster-config-operator, cluster-network-operator) are entirely separate binaries. The failure is caused by transient management cluster infrastructure instability.

Root Cause

Management cluster API server / DNS infrastructure experienced a transient outage affecting ALL hosted clusters simultaneously.

The evidence chain:

1. DNS resolution failures — All 13+ hosted clusters simultaneously failed to resolve their API endpoints (e.g., api-custom-config-94fv7.service.ci.hypershift.devcluster.openshift.com) on the cluster DNS (172.30.0.10:53) with no such host errors.

2. Management cluster API server unreachable — The management cluster's ELB endpoint (a4af76c6dab5f452ba146d289fc8a45e-c54cbae36f78bedc.elb.us-east-1.amazonaws.com:6443) returned TLS handshake timeouts and http2: client connection lost errors across multiple namespaces simultaneously.

3. Cascading container crashes — When the management cluster API server became unreachable, controllers running in the hosted control plane namespaces lost their leader election leases and crashed:

   • control-plane-operator: 33 crash instances (restartCount 3-5 per cluster)
   • capi-provider/manager: 30 crash instances (restartCount ~5 per cluster)
   • hosted-cluster-config-operator: 9 crash instances (restartCount 3-4 per cluster)
   • cluster-network-operator: 9 crash instances

4. Recovery observed — API connections recovered after 1m49s to 4m5s per cluster, confirming the transient nature. But the accumulated restartCount exceeded the EnsureNoCrashingPods threshold (restartCount > 2 within the test window), causing 41 test failures.

5. PR OCPBUGS-84327: Product-cli missing controller-runtime logger initialization causes noisy warning #8491 is unrelated — The PR modifies only product-cli/main.go (+12 lines) and product-cli/main_test.go (+40 lines, new file). It adds ctrl.SetLogger(newLogger()) for controller-runtime logger initialization in the hcp CLI binary. None of the crashing components (control-plane-operator, capi-provider, hosted-cluster-config-operator, cluster-network-operator) are affected by this change.

Recommendations

1. Retest the PR — This is a transient infrastructure failure. Retrigger /retest on the PR to get a clean run.

2. No code changes needed — PR OCPBUGS-84327: Product-cli missing controller-runtime logger initialization causes noisy warning #8491 does not contribute to the failure. The logger initialization change in product-cli/main.go is completely isolated from the HCP control plane components.

3. If failure persists on retest — Check for ongoing management cluster instability in the build01/build09 CI clusters. Simultaneous DNS + TLS failures across all hosted clusters point to management cluster API server or DNS infrastructure issues, not test or product regressions.

Evidence

Failed Test Step: e2e-aws-hypershift-aws-run-e2e-nested (test phase, duration: ~3.2h)

JUnit Summary: 162 tests run, 41 failures, 20 skipped

Failure Pattern: All 41 failures trace to EnsureNoCrashingPods assertions across every hosted cluster created during the test run.

DNS Resolution Failures (simultaneous across all clusters):

dial tcp: lookup api-custom-config-94fv7.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host
dial tcp: lookup api-custom-config-94fv7.service.ci.hypershift.devcluster.openshift.com:6443: i/o timeout

Management Cluster API Server Failures:

Get "https://a4af76c6dab5f452ba146d289fc8a45e-c54cbae36f78bedc.elb.us-east-1.amazonaws.com:6443/...": net/http: TLS handshake timeout
Get "https://a4af76c6dab5f452ba146d289fc8a45e-c54cbae36f78bedc.elb.us-east-1.amazonaws.com:6443/...": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost

Crash-Looping Containers (representative sample from EnsureNoCrashingPods output):

control-plane-operator: restartCount 3-5 across all clusters (33 instances)
capi-provider/manager: restartCount ~5 across all clusters (30 instances)
hosted-cluster-config-operator: restartCount 3-4 across clusters (9 instances)
cluster-network-operator: restartCount ~3 across clusters (9 instances)

Operator Condition Evidence (from TestKarpenterUpgradeControlPlane):

Degraded=True: UnavailableReplicas (capi-provider deployment has 1 unavailable replicas)
KubeAPIServerAvailable=False

PR #8491 Changes (confirmed unrelated):

• product-cli/main.go: +12 lines — adds ctrl.SetLogger(newLogger()) with zap JSON logger
• product-cli/main_test.go: +40 lines — unit test for newLogger() function
• No modifications to any controller, operator, or component that crashed

Key Discriminator: The simultaneous nature of DNS failures and API server timeouts across ALL 13+ independent hosted clusters rules out any test-specific or PR-specific cause. Infrastructure-level instability is the only explanation consistent with this pattern.

---

[amogh-redhat]

/rebase

[github-actions]

🤖 Rebasing PR onto main: workflow run

[github-actions]

The rebase onto the latest main succeeded locally (conflicts in product-cli/main.go and product-cli/main_test.go were resolved preserving the PR's intent), but the force push failed with:

remote rejected: refusing to allow a GitHub App to create or update workflow `.github/workflows/rebase.yaml` without `workflows` permission

The rebase pulled in upstream commits that modified .github/workflows/rebase.yaml, and the CI token does not have the workflows permission required to push changes to workflow files.

To complete the rebase, the PR author should:

1. Ensure "Allow edits from maintainers" is enabled on this PR
2. Retry /rebase, or manually rebase and force push from a local checkout (a personal token with workflows scope can push workflow file changes)

[amogh-redhat]

#8955 was opened for the same issue. closing this one as duplicate.

[openshift-ci-robot]

@amogh-redhat: This pull request references Jira Issue OCPBUGS-84327. The bug has been updated to no longer refer to the pull request using the external bug tracker. All external bug links have been closed. The bug has been moved to the NEW state.

Details

In response to this:

What this PR does / why we need it:

product-cli/main.go does not initialize the controller-runtime logger, while the hypershift CLI (main.go) properly initializes it at line 43 with ctrl.SetLogger(zap.New(...)). This causes a noisy warning to be emitted during operations like create infra when using the product-cli:

[controller-runtime] log.SetLogger(...) was never called; logs will not be displayed.
The hypershift CLI does not produce this warning because the logger is properly initialized.

Added "ctrl.SetLogger(zap.New(...))." to the product-cli/main.go to fix the above mentioned issue.

Which issue(s) this PR fixes:

Fixes : https://redhat.atlassian.net/browse/OCPBUGS-84327

Special notes for your reviewer:

Checklist:

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

Summary by CodeRabbit

• Chores
• CLI now initializes structured JSON logging at startup, improving machine-readability and consistency of log output.
• Timestamps in logs use RFC3339 formatting for better interoperability and easier correlation across systems.
• Result: clearer, more parseable logs to aid debugging and monitoring of CLI behavior.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.