Skip to content

fix(compute-plane): update observability defaults and coverage - #648

Merged
sbaum1994 merged 7 commits into
mainfrom
test/bdd-observability-compute-pr3
Aug 13, 2026
Merged

fix(compute-plane): update observability defaults and coverage#648
sbaum1994 merged 7 commits into
mainfrom
test/bdd-observability-compute-pr3

Conversation

@sbaum1994

@sbaum1994 sbaum1994 commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Why

The compute observability profile needs live split-cluster coverage and published artifacts that work together. NVCA Operator 3.2.0 cannot start with chart 1.14.0 because that chart invokes the removed /tini wrapper.

What changed

  • Adds the compute-profile BDD scenario for ncp-local-cp and ncp-local-compute-1.
  • Updates NVCA Operator and NVCA defaults to 3.2.0 and the compatible operator chart to 1.19.0.
  • Updates the NVCA collector fallback to 0.157.9.
  • Adds render coverage requiring direct operator, mirror, and cleanup binary entrypoints.
  • Verifies releases, Target Allocator, NVCA health, collector configuration, compute monitors, and absent control-plane monitors.

Customer Release Notes

Self-managed compute observability now uses compatible published NVCA and collector artifacts.

Plan Summary

The control-plane cluster receives prerequisites. The compute cluster receives observability and NVCA. Existing version ownership remains in compute-plane and chart defaults.

Usage

Run the observability-compute.feature scenario against the named split ncp-local topology.

Testing

  • Live TestObservabilityCompute: 58/58 steps passed in 14m47s after destructive topology recreation and an idempotent retry of a transient gateway bootstrap check.
  • make test-local passed, including entrypoint and golden-render checks.
  • go test -short ./..., bash -n, and shellcheck passed.
  • Verified chart 1.19.0 deployed with NVCA Operator 3.2.0; both operator containers were Ready and used direct binary arguments.

Notes

Vendor sync remains skipped because the latest-main helper rewrites unrelated nil-safe chart expressions. The checked-in vendor chart is unchanged.

References

Related Pull Requests

Dependencies

Updates internal helm-nvca-operator from 1.14.0 to 1.19.0 and NVCA images to 3.2.0. No third-party license or NOTICE changes.

Issues

Relates to #522
Closes #799

Checklist

  • I am familiar with the Contributing Guidelines.
  • I have signed off my commits for Developer Certificate of Origin (DCO) compliance.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

Summary by CodeRabbit

  • Improvements

    • Updated the OpenTelemetry Collector image to version 0.157.9 across compute-plane and NVCA configurations.
    • Pinned compute-plane NVCA Operator and self-managed NVCA versions to 3.2.0.
    • Updated the NVCA Operator chart to compatible version 1.19.0.
    • Improved configuration handling so supplied operator and observability settings are applied consistently while defaults remain available.
  • Documentation

    • Updated image-version references and deployment guidance for the latest collector release.
  • Tests

    • Added coverage for compute-profile observability, Kubernetes context targeting, image overrides, monitoring, and direct NVCA entrypoints.

@sbaum1994
sbaum1994 requested review from a team as code owners August 4, 2026 00:24
@sbaum1994
sbaum1994 requested a review from estroz August 4, 2026 00:24
@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The change updates NVCA collector defaults, adds compute-plane image and version override handling, introduces context-aware kubectl apply command generation, and adds split-cluster observability BDD coverage.

Changes

Compute observability

Layer / File(s) Summary
NVCA collector defaults
deploy/helm/nvca-operator/..., src/compute-plane-services/nvca/...
The default OpenTelemetry Collector tag changes to 0.157.9 across chart values, schemas, templates, documentation, and image tests.
NVCA override handling
deploy/stacks/nvcf-compute-plane/...
Compute-plane values pin the operator image and NVCA version to 3.1.0. The Helmfile propagates optional image, version, and collector overrides.
Context-aware kubectl command builder
tests/bdd/dsl/...
KubectlApplyCommand validates and quotes manifest paths and adds --context when a context is provided.
Context-aware secret setup
tests/bdd/steps/...
Pull-secret application uses explicit Kubernetes contexts and keeps API keys out of command arguments.
Split-cluster observability scenario
tests/bdd/features/..., tests/bdd/godog_test.go
The BDD scenario installs and validates compute-plane observability, NVCA health, collector settings, monitoring resources, release state, feature gates, and cluster routing.

Estimated code review effort: 4 (Complex) | ~45 minutes

Suggested reviewers: estroz

Sequence Diagram(s)

sequenceDiagram
  participant Godog
  participant ControlPlane
  participant Observability
  participant ComputePlane
  participant NVCA

  Godog->>ControlPlane: Configure split-cluster fixtures
  Godog->>ControlPlane: Register compute cluster
  Godog->>Observability: Install shared observability
  Godog->>ComputePlane: Install NVCA compute plane
  Godog->>NVCA: Inject credential and restart NVCA
  Godog->>ComputePlane: Verify releases and monitoring configuration
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title follows Conventional Commits and accurately describes the compute-plane observability default updates and added coverage.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test/bdd-observability-compute-pr3

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
tests/bdd/godog_test.go (1)

663-670: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Assert manifest applies for both selected contexts.

Line 663 rejects only the ambient-context form. It does not confirm that control-plane manifests use the control context and compute-plane manifests use the compute context.

A regression that sends all manifest applies to one explicit context passes this test. Count the generated apply commands and assert 16 control-plane applies and 4 compute-plane applies.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/bdd/godog_test.go` around lines 663 - 670, Update the
command-validation loop in the relevant Godog test to classify generated
manifest apply commands by explicit control-plane and compute-plane context,
then assert exactly 16 control-plane applies and 4 compute-plane applies. Retain
the existing rejection for ambient-context applies and API-key leakage, using
the test’s existing command/context symbols to distinguish the two apply groups.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@deploy/stacks/nvcf-compute-plane/tests/observability-profile.sh`:
- Around line 78-79: Update the regression test inputs and assertions in
observability-profile.sh to use distinct sentinel values for
global.nvcaOperator.imageTag and global.nvcaOperator.selfManaged.nvcaVersion.
Scope each check specifically to the corresponding image.tag and
selfManaged.nvcaVersion fields, so swapped mappings or values written to
unrelated matching keys fail.

In `@tests/bdd/godog_test.go`:
- Around line 640-653: The wiring test should stop requiring exact matches for
every recorded command. In the test around fakeRunner.runs, retain the existing
status == 0 assertion and replace the loop over registryLoginCommand and related
commands with a single commandRanThatContains assertion targeting the
destructive pod-delete command.

---

Nitpick comments:
In `@tests/bdd/godog_test.go`:
- Around line 663-670: Update the command-validation loop in the relevant Godog
test to classify generated manifest apply commands by explicit control-plane and
compute-plane context, then assert exactly 16 control-plane applies and 4
compute-plane applies. Retain the existing rejection for ambient-context applies
and API-key leakage, using the test’s existing command/context symbols to
distinguish the two apply groups.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e52e56b0-ac5d-4560-a1e6-12bc233ca614

📥 Commits

Reviewing files that changed from the base of the PR and between 892b704 and ffc7151.

📒 Files selected for processing (8)
  • deploy/stacks/nvcf-compute-plane/helmfile.d/02-nvca.yaml.gotmpl
  • deploy/stacks/nvcf-compute-plane/tests/observability-profile.sh
  • tests/bdd/dsl/kubectl.go
  • tests/bdd/dsl/kubectl_test.go
  • tests/bdd/features/observability-compute.feature
  • tests/bdd/godog_test.go
  • tests/bdd/steps/infra_steps.go
  • tests/bdd/steps/steps_test.go

Comment thread deploy/stacks/nvcf-compute-plane/tests/observability-profile.sh Outdated
Comment thread tests/bdd/godog_test.go Outdated
Comment thread tests/bdd/dsl/kubectl.go
Comment thread tests/bdd/features/observability-compute.feature Outdated
Comment thread tests/bdd/features/observability-compute.feature Outdated
Comment thread tests/bdd/features/observability-compute.feature Outdated
Comment thread tests/bdd/features/observability-compute.feature Outdated
Comment thread tests/bdd/features/observability-compute.feature Outdated
@sbaum1994
sbaum1994 force-pushed the test/bdd-observability-compute-pr3 branch from ffc7151 to 5156232 Compare August 4, 2026 06:55
@sbaum1994 sbaum1994 changed the title test(bdd): cover compute observability profile fix(compute-plane): bump NVCA defaults and test observability Aug 4, 2026
@sbaum1994
sbaum1994 force-pushed the test/bdd-observability-compute-pr3 branch from 5156232 to 109cc6d Compare August 5, 2026 05:19
@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@sbaum1994 sbaum1994 changed the title fix(compute-plane): bump NVCA defaults and test observability fix(compute-plane): update observability defaults and coverage Aug 5, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@deploy/stacks/nvcf-compute-plane/README.md`:
- Around line 71-73: Update the README’s chart-defaults description near the
compute-plane image-tag documentation to identify chart defaults as fallback
values, not the source of tested image tags. Keep the existing statement that
compute-plane base values in 02-nvca.yaml.gotmpl provide the tested NVCA, NVCA
operator, and OTel Collector tags.

In `@tests/bdd/godog_test.go`:
- Around line 654-662: Update the command assertions in the test around the
profile-install target loop to also require the register-cluster command and an
explicit KUBECONFIG_FILE on the nvcf-compute-plane install target. Ensure every
compute operation is matched with its expected kubeconfig path so ambient
Kubernetes context cannot satisfy the test.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 6ff523c5-df24-4d29-b5c6-fce01da344ce

📥 Commits

Reviewing files that changed from the base of the PR and between efc98c7 and 109cc6d.

⛔ Files ignored due to path filters (3)
  • deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/deployment.yaml is excluded by !**/testdata/**
  • deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/pre-delete-cleanup-job.yaml is excluded by !**/testdata/**
  • deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/self-managed-nvcfbackend-cm.yaml is excluded by !**/testdata/**
📒 Files selected for processing (20)
  • deploy/helm/nvca-operator/nvca-operator/README.md
  • deploy/helm/nvca-operator/nvca-operator/templates/deployment.yaml
  • deploy/helm/nvca-operator/nvca-operator/values.schema.json
  • deploy/helm/nvca-operator/nvca-operator/values.yaml
  • deploy/helm/nvca-operator/tests/release_image_manifest_test.sh
  • deploy/helm/nvca-operator/tests/self_managed_nvca_image_reference_test.sh
  • deploy/stacks/nvcf-compute-plane/README.md
  • deploy/stacks/nvcf-compute-plane/environments/base.yaml
  • deploy/stacks/nvcf-compute-plane/helmfile.d/02-nvca.yaml.gotmpl
  • deploy/stacks/nvcf-compute-plane/tests/observability-profile.sh
  • src/compute-plane-services/nvca/deployments/nvca-operator/README.md
  • src/compute-plane-services/nvca/deployments/nvca-operator/templates/deployment.yaml
  • src/compute-plane-services/nvca/deployments/nvca-operator/values.schema.json
  • src/compute-plane-services/nvca/deployments/nvca-operator/values.yaml
  • tests/bdd/dsl/kubectl.go
  • tests/bdd/dsl/kubectl_test.go
  • tests/bdd/features/observability-compute.feature
  • tests/bdd/godog_test.go
  • tests/bdd/steps/infra_steps.go
  • tests/bdd/steps/steps_test.go
🚧 Files skipped from review as they are similar to previous changes (8)
  • tests/bdd/steps/steps_test.go
  • tests/bdd/dsl/kubectl.go
  • deploy/stacks/nvcf-compute-plane/tests/observability-profile.sh
  • tests/bdd/features/observability-compute.feature
  • deploy/stacks/nvcf-compute-plane/environments/base.yaml
  • tests/bdd/dsl/kubectl_test.go
  • deploy/stacks/nvcf-compute-plane/helmfile.d/02-nvca.yaml.gotmpl
  • tests/bdd/steps/infra_steps.go

Comment thread deploy/stacks/nvcf-compute-plane/README.md
Comment thread tests/bdd/godog_test.go Outdated
Comment thread deploy/helm/nvca-operator/tests/self_managed_nvca_image_reference_test.sh Outdated
Comment thread deploy/helm/nvca-operator/tests/self_managed_nvca_image_reference_test.sh Outdated
Comment thread deploy/stacks/nvcf-compute-plane/tests/observability-profile.sh Outdated
Comment thread deploy/stacks/nvcf-compute-plane/tests/observability-profile.sh Outdated
Comment thread tests/bdd/features/observability-compute.feature Outdated
Comment thread deploy/stacks/nvcf-compute-plane/environments/base.yaml Outdated
Add explicit kube-context secret setup and optional NVCA image overrides so local split-cluster installs exercise the compute profile with published multi-arch images.

Relates to #522

Signed-off-by: Stephanie Baum <sbaum@nvidia.com>
Make the compute-plane stack own the operator and backend versions instead of overriding them in the live BDD. Remove the redundant Stargate and collector tag overrides.

Relates to #522

Signed-off-by: Stephanie Baum <sbaum@nvidia.com>
Use the latest published multi-architecture NVCF OTel collector. Keep the source and release charts, schema, tests, and compute-plane compatibility default aligned.

Relates-to: #522
Signed-off-by: Stephanie Baum <sbaum@nvidia.com>
Compare rendered collector values to their configured sources and use sentinel overrides for version routing. Keep the BDD wiring assertion focused on the destructive NVCA restart command.

Relates-to: #522
Signed-off-by: Stephanie Baum <sbaum@nvidia.com>
Keep compute-profile coverage focused on collector enablement and the installed monitor resources.

Relates-to: #522
Signed-off-by: Stephanie Baum <sbaum@nvidia.com>
Use the published multi-architecture NVCA operator and backend images. Make the self-managed chart regression select its cluster source explicitly after latest main changed the chart default.

Relates-to: #522
Signed-off-by: Stephanie Baum <sbaum@nvidia.com>
Update helm-nvca-operator from 1.14.0 to 1.19.0 so the chart invokes the NVCA Operator 3.2.0 binaries directly instead of relying on the removed /tini wrapper. Add rendered entrypoint coverage and refresh the chart goldens.

No third-party license or NOTICE changes.

Fixes #799

Signed-off-by: Stephanie Baum <sbaum@nvidia.com>
@sbaum1994
sbaum1994 force-pushed the test/bdd-observability-compute-pr3 branch from e7ab93e to e7143e0 Compare August 13, 2026 03:33
@sbaum1994
sbaum1994 added this pull request to the merge queue Aug 13, 2026
Merged via the queue into main with commit 7a3c8cc Aug 13, 2026
20 checks passed
@sbaum1994
sbaum1994 deleted the test/bdd-observability-compute-pr3 branch August 13, 2026 04:04
@balajinvda

Copy link
Copy Markdown
Contributor

🎉 This PR is included in version helm-nvca-operator-v1.19.3 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NVCA Operator 3.2.0 fails with chart 1.14.0

4 participants