Skip to content

pre-seed metrics from imagebuilds#278

Merged
bennyz merged 1 commit into
centos-automotive-suite:mainfrom
bennyz:pre-pop-metrics
May 10, 2026
Merged

pre-seed metrics from imagebuilds#278
bennyz merged 1 commit into
centos-automotive-suite:mainfrom
bennyz:pre-pop-metrics

Conversation

@bennyz

@bennyz bennyz commented May 7, 2026

Copy link
Copy Markdown
Contributor

To avoid losing data on restarts, pre-seed the metrics based on CRs

Summary

Related Issues

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • CI/CD improvement
  • Refactoring

Testing

  • Unit tests pass (make test)
  • Linter passes (make lint)
  • Manifests are up to date (make manifests generate)
  • Tested on OpenShift cluster (if applicable)

Summary by CodeRabbit

  • New Features

    • Enhanced visibility into build lifecycle with previous phase tracking
    • Improved metrics reliability by seeding from existing resources on operator startup
  • Bug Fixes

    • Metrics now persist across operator pod restarts
  • Tests

    • Added coverage for phase tracking and metric seeding logic

To avoid losing data on restarts, pre-seed the metrics based on CRs

Signed-off-by: Benny Zlotnik <bzlotnik@redhat.com>
Assisted-by: claude-opus-4.6
@coderabbitai

coderabbitai Bot commented May 7, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The PR extends ImageBuild status with a PreviousPhase field to track the prior phase when transitioning to Expired, adds metric seeding functions to initialize Prometheus metrics from existing ImageBuild custom resources on startup (ensuring metrics persist across pod restarts), and updates the reconciler to record and log previous-phase information during expiration.

Changes

ImageBuild Metrics Seeding and Previous Phase Tracking

Layer / File(s) Summary
Data Contract
api/v1alpha1/imagebuild_types.go
ImageBuildStatus gains PreviousPhase field to capture the build phase prior to expiration.
Metrics Helper Functions
internal/controller/imagebuild/metrics.go
adjustActiveBuildsGauge is updated to use phaseBuilding constant. buildMetricStatus maps phases to success/failure statuses, handling Expired by falling back to PreviousPhase. seedMetrics initializes counters and gauges from a slice of ImageBuild CRs. seedMetricsFromCRs is a runnable that syncs cache, lists all ImageBuilds, seeds metrics, and logs results.
Controller Integration
internal/controller/imagebuild/controller.go
checkExpiry logs ttl, anchor, and previousPhase details. updateStatus records Status.PreviousPhase when setting phase to ImageBuildPhaseExpired. SetupWithManager registers seedMetricsFromCRs(mgr) instead of seedActiveBuildsGauge(mgr) for metrics initialization.
Tests
internal/controller/imagebuild/metrics_test.go
New test TestBuildMetricStatus validates phase-to-status mapping including expired cases. TestSeedMetricsFromCRs verifies counter increments for BuildTotal and FlashTotal across various CR states. TestSeedMetricsFromCRs_ActiveBuilds verifies the ActiveBuilds gauge is set correctly from Building-phase CRs.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested reviewers

  • bkhizgiy

Poem

🐰 A rabbit's verse on metrics that persist,
When pods restart, no data lost or missed,
The PreviousPhase marks where builds have been,
Seeded at startup—dashboards stay serene!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title accurately summarizes the main change: pre-seeding metrics from ImageBuild custom resources to persist metric values across restarts.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@bennyz bennyz requested a review from bkhizgiy May 7, 2026 11:07

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/controller/imagebuild/metrics.go`:
- Around line 152-154: The current seeding only increments FlashTotal when
b.Status.FlashTaskRunName != "" (standalone flash); update the seeding logic to
also detect pipeline-based flashes and increment FlashTotal for them.
Concretely, extend the conditional that checks b.Status.FlashTaskRunName to also
call a helper like isPipelineFlashCompleted(b) (or reuse the same
completion-detection logic from recordPipelineFlashMetrics / checkBuildProgress)
so that builds which completed flashing inline in the pipeline are counted
during startup; ensure the helper mirrors the exact completion criteria used by
recordPipelineFlashMetrics/checkBuildProgress to avoid double-counting.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 82c84b19-3f85-4768-8f71-271b3cc97d8e

📥 Commits

Reviewing files that changed from the base of the PR and between 435f86d and e078972.

⛔ Files ignored due to path filters (1)
  • config/crd/bases/automotive.sdv.cloud.redhat.com_imagebuilds.yaml is excluded by !config/crd/bases/**
📒 Files selected for processing (4)
  • api/v1alpha1/imagebuild_types.go
  • internal/controller/imagebuild/controller.go
  • internal/controller/imagebuild/metrics.go
  • internal/controller/imagebuild/metrics_test.go

Comment on lines +152 to +154
if b.Status.FlashTaskRunName != "" {
FlashTotal.WithLabelValues(target, status).Add(1)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

FlashTotal seeding misses pipeline-based flash builds.

b.Status.FlashTaskRunName != "" is only set by the standalone flash path (createFlashTaskRunhandleFlashingState). For builds where flash runs inline in the build pipeline (recordPipelineFlashMetrics / checkBuildProgress), FlashTaskRunName is never populated. After an operator restart, those builds' flash counts won't be pre-seeded into FlashTotal.

A targeted fix for pipeline flash (where flash completing implies build success):

🛠️ Proposed fix
-		if b.Status.FlashTaskRunName != "" {
-			FlashTotal.WithLabelValues(target, status).Add(1)
-		}
+		// Standalone flash (separate TaskRun) — FlashTaskRunName is set directly.
+		// Pipeline flash (inline task) — infer from spec: if flash was enabled and
+		// the build succeeded, the pipeline flash step must have completed.
+		flashRan := b.Status.FlashTaskRunName != "" ||
+			(b.Spec.IsFlashEnabled() && status == buildStatusSuccess)
+		if flashRan {
+			FlashTotal.WithLabelValues(target, status).Add(1)
+		}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if b.Status.FlashTaskRunName != "" {
FlashTotal.WithLabelValues(target, status).Add(1)
}
// Standalone flash (separate TaskRun) — FlashTaskRunName is set directly.
// Pipeline flash (inline task) — infer from spec: if flash was enabled and
// the build succeeded, the pipeline flash step must have completed.
flashRan := b.Status.FlashTaskRunName != "" ||
(b.Spec.IsFlashEnabled() && status == buildStatusSuccess)
if flashRan {
FlashTotal.WithLabelValues(target, status).Add(1)
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/controller/imagebuild/metrics.go` around lines 152 - 154, The
current seeding only increments FlashTotal when b.Status.FlashTaskRunName != ""
(standalone flash); update the seeding logic to also detect pipeline-based
flashes and increment FlashTotal for them. Concretely, extend the conditional
that checks b.Status.FlashTaskRunName to also call a helper like
isPipelineFlashCompleted(b) (or reuse the same completion-detection logic from
recordPipelineFlashMetrics / checkBuildProgress) so that builds which completed
flashing inline in the pipeline are counted during startup; ensure the helper
mirrors the exact completion criteria used by
recordPipelineFlashMetrics/checkBuildProgress to avoid double-counting.

@bennyz bennyz merged commit 4105ec4 into centos-automotive-suite:main May 10, 2026
4 checks passed
maboras-rh added a commit to maboras-rh/automotive-dev-operator that referenced this pull request May 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants