feat(analytics): metric definitions listing endpoint

[aleksdotbar]

Adds read-only GET /v1/metric-definitions — lists every metric definition visible to the tenant (tenant override wins per key), sorted by metric_key, display fields only (no computation internals on the wire).

Also decouples schema validity from data freshness:

• Schema probe now checks measure existence over all history, so schema_status is purely structural (a quiet source no longer reads as broken).
• New last_observed_date records max(metric_date) per definition as a separate freshness signal.
• schema_error_code surfaced so an error status carries its cause.
• Dimension coverage anchored at the newest observed row instead of today().

OpenAPI regenerated; unit + e2e contract tests added.

Closes #1859

Summary by CodeRabbit

• New Features

  • Added an authenticated endpoint for listing metric definitions.
  • Responses include labels, descriptions, formats, directions, units, dimensions, enablement, schema status, errors, and observation dates.
  • Tenant-specific metric definitions now override product defaults where applicable.
  • Metric definitions are returned in a consistent, sorted order.
  • Schema validation now reports per-measure freshness and dimension coverage more accurately.

• Documentation

  • Added OpenAPI documentation for the new endpoint and response fields.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 9c7a439c-d765-477f-bc87-793c7aef0b5b

📥 Commits

Reviewing files that changed from the base of the PR and between 0bf278e5f91178a4915067b7ad943925b7ec723b and e898fec.

📒 Files selected for processing (12)

• docs/components/backend/analytics/openapi.json
• src/backend/services/analytics/src/api/mod.rs
• src/backend/services/analytics/src/domain/metric_definitions/error_code.rs
• src/backend/services/analytics/src/domain/metric_definitions/listing.rs
• src/backend/services/analytics/src/domain/metric_definitions/live_tests.rs
• src/backend/services/analytics/src/domain/metric_definitions/mod.rs
• src/backend/services/analytics/src/domain/metric_definitions/repository.rs
• src/backend/services/analytics/src/domain/metric_definitions/validator.rs
• src/backend/services/analytics/src/migration/m20260722_000001_metric_definition_last_observed.rs
• src/backend/services/analytics/src/migration/mod.rs
• src/ingestion/tests/e2e/api/conftest.py
• src/ingestion/tests/e2e/api/test_metric_definitions.py

🚧 Files skipped from review as they are similar to previous changes (8)

• src/backend/services/analytics/src/migration/m20260722_000001_metric_definition_last_observed.rs
• src/backend/services/analytics/src/domain/metric_definitions/mod.rs
• src/backend/services/analytics/src/domain/metric_definitions/error_code.rs
• src/ingestion/tests/e2e/api/test_metric_definitions.py
• src/backend/services/analytics/src/api/mod.rs
• src/backend/services/analytics/src/migration/mod.rs
• docs/components/backend/analytics/openapi.json
• src/backend/services/analytics/src/domain/metric_definitions/listing.rs

---

📝 Walkthrough

Walkthrough

Adds an authenticated GET /v1/metric-definitions endpoint with tenant override resolution, typed metadata, schema status, dimensions, and freshness. Validation now tracks per-measure observation dates, anchors coverage checks, and persists last_observed_date, with migration and integration coverage.

Changes

Metric catalog API

Layer / File(s)	Summary
Listing contract and data assembly src/backend/services/analytics/src/domain/metric_definitions/*	Defines metric-definition response DTOs, decodes stored enum and error values, resolves tenant overrides, attaches dimensions, sorts results, and validates configuration data.
Endpoint and OpenAPI wiring src/backend/services/analytics/src/api/..., docs/components/backend/analytics/openapi.json	Registers the authenticated listing route and documents its response schemas, enum values, error responses, and bearer authentication.
Anchored validation and freshness persistence src/backend/services/analytics/src/domain/metric_definitions/validator.rs, src/backend/services/analytics/src/domain/metric_definitions/repository.rs, src/backend/services/analytics/src/migration/...	Uses per-measure observation dates for freshness classification and dimension coverage windows, then stores monotonic last_observed_date values through the new migration and repository update path.
Integration and API contract validation src/backend/services/analytics/src/domain/metric_definitions/live_tests.rs, src/ingestion/tests/e2e/api/...	Tests tenant override selection, freshness monotonicity, response field types, enum values, ordering, uniqueness, error consistency, and omission of computation internals.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related issues

• INGEST 1: Unified metric system (Metrics catalog) #1561 — Concerns the same unified metric-definition catalog, including tenant overrides, validation, and metadata-driven API exposure.

Possibly related PRs

• constructorfabric/insight#1548 — Modifies the same analytics route and OpenAPI generation wiring.
• constructorfabric/insight#1656 — Introduces the metric-definition validator and repository framework extended by this change.
• constructorfabric/insight#1661 — Adds related status-aware API contract coverage requirements.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Linked Issues check	⚠️ Warning	Issue #1859 is only partially covered: the backend listing, schema status, and last_observed_date are implemented, but the required /metrics frontend catalog is missing.	Add the /metrics frontend catalog with domain grouping, search, counts, direction, dimensions, schema-error tooltips, last-data column, and metrics-v2 gating.
Docstring Coverage	⚠️ Warning	Docstring coverage is 40.54% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title is concise and accurately describes the main change: adding the metric definitions listing endpoint.
Out of Scope Changes check	✅ Passed	The changes stay aligned with #1859 and supporting work for the metric definitions listing, freshness tracking, migrations, and tests.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/metric-definitions-endpoint

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 Checkov (3.3.8)

docs/components/backend/analytics/openapi.json

Traceback (most recent call last):
File "/usr/local/bin/checkov", line 2, in
from checkov.main import Checkov
ModuleNotFoundError: No module named 'checkov'

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

src/backend/services/analytics/src/domain/metric_definitions/repository.rs (1)

619-653: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

COALESCE doesn't fully enforce the stated monotonic-freshness invariant.

The comment says last_observed_date is monotonic knowledge, not per-sweep state, but COALESCE(?, last_observed_date) only protects the "no observation produced" (NULL) case. If a sweep computes a last_observed that is older than the currently stored value (e.g., a ClickHouse TTL/partition drop or backfill temporarily lowers max(metric_date) for a measure), this silently regresses the freshness marker instead of preserving the higher-water mark.

🔧 Suggested direction: never let the marker move backward

-             last_observed_date = COALESCE(?, last_observed_date), \
+             last_observed_date = CASE \
+                 WHEN ? IS NULL THEN last_observed_date \
+                 WHEN last_observed_date IS NULL OR ? > last_observed_date THEN ? \
+                 ELSE last_observed_date \
+             END, \
              updated_at = updated_at \
          WHERE id = ?",
         [
             Value::from(status.as_db()),
             match error_code {
                 Some(code) => Value::from(code.as_db()),
                 None => Value::String(None),
             },
-            match last_observed {
-                Some(date) => Value::from(date.to_string()),
-                None => Value::String(None),
-            },
+            // bound three times, matching the three `?` placeholders in the CASE above
+            last_observed_value(last_observed),
+            last_observed_value(last_observed),
+            last_observed_value(last_observed),
             uuid_value(definition_id),
         ],

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/backend/services/analytics/src/domain/metric_definitions/repository.rs`
around lines 619 - 653, Update update_definition_status so last_observed_date
remains monotonic: preserve the existing database value when the supplied
last_observed date is NULL or older than the stored date, and only advance it
when the supplied date is newer. Implement this in the SQL update expression
while keeping the existing status, error code, and timestamp behavior unchanged.

🧹 Nitpick comments (1)

src/ingestion/tests/e2e/api/test_metric_definitions.py (1)

37-45: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

No test asserts tenant-override values actually win, only that keys collapse to one row.

test_list_metric_definitions_sorted_and_unique proves uniqueness/sorting but not that an override's fields (e.g. label) are the ones returned rather than the product-default's. Given this precedence rule is central to the endpoint's contract, consider seeding a tenant override for a known metric_key and asserting the returned row reflects the override, not the default.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/ingestion/tests/e2e/api/test_metric_definitions.py` around lines 37 - 45,
Extend test_list_metric_definitions_sorted_and_unique to seed a tenant override
for a known metric_key with a distinguishable field value, then locate that
returned metric and assert its overridden field, such as label, is returned
instead of the product-default value. Preserve the existing sorted-key and
uniqueness assertions.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/backend/services/analytics/src/domain/metric_definitions/validator.rs`:
- Around line 223-257: Update check_dimension_coverage and its
dimension_coverage query to anchor each measure’s coverage window using that
measure’s own entry in last_dates, rather than the definition-wide last_observed
maximum. Preserve batching where practical, but ensure stale measures are
evaluated against their own observation date so freshness differences cannot
produce a false DimensionNotCovered result; retain the existing behavior for
measures without observed dates.

---

Outside diff comments:
In `@src/backend/services/analytics/src/domain/metric_definitions/repository.rs`:
- Around line 619-653: Update update_definition_status so last_observed_date
remains monotonic: preserve the existing database value when the supplied
last_observed date is NULL or older than the stored date, and only advance it
when the supplied date is newer. Implement this in the SQL update expression
while keeping the existing status, error code, and timestamp behavior unchanged.

---

Nitpick comments:
In `@src/ingestion/tests/e2e/api/test_metric_definitions.py`:
- Around line 37-45: Extend test_list_metric_definitions_sorted_and_unique to
seed a tenant override for a known metric_key with a distinguishable field
value, then locate that returned metric and assert its overridden field, such as
label, is returned instead of the product-default value. Preserve the existing
sorted-key and uniqueness assertions.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9b200c7d-d1ae-48d3-b132-e998f2c084ba

📥 Commits

Reviewing files that changed from the base of the PR and between fd92d5b and 0bf278e5f91178a4915067b7ad943925b7ec723b.

📒 Files selected for processing (12)

• docs/components/backend/analytics/openapi.json
• src/backend/services/analytics/src/api/metric_definitions.rs
• src/backend/services/analytics/src/api/mod.rs
• src/backend/services/analytics/src/domain/metric_definitions/definition.rs
• src/backend/services/analytics/src/domain/metric_definitions/error_code.rs
• src/backend/services/analytics/src/domain/metric_definitions/listing.rs
• src/backend/services/analytics/src/domain/metric_definitions/mod.rs
• src/backend/services/analytics/src/domain/metric_definitions/repository.rs
• src/backend/services/analytics/src/domain/metric_definitions/validator.rs
• src/backend/services/analytics/src/migration/m20260722_000001_metric_definition_last_observed.rs
• src/backend/services/analytics/src/migration/mod.rs
• src/ingestion/tests/e2e/api/test_metric_definitions.py