test(e2e): metric-coverage gate + full bullet-stat assertions (p25/p75) with enforcement

[ktursunov]

What

Two complementary layers of e2e coverage for the analytics-api bullet metrics:

1. Breadth — a metric-coverage gate: every product metric_key the catalog exposes must have its value asserted by a test (or be explicitly skip-listed). A CI job, not a pytest test.
2. Depth — full bullet-stat assertions: every metric test asserts every statistic the API returns for that bullet — value, median, range_min, range_max, and (where the served query computes them) p25/p75 — enforced at runtime so a test can't silently skip a returned number.

Breadth: the metric-coverage gate

Verdict is binary, per metric_key (each number on a bullet is validated independently):

• value-tested — a metrics/*.test.yaml asserts its value (find: {metric_key: …} paired with equal/assert) → PASS
• skip-listed — covered by SKIP_TABLES (a connector-less storage vector — e.g. all ai_bullet_rows.* need a Cursor/Claude/ChatGPT connector) or an explicit SKIP_LIST entry → PASS (accepted baseline)
• neither → FAIL — a number nobody validates must get an assertion or a skip entry.

A stale skip (key no longer in the catalog) or a redundant one (now value-tested) also fails, so the baseline can't rot.

How it runs (no second compose, no extra app boot): the universe is read from the live catalog (POST /v1/catalog/get_metrics), not a fragile static parse of the seed migrations. The e2e suite snapshots the catalog while analytics-api is already up (lib/collect_metrics.py, from the analytics_api fixture teardown) into .artifacts/catalog_metrics.json, uploaded as the coverage-inputs artifact. A separate pure-Python metric-coverage-gate job reads that file (--universe-file) + the test YAMLs and renders the verdict — no Docker, no analytics-api boot in the gate.

Coverage today: 44/96 metric_keys value-tested · 52 baseline-skipped · 0 missing → PASS.

Depth: full bullet-stat assertions (p25/p75) + runtime enforcement

All 34 bullet-metric fixtures assert the complete stat set the API returns:

• A 5-member department so p25/median/p75 are interior quartiles (not collapsed to min/max), seeded over a single custom window + an empty window (proves the date bound excludes out-of-range rows).
• Asserts value/median/range_min/range_max/p25/p75 for collab & task_delivery; the 4-stat subset for wiki & AI/CC, whose served query_refs don't compute quartiles.

No-unasserted-stat enforcement (lib/expect_engine.py): within a case, any find-matched row that carries a stat field MUST assert it (via equal or a CEL assert) — otherwise the rule fails. It is self-scoping: a stat absent from the response is never required, so the 4-stat vectors pass without p25/p75, and empty-window cases (no matched row) are exempt. The equal check compares numbers with tolerance (math.isclose, abs_tol=1e-6) so a fractional stat (ratio / average) isn't rejected by binary float drift, while a real ≥1e-4 difference still fails.

Notable findings (worked around in fixtures; no backend changes)

• AI & wiki IC bullets return only the 4-stat set (no p25/p75) — the self-scoping gate handles this uniformly.
• bronze_zoom.participants is ReplacingMergeTree ORDER BY email, so a person collapses to one Zoom row regardless of distinct meetings → meetings_count is Teams-only (cross-source proof lives in meeting_hours).
• silver.class_* is truncated only at session start, not per-test → a fixture's rows can leak into the next; wiki fixtures use disjoint per-fixture windows so any leaked foreign row falls outside the window (zero rig change).

Contents

• lib/metric_coverage.py — gate logic + SKIP_TABLES (7 vectors) / SKIP_LIST (20 per-key; single source of truth, no side-car).
• lib/collect_metrics.py — live-catalog snapshot script.
• lib/expect_engine.py — no-unasserted-stat enforcement + float-tolerant equal.
• 34 metrics/*.test.yaml redesigned to the full-stat / single-custom-window shape, + 4 shared templates (5-member departments).
• .github/workflows/e2e-bronze-to-api.yml (the e2e job uploads the catalog; the metric-coverage-gate job analyses it), conftest.py, e2e.sh (./e2e.sh gates), README.md, .gitignore.

Validation

• Full suite green through the real bronze→dbt→CH→API pipeline (34 fixtures × custom + empty window).
• Metric-coverage gate: 44/96, 0 missing; FAIL paths verified (uncovered-not-skipped, stale skip, redundant skip).
• Enforcement gate verified self-scoping: 6-stat collab/task fixtures, 4-stat wiki/AI fixtures, and a missing-stat fixture all behave as expected.

Local: ./e2e.sh test (snapshots the catalog) → ./e2e.sh gates.

Summary by CodeRabbit

• Bug Fixes
  • Strengthened E2E assertions to require every returned stat field to be explicitly validated, including per-row completeness checks and tolerant numeric comparisons.
• New Features
  • Added an E2E metric coverage gate that snapshots the metric catalog, evaluates whether metrics are fully covered by tests vs skips, and fails on uncovered, stale, or inconsistent coverage.
  • Added a local “gates” mode to run the metric coverage check inside the runner container.
• Documentation / Tests
  • Updated the E2E framework README and refreshed numerous metric fixtures/expected outputs for the new department/window scenarios.

[coderabbitai]

Warning

Review limit reached

@ktursunov, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 17 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: bb8f2528-558b-4b50-9c28-f69db6dc91e9

📥 Commits

Reviewing files that changed from the base of the PR and between 64ecc5d5399242df511bd56fe012e031883c0591 and bd3d802.

📒 Files selected for processing (53)

• .github/workflows/e2e-bronze-to-api.yml
• src/ingestion/tests/e2e/.gitignore
• src/ingestion/tests/e2e/README.md
• src/ingestion/tests/e2e/conftest.py
• src/ingestion/tests/e2e/e2e.sh
• src/ingestion/tests/e2e/lib/collect_metrics.py
• src/ingestion/tests/e2e/lib/expect_engine.py
• src/ingestion/tests/e2e/lib/metric_coverage.py
• src/ingestion/tests/e2e/meta/test_expect_engine.py
• src/ingestion/tests/e2e/metrics/ai_ai_loc_share2.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_active.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_cost.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_lines.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_overage.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_sessions.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_tool_accept.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_tool_acceptance.test.yaml
• src/ingestion/tests/e2e/metrics/ai_claude_web.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cursor_acceptance.test.yaml
• src/ingestion/tests/e2e/metrics/ai_prs_total.test.yaml
• src/ingestion/tests/e2e/metrics/ai_prs_with_cc.test.yaml
• src/ingestion/tests/e2e/metrics/collab_active_days.test.yaml
• src/ingestion/tests/e2e/metrics/collab_emails_read.test.yaml
• src/ingestion/tests/e2e/metrics/collab_emails_received.test.yaml
• src/ingestion/tests/e2e/metrics/collab_emails_sent.test.yaml
• src/ingestion/tests/e2e/metrics/collab_files_engaged.test.yaml
• src/ingestion/tests/e2e/metrics/collab_files_shared_external.test.yaml
• src/ingestion/tests/e2e/metrics/collab_files_shared_internal.test.yaml
• src/ingestion/tests/e2e/metrics/collab_meeting_free.test.yaml
• src/ingestion/tests/e2e/metrics/collab_meeting_hours.test.yaml
• src/ingestion/tests/e2e/metrics/collab_meetings_count.test.yaml
• src/ingestion/tests/e2e/metrics/collab_teams_chats.test.yaml
• src/ingestion/tests/e2e/metrics/collab_teams_meeting_hours.test.yaml
• src/ingestion/tests/e2e/metrics/collab_teams_meetings.test.yaml
• src/ingestion/tests/e2e/metrics/collab_zulip_chat.test.yaml
• src/ingestion/tests/e2e/metrics/task_delivery_bugs_to_task_ratio_jira.test.yaml
• src/ingestion/tests/e2e/metrics/task_delivery_due_date_compliance_jira.test.yaml
• src/ingestion/tests/e2e/metrics/task_delivery_tasks_completed_jira.test.yaml
• src/ingestion/tests/e2e/metrics/team_bullet_collab_emails_sent.test.yaml
• src/ingestion/tests/e2e/metrics/team_bullet_task_delivery_tasks_completed.test.yaml
• src/ingestion/tests/e2e/metrics/templates/confluence_wiki_pages.yaml
• src/ingestion/tests/e2e/metrics/templates/jira_task.yaml
• src/ingestion/tests/e2e/metrics/templates/m365_email.yaml
• src/ingestion/tests/e2e/metrics/templates/outline_wiki_pages.yaml
• src/ingestion/tests/e2e/metrics/templates/people.yaml
• src/ingestion/tests/e2e/metrics/wiki_confluence_active_authors.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_confluence_comments.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_confluence_edits.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_confluence_pages_created.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_outline_active_authors.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_outline_comments.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_outline_edits.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_outline_pages_created.test.yaml

📝 Walkthrough

Walkthrough

Adds a metric coverage gate and catalog snapshot artifact flow, tightens expectation checks so all returned stat fields must be asserted, and updates shared templates plus AI, collaboration, task-delivery, and wiki E2E fixtures to department-of-5 scenarios with empty-window cases.

Changes

Metric Coverage Gate and E2E Fixture Refresh

Layer / File(s)	Summary
Catalog snapshot collection src/ingestion/tests/e2e/lib/collect_metrics.py, src/ingestion/tests/e2e/conftest.py, src/ingestion/tests/e2e/.gitignore	collect_metrics.py posts to /v1/catalog/get_metrics and writes catalog_metrics.json; conftest.py runs it during analytics_api teardown before stopping the process, and .artifacts/ is ignored.
Metric coverage gate script src/ingestion/tests/e2e/lib/metric_coverage.py	The new gate loads the metric universe, scans YAML tests for asserted metric keys, resolves skip rules, and renders a Markdown coverage report with pass/fail exit handling.
Workflow and local gate wiring .github/workflows/e2e-bronze-to-api.yml, src/ingestion/tests/e2e/e2e.sh, src/ingestion/tests/e2e/README.md	The workflow uploads the metric snapshot artifact and runs a downstream coverage job; e2e.sh adds gates, and the README documents the gate flow and local commands.
Expect engine stat enforcement src/ingestion/tests/e2e/lib/expect_engine.py, src/ingestion/tests/e2e/meta/test_expect_engine.py	evaluate_case now tracks asserted stat fields per matched row and fails when any present stat field is left unasserted; tests were expanded to cover the stricter checks.
Shared fixture templates src/ingestion/tests/e2e/metrics/templates/people.yaml, src/ingestion/tests/e2e/metrics/templates/m365_email.yaml, src/ingestion/tests/e2e/metrics/templates/confluence_wiki_pages.yaml, src/ingestion/tests/e2e/metrics/templates/outline_wiki_pages.yaml, src/ingestion/tests/e2e/metrics/templates/jira_task.yaml	Adds dave and erin template records for people, email, wiki, outline, and Jira fixtures.
AI metric fixtures src/ingestion/tests/e2e/metrics/ai_*.test.yaml	AI metric fixtures are updated to five-member Engineering scenarios, revised bronze seeds, full stat assertions, and empty-window cases.
Collaboration metric fixtures src/ingestion/tests/e2e/metrics/collab_*.test.yaml, src/ingestion/tests/e2e/metrics/team_bullet_collab_emails_sent.test.yaml	Collaboration fixtures are rewritten for five-member department scenarios with revised bronze data, distribution assertions, and empty-window tests.
Task delivery and wiki metric fixtures src/ingestion/tests/e2e/metrics/task_delivery_*.test.yaml, src/ingestion/tests/e2e/metrics/team_bullet_task_delivery_tasks_completed.test.yaml, src/ingestion/tests/e2e/metrics/wiki_*.test.yaml	Jira task-delivery and wiki fixtures are updated to department-of-5 scenarios with distribution assertions and empty-window cases, plus new team-bullet coverage.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related issues

• Metric smoke: Collaboration metrics #1441: The changes heavily modify the same collaboration metric E2E YAMLs and coverage logic, but around validation and gating.

Possibly related PRs

• constructorfabric/insight#1287: Related through the analytics-api tenant header behavior used by the new catalog snapshot step.
• constructorfabric/insight#1371: Aligns with the ai_cc_overage E2E fixture updates.
• constructorfabric/insight#1509: Directly overlaps with the expect_engine stat-field enforcement changes.

Suggested reviewers: cyberantonz, mitasovr

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 61.29% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the two main changes: the metric-coverage gate and enforcement of full bullet-stat assertions, including p25/p75.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Caution

Review failed

An error occurred during the review process. Please try again later.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 11

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/ingestion/tests/e2e/lib/expect_engine.py`:
- Around line 50-52: The stat detection in _stat_refs_in_cel is too broad
because it matches any whole-word stat token, even inside strings or on other
objects. Update _stat_refs_in_cel to detect only actual it.<stat> field access
in the CEL expression, and keep the result flowing into the no-unasserted-stat
check in the code that uses it. Use the existing _STAT_FIELDS and the caller
around the ledger update to ensure only real reads of it.value, it.median,
it.p25, etc. are counted as covered.

In `@src/ingestion/tests/e2e/lib/metric_coverage.py`:
- Around line 106-129: The skip hygiene logic only tracks explicit SKIP_LIST
entries, so SKIP_TABLES entries can become stale or fully redundant without
being flagged. Update the skip-gate flow around resolve_skips so table-based
skips are also included in the inputs checked by stale_skips and
redundant_skips, and make sure the baseline comparison treats SKIP_TABLES-backed
keys as first-class skip entries. Use the existing resolve_skips helper and the
stale_skips/redundant_skips call sites to locate the change and keep the
duplicate/doubly-specified validation intact.
- Around line 174-180: The asserted_keys_from_tests() scan is only looking at
top-level files with glob("*.test.yaml"), so nested metric fixtures under
subdirectories are missed. Update the file discovery in metric_coverage.py to
recurse through metrics_dir (and keep the existing parsing in
asserted_keys_from_tests()) so it matches the metrics/**/*.test.yaml layout and
collects metric_key values from all nested fixtures.

In `@src/ingestion/tests/e2e/metrics/collab_files_engaged.test.yaml`:
- Around line 30-57: The current m365_files_engaged fixture only validates
OneDrive aggregation, so it can miss regressions where SharePoint rows are
dropped or not summed with OneDrive. Update the collab_files_engaged test to add
a SharePoint-backed case or assertion alongside the existing
bronze_m365.onedrive_activity coverage, using the m365_files_engaged metric_key
and a query that verifies additive behavior across both products. Make sure the
new expectation is anchored in the existing test cases structure so it exercises
the same metric path with SharePoint data included.

In `@src/ingestion/tests/e2e/metrics/collab_files_shared_external.test.yaml`:
- Around line 29-56: The current `m365_files_shared_external` fixture only
exercises the OneDrive path, so it no longer covers the intended cross-product
aggregation. Add at least one SharePoint ingestion row and a corresponding
expectation in `collab_files_shared_external.test.yaml` using the same
`m365_files_shared_external` metric query, so `bronze_m365.sharepoint_activity`
is validated alongside `bronze_m365.onedrive_activity` and any additivity or
SharePoint regression is caught.

In `@src/ingestion/tests/e2e/metrics/collab_files_shared_internal.test.yaml`:
- Around line 29-56: The e2e case for m365_files_shared_internal only seeds and
checks bronze_m365.onedrive_activity, so it does not verify the SharePoint side
of the combined metric. Update this test to also seed SharePoint activity rows
alongside the existing OneDrive fixtures, and extend the expectation so
m365_files_shared_internal proves both sources are included and summed together
for the selected person. Use the existing collaboration query case and
metric_key assertion to keep the check in the same test while covering both
products.

In `@src/ingestion/tests/e2e/metrics/collab_meeting_hours.test.yaml`:
- Around line 28-53: The collab_meeting_hours test currently covers only the
Teams-backed `meeting_hours` path, so a Zoom or cross-source aggregation
regression could slip through. Update
`src/ingestion/tests/e2e/metrics/collab_meeting_hours.test.yaml` to keep at
least one `meeting_hours` assertion that exercises Zoom-backed or Teams+Zoom
additive behavior alongside the existing `collaboration` case, using the same
metric query/expect structure so the `meeting_hours` calculation is validated
across sources.

In `@src/ingestion/tests/e2e/metrics/collab_meetings_count.test.yaml`:
- Around line 18-24: The coverage note in this test is now stale because the
referenced additive proof no longer exists in collab_meeting_hours either.
Update the comment in collab_meetings_count.test.yaml to reflect the current
Teams-only fixture behavior, and add or point to a different test fixture that
actually exercises both Zoom and Teams together if cross-source additivity still
needs coverage. Use the existing metric test names and fixture setup in
collab_meetings_count and collab_meeting_hours to locate the affected assertions
and comments.

In `@src/ingestion/tests/e2e/metrics/wiki_outline_edits.test.yaml`:
- Around line 46-48: The wiki_edits expectation in this test fixture only checks
value, median, and range, so add explicit quartile assertions for the same
5-item distribution used elsewhere. Update the wiki metric block in
wiki_outline_edits.test.yaml to include p25 and p75 alongside the existing equal
checks, using the same quartile values that correspond to the {1,2,3,4,5}
dataset so the no-unasserted-stat gate is satisfied.

In `@src/ingestion/tests/e2e/metrics/wiki_outline_pages_created.test.yaml`:
- Around line 56-58: The wiki_pages_created metrics fixture is missing quartile
assertions for the selected wiki row, so update the expectation in the wiki
outline pages created test to include p25 and p75 alongside value, median,
range_min, and range_max. Use the existing metric_key match for
wiki_pages_created and align the expected quartiles with the underlying
{1,2,3,4,5} distribution so the new expect-engine checks pass.

In `@src/ingestion/tests/e2e/README.md`:
- Around line 95-99: Update the command examples in the README snippet to match
the current e2e workflow: the `./e2e.sh gates` example should describe the
metric-coverage gate only, and the `metric_coverage.py` example should use its
supported arguments instead of `--md`. Adjust the surrounding usage note so it
no longer implies all three gates run or that the ad hoc command will work with
the removed flag.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 7039e68d-dbbb-4918-b62d-b15e67bcfac3

📥 Commits

Reviewing files that changed from the base of the PR and between 1cb8c6a and de6b0cf0b734e831122f271e7c74baddf38aff58.

📒 Files selected for processing (53)

• .github/workflows/e2e-bronze-to-api.yml
• src/ingestion/tests/e2e/.gitignore
• src/ingestion/tests/e2e/README.md
• src/ingestion/tests/e2e/conftest.py
• src/ingestion/tests/e2e/e2e.sh
• src/ingestion/tests/e2e/lib/collect_metrics.py
• src/ingestion/tests/e2e/lib/expect_engine.py
• src/ingestion/tests/e2e/lib/metric_coverage.py
• src/ingestion/tests/e2e/meta/test_expect_engine.py
• src/ingestion/tests/e2e/metrics/ai_ai_loc_share2.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_active.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_cost.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_lines.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_overage.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_sessions.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_tool_accept.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_tool_acceptance.test.yaml
• src/ingestion/tests/e2e/metrics/ai_claude_web.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cursor_acceptance.test.yaml
• src/ingestion/tests/e2e/metrics/ai_prs_total.test.yaml
• src/ingestion/tests/e2e/metrics/ai_prs_with_cc.test.yaml
• src/ingestion/tests/e2e/metrics/collab_active_days.test.yaml
• src/ingestion/tests/e2e/metrics/collab_emails_read.test.yaml
• src/ingestion/tests/e2e/metrics/collab_emails_received.test.yaml
• src/ingestion/tests/e2e/metrics/collab_emails_sent.test.yaml
• src/ingestion/tests/e2e/metrics/collab_files_engaged.test.yaml
• src/ingestion/tests/e2e/metrics/collab_files_shared_external.test.yaml
• src/ingestion/tests/e2e/metrics/collab_files_shared_internal.test.yaml
• src/ingestion/tests/e2e/metrics/collab_meeting_free.test.yaml
• src/ingestion/tests/e2e/metrics/collab_meeting_hours.test.yaml
• src/ingestion/tests/e2e/metrics/collab_meetings_count.test.yaml
• src/ingestion/tests/e2e/metrics/collab_teams_chats.test.yaml
• src/ingestion/tests/e2e/metrics/collab_teams_meeting_hours.test.yaml
• src/ingestion/tests/e2e/metrics/collab_teams_meetings.test.yaml
• src/ingestion/tests/e2e/metrics/collab_zulip_chat.test.yaml
• src/ingestion/tests/e2e/metrics/task_delivery_bugs_to_task_ratio_jira.test.yaml
• src/ingestion/tests/e2e/metrics/task_delivery_due_date_compliance_jira.test.yaml
• src/ingestion/tests/e2e/metrics/task_delivery_tasks_completed_jira.test.yaml
• src/ingestion/tests/e2e/metrics/team_bullet_collab_emails_sent.test.yaml
• src/ingestion/tests/e2e/metrics/team_bullet_task_delivery_tasks_completed.test.yaml
• src/ingestion/tests/e2e/metrics/templates/confluence_wiki_pages.yaml
• src/ingestion/tests/e2e/metrics/templates/jira_task.yaml
• src/ingestion/tests/e2e/metrics/templates/m365_email.yaml
• src/ingestion/tests/e2e/metrics/templates/outline_wiki_pages.yaml
• src/ingestion/tests/e2e/metrics/templates/people.yaml
• src/ingestion/tests/e2e/metrics/wiki_confluence_active_authors.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_confluence_comments.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_confluence_edits.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_confluence_pages_created.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_outline_active_authors.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_outline_comments.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_outline_edits.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_outline_pages_created.test.yaml

[ktursunov]

@coderabbitai review

The 5 actionable findings from the previous review are addressed in the latest commits:

• expect_engine _stat_refs_in_cel now matches only real it.<field> access (string-literal stat names no longer count).
• metric_coverage flags a stale SKIP_TABLES vector, and discovers fixtures with rglob (**/*.test.yaml).
• Fixed the stale cross-source note in collab_meetings_count and the stale README gate examples (single gate, no --md, 44/96).

The wiki-quartile comments were withdrawn (wiki returns no p25/p75). The SharePoint/Zoom cross-source comments are intentional per the single-scope simplification (see replies).

[coderabbitai]

@ktursunov Thanks for the detailed summary. Kicking off a fresh review to verify the fixes to _stat_refs_in_cel, the SKIP_TABLES staleness check, rglob-based fixture discovery, the corrected cross-source note in collab_meetings_count, and the refreshed README gate examples. I've also noted the withdrawal of the wiki-quartile and SharePoint/Zoom cross-source comments per your explanations.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

🧹 Nitpick comments (1)

.github/workflows/e2e-bronze-to-api.yml (1)

51-51: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Disable credential persistence on checkout.

Neither job pushes back to the repo; persisting the token in .git/config for the rest of the job is unnecessary exposure. Flagged by static analysis (artipacked).

🔒 Proposed fix

       - uses: actions/checkout@v4
+        with:
+          persist-credentials: false

Apply to both the e2e job (line 51) and the metric-coverage-gate job (line 126).

Also applies to: 126-126

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/e2e-bronze-to-api.yml at line 51, Disable credential
persistence in the checkout steps used by the e2e-bronze-to-api workflow. Update
both occurrences of actions/checkout@v4 in the e2e and metric-coverage-gate jobs
to prevent the token from being stored in .git/config, since neither job needs
to push changes back to the repository.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In @.github/workflows/e2e-bronze-to-api.yml:
- Line 51: Disable credential persistence in the checkout steps used by the
e2e-bronze-to-api workflow. Update both occurrences of actions/checkout@v4 in
the e2e and metric-coverage-gate jobs to prevent the token from being stored in
.git/config, since neither job needs to push changes back to the repository.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b412bb2c-b04a-4bd9-aee6-0eefa358d481

📥 Commits

Reviewing files that changed from the base of the PR and between b1459124aa9fdecd4022317841ec9a6d8980914e and 830a0061ea641ead74f711a398e785e3a4bafea7.

📒 Files selected for processing (53)

• .github/workflows/e2e-bronze-to-api.yml
• src/ingestion/tests/e2e/.gitignore
• src/ingestion/tests/e2e/README.md
• src/ingestion/tests/e2e/conftest.py
• src/ingestion/tests/e2e/e2e.sh
• src/ingestion/tests/e2e/lib/collect_metrics.py
• src/ingestion/tests/e2e/lib/expect_engine.py
• src/ingestion/tests/e2e/lib/metric_coverage.py
• src/ingestion/tests/e2e/meta/test_expect_engine.py
• src/ingestion/tests/e2e/metrics/ai_ai_loc_share2.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_active.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_cost.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_lines.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_overage.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_sessions.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_tool_accept.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_tool_acceptance.test.yaml
• src/ingestion/tests/e2e/metrics/ai_claude_web.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cursor_acceptance.test.yaml
• src/ingestion/tests/e2e/metrics/ai_prs_total.test.yaml
• src/ingestion/tests/e2e/metrics/ai_prs_with_cc.test.yaml
• src/ingestion/tests/e2e/metrics/collab_active_days.test.yaml
• src/ingestion/tests/e2e/metrics/collab_emails_read.test.yaml
• src/ingestion/tests/e2e/metrics/collab_emails_received.test.yaml
• src/ingestion/tests/e2e/metrics/collab_emails_sent.test.yaml
• src/ingestion/tests/e2e/metrics/collab_files_engaged.test.yaml
• src/ingestion/tests/e2e/metrics/collab_files_shared_external.test.yaml
• src/ingestion/tests/e2e/metrics/collab_files_shared_internal.test.yaml
• src/ingestion/tests/e2e/metrics/collab_meeting_free.test.yaml
• src/ingestion/tests/e2e/metrics/collab_meeting_hours.test.yaml
• src/ingestion/tests/e2e/metrics/collab_meetings_count.test.yaml
• src/ingestion/tests/e2e/metrics/collab_teams_chats.test.yaml
• src/ingestion/tests/e2e/metrics/collab_teams_meeting_hours.test.yaml
• src/ingestion/tests/e2e/metrics/collab_teams_meetings.test.yaml
• src/ingestion/tests/e2e/metrics/collab_zulip_chat.test.yaml
• src/ingestion/tests/e2e/metrics/task_delivery_bugs_to_task_ratio_jira.test.yaml
• src/ingestion/tests/e2e/metrics/task_delivery_due_date_compliance_jira.test.yaml
• src/ingestion/tests/e2e/metrics/task_delivery_tasks_completed_jira.test.yaml
• src/ingestion/tests/e2e/metrics/team_bullet_collab_emails_sent.test.yaml
• src/ingestion/tests/e2e/metrics/team_bullet_task_delivery_tasks_completed.test.yaml
• src/ingestion/tests/e2e/metrics/templates/confluence_wiki_pages.yaml
• src/ingestion/tests/e2e/metrics/templates/jira_task.yaml
• src/ingestion/tests/e2e/metrics/templates/m365_email.yaml
• src/ingestion/tests/e2e/metrics/templates/outline_wiki_pages.yaml
• src/ingestion/tests/e2e/metrics/templates/people.yaml
• src/ingestion/tests/e2e/metrics/wiki_confluence_active_authors.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_confluence_comments.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_confluence_edits.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_confluence_pages_created.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_outline_active_authors.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_outline_comments.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_outline_edits.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_outline_pages_created.test.yaml

✅ Files skipped from review due to trivial changes (7)

• src/ingestion/tests/e2e/metrics/templates/outline_wiki_pages.yaml
• src/ingestion/tests/e2e/metrics/ai_prs_with_cc.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_outline_active_authors.test.yaml
• src/ingestion/tests/e2e/metrics/ai_ai_loc_share2.test.yaml
• src/ingestion/tests/e2e/metrics/ai_claude_web.test.yaml
• src/ingestion/tests/e2e/metrics/templates/confluence_wiki_pages.yaml
• src/ingestion/tests/e2e/.gitignore

🚧 Files skipped from review as they are similar to previous changes (37)

• src/ingestion/tests/e2e/metrics/ai_cursor_acceptance.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_cost.test.yaml
• src/ingestion/tests/e2e/metrics/ai_prs_total.test.yaml
• src/ingestion/tests/e2e/e2e.sh
• src/ingestion/tests/e2e/metrics/templates/m365_email.yaml
• src/ingestion/tests/e2e/metrics/collab_files_shared_internal.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_overage.test.yaml
• src/ingestion/tests/e2e/meta/test_expect_engine.py
• src/ingestion/tests/e2e/metrics/collab_zulip_chat.test.yaml
• src/ingestion/tests/e2e/metrics/collab_active_days.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_active.test.yaml
• src/ingestion/tests/e2e/metrics/templates/people.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_tool_accept.test.yaml
• src/ingestion/tests/e2e/metrics/team_bullet_collab_emails_sent.test.yaml
• src/ingestion/tests/e2e/metrics/collab_emails_received.test.yaml
• src/ingestion/tests/e2e/metrics/collab_meeting_free.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_sessions.test.yaml
• src/ingestion/tests/e2e/metrics/collab_meetings_count.test.yaml
• src/ingestion/tests/e2e/metrics/collab_emails_read.test.yaml
• src/ingestion/tests/e2e/metrics/collab_teams_chats.test.yaml
• src/ingestion/tests/e2e/metrics/templates/jira_task.yaml
• src/ingestion/tests/e2e/metrics/wiki_confluence_pages_created.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_lines.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_outline_edits.test.yaml
• src/ingestion/tests/e2e/metrics/ai_cc_tool_acceptance.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_outline_pages_created.test.yaml
• src/ingestion/tests/e2e/metrics/collab_files_shared_external.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_outline_comments.test.yaml
• src/ingestion/tests/e2e/metrics/collab_teams_meetings.test.yaml
• src/ingestion/tests/e2e/metrics/collab_files_engaged.test.yaml
• src/ingestion/tests/e2e/metrics/collab_teams_meeting_hours.test.yaml
• src/ingestion/tests/e2e/metrics/collab_meeting_hours.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_confluence_edits.test.yaml
• src/ingestion/tests/e2e/metrics/collab_emails_sent.test.yaml
• src/ingestion/tests/e2e/metrics/wiki_confluence_comments.test.yaml
• src/ingestion/tests/e2e/lib/metric_coverage.py
• src/ingestion/tests/e2e/metrics/wiki_confluence_active_authors.test.yaml