Skip to content

ci(backend): calculate code coverage baseline in backend checks - #1429

Merged
ktursunov merged 7 commits into
constructorfabric:mainfrom
ktursunov:ci/backend-coverage-baseline
Jun 23, 2026
Merged

ci(backend): calculate code coverage baseline in backend checks#1429
ktursunov merged 7 commits into
constructorfabric:mainfrom
ktursunov:ci/backend-coverage-baseline

Conversation

@ktursunov

@ktursunov ktursunov commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

What

Adds report-only, self-contained code-coverage reporting for src/backend to the Backend Lint & Test workflow — a coverage summary on every backend PR, with no external service and no plan requirement (works on Free).

  • Rust workspace (analytics-api, api-gateway, insight-clickhouse, oidc-authn-plugin): cargo-llvm-cov runs the suite under instrumentation (replacing plain cargo test, so tests still run) and emits a single Cobertura report.
  • Identity (.NET): dotnet test --collect:"XPlat Code Coverage" on the unit + integration projects; the two Cobertura files are merged with dotnet-coverage.
  • Each is rendered to markdown by irongut/CodeCoverageSummary and posted as a sticky PR comment + appended to the Actions job summary. No HTML, no artifacts, no Codecov.

Adds coverlet.collector (dev-only, PrivateAssets=all) to the two Identity test projects so the XPlat Code Coverage collector emits Cobertura.

Baseline (from CI, all tests green)

Component Line Notes
Rust workspace 62% (4392/7129) insight-clickhouse 89%, oidc-authn-plugin 86%, api-gateway 0% (thin binary, no tests), analytics-api mixed
Identity (.NET) 87% (2618/3011), branch 81% Domain 97%, Api 86%, Infrastructure 83%

Scope / non-goals

Deliberately thin — only calculates and exposes the baseline. Out of scope (future follow-ups under #1384): coverage gates / ratchet, a shared task-runner so local == CI, cargo-nextest, connector (Python) coverage.

GitHub-native coverage (Code Quality) was evaluated and rejected for now: it's a paid Team/Enterprise-Cloud product (GA 2026-07-20, $10/active-committer/mo), not free for public repos, and this org is on Free — the upload API returns HTTP 403 ("Code quality is not enabled for this repository"). The Cobertura we emit would feed it directly (actions/upload-code-coverage) if the org ever upgrades.

Test plan

  • backend-checks.yml runs on this PR (touches src/backend/** + the workflow). Both jobs run the full unit (+ integration) suites and post their coverage comments.
  • Verified on CI and locally in containers: Rust 315 tests pass; Identity unit + integration pass.

Closes #1384

Summary by CodeRabbit

  • Chores
    • Updated backend CI to gate backend-related PRs and cancel in-progress runs for the same workflow/ref, improving build efficiency.
    • Added automated code coverage instrumentation and reporting for backend test runs, including Cobertura output and summarized results in job run notes.
    • Enhanced unit and integration test coverage collection to consistently generate merged coverage artifacts for clearer visibility.

@coderabbitai

coderabbitai Bot commented Jun 22, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 7d2bc879-47fd-449d-ab53-2d57cb4d51f6

📥 Commits

Reviewing files that changed from the base of the PR and between 09003d4 and 151f392.

📒 Files selected for processing (1)
  • .github/workflows/backend-checks.yml
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/backend-checks.yml

📝 Walkthrough

Walkthrough

The backend CI workflow gains concurrency cancellation and minimal token permissions. Rust test execution is replaced with cargo llvm-cov to produce a Cobertura XML report summarised as markdown in the job summary. Both .NET Identity test projects add the coverlet.collector package, and the CI workflow collects, merges, and summarises .NET coverage the same way.

Changes

Backend CI Code Coverage

Layer / File(s) Summary
Workflow concurrency and permissions
.github/workflows/backend-checks.yml
Adds concurrency group with cancel-in-progress: true, restricts permissions to contents: read, and documents PR/manual dispatch gating.
Rust coverage instrumentation and summary
.github/workflows/backend-checks.yml
Adds llvm-tools-preview to the Rust toolchain components; replaces cargo test --all with cargo llvm-cov (Cobertura XML output); appends a markdown coverage table to $GITHUB_STEP_SUMMARY via CodeCoverageSummary with a missing-file fallback.
coverlet.collector package references
src/backend/services/identity/tests/Insight.Identity.Tests.Integration/Insight.Identity.Tests.Integration.csproj, src/backend/services/identity/tests/Insight.Identity.Tests.Unit/Insight.Identity.Tests.Unit.csproj
Adds coverlet.collector 6.0.2 with PrivateAssets=all to both .NET test projects so dotnet test can emit Cobertura coverage without transitive exposure.
.NET coverage collection, merge, and summary
.github/workflows/backend-checks.yml
Extends integration test step with --collect "XPlat Code Coverage" into ./coverage; adds dotnet-coverage merge step producing merged.cobertura.xml; generates and appends markdown summary to the job summary with if: always() and fallback.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related issues

  • #1384 – Add code coverage calculation for src/backend (Rust + .NET): This PR directly implements all acceptance criteria from that issue: cargo-llvm-cov for Rust, coverlet.collector + dotnet-coverage merge for .NET Identity, and per-stack summaries posted to the GitHub Actions job summary.
  • #1410: The PR adds exactly the cargo llvm-cov and coverlet.collector coverage infrastructure that issue #1410 proposes for the backend, laying the groundwork for coverage-based gating and enforcement.

Poem

🐇 Hop hop, the coverage grows,
In XML and markdown it flows.
Rust runs cov, .NET collects,
Merged and merged until it checks —
The job summary blooms with stats,
All green! No room for coverage gaps! 🌿

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title accurately summarizes the main change: adding automated code coverage calculation to the backend CI workflow as a foundational, report-only baseline.
Linked Issues check ✅ Passed All acceptance criteria from #1384 are met: coverage is collected for both Rust (cargo-llvm-cov) and .NET (Coverlet), summaries are rendered to markdown and posted via job summary, coverage is report-only and never fails backend-checks, and baseline coverage is established.
Out of Scope Changes check ✅ Passed All changes are directly scoped to implementing code coverage collection and reporting for the backend, aligning with #1384 objectives—no unrelated modifications are present.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ktursunov
ktursunov force-pushed the ci/backend-coverage-baseline branch 2 times, most recently from 6002adc to 33dc427 Compare June 22, 2026 13:50
Konstantin Tursunov added 4 commits June 22, 2026 16:53
Wire coverage into backend-checks.yml as a report-only signal to establish
a baseline (no gates, no new required checks):

- Rust workspace: cargo-llvm-cov wraps the existing test pass
  (--all-features --workspace), emitting an HTML report + Cobertura, with a
  markdown summary rendered via ReportGenerator.
- Identity (.NET): dotnet test --collect:"XPlat Code Coverage" on the unit +
  integration projects, merged into one report.

Both publish an HTML artifact and a sticky PR comment. Report-only — the
comment step is continue-on-error and nothing fails the job. Thresholds /
ratchet and broader surfaces (connectors) are deferred.

Adds coverlet.collector (dev-only, PrivateAssets=all; never shipped) to the
two Identity test projects so the XPlat Code Coverage collector emits
Cobertura.

Closes constructorfabric#1384

Signed-off-by: Konstantin Tursunov <Konstantin.Tursunov@constructor.tech>
A rebase/force-push left the prior in-flight run executing in parallel
(no concurrency control), double-spending runner minutes on the slow
instrumented coverage build. Add a per-ref concurrency group with
cancel-in-progress so a new push obsoletes the older run.

Refs constructorfabric#1384

Signed-off-by: Konstantin Tursunov <Konstantin.Tursunov@constructor.tech>
Replace the self-contained ReportGenerator/HTML/artifact + sticky-comment
pipeline with GitHub's native code coverage (Code Quality): upload the
Cobertura report via actions/upload-code-coverage; the github-code-quality
bot posts the PR coverage comment.

- Rust: cargo-llvm-cov emits one cobertura.xml -> upload (language Rust).
- Identity: merge the unit + integration cobertura via dotnet-coverage ->
  upload (language C#).
- Add push:main trigger so the native feature has a default-branch baseline
  to compare PRs against; check out the PR head so line numbers map.
- permissions: code-quality:write (dropped pull-requests:write). Uploads are
  fail-on-error:false (report-only) and skip fork PRs automatically.

NOTE: GitHub Code Quality requires Team/Enterprise Cloud; on the Free plan the
upload no-ops (warning only, build stays green).

Refs constructorfabric#1384

Signed-off-by: Konstantin Tursunov <Konstantin.Tursunov@constructor.tech>
… plan)

GitHub-native code coverage (Code Quality) returns HTTP 403 here: it's a
Team/Enterprise-Cloud product (paid at GA, $10/committer/mo), not free for
public repos, and the org is on the Free plan. Switch to a self-contained,
report-only summary that works on any plan:

- Cobertura (already produced) -> markdown via irongut/CodeCoverageSummary
  (no HTML, no .NET-SDK-in-Rust-job), posted as a sticky PR comment and
  appended to the GitHub Actions job summary.
- Identity: merge unit + integration Cobertura via dotnet-coverage first.
- Drop the native upload step, the push:main trigger, and code-quality:write.

Refs constructorfabric#1384

Signed-off-by: Konstantin Tursunov <Konstantin.Tursunov@constructor.tech>
Fork PRs get a read-only GITHUB_TOKEN, so the sticky-comment step 403s
("Resource not accessible by integration"). Remove the comment step from both
jobs and report coverage solely via the GitHub Actions job summary, which
needs no token. Drop the now-unnecessary pull-requests:write permission, and
guard the job-summary step so a skipped merge (after a test failure) doesn't
error on a missing file.

Refs constructorfabric#1384

Signed-off-by: Konstantin Tursunov <Konstantin.Tursunov@constructor.tech>
@ktursunov
ktursunov marked this pull request as ready for review June 22, 2026 14:37
@ktursunov
ktursunov requested a review from a team as a code owner June 22, 2026 14:37

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/backend-checks.yml:
- Line 59: The GitHub Actions in the workflow file use floating version
references instead of full commit SHAs, which violates supply-chain integrity
policy. Replace the floating refs for taiki-e/install-action@cargo-llvm-cov and
any other actions pinned to version tags like `@v1.3.0` with their corresponding
full commit SHA hashes. This applies to all instances across the workflow file
where actions use floating refs to ensure all third-party actions are pinned to
immutable commit SHAs.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: cc4ff2eb-7973-4d3a-9b30-ef37848b1bbf

📥 Commits

Reviewing files that changed from the base of the PR and between cf3161e and 09003d4.

📒 Files selected for processing (3)
  • .github/workflows/backend-checks.yml
  • src/backend/services/identity/tests/Insight.Identity.Tests.Integration/Insight.Identity.Tests.Integration.csproj
  • src/backend/services/identity/tests/Insight.Identity.Tests.Unit/Insight.Identity.Tests.Unit.csproj


- name: Run tests
run: cargo test --all
- uses: taiki-e/install-action@cargo-llvm-cov

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify no non-SHA action refs remain in this workflow.
# Expected: zero matches for third-party `uses:` refs not pinned to 40-char SHA.
rg -nP '^\s*-\s*uses:\s*[^@\s]+@(?!(?:[0-9a-f]{40})$).+$' .github/workflows/backend-checks.yml

Repository: constructorfabric/insight

Length of output: 327


🏁 Script executed:

# Display the workflow file around the mentioned lines with context
sed -n '55,145p' .github/workflows/backend-checks.yml | cat -n

Repository: constructorfabric/insight

Length of output: 4069


🏁 Script executed:

# Display the workflow file around line 139 to verify the third unpinned action
sed -n '135,145p' .github/workflows/backend-checks.yml | cat -n

Repository: constructorfabric/insight

Length of output: 569


Pin third-party GitHub Actions to full commit SHAs.

Lines 59, 73, and 139 use floating refs (@cargo-llvm-cov, @v1.3.0), which violates the current policy and weakens supply-chain integrity.

Suggested changes
-      - uses: taiki-e/install-action@cargo-llvm-cov
+      - uses: taiki-e/install-action@<FULL_40_CHAR_COMMIT_SHA>

-      - name: Coverage summary
-        uses: irongut/CodeCoverageSummary@v1.3.0
+      - name: Coverage summary
+        uses: irongut/CodeCoverageSummary@<FULL_40_CHAR_COMMIT_SHA>

-      - name: Coverage summary
-        uses: irongut/CodeCoverageSummary@v1.3.0
+      - name: Coverage summary
+        uses: irongut/CodeCoverageSummary@<FULL_40_CHAR_COMMIT_SHA>
🧰 Tools
🪛 zizmor (1.25.2)

[error] 59-59: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)

(unpinned-uses)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/backend-checks.yml at line 59, The GitHub Actions in the
workflow file use floating version references instead of full commit SHAs, which
violates supply-chain integrity policy. Replace the floating refs for
taiki-e/install-action@cargo-llvm-cov and any other actions pinned to version
tags like `@v1.3.0` with their corresponding full commit SHA hashes. This applies
to all instances across the workflow file where actions use floating refs to
ensure all third-party actions are pinned to immutable commit SHAs.

Source: Linters/SAST tools

Flip CodeCoverageSummary fail_below_min false -> true in both jobs so
backend-checks fails when overall line coverage drops below the 60% lower
threshold. Coverage is no longer report-only. Current: Rust ~62%, Identity ~87%.

Refs constructorfabric#1384

Signed-off-by: Konstantin Tursunov <Konstantin.Tursunov@constructor.tech>
@ktursunov
ktursunov merged commit 6165649 into constructorfabric:main Jun 23, 2026
20 checks passed
@ktursunov
ktursunov deleted the ci/backend-coverage-baseline branch August 13, 2026 02:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add code coverage calculation for src/backend (Rust + .NET)

2 participants