feat(bench): freeze demo corpus as a release asset + wire bench capture: otel-demo (PR-N4)

[jensholdgaard]

What

Wires the validated OTel Demo corpus into the bench as a frozen, reproducible release asset. Two halves:

1. capture-otel-demo-corpus.yml — publish a release

• New release_tag input. When set, the workflow gzips logs.jsonl + manifest.md and publishes them as assets on a GitHub release at that tag — creating the release, or --clobbering assets on re-mint.
• Provenance in the release notes: demo_ref, locust_users, duration_seconds, source commit, record count, uncompressed bytes.
• Job permission raised to contents: write (manual-dispatch workflow only).

2. bench.yml — capture: otel-demo

• New otel-demo choice + corpus_release_tag input (default corpus/otel-demo-v1).
• When selected: gh release download the frozen corpus, gunzip, bench it. Every run sees identical bytes (RFC 0006 §J4 reproducibility) — unlike a live capture.
• The fetch step emits corpus_dir exactly like the telemetrygen step; BENCH_CORPUS prefers whichever capture step ran (capture ∥ fetch ∥ input). telemetrygen + none paths untouched.

Why a frozen release, not capture-and-bench in one run

A live capture produces different bytes every run (timing, ordering, load variation), which breaks A1/C1/C2 reproducibility — the bench would be a smoke test, not a thesis gate. Minting the corpus once and freezing it as a versioned asset makes the bench deterministic and the §9 numbers re-derivable. Re-minting (e.g. v2) is a deliberate act with its own provenance.

Validation (in progress)

Branch-dispatching the mint (capture-otel-demo-corpus.yml with release_tag=corpus/otel-demo-v1, 40 users / 1800s) then bench.yml capture=otel-demo to get the first real A1/C1/C2 on the demo corpus. Will post the numbers before merging.

Notes

• No Rust changes; both YAML files validate, cargo fmt --all --check clean.
• Security: corpus_release_tag flows via env:, not ${{ }} splicing, per the established template-injection rule.

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Added otel-demo as a corpus capture strategy for benchmarking workflows
  • Can fetch frozen corpus assets from GitHub releases via a new release-tag input
  • Can publish captured corpus data to GitHub releases with generated manifest, counts and size info

• Bug Fixes / Improvements

  • Bench execution and results summary now consistently select the effective corpus source (capture output, fetched release, or provided corpus)

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: adf779a6-e8c6-498c-98fa-3888f1048d09

📥 Commits

Reviewing files that changed from the base of the PR and between 529d242 and dce0ca3.

📒 Files selected for processing (1)

• .github/workflows/capture-otel-demo-corpus.yml

🚧 Files skipped from review as they are similar to previous changes (1)

• .github/workflows/capture-otel-demo-corpus.yml

---

📝 Walkthrough

Walkthrough

Bench workflow gains otel-demo capture support and a corpus_release_tag input to download a frozen logs.jsonl.gz corpus; bench corpus selection prefers telemetrygen output, then otel-demo fetch, then operator inputs.corpus_dir. The capture workflow can optionally publish gzipped corpus and manifest to a release tag.

Changes

Frozen OTel Demo Corpus Release Support

Layer / File(s)	Summary
Bench corpus inputs and download step .github/workflows/bench.yml	capture input extended with otel-demo choice; new corpus_release_tag input added. New step fetches logs.jsonl.gz from specified release, unzips to bench-corpus/otel-demo-v1, and outputs corpus_dir.
Bench corpus source selection .github/workflows/bench.yml	BENCH_CORPUS env variable selection in bench run and job summary steps prioritizes telemetrygen output, then otel-demo fetch output, then operator-supplied inputs.corpus_dir.
Capture workflow release publishing .github/workflows/capture-otel-demo-corpus.yml	New release_tag input and permissions.contents elevated to write. Conditional step gzips logs.jsonl, derives record/byte counts, formats structured release notes (demo ref, locust users, duration, source commit, records, size), and publishes assets to the specified release tag (upload or create).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• jensholdgaard/ourios#73: Adds the capture-otel-demo-corpus workflow that produces captured/logs.jsonl consumed by this PR's download step.
• jensholdgaard/ourios#70: Earlier changes to bench.yml that add telemetrygen capture wiring and corpus selection logic extended here to include otel-demo.
• jensholdgaard/ourios#60: Introduced parameterized corpus_dir handling in bench.yml; this PR extends selection to include fetched otel-demo corpus.

Poem

🐰 I zipped the demo logs so neat,
Tuned releases for each frozen beat,
Bench fetches, unpacks, and runs with glee,
Capture publishes proof for history,
Hooray — small hops, big telemetry!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically describes the main changes: freezing a demo corpus as a release asset and wiring it into the bench workflow with otel-demo capture support.
Description check	✅ Passed	The description comprehensively covers the changes (two workflow modifications), rationale, validation status, and technical notes. It matches the template structure with sections for What, Why, and Validation, though the Checklist section is not included.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/bench-otel-demo-corpus-wiring

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

Wires the validated OTel Demo log corpus into the thesis-gate bench as a frozen, versioned GitHub release asset, so each bench run consumes byte-identical input (preserving RFC 0006 §J4 reproducibility) instead of regenerating a noisy live capture.

Changes:

• capture-otel-demo-corpus.yml: adds an optional release_tag input and a publish step that gzips logs.jsonl, writes provenance into release notes, and creates the release or --clobbers its assets; bumps contents permission to write.
• bench.yml: adds an otel-demo capture choice with a corpus_release_tag input, a new fetch step that gh release downloads + gunzips the frozen corpus, and updates BENCH_CORPUS fallback to consider steps.fetch.outputs.corpus_dir.
• Tag is plumbed via env: rather than ${{ }} shell splicing, matching the existing template-injection mitigation pattern.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
.github/workflows/capture-otel-demo-corpus.yml	New release_tag input + publish-to-release step; contents: write permission.
.github/workflows/bench.yml	New otel-demo capture path + corpus_release_tag input; BENCH_CORPUS selects across capture/fetch/input.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[coderabbitai]

🧹 Nitpick comments (1)

.github/workflows/capture-otel-demo-corpus.yml (1)

290-294: ⚡ Quick win

Release notes become stale when re-minting an existing tag.

When the release already exists, gh release upload --clobber updates the assets but the $notes with current provenance (demo_ref, duration, commit, etc.) is discarded. Re-minting with different parameters would leave release notes showing outdated metadata while manifest.md reflects the new corpus.

🔧 Proposed fix to update release notes on re-mint

 if gh release view "$RELEASE_TAG" >/dev/null 2>&1; then
   gh release upload "$RELEASE_TAG" logs.jsonl.gz manifest.md --clobber
+  gh release edit "$RELEASE_TAG" --notes "$notes"
 else
   gh release create "$RELEASE_TAG" logs.jsonl.gz manifest.md \
     --title "$RELEASE_TAG" --notes "$notes"
 fi

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/capture-otel-demo-corpus.yml around lines 290 - 294, The
current workflow uploads assets with gh release upload --clobber when
RELEASE_TAG exists but does not update the release notes, causing stale
metadata; after the existing-release branch where gh release upload
"$RELEASE_TAG" logs.jsonl.gz manifest.md --clobber is executed, run gh release
edit "$RELEASE_TAG" --notes "$notes" (and optionally --title "$RELEASE_TAG") to
update the release metadata so the release notes match the new manifest.md and
assets; ensure you reference the existing RELEASE_TAG and notes variables and
run edit after upload to avoid losing provenance when re-minting the tag.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In @.github/workflows/capture-otel-demo-corpus.yml:
- Around line 290-294: The current workflow uploads assets with gh release
upload --clobber when RELEASE_TAG exists but does not update the release notes,
causing stale metadata; after the existing-release branch where gh release
upload "$RELEASE_TAG" logs.jsonl.gz manifest.md --clobber is executed, run gh
release edit "$RELEASE_TAG" --notes "$notes" (and optionally --title
"$RELEASE_TAG") to update the release metadata so the release notes match the
new manifest.md and assets; ensure you reference the existing RELEASE_TAG and
notes variables and run edit after upload to avoid losing provenance when
re-minting the tag.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 839b5500-46f1-4ba5-8cc1-3f5e7d48e334

📥 Commits

Reviewing files that changed from the base of the PR and between 1e3a4d4 and 529d242.

📒 Files selected for processing (2)

• .github/workflows/bench.yml
• .github/workflows/capture-otel-demo-corpus.yml