feat(bench): wire telemetrygen OTLP capture into bench.yml (PR-N1)

[jensholdgaard]

What

First step toward flipping docs/benchmarks.md §9 from "no benchmark has been run" to a real measured row. Wires an in-CI corpus capture path into bench.yml so a manual dispatch produces actual A1/C1/C2 numbers end-to-end, with zero pre-staged corpus state.

• New .github/workflows/bench-collector.yaml — minimal OTel collector config: OTLP-gRPC + OTLP-HTTP receiver → fileexporter writing OTLP/JSON Lines to /tmp/otel-capture/logs.jsonl (the wire shape ourios-bench's *.jsonl loader consumes per RFC 0006 §3.1).
• .github/workflows/bench.yml:
  • New capture workflow input. telemetrygen (default) drives the in-CI capture; none preserves the existing manual-corpus path.
  • Installs otelcol-contrib (binary download from the collector-releases repo) + telemetrygen (go install against the contrib repo) at a workflow-level pinned OTEL_VER = v0.115.0.
  • Boots the collector, waits for OTLP gRPC, generates telemetrygen_logs records (default 1M), flushes, kills the collector, snapshots into bench-corpus/telemetrygen-synthetic-v1/.
  • corpus_dir default changed to bench-corpus/telemetrygen-synthetic-v1 (the captured path).
  • hardware_kind default changed from the unknown tag github-ubuntu-4vcpu to the RFC 0006 §3.5 RFC-blessed ci-runner.
  • Job summary explicitly tags the run as synthetic-on-content; the existing "indicative only" hardware disclaimer is expanded with the corpus disclaimer.

Why telemetrygen, not Loghub or the OTel Demo

Considered three corpus sources:

1. Loghub wrapped in OTLP envelopes — synthetic envelopes around real text. Inflates A1 (single dictionary value across all Resources) and C1 (envelope round-trips trivially). Rejected as masquerading per the maintainer's note.
2. OTel Demo capture — real envelopes + real bodies; the most representative corpus we can produce. Significantly more cost: ~20-container compose stack, ~6 GiB RAM, multi-PR setup with release-asset hosting. The right eventual corpus.
3. telemetrygen — real OTLP wire envelopes (real codec, real Resource/Scope structure, real batch boundaries) but uniform synthetic bodies. Runs entirely in-CI, no committed corpus, no release-asset plumbing. Confirmed by the OpenTelemetry knowledge base as the intended-purpose tool for this kind of fixture generation.

Picked #3 as the MVP because it gets us the first §9 row with honest tagging today, and #2 lands as a follow-up capture option with the same workflow surface.

What this run won't claim

• Not the canonical hardware. ci-runner is the RFC 0006 §3.5 tag; the §1 baseline is baseline-8vcpu-32gib. Per the §7 escalation rule, a thesis-gate failure on ci-runner does not trigger pillar-level escalation — only failures on the §1 baseline + a representative corpus do.
• Not a representative corpus. telemetrygen-synthetic-v1 will compress unrealistically well (uniform body content). The expected reading of an inflated A1 number is "the writer-side compression path works end-to-end", not "the thesis holds for production logs". corpus: telemetrygen-synthetic-v1 in the results JSON is the warning label.

Out of scope (follow-ups)

• PR-N2: dispatch this workflow once, surface the actual A1/C1/C2 numbers, then add a real §9 row in docs/benchmarks.md with the ci-runner + telemetrygen-synthetic-v1 annotation.
• PR-N3 (later): add capture: otel-demo — runs the OpenTelemetry Demo in CI, captures real envelopes + real bodies for ~30 min, uploads as a corpus/otel-demo-vN release asset, bench fetches it. The representative-content corpus.
• PR-N4 (much later): provisioned cloud VM at baseline-8vcpu-32gib so the §7 escalation rule has teeth. This is the row that flips the project's thesis claim from "we believe" to "the thesis survives one run."

CLAUDE.md §6.6

• cargo fmt --all --check — clean.
• cargo clippy --all-targets --all-features -- -D warnings — clean.
• cargo test --all-features — 322 passed / 0 failed / 44 ignored (no Rust changes; baseline preserved).
• mdbook build — clean.
• YAML syntax of both files validated with yaml.safe_load.
• The workflow itself won't run on this PR (it's workflow_dispatch-only); first real dispatch happens after merge.

Invariants touched

None — this PR is bench-only, no change to the writer / reader / WAL / miner contracts.

🤖 Generated with Claude Code

Summary by CodeRabbit

• Chores
  • Added an OpenTelemetry Collector configuration to capture telemetry logs for benchmark runs.
  • Enhanced benchmark workflow to support on-demand corpus capture driven by a new capture input and telemetrygen integration.
  • Pinned collector/telemetry tooling versions and adjusted workflow defaults for predictable corpus selection and hardware targeting.
  • Expanded job summary to report the chosen capture strategy.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 970c6911-fad9-436a-9da7-1485905c576e

📥 Commits

Reviewing files that changed from the base of the PR and between 027109f and 906394d.

📒 Files selected for processing (1)

• .github/workflows/bench.yml

🚧 Files skipped from review as they are similar to previous changes (1)

• .github/workflows/bench.yml

---

📝 Walkthrough

Walkthrough

Adds an OTEL Collector config and extends the bench GitHub Actions workflow with a new capture mode (default telemetrygen) that can run telemetrygen against a local collector to produce OTLP/JSONL logs, move them into the bench corpus, and report capture choices in job summaries.

Changes

Telemetrygen corpus capture integration

Layer / File(s)	Summary
Collector configuration for OTLP export .github/workflows/bench-collector.yaml	New YAML config defines OTLP gRPC + HTTP receivers on localhost (4317/4318) and a logs pipeline exporting JSON Lines with flush_interval, disabled self-metrics, and log level warn.
Workflow inputs and OTEL version pinning .github/workflows/bench.yml	Adds capture input (default telemetrygen), introduces telemetrygen_logs, changes corpus_dir default to the telemetrygen synthetic corpus, updates hardware_kind default, and pins OTEL_VER at workflow level.
Telemetrygen capture pipeline steps .github/workflows/bench.yml	Conditional steps for capture=telemetrygen: download/install pinned otelcol-contrib and telemetrygen, start collector, wait for OTLP readiness, run telemetrygen logs to generate synthetic records, stop collector, move logs.jsonl into bench-corpus/telemetrygen-synthetic-v1/, and emit corpus_dir.
Bench corpus environment precedence .github/workflows/bench.yml	Bench step BENCH_CORPUS env now prefers capture output (steps.capture.outputs.corpus_dir) with fallback to inputs.corpus_dir.
Capture mode reporting in job summary .github/workflows/bench.yml	Job summary sets BENCH_CAPTURE, computes BENCH_CORPUS with capture-or-input precedence, and updates summary text to include the capture mode and revised indicative messaging.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

• jensholdgaard/ourios#60: Updates corpus_dir workflow input handling; this PR adjusts its default to align with the new telemetrygen synthetic corpus path.
• jensholdgaard/ourios#58: Adds OTLP/JSONL log parsing to the bench corpus loader; this PR generates the synthetic OTLP/JSONL logs that feed into that parsing.
• jensholdgaard/ourios#57: Extends the bench workflow with on-demand workflow_dispatch; this PR adds the telemetrygen-based capture phase to that workflow.

Poem

A little rabbit taps the keys with glee,
Starting collectors, logs flow free,
telemetrygen hums a patterned tune,
JSON Lines pile up beneath the moon,
Benchmarks bloom — hop, capture, see! 🐇

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically describes the main change: wiring telemetrygen OTLP capture into the bench.yml workflow.
Description check	✅ Passed	The description provides comprehensive details covering what changed, why each decision was made, and what is out of scope, though it does not follow the template structure with explicit sections.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/bench-telemetrygen-capture

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

Adds an in-CI synthetic corpus capture path to the manual bench workflow so a workflow_dispatch run can generate an OTLP/JSONL corpus (via telemetrygen + an OTel collector fileexporter) and then run ourios-bench end-to-end without requiring a pre-staged corpus.

Changes:

• Extends bench.yml with a new capture mode (telemetrygen | none) and parameters for synthetic log generation.
• Downloads and runs otelcol-contrib + telemetrygen, captures OTLP/JSONL logs, and snapshots them into bench-corpus/telemetrygen-synthetic-v1/.
• Adds a minimal OTel collector config (bench-collector.yaml) to receive OTLP and write JSONL to disk for the bench loader.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.

File	Description
.github/workflows/bench.yml	Adds workflow inputs and a capture phase that generates a synthetic OTLP/JSONL corpus before running ourios-bench.
.github/workflows/bench-collector.yaml	Defines the minimal collector pipeline (OTLP receiver → fileexporter to JSONL) used by the capture phase.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[coderabbitai]

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

.github/workflows/bench-collector.yaml (1)

1-45: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Move bench-collector.yaml out of .github/workflows/.

GitHub treats .github/workflows/*.yml|*.yaml files as workflow candidates and validates workflow syntax (including on and jobs). This file is only OTel Collector config (receivers/exporters/service) and lacks those keys, so it can trigger “invalid workflow” / missing on/jobs errors. Move it to a non-workflow path and update bench.yml’s --config reference accordingly.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/bench-collector.yaml around lines 1 - 45, The
bench-collector.yaml file (contains receivers/exporters/service settings and the
file/corpus exporter writing to /tmp/otel-capture/logs.jsonl) must be moved out
of .github/workflows/ because GH Actions validates YAML in that directory as
workflows; relocate bench-collector.yaml to a non-workflow path (e.g., a
configs/ or infra/ directory) and then update the bench.yml invocation that
supplies --config to point to the new path so the OTLP receiver (otlp grpc/http
endpoints) and fileexporter remain referenced correctly. Ensure bench.yml still
references the same config filename and that the exporters/receivers/service
keys are unchanged.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/bench.yml:
- Around line 112-115: The capture step currently writes to a fixed path
(bench-corpus/telemetrygen-synthetic-v1) but downstream "bench" and "summary"
steps still read inputs.corpus_dir; update the capture step (give it an id,
e.g., id: capture_corpus_telemetrygen) to emit an outputs.corpus_dir set to the
actual written path (use the computed path string as the output), then change
downstream references to use
steps.capture_corpus_telemetrygen.outputs.corpus_dir when inputs.capture ==
'telemetrygen' (or fall back to inputs.corpus_dir otherwise); apply the same
pattern to the other capture steps mentioned so bench and summary always consume
the produced corpus path rather than the raw input.

---

Outside diff comments:
In @.github/workflows/bench-collector.yaml:
- Around line 1-45: The bench-collector.yaml file (contains
receivers/exporters/service settings and the file/corpus exporter writing to
/tmp/otel-capture/logs.jsonl) must be moved out of .github/workflows/ because GH
Actions validates YAML in that directory as workflows; relocate
bench-collector.yaml to a non-workflow path (e.g., a configs/ or infra/
directory) and then update the bench.yml invocation that supplies --config to
point to the new path so the OTLP receiver (otlp grpc/http endpoints) and
fileexporter remain referenced correctly. Ensure bench.yml still references the
same config filename and that the exporters/receivers/service keys are
unchanged.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a66ea8ea-8f8b-45ad-a008-343fbbd592dd

📥 Commits

Reviewing files that changed from the base of the PR and between dd0c0c6 and 027109f.

📒 Files selected for processing (2)

• .github/workflows/bench-collector.yaml
• .github/workflows/bench.yml