ci(capture): failure feature-flag input for OTel-Demo corpus captures

[jensholdgaard]

Why

The B1 thesis bench (severity-predicate pushdown, level='ERROR') needs a real corpus with an error band, but the OTel Demo's default traffic emits no error logs — verified on corpus/otel-demo-v4: zero literal-"ERROR" body texts and zero severity numbers in 17..=20, so the merged b1/real-corpus arm skips it.

The demo ships failure feature-flags via flagd (adFailure, cartFailure, paymentFailure, productCatalogFailure, …) that make services emit genuine errors under load. This PR exposes them as a capture input so we can mint a corpus/otel-demo-v6 with a realistic error band.

What

• New workflow_dispatch input failure_flags (string, default ''): space/comma-separated flagd flag names to force on for the capture. Empty input = byte-for-byte identical behavior to today — the new step is if: inputs.failure_flags != ''.
• A patch step (mirroring the collector-overlay injection) that edits demo/src/flagd/demo.flagd.json with jq before bring-up. flagd bind-mounts ./src/flagd → /etc/flagd and reads demo.flagd.json from it (verified in the 2.2.0 compose), so patching the file pre-up is the whole injection.
• For each named flag, defaultVariant flips to the failing variant: "on" where it exists, else "100%" (the fractional flags, e.g. paymentFailure at 2.2.0, have 10%..100% variants instead of on).
• Hardening, same bar as the corpus-tag validation in query-bench.yml: input arrives via env: (no inline ${{ }} splicing), each name is validated against ^[A-Za-z0-9_-]+$, and an unknown flag fails with ::error:: listing the flags available at that demo ref.
• Provenance: failure_flags is recorded in the diversity manifest and the release notes (none when empty), so a corpus's error band is auditable.

Flags available at demo tag 2.2.0 (src/flagd/demo.flagd.json)

adFailure, adHighCpu, adManualGc, cartFailure, emailMemoryLeak, failedReadinessProbe, imageSlowLoad, kafkaQueueProblems, llmInaccurateResponse, llmRateLimitError, loadGeneratorFloodHomepage, paymentFailure (fractional), paymentUnreachable, productCatalogFailure, recommendationCacheFailure

jq sanity test (run locally against the real 2.2.0 file)

Input failure_flags: 'adFailure,paymentFailure cartFailure' (mixed separators) yields:

-  "adFailure": "off",
+  "adFailure": "on",
-  "cartFailure": "off",
+  "cartFailure": "on",
-  "paymentFailure": "off",
+  "paymentFailure": "100%",

Unknown-flag path (notAFlag) exits 1 with ::error::flagd flag 'notAFlag' not in … ; available: adFailure, adHighCpu, ….

Verification

• actionlint .github/workflows/capture-otel-demo-corpus.yml — clean (exit 0)
• Full patch-step shell logic exercised end-to-end against a fresh opentelemetry-demo clone at 2.2.0 (happy path incl. the fractional flag, comma/space separators, unknown-flag error path)
• cargo test --all-features — all 77 suites green (no Rust code touched; confirms the tree is unaffected)
• No change to query-bench.yml or bench code; defaults unchanged when the input is empty

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features
  • Added support for injecting failure feature flags when manually triggering the OpenTelemetry Demo corpus capture workflow, with automatic validation and application to the demo configuration.
  • Failure flags are now included in generated manifest and release documentation.

[jensholdgaard]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Warning

Review limit reached

@jensholdgaard, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 48 minutes and 47 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 10578647-bdcf-4267-858e-d6d116e5edcc

📥 Commits

Reviewing files that changed from the base of the PR and between 9443aa4 and 38e6d96.

📒 Files selected for processing (1)

• .github/workflows/capture-otel-demo-corpus.yml

📝 Walkthrough

Walkthrough

The workflow now supports injecting OpenTelemetry Demo failure feature flags via a workflow_dispatch input. When provided, the workflow validates each flag name, verifies it exists in the demo's flag configuration with an "on" or "100%" variant, patches the default variant, and records the failure flags in both the generated manifest and release metadata.

Changes

Failure Flags Feature

Layer / File(s)	Summary
Input declaration and flag enablement .github/workflows/capture-otel-demo-corpus.yml	Adds failure_flags workflow_dispatch input and implements a new step that parses, validates, and applies each flag to the demo's demo.flagd.json by checking existence, selecting the appropriate variant, and patching defaultVariant in-place.
Manifest and release metadata propagation .github/workflows/capture-otel-demo-corpus.yml	Extends the "Build diversity manifest" step to record failure_flags (or none) in manifest.md and the "Publish corpus release" step to include failure_flags in generated release notes/provenance.

Possibly Related PRs

• jensholdgaard/ourios#79: Both PRs modify the same workflow's workflow_dispatch inputs and release publishing logic, with PR #79 establishing the release tag publication mechanism that this PR extends.
• jensholdgaard/ourios#73: Both PRs extend the same workflow file and wire metadata into manifest and release outputs.

Estimated Code Review Effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

🐰 A workflow gains new flags to play,
With failure_flags in disarray,
Validation checks that none are spurious,
Recording traces for the corpus curious—
The demo dances, chaos made glorious!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately reflects the main change: adding a failure feature-flag input to the OTel-Demo corpus capture workflow.
Description check	✅ Passed	The description includes a Why section (rationale for error band corpus), a What section (feature details, hardening, provenance), example usage with jq, available flags, and verification steps.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch ci/otel-demo-failure-flags

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

Adds an opt-in failure_flags input to the OTel Demo corpus capture workflow so operators can force specific flagd “failure” feature flags on during capture, enabling generation of corpora with a realistic error band while keeping the default capture behavior unchanged when the input is empty.

Changes:

• Introduces a new workflow_dispatch input failure_flags (string, default empty).
• Adds a pre-bring-up patch step that uses jq to flip defaultVariant for specified flagd flags to the failing variant.
• Records failure_flags provenance in the capture manifest and (when publishing) in the release notes.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[coderabbitai]

Actionable comments posted: 1

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: fdbb140f-027f-4474-8528-18303532a1e7

📥 Commits

Reviewing files that changed from the base of the PR and between da4da0e and 9443aa4.

📒 Files selected for processing (1)

• .github/workflows/capture-otel-demo-corpus.yml

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.