-
Notifications
You must be signed in to change notification settings - Fork 0
Wave-32 C07: sustained fuzz cadence soft job #266
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| # Soft / nightly sustained fuzz cadence for C07 L67 beyond PR fuzz-smoke. | ||
| # PR path: hermetic SelfCheck only (keeps default PR CI fast). | ||
| # Schedule / dispatch: 120s per target + crash artifact upload on failure. | ||
| # continue-on-error: soft gate; does not block merges. | ||
| name: fuzz cadence | ||
|
|
||
| on: | ||
| schedule: | ||
| - cron: "57 4 * * *" # nightly 04:57 UTC (offset from miri/loom soft jobs) | ||
| workflow_dispatch: | ||
| pull_request: | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| env: | ||
| CARGO_TERM_COLOR: always | ||
|
|
||
| jobs: | ||
| fuzz-selfcheck: | ||
| name: soft fuzz · SelfCheck | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 5 | ||
| continue-on-error: true | ||
| steps: | ||
| - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7 | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win Disable credential persistence in checkout steps. Both checkout steps rely on the default behavior that leaves the
🧰 Tools🪛 zizmor (1.26.1)[warning] 30-30: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false (artipacked) 📍 Affects 1 file
🤖 Prompt for AI AgentsSource: Linters/SAST tools |
||
| - name: fuzz cadence SelfCheck | ||
| shell: pwsh | ||
| run: ./scripts/fuzz-cadence-check.ps1 -SelfCheck | ||
|
|
||
| fuzz-sustained: | ||
| name: soft fuzz · sustained 120s | ||
| if: github.event_name != 'pull_request' | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 45 | ||
| continue-on-error: true | ||
| # cargo-fuzz requires nightly. rust-toolchain.toml pins stable, so force | ||
| # nightly via RUSTUP_TOOLCHAIN. Clear inherited RUSTFLAGS. Pin gnu host | ||
| # target — cargo-fuzz otherwise may pick musl, which cannot use ASAN. | ||
| env: | ||
| RUSTFLAGS: "" | ||
| RUSTUP_TOOLCHAIN: nightly | ||
| steps: | ||
| - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7 | ||
| - uses: dtolnay/rust-toolchain@4be7066ada62dd38de10e7b70166bc74ed198c30 # nightly for fuzz | ||
| with: | ||
| toolchain: nightly | ||
| components: rust-src | ||
| targets: x86_64-unknown-linux-gnu | ||
| - name: install cargo-fuzz | ||
| uses: taiki-e/install-action@e28ac56891501ddb0600608470dbe94544964ed4 # cargo-fuzz | ||
| with: | ||
| tool: cargo-fuzz | ||
| - name: sustained fuzz OKF parse and roundtrip (120s) | ||
| run: cargo +nightly fuzz run okf_roundtrip --sanitizer address --target x86_64-unknown-linux-gnu -- -max_total_time=120 | ||
| - name: sustained fuzz JSONL ingest parse (120s) | ||
| run: cargo +nightly fuzz run jsonl_ingest --sanitizer address --target x86_64-unknown-linux-gnu -- -max_total_time=120 | ||
| - name: upload fuzz crash artifacts | ||
| if: failure() | ||
| uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 | ||
| with: | ||
| name: fuzz-crash-artifacts | ||
| path: fuzz/artifacts/ | ||
| if-no-files-found: ignore | ||
| retention-days: 14 | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,90 @@ | ||
| # Fuzz cadence (C07 L67) | ||
|
|
||
| SessionLedger ships two `cargo-fuzz` targets with a **seeded corpus** and a | ||
| **blocking PR smoke** (`ci.yml` → `fuzz-smoke`, 10 seconds per target). This | ||
| page is the SSOT for the **sustained / soft longer cadence** beyond that smoke: | ||
| nightly campaigns, crash artifact triage, and how to keep PR CI fast. | ||
|
|
||
| Related: [`test-pyramid.md`](test-pyramid.md) (pyramid layer), | ||
| [`fuzz/`](../../fuzz/), [`.github/workflows/ci.yml`](../../.github/workflows/ci.yml) | ||
| (`fuzz-smoke`), [`.github/workflows/fuzz-cadence.yml`](../../.github/workflows/fuzz-cadence.yml). | ||
|
|
||
| ## Cadence map | ||
|
|
||
| | Lane | Duration | When | Gate | | ||
| |------|----------|------|------| | ||
| | PR smoke | 10 s / target | every PR / push via `ci.yml` → `fuzz-smoke` | **blocking** | | ||
| | Sustained soft | 120 s / target | nightly schedule + `workflow_dispatch` | **soft** (`continue-on-error`) | | ||
| | Local campaign | operator-chosen | maintainer machine | manual | | ||
|
|
||
| PR smoke stays short on purpose. Sustained runs **do not** join the default PR | ||
| required-check surface — they live in `fuzz-cadence.yml` and stay soft so a | ||
| flaky libFuzzer campaign cannot block merges. | ||
|
|
||
| ## Targets and corpora | ||
|
|
||
| | Target | Seed corpus | Exercises | | ||
| |--------|-------------|-----------| | ||
| | `okf_roundtrip` | `fuzz/corpus/okf_roundtrip/` | OKF parse + roundtrip invariants | | ||
| | `jsonl_ingest` | `fuzz/corpus/jsonl_ingest/` | JSONL ingest parse paths | | ||
|
|
||
| ## Soft sustained workflow | ||
|
|
||
| [`fuzz-cadence.yml`](../../.github/workflows/fuzz-cadence.yml) is **non-blocking** | ||
| (`continue-on-error: true`). It: | ||
|
|
||
| 1. Runs `scripts/fuzz-cadence-check.ps1 -SelfCheck` (docs/workflow/path anchors). | ||
| 2. On schedule / `workflow_dispatch` only (skipped on `pull_request`): runs each | ||
| target for `-max_total_time=120` with ASAN on `x86_64-unknown-linux-gnu` | ||
| (same toolchain pins as `fuzz-smoke`). | ||
| 3. On failure, uploads `fuzz/artifacts/` for crash corpus triage. | ||
|
|
||
| Schedule: nightly UTC (offset from miri/loom soft jobs) + `workflow_dispatch`. | ||
| `pull_request` only exercises the hermetic SelfCheck job so default PR CI is | ||
| not lengthened by the 120 s campaigns. | ||
|
|
||
| ## Crash corpus triage | ||
|
|
||
| When a sustained (or local) run finds a crash, libFuzzer writes under | ||
| `fuzz/artifacts/<target>/` (for example `crash-*`). Triage steps: | ||
|
|
||
| 1. Download the workflow artifact `fuzz-crash-artifacts` (or copy the local | ||
| `fuzz/artifacts/` tree). | ||
| 2. Reproduce with the failing input: | ||
| `cargo +nightly fuzz run <target> fuzz/artifacts/<target>/<crash-file>`. | ||
| 3. Minimize when useful: | ||
| `cargo +nightly fuzz tmin <target> fuzz/artifacts/<target>/<crash-file>`. | ||
| 4. Reduce to a focused regression (unit/property test or a small corpus seed | ||
| under `fuzz/corpus/<target>/`) and open a fix PR. | ||
| 5. Do **not** commit raw unbounded crash dumps or corpus growth from CI without | ||
| review — keep seeds small and intentional. | ||
|
|
||
| ## Done gates | ||
|
|
||
| | Gate | Status | Evidence | | ||
| |------|--------|----------| | ||
| | Fuzz cadence SelfCheck | **done** | `scripts/fuzz-cadence-check.ps1 -SelfCheck` (+ `tests/fuzz_cadence.rs`) | | ||
| | Soft sustained fuzz CI | **done** | `.github/workflows/fuzz-cadence.yml` (`continue-on-error`, 120 s / target) | | ||
| | PR `fuzz-smoke` (10 s) | **done** | `.github/workflows/ci.yml` (unchanged; stays blocking + short) | | ||
| | Auto corpus promotion from CI crashes | **unpaid** | Triage remains maintainer-driven (see above) | | ||
|
|
||
| ## Machine verification (SelfCheck) | ||
|
|
||
| Hermetic docs + path + workflow anchors (no `cargo fuzz`, no network): | ||
|
|
||
| ```powershell | ||
| pwsh ./scripts/fuzz-cadence-check.ps1 -SelfCheck | ||
| ``` | ||
|
|
||
| ## Local sustained campaign | ||
|
|
||
| Nightly toolchain + `cargo-fuzz` (same flags as soft CI): | ||
|
|
||
| ```powershell | ||
| $env:CARGO_TARGET_DIR = Join-Path $PWD "target-w32-c07-fuzz" | ||
| cargo +nightly fuzz run okf_roundtrip --sanitizer address --target x86_64-unknown-linux-gnu -- -max_total_time=120 | ||
| cargo +nightly fuzz run jsonl_ingest --sanitizer address --target x86_64-unknown-linux-gnu -- -max_total_time=120 | ||
| ``` | ||
|
|
||
| On Windows hosts without ASAN/gnu, drop `--sanitizer` / `--target` and use a | ||
| shorter local time budget; prefer Linux (or the soft CI job) for ASAN campaigns. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,146 @@ | ||
| <# | ||
| .SYNOPSIS | ||
| Machine-check fuzz cadence SSOT anchors (C07 L67). | ||
|
|
||
| .DESCRIPTION | ||
| Verifies docs/ops/fuzz-cadence.md documents sustained soft fuzz beyond PR | ||
| smoke, and that the fuzz-cadence workflow, fuzz targets/corpus, and this | ||
| script stay wired. Hermetic: no cargo-fuzz, no network. | ||
|
|
||
| Does not claim blocking sustained fuzz or automatic corpus promotion. | ||
|
|
||
| .PARAMETER SelfCheck | ||
| Explicit docs/path smoke (CI unit proof). Same checks as the default path. | ||
|
|
||
| .EXAMPLE | ||
| pwsh ./scripts/fuzz-cadence-check.ps1 -SelfCheck | ||
| #> | ||
| [CmdletBinding()] | ||
| param( | ||
| [switch]$SelfCheck | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. SUGGESTION: The Reply with |
||
| ) | ||
|
|
||
| Set-StrictMode -Version Latest | ||
| $ErrorActionPreference = "Stop" | ||
|
|
||
| $repoRoot = Split-Path -Parent $PSScriptRoot | ||
| $docPath = Join-Path $repoRoot "docs/ops/fuzz-cadence.md" | ||
| $workflowPath = Join-Path $repoRoot ".github/workflows/fuzz-cadence.yml" | ||
| $ciPath = Join-Path $repoRoot ".github/workflows/ci.yml" | ||
| $selfPath = Join-Path $repoRoot "scripts/fuzz-cadence-check.ps1" | ||
| $okfTarget = Join-Path $repoRoot "fuzz/fuzz_targets/okf_roundtrip.rs" | ||
| $jsonlTarget = Join-Path $repoRoot "fuzz/fuzz_targets/jsonl_ingest.rs" | ||
| $okfCorpus = Join-Path $repoRoot "fuzz/corpus/okf_roundtrip/minimal.json" | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. SUGGESTION: SelfCheck pins specific corpus seed filenames
Reply with |
||
| $jsonlCorpus = Join-Path $repoRoot "fuzz/corpus/jsonl_ingest/two_sessions.jsonl" | ||
|
|
||
| function Assert-File { | ||
| param( | ||
| [Parameter(Mandatory = $true)][string]$Path, | ||
| [Parameter(Mandatory = $true)][string]$Label | ||
| ) | ||
| if (-not (Test-Path -LiteralPath $Path -PathType Leaf)) { | ||
| throw "Missing $Label at '$Path'." | ||
| } | ||
| } | ||
|
|
||
| function Write-Check { | ||
| param([string]$Label, [bool]$Ok) | ||
| $mark = if ($Ok) { "PASS" } else { "FAIL" } | ||
| Write-Host " [$mark] $Label" | ||
| return $Ok | ||
| } | ||
|
|
||
| function Test-DocContains { | ||
| param( | ||
| [Parameter(Mandatory = $true)][string]$Doc, | ||
| [Parameter(Mandatory = $true)][string]$Needle, | ||
| [Parameter(Mandatory = $true)][string]$Label, | ||
| [string]$Context = "docs/ops/fuzz-cadence.md" | ||
| ) | ||
| $ok = $Doc.Contains($Needle) | ||
| [void](Write-Check -Label $Label -Ok $ok) | ||
| if (-not $ok) { | ||
| throw "$Context missing required anchor: '$Needle'" | ||
| } | ||
| } | ||
|
|
||
| Write-Host "Fuzz cadence check (C07 L67)" | ||
| if ($SelfCheck) { | ||
| Write-Host "Mode: SelfCheck (docs + workflow + corpus anchors; no cargo-fuzz / no network)" | ||
| } | ||
|
|
||
| Assert-File -Path $docPath -Label "fuzz cadence doc" | ||
| Assert-File -Path $workflowPath -Label "fuzz-cadence workflow" | ||
| Assert-File -Path $ciPath -Label "ci.yml" | ||
| Assert-File -Path $selfPath -Label "fuzz cadence check script" | ||
| Assert-File -Path $okfTarget -Label "okf_roundtrip fuzz target" | ||
| Assert-File -Path $jsonlTarget -Label "jsonl_ingest fuzz target" | ||
| Assert-File -Path $okfCorpus -Label "okf_roundtrip corpus seed" | ||
| Assert-File -Path $jsonlCorpus -Label "jsonl_ingest corpus seed" | ||
|
|
||
| $doc = Get-Content -LiteralPath $docPath -Raw | ||
| $workflow = Get-Content -LiteralPath $workflowPath -Raw | ||
| $ci = Get-Content -LiteralPath $ciPath -Raw | ||
|
|
||
| Write-Host "Fuzz cadence doc anchors:" | ||
| Test-DocContains -Doc $doc -Needle "Fuzz cadence (C07 L67)" ` | ||
| -Label "doc heading" | ||
| Test-DocContains -Doc $doc -Needle "scripts/fuzz-cadence-check.ps1" ` | ||
| -Label "SelfCheck script reference" | ||
| Test-DocContains -Doc $doc -Needle "-SelfCheck" ` | ||
| -Label "SelfCheck invocation" | ||
| Test-DocContains -Doc $doc -Needle "Fuzz cadence SelfCheck | **done**" ` | ||
| -Label "SelfCheck gate marked done" | ||
| Test-DocContains -Doc $doc -Needle "fuzz-cadence.yml" ` | ||
| -Label "fuzz-cadence workflow reference" | ||
| Test-DocContains -Doc $doc -Needle "continue-on-error" ` | ||
| -Label "soft continue-on-error note" | ||
| Test-DocContains -Doc $doc -Needle "max_total_time=120" ` | ||
| -Label "sustained 120s budget" | ||
| Test-DocContains -Doc $doc -Needle "fuzz-smoke" ` | ||
| -Label "PR fuzz-smoke reference" | ||
| Test-DocContains -Doc $doc -Needle "## Crash corpus triage" ` | ||
| -Label "crash corpus triage section" | ||
| Test-DocContains -Doc $doc -Needle "fuzz/artifacts/" ` | ||
| -Label "crash artifacts path" | ||
| Test-DocContains -Doc $doc -Needle "Auto corpus promotion from CI crashes | **unpaid**" ` | ||
| -Label "auto corpus promotion unpaid gate" | ||
| Test-DocContains -Doc $doc -Needle "okf_roundtrip" ` | ||
| -Label "okf_roundtrip target" | ||
| Test-DocContains -Doc $doc -Needle "jsonl_ingest" ` | ||
| -Label "jsonl_ingest target" | ||
|
|
||
| Write-Host "Workflow soft-gate anchors:" | ||
| if ($workflow -notmatch 'continue-on-error:\s*true') { | ||
| throw "fuzz-cadence.yml must set continue-on-error: true (soft gate)." | ||
| } | ||
| [void](Write-Check -Label "workflow continue-on-error: true" -Ok $true) | ||
|
|
||
| if ($workflow -notmatch 'max_total_time=120') { | ||
| throw "fuzz-cadence.yml must run sustained fuzz with -max_total_time=120." | ||
| } | ||
| [void](Write-Check -Label "workflow max_total_time=120" -Ok $true) | ||
|
|
||
| if ($workflow -notmatch 'github\.event_name != ''pull_request''') { | ||
| throw "fuzz-cadence.yml must skip sustained job on pull_request (keep PR CI fast)." | ||
| } | ||
| [void](Write-Check -Label "sustained job skips pull_request" -Ok $true) | ||
|
|
||
| if ($workflow -notmatch 'fuzz-crash-artifacts') { | ||
| throw "fuzz-cadence.yml must upload fuzz-crash-artifacts on failure." | ||
| } | ||
| [void](Write-Check -Label "workflow crash artifact upload" -Ok $true) | ||
|
|
||
| if ($workflow -notmatch 'okf_roundtrip' -or $workflow -notmatch 'jsonl_ingest') { | ||
| throw "fuzz-cadence.yml must exercise okf_roundtrip and jsonl_ingest." | ||
| } | ||
| [void](Write-Check -Label "workflow exercises both fuzz targets" -Ok $true) | ||
|
|
||
| Write-Host "PR smoke stays short:" | ||
| if ($ci -notmatch 'max_total_time=10') { | ||
| throw "ci.yml fuzz-smoke must keep -max_total_time=10 (do not slow PR CI here)." | ||
| } | ||
| [void](Write-Check -Label "ci.yml fuzz-smoke max_total_time=10" -Ok $true) | ||
|
|
||
| Write-Host "Fuzz cadence SelfCheck passed" | ||
| exit 0 | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| //! Hermetic `SelfCheck` for fuzz cadence SSOT anchors (C07 L67). | ||
| //! | ||
| //! Local: `pwsh ./scripts/fuzz-cadence-check.ps1 -SelfCheck` | ||
| //! Does not run cargo-fuzz — safe under default Windows `cargo test`. | ||
|
|
||
| use std::path::PathBuf; | ||
| use std::process::Command; | ||
|
|
||
| fn repo_root() -> PathBuf { | ||
| PathBuf::from(env!("CARGO_MANIFEST_DIR")) | ||
| } | ||
|
|
||
| #[test] | ||
| fn fuzz_cadence_doc_self_check_validates_anchors() { | ||
| let script = repo_root().join("scripts/fuzz-cadence-check.ps1"); | ||
| assert!(script.is_file(), "expected fuzz cadence check script at {}", script.display()); | ||
|
|
||
| let output = Command::new("pwsh") | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. SUGGESTION: Test hard-depends on This integration test spawns Reply with |
||
| .args(["-NoProfile", "-File", script.to_str().expect("utf-8 script path"), "-SelfCheck"]) | ||
| .output() | ||
| .unwrap_or_else(|error| panic!("failed to spawn pwsh for SelfCheck: {error}")); | ||
|
|
||
| let stdout = String::from_utf8_lossy(&output.stdout); | ||
| let stderr = String::from_utf8_lossy(&output.stderr); | ||
| assert!( | ||
| output.status.success(), | ||
| "fuzz-cadence-check.ps1 -SelfCheck failed\nstdout:\n{stdout}\nstderr:\n{stderr}" | ||
| ); | ||
| assert!( | ||
| stdout.contains("Fuzz cadence SelfCheck passed"), | ||
| "expected SelfCheck success line, got:\n{stdout}" | ||
| ); | ||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
WARNING:
permissions: contents: readlikely breaks the crash-artifact uploadSetting
permissions:at workflow scope overrides the default GITHUB_TOKEN grants and leaves the token with onlycontents: read.actions/upload-artifact@v7(v4) needsactions: readto query the run/API, and sibling uploads inci.ymlwork precisely because they set nopermissions:block (so they inherit defaults). Because the upload step isif: failure()inside acontinue-on-error: truejob, a permission failure here is silent — crashing fuzz runs won't produce the triage artifacts this PR exists to capture. Addactions: read(andid-token: writeif OIDC upload is used) to the permissions block.Reply with
@kilocode-bot fix itto have Kilo Code address this issue.