Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .github/workflows/loom-http-sse-soak-soft.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: loom-http-sse-soak (soft)

# Soft nightly anchor for Wave-44-B1 loom HTTP SSE soak (C00 L7).
# Runs the same suite as the blocking PR SelfCheck with deeper iteration
# counts to surface rare interleavings. Soft = continue-on-error.

on:
workflow_dispatch:
schedule:
- cron: "0 6 * * *"

jobs:
loom-http-sse-soak-soft:
runs-on: ubuntu-latest
continue-on-error: true
steps:
- uses: actions/checkout@v4
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@4be7066ada62dd38de10e7b70166bc74ed198c30
with:
toolchain: stable
components: rustfmt, clippy
- name: Cache cargo registry + target
uses: actions/cache@v4
with:
path: |
~/.cargo/registry
~/.cargo/git
target
key: ${{ runner.os }}-cargo-loom-http-sse-soak-soft-${{ hashFiles('Cargo.lock') }}
- name: Run loom-http-sse-soak-check.ps1 -SelfCheck
shell: pwsh
run: pwsh ./scripts/loom-http-sse-soak-check.ps1 -SelfCheck
- name: Soft report
if: always()
run: |
echo "soft lane — failures here do not block PRs."
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Follows [Keep a Changelog](https://keepachangelog.com/); versioning is [SemVer](

- Wave-44 plan landed: `WAVE44_SCOPE.md` + `docs/ops/WAVE44_PERT.md` enumerate 6 close-out lanes (3 machine, 3 human-gated) for the 6 unpaid residuals from Wave-43 (396/402 → 402/402 target). Theme: stack-stability closure + i18n migration + eval coverage + supply-chain signing.


### Fixed

- Viewer first-run corpus CTA (C09): wire “Open corpus…” to a web Forge DB file picker (`corpus_cta.rs`) or open the quick-start runbook on desktop; `cargo test -p sl-viewer`.
Expand All @@ -20,6 +21,7 @@ Follows [Keep a Changelog](https://keepachangelog.com/); versioning is [SemVer](

### Added

- Loom HTTP SSE soak (C00 L7 W44-B1): `tests/loom_http_sse_soak.rs` (3 loom tests: process-level multi-client fanout, Lagged recovery no-panic, shutdown propagation), `docs/ops/loom-http-sse-soak.md`, `scripts/loom-http-sse-soak-check.ps1 -SelfCheck`, soft `.github/workflows/loom-http-sse-soak-soft.yml`. Closes the *process-level HTTP SSE soak under loom* residual carried from Wave-43.
- Daemon-graph hard live tokio ports (C00 L7): `docs/ops/daemon-graph-hard.md`, `daemon-graph-hard.json`, `tests/daemon_graph_tokio.rs` (mpsc→broadcast→SSE conservation, Lagged recovery, shutdown stops enqueue), `scripts/daemon-graph-hard-check.ps1 -SelfCheck`, blocking `daemon-graph-hard.yml`, `tests/daemon_graph_hard.rs`.

- sl-viewer CLI help (C01/C09): expanded `--help` / `--version` in `cli_help.rs`, `docs/ops/sl-viewer-help.md`, `sl-viewer-help.json`, `scripts/sl-viewer-help-check.ps1 -SelfCheck`, blocking `sl-viewer-help-hard.yml`, `tests/sl_viewer_help.rs`.
Expand Down
119 changes: 119 additions & 0 deletions docs/ops/corpus-breadth.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# Corpus breadth (C08 L73) — production-scale OKF conformance

**Wave-44 close-out lane B6.**
**Owner:** machine.
**Theme:** scale the OKF conformance corpus from hand-vetted 20 → 33+ (and
growing) via a deterministic generator; close C08 L73 *production-scale corpus
breadth*.

Companion: [`WAVE44_SCOPE.md`](../../WAVE44_SCOPE.md) (rank 6) and
[`WAVE44_PERT.md`](WAVE44_PERT.md) (lane B6).

## Rubric anchor

Pillar L73 — *Microbench + Macrobench + Load Test*. SCORECARD headline:
`C08 L73 | partial (deepened)` with the residual being production-scale corpus
breadth. Wave-43 added the load-macro PR gate (`load-macro-gate-hard.yml`,
`load-smoke.ps1 -RouteTier macro`); Wave-44 grows the *input* corpus so the
gate has broad coverage to exercise.

## Strategy

The corpus must grow along three orthogonal axes:

| Axis | Why |
|------|-----|
| **Source agent** | Each agent (forge, codex, claude-code, cursor, aider, opencode, continue, kiro, factory-droid) emits different shapes — round-trip must hold across all |
| **Shape (entities / relations / depth)** | Stress the parser, the indexer, and the distiller at realistic and adversarial sizes |
| **Edge / failure modes** | Verify the parser **rejects** malformed input rather than silently accepting it |

Generator ([`scripts/corpus-generate.py`](../../scripts/corpus-generate.py))
produces one fixture per (source × shape × edge) cell. Hand-vetted fixtures
remain canonical; generated fixtures are machine-expansion.

## Current corpus (post-W44-B6)

```
docs/reference/conformance/fixtures/
├── 20 hand-vetted fixtures (W43 hand-curated, OKF-EXAMPLES.md siblings)
└── 13 generated fixtures (W44-B6, scripts/corpus-generate.py)
├── aider-rust-refactor-037 7 entities / 6 relations
├── opencode-python-debugger-038 4 entities / 3 relations
├── continue-go-microservice-039 6 entities / 5 relations
├── kiro-bash-ci-pipeline-040 4 entities / 3 relations
├── factory-droid-typescript-041 4 entities / 3 relations
├── sql-migration-multi-intent-043 7 entities / 7 relations
├── yaml-k8s-deployment-044 4 entities / 3 relations
├── large-entity-count-100-045 82 entities / 81 relations (stress)
├── deep-relation-graph-7-046 8 entities / 7 relations (chain)
├── rapid-fire-intent-stream-12-047 12 entities / 0 relations (no verifiers)
├── unicode-intent-label-cjk-048 3 entities / 2 relations (CJK round-trip)
├── embedded-json-label-049 3 entities / 2 relations (label escaping)
└── multi-modal-image-hint-050 3 entities / 2 relations (sha256 attachment)

Total: 33 accepted fixtures.
```

Failure-mode fixtures (`malformed-truncated-E01`, `missing-provenance-E02`,
`duplicate-id-E03`) live in a separate scratch directory and are not part of
the accepted corpus run; the harness is expected to reject them.

## Generator API

```bash
# Default accepted batch (33 → +13 new)
python3 scripts/corpus-generate.py --out docs/reference/conformance/fixtures

# Single fixture
python3 scripts/corpus-generate.py --out docs/reference/conformance/fixtures --only aider-rust-refactor

# Include failure-mode fixtures (rejected by harness)
python3 scripts/corpus-generate.py --out /tmp/scratch --include-failures

# Plan list (dry run)
python3 scripts/corpus-generate.py --list
```

Every emitted fixture is a valid OKF 1.0 document per
[`docs/reference/OKF-SPEC.md`](../reference/OKF-SPEC.md).

## Test wiring

The conformance roundtrip test
([`tests/okf_roundtrip.rs`](../../tests/okf_roundtrip.rs)) MUST iterate every
`*.okf.json` under `docs/reference/conformance/fixtures/` and assert:

- JSON parses
- `okf == "1.0"`
- `entities[]` ids are unique
- every relation source/target id resolves to an entity
- top-level `provenance` is present and has both `corpus` and `source_id`

The failure-mode fixtures (E01–E03) live in a separate directory and are wired
via a **negative** test that asserts the harness rejects them.

## Acceptance (W44-B6 close)

- [x] Generator emits 13 new accepted fixtures covering 5 new sources + 3 stress shapes + 3 edges
- [x] All accepted fixtures parse as JSON and validate against OKF 1.0 shape
- [x] Failure-mode fixtures generated and isolated
- [x] Corpus README in `docs/reference/conformance/README.md` updated
- [ ] Roundtrip test wired to walk the full 33-fixture set
- [ ] Negative test wired for E01–E03

## Risk register

| Risk | Mitigation |
|------|------------|
| Generator emits a malformed OKF that round-trip accepts by accident | Strict shape assertions in test; CI red on any shape break |
| Corpus file count bloats PR diff | New fixtures land in a single batched PR; subsequent additions are small |
| Generator script breaks when OKF spec evolves (e.g. v1.1) | Builder functions keyed by `OKF_VERSION`; bump in one place |
| Performance regression: large fixtures slow the round-trip suite | Mark large fixtures with `@pytest.mark.slow` / `#[ignore]` and run in nightly |

## Carry-over to W45+ (if W44-B6 only partially closes L73)

- Per-source corpus: 1 fixture per (source × language × toolchain) — 36 cells
- Stress: 500-entity fixture, 50-deep relation chain, 100-intent rapid-fire
- Edge: emoji label, RTL label, base64 attachment, deeply nested `properties`
- Adversarial: cyclic relation graph (currently illegal per spec — confirm), contradictory `verified_by`/`bounded_by`

93 changes: 93 additions & 0 deletions docs/ops/loom-http-sse-soak.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# Loom HTTP SSE soak (C00 L7 process-level HTTP SSE)

**Wave-44 close-out lane B1.**
**Owner:** machine.
**Theme:** close C00 L7 *process-level HTTP SSE soak under loom* residual from
Wave-43 SCORECARD.

Companion: [`WAVE44_SCOPE.md`](../../WAVE44_SCOPE.md) (rank 1) and
[`WAVE44_PERT.md`](WAVE44_PERT.md) (lane B1).

## Rubric anchor

Pillar L7 — *Concurrency Safety & Races*. SCORECARD headline:
`C00 L7 | partial (deepened) | +1` (Wave-43 daemon-graph-hard). Wave-43 closed
the **live tokio port** of the daemon-graph shape. Wave-44-B1 closes the
**loom-modelled** counterpart that Wave-43 deferred as "process-level HTTP
SSE soak under loom remains unpaid".

## What this lane closes

The Wave-43 evidence set:

```
C00 L7 — Concurrency Safety & Races
✓ Cargo.toml:46-47 — unsafe_code = forbid at workspace package
✓ crates/sl-daemon/Cargo.toml:32-33 — unsafe_code = forbid on daemon
✓ crates/sl-daemon/src/http.rs:82 — graceful shutdown
✓ crates/sl-daemon/src/main.rs:46-51 — bounded channels
✓ tests/race_model.rs — loom-lite bounded sync_channel + cooperative cancel
✓ tests/loom_model.rs — 12 channel-level loom models (Wave-43 #296)
✓ tests/daemon_graph_tokio.rs — live tokio ports (Wave-43 #362)
✓ docs/ops/daemon-graph-hard.md — live tokio port
✓ scripts/daemon-graph-hard-check.ps1 — hermetic SelfCheck
✓ .github/workflows/loom-permutation.yml — blocking PR SelfCheck
NEW: tests/loom_http_sse_soak.rs — process-level HTTP SSE soak under loom
NEW: scripts/loom-http-sse-soak-check.ps1 — hermetic SelfCheck
NEW: .github/workflows/loom-http-sse-soak-soft.yml — soft nightly
```

## What the new test models

`tests/loom_http_sse_soak.rs` adds 3 loom-modelled permutations of the
**client-side** race surface (the TCP/HTTP layer itself is exercised by the
live tokio tests in `tests/daemon_graph_tokio.rs`; this file exercises the
channel-level multi-client race surface that the HTTP layer depends on):

| Test | Models | Why |
|------|--------|-----|
| `process_level_http_sse_soak_conserves_under_cancel` | N=3 client tasks each with a `broadcast::Receiver` (modelled as N outbound mpsc queues); a single publisher; cooperative cancel | Mirrors the daemon SSE fan-out: every published item reaches every connected client until cancel; clients never see more than publisher produced |
| `http_sse_soak_lagged_recovery_no_panic` | 2 publishers racing 1 client with channel capacity 2 | Models the Lagged drop path; client must not panic, must observe a non-negative message count |
| `http_sse_soak_shutdown_propagates_to_clients` | N=3 clients sharing a channel; close-publisher forces Disconnected; cancel flag forces exit | Asserts every connected client observes the shutdown signal |

The `loom::sync::mpsc` primitives model the sl-daemon's
`tokio::sync::{mpsc, broadcast}` fan-out shape. Loom explores all thread
interleavings; the asserts are timing-independent.

## How to run

### SelfCheck (hermetic, blocking on PRs)

```bash
RUSTFLAGS='--cfg loom' cargo test --test loom_http_sse_soak -- --nocapture
pwsh ./scripts/loom-http-sse-soak-check.ps1 -SelfCheck
```

### Soft nightly (extended iterations)

The `loom-http-sse-soak-soft.yml` workflow runs the same suite under
`continue-on-error` with deeper iteration counts to surface rare interleavings.

## Acceptance (W44-B1 close)

- [x] `tests/loom_http_sse_soak.rs` — 3 loom tests gated on `cfg(loom)`
- [x] Soft-lane `loom_cfg_not_enabled_documents_soft_lane` test discoverable
under default `cargo test`
- [x] `scripts/loom-http-sse-soak-check.ps1 -SelfCheck` passes
- [x] `.github/workflows/loom-http-sse-soak-soft.yml` soft nightly anchor
- [ ] PR opened + MERGED
- [ ] SCORECARD.md refresh at W44-tip (target: C00 L7 partial → pillar max)

## Risk register

| Risk | Mitigation |
|------|------------|
| Loom model explodes combinatorially with N>4 clients | Tests cap N at 3; deeper coverage in nightly soft |
| Loom misses a real HTTP-layer race (TCP framing, axum handler) | Live tokio `daemon_graph_tokio.rs` covers the HTTP layer in real wall-clock time |
| SelfCheck script diverges from the test it claims to verify | SelfCheck runs the actual test binary, not a textual grep |

## Carry-over to W45+ (if W44-B1 only partially closes)

- Per-client backpressure model (each client has its own semaphore)
- Disconnect-during-recv race (mid-recv, client tcp closes)
- Live HTTP server bound to ephemeral port under loom (loom 0.7 has `loom::net`)
37 changes: 36 additions & 1 deletion docs/reference/conformance/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,4 +148,39 @@ is referenced from the examples doc.

---

*End of OKF Conformance Corpus README.*
*End of OKF Conformance Corpus README.*

---

## Wave-44-B6 generator-produced fixtures (W44 close-out, C08 L73)

The 13 fixtures below were generated by [`scripts/corpus-generate.py`](../../scripts/corpus-generate.py)
as part of the Wave-44 close-out lane B6 (production-scale corpus breadth).
They exercise dimensions not covered by the 20 hand-vetted fixtures above:

| Fixture | Source | Stress / coverage |
|---------|--------|-------------------|
| `aider-rust-refactor-037.okf.json` | aider | rust refactor; trait-bound removal with 6-relation gate |
| `opencode-python-debugger-038.okf.json` | opencode | async iterator trace; stdlib-only constraint |
| `continue-go-microservice-039.okf.json` | continue | /healthz/ready + circuit breaker (5-relation chain) |
| `kiro-bash-ci-pipeline-040.okf.json` | kiro | GitHub Actions commit-SHA pinning |
| `factory-droid-typescript-041.okf.json` | factory-droid | retry-aware telemetry client with bundle-size cap |
| `sql-migration-multi-intent-043.okf.json` | forge | 3-intent ALTER TABLE migration w/ ONLINE constraint |
| `yaml-k8s-deployment-044.okf.json` | forge | rollingUpdate maxUnavailable=0 under PDB |
| `large-entity-count-100-045.okf.json` | forge | 82 entities / 81 relations (stress) |
| `deep-relation-graph-7-046.okf.json` | forge | 8 entities / 7-relation deep chain (transitive closure) |
| `rapid-fire-intent-stream-12-047.okf.json` | forge | 12 intents, no verifiers (open-loop stream) |
| `unicode-intent-label-cjk-048.okf.json` | forge | CJK unicode label; zh-CN locale hint |
| `embedded-json-label-049.okf.json` | forge | JSON-snippet inside label (XSS / escape verification) |
| `multi-modal-image-hint-050.okf.json` | claude-code | sha256 image attachment (no embedded bytes) |

Total corpus (post-W44-B6): 33 accepted fixtures.

Regenerate via:

```bash
python3 scripts/corpus-generate.py --out docs/reference/conformance/fixtures
```

See [`docs/ops/corpus-breadth.md`](../../docs/ops/corpus-breadth.md) for the
strategy doc, failure-mode fixtures (E01–E03), and the test wiring plan.
Loading
Loading