Recapture the gallery stills so the preview digest matches main - #156
Recapture the gallery stills so the preview digest matches main#156leaf-agent wants to merge 1 commit into
Conversation
PR #134 captured the stills from a branch based at #149, before #152 rewrote skills/leaf/assets/runtime/chrome-style.js. The squash merge kept main's stylesheet and the branch's stills, so docs/example-previews.json recorded an inputs_sha256 for a tree that no longer exists and test_site.py's catalog check went red on main. Rerun scripts/example-previews.py against current main. Eight of the nine JPEGs re-encode byte-identically; only example-ship-review.jpg moves, by an 8x8 pixel region in the key line corner that #152's keycap box accounts for.
|
Reproduced this capture independently and it comes back byte-identical, which is the thing a binary diff cannot show on its own. I was dispatched for run 33344080789 (21332fc, "Simplify tab selection"), whose 32 failures leave three on What was compared
Fresh
|
|
This recapture no longer satisfies the gate at Measured here with
Since none of the nine stills nor I am not recapturing from CI: the fonts here substitute, per the |
|
Folded into #161 at Closing this duplicate in favor of #161.
|
… is hidden (#160) ## Problem `ci` on [72be4be](https://github.com/max-sixty/leaf/actions/runs/33345068975) failed with 35 nightly tests red. Rerunning all 35 node ids against `c0a5b65` leaves four: the catalog digest (#156), the two page-map chord reads (#157), and this one, which nothing owns. ``` FAILED tests/test_render_controls.py::test_a_wide_banner_spends_status_copy_before_action_reach AssertionError: Locator expected to be hidden 6 × resolved to <button ... class="lf-btn lf-answer-all ...">✓ Accept all (3)</button> 3 × resolved to ... ✓ Accept all (2) 4 × resolved to ... ✓ Accept all (1) ``` The count is walking down and running out of budget. The blanket answer [decides its decisions one at a time](https://github.com/max-sixty/leaf/blob/8683b9b1d2b9d408506c9f88bf3baf5a4f84685c/skills/leaf/assets/runtime/decisions/view.js#L68-L71) — `await source[verb]?.()` per decision — so the press owes one round trip per decision, and the test released the held first answer straight into `expect(answer_all).to_be_hidden()`. That single 5s budget was covering three sequential server trips, not the repaint it is for. `tests/CLAUDE.md` names this exactly: "Letting `expect` absorb the page's next read hides which mechanism supplied the wait and spends its timeout budget on transport rather than on the assertion." Instrumenting the sequence on an idle runner gives `event → state → event → state → event`, 1.89s end to end. Three trips at ~0.6s each fit inside 5s on an idle machine and do not on a loaded one, which is why this is nightly-and-under-load only. `test_accept_all_decides_every_pending_suggestion` drives the same control and does not have the bug: it waits for each widget's own settle first, so its `to_be_hidden` only has to cover the last repaint, and [its comment says why](https://github.com/max-sixty/leaf/blob/8683b9b1d2b9d408506c9f88bf3baf5a4f84685c/tests/test_render_widgets.py#L1541-L1543). This site was the one that skipped the staging. ## Solution State the transport with `_until` on the `Traffic` counters — the same causal helper this test already uses eight lines above to hold the answer in the wire — then let `to_be_hidden` assert only the repaint it is about. The number of trips owed is read off the control's own face (`✓ Accept all (3)`) rather than written out, so the fixture's arithmetic stays in one place, and an `owed > 1` guard fails loudly if a fixture change ever leaves this test a single trip and stops it exercising the sequence at all. This is not a raised timeout. `_until` is bounded at 30s with the counters in its failure message, so a genuinely stuck answer now fails naming the missing evidence instead of pointing at a control that "never went". ## Testing Playwright driving real Chromium on this runner, at `8683b9b`: - `uv run pytest tests` — **740 passed, 6 skipped**. This is the gate `ci` runs on the PR; the test fixed here is nightly-only and not exercised by it. - `uv run pytest tests/test_render_controls.py --run-nightly -n4` — **64 passed**, three times consecutively on an otherwise-idle machine. - `ruff check` and `ruff format --check` on the changed file, at the `v0.16.1` `.pre-commit-config.yaml` pin — clean. <details><summary>Reproducing the failure, and the mutation checks</summary> The test passes 6/6 at `-n0` on `c0a5b65` unmodified, so `-n0` alone leaves it unclassified. Stalling the driver for 300ms between `held[0].continue_()` and the hide assertion reproduces the CI shape exactly on the unfixed test — the button walks `(3) → (2) → (1)` and the expect runs out: ``` 5 × ... ✓ Accept all (3) 5 × ... ✓ Accept all (2) 3 × ... ✓ Accept all (1) ``` With this change in place the same stall passes. Two mutations confirm the new wait bites rather than just returning: | mutation | result | | --- | --- | | `traffic.sends == owed` → `owed + 1` | fails: `the page never settled every one of the 3 answers the blanket press owed: the wait began on sends=1 acked=0 pending=1 asked=3 heard=3 and gave up on sends=3 acked=3 pending=0 asked=6 heard=6` — and this also confirms `owed` reads 3 and that the predicate's own terms are what carry it | | driver stall re-applied to the fixed test | passes in 8.55s | Timed trace of the released sequence on an idle runner, seconds from `continue_()`: ``` 0.38 FIN event 0.66 FIN state 1.17 FIN event 1.38 FIN state 1.81 FIN event → hidden at 1.89 ``` The other `.lf-answer-all` sites were checked for the same shape and do not have it: `test_render_gate.py` navigates away, `test_render_startup.py` puts `wait_for_function(BOTH_STAMPS)` between the release and the hide assertion and has one decision rather than three, and `test_render_projection.py` asserts the control is absent. </details> --- Automated fix for [failed run](https://github.com/max-sixty/leaf/actions/runs/33345068975) Co-authored-by: leaf-agent <318509791+leaf-agent@users.noreply.github.com>
2 participants
Problem
The nightly
cirun on7d219d1reported 30 failures. 29 of them were fixed by the 15 commits that landed onmainafterwards (#137, #145, #148–#153); rerunning all 30 node ids againstc0a5b65leaves exactly one still red, and it is red for a reason none of those commits addressed:The cause is a merge-order gap, not a flake. #134 recaptured the gallery stills from a branch based at
4b5b4b8(#149) — before6d68a1b(#152) rewroteskills/leaf/assets/runtime/chrome-style.jsto hoist the keycap box intoKEY_BOX. The squash merge tookmain's stylesheet and the branch's stills together, sodocs/example-previews.jsonshipped aninputs_sha256computed over a tree that no longer exists.capture_input_files()digests that stylesheet, so the manifest and the checkout disagreed from the moment #134 landed.Isolating the single differing input
Enumerating
capture_input_files()at #134's head and atc0a5b65and diffing per-file hashes gives exactly one row:git merge-base cd998bf c0a5b65is4b5b4b8, which is an ancestor of6d68a1b— the branch predates the stylesheet change.Solution
Rerun
scripts/example-previews.pyagainst currentmain, which is the remedy the assertion message names. This is the same fix #134 applied, redone from a base that has #152 in it — #134 merged rather than being turned down, so the approach stands; only its base was stale.Eight of the nine JPEGs re-encode byte-identically, so the capture is reproducible here and no font or renderer drift is being introduced. Only
example-ship-review.jpgmoves, and only across an 8×8 pixel region at (776, 520) with a maximum channel difference of 8 — the key line corner that #152's keycap box accounts for. The manifest'sinputs_sha256now reads3c23a202…, matching the digest computed over the checkout.Left alone, and worth a maintainer's call rather than a bot's: nothing stops this recurring. Any recapture PR silently goes stale if a commit touching a capture input merges between its base and its own merge, and the staleness only surfaces on
mainafter the fact. Requiring branches to be up to date before merging would close it, but that is a repository setting with effects well past this script.Testing
uv run pytest --run-nightlyover all 30 node ids from the failed run atc0a5b65: 29 passed, 1 failed — establishing this as the only surviving failure.uv run pytest --run-nightly tests/test_site.py— 20 passed on this branch (was 1 failed onmain).uv run pytest tests— 740 passed, 6 skipped.preview_build.digest(preview_build.capture_input_files())against the working tree and confirmed it equals the committedinputs_sha256.Automated fix for failed run