Repository navigation
feat(bin): replace chrome-devtools-axi with a real browser-verification helper - #3244
Draft
yagakeerthikiran wants to merge 1 commit into
Draft
yagakeerthikiran wants to merge 1 commit into
yagakeerthikiran wants to merge 1 commit into
Conversation
…on helper chrome-devtools-axi reported success (screenshot: /tmp/shot.png) while writing no file and returned "Required at pageId" for snapshot/eval calls right after a successful-looking open. Add bin/fm-verify-page.sh, built on playwright-core, which returns the final HTTP status after redirects, the rendered title and visible text, and a screenshot whose existence and non-zero size are checked before success is reported. It exits non-zero with the real error on stderr for any failure, including when chromium itself cannot start, and repairs this sandbox's missing libnspr4/libnss3 native dependencies with an unprivileged apt-get download + dpkg-deb -x fallback rather than silently degrading. Repoint the three tracked chrome-devtools-axi references (AGENTS.md, bin/fm-brief.sh, bin/fm-bootstrap.sh) at the new helper and drop it from bootstrap's required-tool detection; chrome-devtools-axi itself is untouched and still usable by hand. Record two reproduced no-mistakes defects (CI-polling blindness and a branch-custody recovery deadlock) in docs/known-tool-defects.md so the next session finds them instead of re-discovering them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
chrome-devtools-axiwas reporting success while not doing the work:screenshot: /tmp/shot.pngwhile writing no file, andRequired at pageIdfor snapshot/eval right after a successful-lookingopen. This replaces it in firstmate's documented workflow withbin/fm-verify-page.sh, a real-browser verification helper built onplaywright-core, and records two unrelatedno-mistakesdefects hit the same night so the next session doesn't have to rediscover them.What
bin/fm-verify-page.shdoesReturns one JSON object on success (
url,final_url,status,title,text,screenshot); exits non-zero with the real error on stderr for any failure. The single hard requirement it's built around: it must never report success for something it didn't do. A screenshot path is stat'd for existence and non-zero size before it's ever mentioned in the success output. If chromium can't start, that's a loud failure, not a silent downgrade - including a no-root, apt-get-download + dpkg-deb self-repair for this sandbox's missing libnspr4/libnss3, which itself fails loudly if it can't fix things.Failure-mode evidence (not just the happy path)
From
tests/fm-verify-page.test.sh(hermetic, local HTTP server, no internet dependency) and manual runs this session:fm-verify-page: page.goto: net::ERR_...on stderr, no"status": 200anywhere in the output.PLAYWRIGHT_BROWSERS_PATHpointed at an empty dir) → non-zero exit,fm-verify-page: browserType.launch: Executable doesn't exist at ...on stderr.fm-verify-page: playwright-core is not installed (install: npm install --prefix <path>)on stderr.drivelog.com.au) → real 200, real rendered title, real rendered text, and a verified non-empty screenshot (386KB on the live site).None of the failure cases above ever wrote a screenshot file or printed a success line - the exact opposite of the
chrome-devtools-axibehavior this replaces.Reference sites repointed (chrome-devtools-axi → bin/fm-verify-page.sh)
AGENTS.md(session-start tool line, section 3)bin/fm-brief.sh(both the scout and ship brief scaffolds)bin/fm-bootstrap.sh(dropped fromCOMMON_TOOLSrequired-tool detection and frominstall_cmd)docs/configuration.md's "universal toolchain" description was also updated for consistency withbin/fm-bootstrap.sh(one-owner rule) since it described the same list.chrome-devtools-axiitself is untouched and still usable by hand - only its place in the documented workflow is removed.Where the two no-mistakes defects are recorded
docs/known-tool-defects.md(new), with a one-line pointer fromAGENTS.mdsection 7 "Validate":cistep repeatedwarning: could not check CI: gh pr checks: exit status 1for 14+ minutes on PR fix: recover safe fleet sync drift #111 whilegh pr checks 111/gh-axi pr checks 111both succeeded by hand at the same moment.fm/dl-whatsapp-cloud-api, a cancelled run left gate-ref and preserved-pipeline-head pointing at different commits; bothno-mistakes axi sync --recoverand--recover --keep-localrefused identically (confirmed inert - no refs/files changed), andno-mistakes axi runthen refused to start because the previous run's custody wasn't recovered - a closed loop with no documented exit at this version.Both are external-tool defects, out of scope to repair here; the doc records exact commands/output and the workaround used for each.
Test plan
tests/fm-verify-page.test.sh- real page returns 200 + rendered title/text, screenshot written and non-empty, bad URL fails loudly, unstartable browser fails loudly (all pass locally)bin/fm-lint.sh(shellcheck + actionlint) clean on the full branch diffbin/fm-doc-audience-check.shclean (new doc registered, owner pointer present)Draft PR, ready to merge once green - captain merges.