Skip to content

G D.10 + G D.11 — Verifier hardening (source fixes + alert-model + display refresh)#14

Merged
blackbrowed-labs merged 11 commits into
mainfrom
dev
May 12, 2026
Merged

G D.10 + G D.11 — Verifier hardening (source fixes + alert-model + display refresh)#14
blackbrowed-labs merged 11 commits into
mainfrom
dev

Conversation

@larsweiser
Copy link
Copy Markdown
Collaborator

Summary

  • Restore the Cloudflare-fact verifier to working state after both source URLs broke between 2026-04-30 and 2026-05-11 (CWA URL retired; DPF rewrote as a React SPA).
  • Rewrite the alert-model so the verifier is silent when healthy, opens a PR only on value changes, opens an Issue only on parser/fetch failures.
  • Decouple the displayed "last verified" date from git: build workflows read a GitHub Actions repo variable that the verifier updates after every clean run — no commits, no merges to refresh the date on dev or prod.

Gates landed

  • G D.10.0 — Per-check fixture extension support in scripts/run-verifier.mjs (68ad168).
  • G D.10.1 (A) — CWA URL + parser fix for the FAQ-page shape (fe0b68a).
  • G D.10.3a (B) — DPF rewrite to JSON API at dpfapi.azurewebsites.net (POST + filter body, 132a6ea).
  • G D.11.1 (C) — Three-channel alert-model rewrite in orchestrator + workflow YAML (595d139).
  • G D.11.2 (D) — env-var-driven display refresh + new rebuild-nightly-staging.yml + freshness gate moved from Node script to CI step (cf07bb3).
  • Fix-up Pass 1 follow-ups: sticky header + mobile scroll lock + docs #1 — Three Minors from final integration review: dead hasDiff variable, stale README reference, JSDoc note on per-fact meta-marker intentionality (055a851).
  • Fix-up [ci-bootstrap] CI smoke-test — close without merging #2 — Use fine-grained PAT (VERIFIER_VARIABLE_TOKEN) for the variable-update step; GITHUB_TOKEN with actions: write returns HTTP 403 on the Actions Variables API (60b0b40).

Test plan / staging evidence

  • All four implementer reviews + final integration review: approved.
  • Live verifier dispatch against dev: variable updated 2026-04-29T00:00:00Z2026-05-12T10:01:17Z, no PR opened, no Issue opened — exactly the silent-healthy design intent.
  • Drift tests (mock_scenario=cwa-changed-figure and cwa-parser-broken): channel gates correctly skip on dry_run=true; stale-escalation still runs cleanly.
  • Staging redeploy + curl dev.blackbrowedlabs.com/datenschutz + /en/privacy: top-level bbl-verified-at meta + visible "zuletzt geprüft am" / "last verified on" prose all show today's date in both locales; per-fact bbl-verified-at-dpf/-cwa markers correctly stay pinned to JSON's 2026-04-29 ground truth.
  • npm run build + astro check: clean.

Notable design

  • Three-channel alert-model: all_ok → silent variable refresh; has_value_diff → PR; has_failure → Issue. Channels independent; a run with one fact changed and another parser-broken fires both.
  • Variable-write requires a separate PAT: GITHUB_TOKEN with actions: write does NOT cover the Actions Variables API. The fine-grained PAT VERIFIER_VARIABLE_TOKEN is scoped to this repo with Variables: read/write + Metadata: read only.
  • Freshness-gate semantic shift: moved from a Node build-step reading _meta.last_check_attempt (any verifier run, 30d) to a CI shell step reading the VERIFIER_LAST_OK_AT variable (clean runs only, 90d on deploy/nightly, 60d on PR). Stricter signal + wider threshold; deliberate widening.
  • Parallelization: four implementers dispatched simultaneously in isolated worktrees (controlled exception to the project's no-worktree default). Each implementer self-rebased onto the G D.10.0 prep commit; the JSON file's two cwa/dpf source_url fields didn't conflict despite shared ownership.

Refs

🤖 Generated with Claude Code

blackbrowed-labs and others added 11 commits May 12, 2026 11:10
Lift the .html hard-code in fixturePathFor so individual check
modules can declare their preferred fixture format via a
fixtureExtension export. Backwards-compatible default 'html' keeps
existing dpf-*.html and cwa-*.html fixtures resolving as before.

Enables G D.10.3a's DPF JSON-API switch (.json fixtures for the
rewritten check module) without coupling the orchestrator to
per-fact formats.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Cloudflare Web Analytics retention info moved from the dedicated
/web-analytics/data-retention/ page (now HTTP 404) to a single FAQ
answer at /web-analytics/faq/ under "What is the period of time I can
access data in Web Analytics?". The prose shape also changed: from
"retains aggregated page view counts for 6 months" to "you can access
data for the previous six months" — same 6-month figure, spelled-out
number, different surrounding keywords.

This commit:
  - updates sourceUrl in scripts/checks/cwa-retention.mjs
  - updates cwa_retention.source_url in src/data/cloudflare-facts.json
  - replaces the cwa-active.html fixture with the new FAQ shape
  - adds a fourth aggregatedPatterns variant matching the FAQ prose
    ("access/view/see ... previous/past/last <N> months")
  - extracts a NUMBER_WORDS map + toMonths() helper so the variant
    accepts both digits and spelled-out numbers up to twenty-four

The cwa-parser-broken.html fixture still triggers parser-broken under
the new permissive regex (no numeric retention phrase present), so no
fixture tightening was needed.
Replace the has_diff/has_attention two-flag model with three
independent channels:
  - all_ok          → update VERIFIER_LAST_OK_AT repo variable (silent)
  - has_value_diff  → open PR on dev with the value diff
  - has_failure     → open verifier-alert Issue

Channels are independent; a run can fire >1. Drops the post-deploy
smoke step (out of scope under the new model; may return inside
deploy-production.yml later). Drops the auto-PR/Issue cascading
gate so a permission failure in one channel no longer silences
the others.

JSON disk write is now gated on has_value_diff — clean runs don't
mutate the file in the runner. Closes the weekly-PR-noise concern.
…ly parity

Switch the privacy pages' displayed verified-at date from a JSON-driven
build-time read to an env-driven build-time read so prod/staging can
refresh the rendered date from the VERIFIER_LAST_OK_AT GitHub Actions
repo variable without any git operations.

- getEffectiveVerifiedDate prefers import.meta.env.VERIFIED_AT over the
  JSON's per-fact verified_at values. Date.parse() guard falls back to
  JSON on empty/malformed env. Project convention is import.meta.env
  (not process.env); same shape that github-api.ts and env.ts already
  use. Build-time only; no client-JS impact.
- All four build workflows (deploy-staging, deploy-production,
  rebuild-nightly, ci-pr) read \${{ vars.VERIFIER_LAST_OK_AT }} and pass
  it as VERIFIED_AT env to npm run build.
- New rebuild-nightly-staging.yml mirrors rebuild-nightly.yml against
  dev → staging so the dev environment refreshes daily too (closes the
  prod-only nightly asymmetry).
- Freshness-gate moves from a Node build-step
  (scripts/check-cloudflare-facts-freshness.mjs, now deleted) to a CI
  step that reads the repo variable's age. Deploy/nightly workflows
  fail above 90 days (hard gate); ci-pr warns above 60 days (so
  editorial PRs aren't blocked by verifier health).

Build smokes (unset / valid / malformed VERIFIED_AT) confirm the env
override, the JSON fallback, and the Date.parse guard behave as
specified.
Brings Implementer A's branch (commit fe0b68a) into the integration
branch. Reviewer (superpowers:code-reviewer) verdict: approved with
two cosmetic Minors deferred to backlog (JSDoc-comment URL drift on
cwa-retention.mjs:8; twentyfour map-entry redundancy).
Brings Implementer C's branch (commit 595d139) into the integration
branch. Reviewer (superpowers:code-reviewer) verdict: approved with
two Minors deferred to backlog (hasDiff now dead code on
run-verifier.mjs:207; PR-body "went absent" wording reads naturally
for DPF less so for CWA-style facts).
…(Implementer D, approved-with-notes)

Brings Implementer D's branch (commit cf07bb3) into the integration
branch. Reviewer (superpowers:code-reviewer) verdict: approved-with-
notes. Two Minors deferred to backlog (README.md:56 stale freshness-
script reference; plan §13 wording references process.env where
import.meta.env is the correct Astro/Vite pattern).

Notable deviations rolled in:
  - import.meta.env.VERIFIED_AT (not process.env) — astro check
    correctness; matches codebase pattern in github-api.ts / env.ts
  - Astro pages NOT directly edited — both already route through
    getEffectiveVerifiedDate; helper-only change covers them cleanly
The dpf.gov/list page rewrote as a client-side React SPA between
2026-02-10 and 2026-05-04, so the verifier's static-fetch HTML parse
broke. Discovery in g-d-10/g-d-10-2/dpf-investigation.md identified
the backing JSON API. This commit switches dpf.mjs from HTML parse to
JSON parse against that API, replaces the three dpf-* fixtures with
JSON equivalents, and exports fixtureExtension='json' (the per-check
extension support landed in G D.10.0 prep).

The 'Cloudflare, Inc.' active-listing fact itself is unchanged.
Brings Implementer B's branch (commit 132a6ea) into the integration
branch. Reviewer (superpowers:code-reviewer) verdict: approved. Two
cosmetic Minors deferred to backlog (dpf.mjs:11 spec-section reference;
dpf-parser-broken.json could exercise SumCount===0 case too).

Notable design choice rolled in:
  - sourceUrl stays at the user-facing SPA URL (dataprivacyframework.gov/list)
    so JSON's dpf.source_url citation is human-readable; a module-private
    API_ENDPOINT carries the actual API URL (dpfapi.azurewebsites.net).
Three small fix-ups from the final integration review:

1. scripts/run-verifier.mjs — drop the dead `hasDiff` local. It was
   computed but never read; the JSON write at the next block uses
   `hasValueDiff`, which is the correct signal.
2. src/lib/cloudflare-facts.ts — extend the file-level JSDoc with an
   explicit note that the per-fact `bbl-verified-at-dpf` /
   `bbl-verified-at-cwa` meta markers deliberately read JSON's
   per-fact `verified_at` directly rather than routing through
   `getEffectiveVerifiedDate()`. The asymmetry is intentional
   (diagnostic markers tracking per-fact JSON ground truth, vs the
   unified display date that's env-preferred).
3. README.md — drop the stale bullet for the deleted
   `scripts/check-cloudflare-facts-freshness.mjs`; add a one-paragraph
   note pointing at the new CI-side freshness gate that reads the
   `VERIFIER_LAST_OK_AT` repo variable.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…e updates

The VERIFIER_LAST_OK_AT variable-update step in verify-cloudflare-facts.yml
failed with HTTP 403 against the default GITHUB_TOKEN, even with
`actions: write` declared in the permissions block. The workflow-
permissions YAML does not actually expose a `variables` scope; updating
Actions Variables requires either a classic PAT with `repo` scope or a
fine-grained PAT with Variables: read/write.

Resolution: switch the variable-update step's GH_TOKEN to a new
VERIFIER_VARIABLE_TOKEN repo secret (fine-grained PAT scoped to this
repo only, Variables: read/write + Metadata: read). Drop the unused
`actions: write` from the workflow's permissions block; replace with
an explanatory comment so future maintainers don't re-add it. The PR
and Issue steps continue to use GITHUB_TOKEN — they only need
pull-requests: write and issues: write, which the default token carries.

Surfaced when the first live workflow_dispatch on dev's new YAML failed
at step 6. The mock-scenario drift tests passed because their channel
steps gate on dry_run=false and were correctly skipped end-to-end.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@blackbrowed-labs blackbrowed-labs merged commit 55ebda0 into main May 12, 2026
4 checks passed
blackbrowed-labs added a commit that referenced this pull request May 12, 2026
Routine "update branch" per docs/TECH_STACK.md §8.4 merge-commit
workflow. main got PR #14's merge commit (f76833a → ...); this
brings that commit into dev's history so PR #15 (G D.12) is up
to date with main and can be merged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
blackbrowed-labs added a commit that referenced this pull request May 12, 2026
…actForm (#14)

Add margin-inline centring to max-width: 68ch inner-content selectors on
Contact (DE+EN), Impressum + en/legal, Datenschutz + en/privacy, plus the
ContactForm component's .contact-form max-width: 32rem block. Body content
now centres within the 1200px outer container at desktop widths instead of
sitting flush-left with a ~700px right-side gap.

- 18 CSS rule modifications across 6 page files (kontakt, en/contact,
  impressum, en/legal, datenschutz, en/privacy) plus 1 in ContactForm.astro
  (Option B per locked pre-flight decision 2026-05-12).
- Selectors using the margin shorthand (kontakt/en-contact h1, content,
  content p, content h2 + impressum/legal lede, address, p) get their inline
  slot rewritten 0 → auto. Selectors without an existing margin declaration
  (impressum/legal/datenschutz/privacy __content + ContactForm .contact-form)
  gain a standalone margin-inline: auto.
- Mobile (<768px) unchanged: existing max-width: none resets on inner
  selectors make margin-inline: auto a no-op at single-column viewports.
  Explicit margin: 0 0 1.5rem mobile reset added to .contact h1 (which
  retains max-width: 14ch mobile-side) to preserve the Phase B.1
  flush-left mobile H1.
- Out of scope: ueber / en/about (two-column .about__top grid handles
  centring intentionally) and product-detail pages (already symmetric
  via .product-detail__container { margin: 0 auto; max-width: 70ch; }).

Closes Pass 2 backlog row #14. Phase J's planned CSS extraction will fold
these into shared modules; this gate ships the layout fix only.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants