Skip to content

feat(ci): block stale UI translations when an English value is rewritten - #8574

Merged
diegosouzapw merged 1 commit into
release/v3.8.49from
feat/i18n-ui-value-drift-gate
Jul 27, 2026
Merged

diegosouzapw merged 1 commit into
release/v3.8.49from
feat/i18n-ui-value-drift-gate

Conversation

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Follow-up to #8463: make the bug that PR fixed impossible to reintroduce silently.

The gap

oauthModal.googleOAuthWarning's English value was rewritten when the Antigravity login
helper landed (#5203). 39 of 43 locales kept a translation of the previous English:

"Após a autorização, seu navegador tentará abrir localhost — copie o URL completo e cole-o abaixo."

That flow cannot complete for Google-loopback providers — the consent never redirects, so
there is no URL to copy. Non-English users read confident, wrong instructions for months.
No gate noticed, because none of the three can see this class:

Existing gate Why it misses a stale value
sync-ui-keys.mjs Backfills keys that are absent; a key that exists with an outdated translation is invisible to it.
check-ui-keys-coverage.mjs Counts key presence. All 43 locales reported 99.6% coverage the whole time.
check-translation-drift.mjs Tracks docs/i18n/<locale>/**.md — the documentation mirrors. It never reads src/i18n/messages/*.json.

Worth stating plainly: bootstrapping check-translation-drift's missing .i18n-state.json
would not have caught this. Wrong surface — it is a docs-translation gate.

The gate

scripts/i18n/check-ui-value-drift.mjs is diff-aware, not baseline-backed. It compares
en.json at the merge base against the working tree; for every key whose English value
changed, it reports any locale still holding an untouched translation.

[i18n-value-drift] 1 English value(s) rewritten with 39 stale translation(s):
  - oauthModal.remoteAccessInfo
      stale in 39 locale(s): ar, az, bg, bn, cs, da, de, es, fa, fi, fr, …

  Fix either way:
    1. update those translations to match the new English, or
    2. set them to "__MISSING__:<new english>" so the runtime serves
       the corrected English until the translation pipeline catches up.

  If the string's MEANING changed, prefer renaming the key instead — a new key
  cannot inherit a stale translation (see #8463).

Why diff-aware rather than a baseline

It freezes pre-existing debt by construction: a diff cannot reveal which old English a
long-standing translation was derived from, so the gate judges only what the current change
touches and unrelated PRs never pay for historical drift.

The alternative — a per-key hash baseline over 11,207 keys — would have cost a ~600 KB
generated file (3× file-size-baseline.json, the largest existing baseline) that churns on
every i18n PR. Same protection, none of the artifact.

Consequence to be explicit about: already-stale translations stay invisible. This gate
stops the bleeding; it does not retroactively audit history.

CI wiring

Blocking in i18n-ui-coverage, not i18n — the i18n job is
continue-on-error: true, so a gate placed there could not block anything.

That job gains fetch-depth: 0: the gate needs the base ref, and on a shallow clone it
self-skips with SKIP reason=base-unresolved (mirroring check-openapi-breaking, which
carries the same comment for the same reason). BASE_REF is passed via env: and reaches
git only through execFileSync argv — never interpolated into a shell string.

npm run i18n:check-value-drift          # strict — what CI runs
npm run i18n:check-value-drift:warn     # report only

Tests

11 tests, written failing-first, against the exported pure core:

  • an edited EN value with an untouched translation → stale;
  • refreshed in the same diff → not stale;
  • demoted to __MISSING__: → not stale;
  • never translated in that locale → not stale (coverage's job, not this gate's);
  • brand-new key → not stale;
  • unchanged EN value → never flags, however stale the translation actually is (the
    frozen-debt semantics, asserted rather than assumed);
  • renamed key (delete + add) → flags nothing, i.e. the fix(oauth): actionable guidance for LAN-origin loopback mismatches (codex + Antigravity) #8463 fix pattern stays legal;
  • en is never its own target, and every affected locale is reported;
  • whitespace-only edits count (conservative on purpose — a reword must not hide behind an
    innocuous diff);
  • non-string leaves (numbers, booleans, null) compare without throwing.

End-to-end verification against the real defect

Rewrote an English value in the working tree, leaving translations untouched:

Run Result
check-ui-value-drift (strict) exit 1, reports 39 stale locales — matching the real #5203 count
--warn exit 0, same report
--json {"ok": false, …} with the resolved diffBase
BASE_REF=refs/heads/nao-existe exit 0, SKIP reason=base-unresolved
clean tree exit 0, PASS

Validation

Gate Result
new unit suite 11/11 passed
i18n:check-value-drift on this branch PASS
i18n:check-ui-coverage PASS — 42 locales ≥ 65%
check-docs-sync PASS
check:any-budget:t11 / check:tracked-artifacts PASS
eslint clean (script path is eslint-ignored; 0 errors)

check-workflows ratchet — pre-existing, not from this PR. It reports
180 zizmor findings > baseline 176. Measured origin/release/v3.8.49 in a detached
worktree: also 180. My workflow edit adds zero findings; the +4 is base drift that
predates this branch and needs either a fix or a re-baseline independently.

Docs

docs/architecture/QUALITY_GATES.md — the i18n-ui-coverage table gains the gate, plus a
subsection covering the incident, why the other three gates miss it, the diff-aware
trade-off, and the two remedies.

Closes the gap that let #8463 ship. `oauthModal.googleOAuthWarning`'s English value
was rewritten when the Antigravity login helper landed (#5203); 39 of 43 locales kept a
translation of the PREVIOUS English, which told operators to "copy the full URL and
paste it below" — a flow that cannot complete for that provider family. Non-English
users read confident, wrong instructions for months and no gate noticed.

None of the three existing gates can see this class:

- `sync-ui-keys.mjs` only backfills keys that are ABSENT, never ones that are STALE;
- `check-ui-keys-coverage.mjs` counts key PRESENCE, so a stale translation scores as
  fully covered (all 43 locales sat at 99.6% throughout);
- `check-translation-drift.mjs` tracks the `docs/i18n/<locale>/**.md` documentation
  mirrors — it never reads `src/i18n/messages/*.json` at all. (Its `.i18n-state.json` is
  also absent, so it self-skips, but bootstrapping it would not have helped: wrong
  surface.)

New gate `scripts/i18n/check-ui-value-drift.mjs` is DIFF-AWARE rather than
baseline-backed: it compares `en.json` at the merge base against the working tree, and
for every key whose English value changed, reports any locale still holding an untouched
translation.

That choice deliberately freezes pre-existing debt — a diff cannot reveal which old
English a long-standing translation came from, so the gate judges only what the current
change touches, and unrelated PRs never pay for historical drift. The alternative, a
per-key hash baseline over 11207 keys, would have cost a ~600 KB generated file (3x the
largest existing baseline) churning on every i18n PR.

Two ways to satisfy it: refresh the translations, or set them to
`__MISSING__:<new english>` so the runtime serves the corrected English (#7258) while
the key queues for translation. When the string's MEANING changes, renaming the key is
better still — a new key cannot inherit a stale translation, which is what #8463 did.

Wired blocking into the `i18n-ui-coverage` job (the `i18n` job is
`continue-on-error: true`, so a gate there could not block anything). That job gains
`fetch-depth: 0` because the gate needs the base ref; without it the gate self-skips with
`base-unresolved`, mirroring `check-openapi-breaking`. `BASE_REF` is passed via `env:`
and reaches git only through `execFileSync` argv — never a shell string.

Verified against the real defect: rewriting an English value with translations left
behind reports exactly 39 stale locales and exits 1; `--warn` exits 0; an unresolvable
base exits 0 with `SKIP reason=base-unresolved`.
@diegosouzapw
diegosouzapw merged commit 803e737 into release/v3.8.49 Jul 27, 2026
10 checks passed
@diegosouzapw
diegosouzapw deleted the feat/i18n-ui-value-drift-gate branch July 27, 2026 22:06
HouMinXi pushed a commit to HouMinXi/OmniRoute that referenced this pull request Aug 2, 2026
…ten (cheaper-inference#8574)

Closes the gap that let cheaper-inference#8463 ship. `oauthModal.googleOAuthWarning`'s English value
was rewritten when the Antigravity login helper landed (cheaper-inference#5203); 39 of 43 locales kept a
translation of the PREVIOUS English, which told operators to "copy the full URL and
paste it below" — a flow that cannot complete for that provider family. Non-English
users read confident, wrong instructions for months and no gate noticed.

None of the three existing gates can see this class:

- `sync-ui-keys.mjs` only backfills keys that are ABSENT, never ones that are STALE;
- `check-ui-keys-coverage.mjs` counts key PRESENCE, so a stale translation scores as
  fully covered (all 43 locales sat at 99.6% throughout);
- `check-translation-drift.mjs` tracks the `docs/i18n/<locale>/**.md` documentation
  mirrors — it never reads `src/i18n/messages/*.json` at all. (Its `.i18n-state.json` is
  also absent, so it self-skips, but bootstrapping it would not have helped: wrong
  surface.)

New gate `scripts/i18n/check-ui-value-drift.mjs` is DIFF-AWARE rather than
baseline-backed: it compares `en.json` at the merge base against the working tree, and
for every key whose English value changed, reports any locale still holding an untouched
translation.

That choice deliberately freezes pre-existing debt — a diff cannot reveal which old
English a long-standing translation came from, so the gate judges only what the current
change touches, and unrelated PRs never pay for historical drift. The alternative, a
per-key hash baseline over 11207 keys, would have cost a ~600 KB generated file (3x the
largest existing baseline) churning on every i18n PR.

Two ways to satisfy it: refresh the translations, or set them to
`__MISSING__:<new english>` so the runtime serves the corrected English (cheaper-inference#7258) while
the key queues for translation. When the string's MEANING changes, renaming the key is
better still — a new key cannot inherit a stale translation, which is what cheaper-inference#8463 did.

Wired blocking into the `i18n-ui-coverage` job (the `i18n` job is
`continue-on-error: true`, so a gate there could not block anything). That job gains
`fetch-depth: 0` because the gate needs the base ref; without it the gate self-skips with
`base-unresolved`, mirroring `check-openapi-breaking`. `BASE_REF` is passed via `env:`
and reaches git only through `execFileSync` argv — never a shell string.

Verified against the real defect: rewriting an English value with translations left
behind reports exactly 39 stale locales and exits 1; `--warn` exits 0; an unresolvable
base exits 0 with `SKIP reason=base-unresolved`.

Co-authored-by: ikelvingo <im.kelvinwong@gmail.com>
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…ten (cheaper-inference#8574)

Closes the gap that let cheaper-inference#8463 ship. `oauthModal.googleOAuthWarning`'s English value
was rewritten when the Antigravity login helper landed (cheaper-inference#5203); 39 of 43 locales kept a
translation of the PREVIOUS English, which told operators to "copy the full URL and
paste it below" — a flow that cannot complete for that provider family. Non-English
users read confident, wrong instructions for months and no gate noticed.

None of the three existing gates can see this class:

- `sync-ui-keys.mjs` only backfills keys that are ABSENT, never ones that are STALE;
- `check-ui-keys-coverage.mjs` counts key PRESENCE, so a stale translation scores as
  fully covered (all 43 locales sat at 99.6% throughout);
- `check-translation-drift.mjs` tracks the `docs/i18n/<locale>/**.md` documentation
  mirrors — it never reads `src/i18n/messages/*.json` at all. (Its `.i18n-state.json` is
  also absent, so it self-skips, but bootstrapping it would not have helped: wrong
  surface.)

New gate `scripts/i18n/check-ui-value-drift.mjs` is DIFF-AWARE rather than
baseline-backed: it compares `en.json` at the merge base against the working tree, and
for every key whose English value changed, reports any locale still holding an untouched
translation.

That choice deliberately freezes pre-existing debt — a diff cannot reveal which old
English a long-standing translation came from, so the gate judges only what the current
change touches, and unrelated PRs never pay for historical drift. The alternative, a
per-key hash baseline over 11207 keys, would have cost a ~600 KB generated file (3x the
largest existing baseline) churning on every i18n PR.

Two ways to satisfy it: refresh the translations, or set them to
`__MISSING__:<new english>` so the runtime serves the corrected English (cheaper-inference#7258) while
the key queues for translation. When the string's MEANING changes, renaming the key is
better still — a new key cannot inherit a stale translation, which is what cheaper-inference#8463 did.

Wired blocking into the `i18n-ui-coverage` job (the `i18n` job is
`continue-on-error: true`, so a gate there could not block anything). That job gains
`fetch-depth: 0` because the gate needs the base ref; without it the gate self-skips with
`base-unresolved`, mirroring `check-openapi-breaking`. `BASE_REF` is passed via `env:`
and reaches git only through `execFileSync` argv — never a shell string.

Verified against the real defect: rewriting an English value with translations left
behind reports exactly 39 stale locales and exits 1; `--warn` exits 0; an unresolvable
base exits 0 with `SKIP reason=base-unresolved`.

Co-authored-by: ikelvingo <im.kelvinwong@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants