fix(i18n): reject translated chunks that leak reasoning or lose table rows - #14731
Merged
Merged
Conversation
… rows The drift check only compares hashes, so broken mirrors passed it. In the 2026-09-23/24 refreshes, fallback models leaked their <think> block and English meta-prose into about 90 mirrors, and dropped table rows in about 60 (es PROVIDER_REFERENCE lost 23 rows). validateTranslatedChunk() strips <think> blocks and a wrapping ```markdown fence. It then rejects the chunk when the code-fence count or the table-row count differs from the source chunk, or when English meta phrases appear more often than in the source. translateBody() retries a rejected chunk up to 3 times and then fails the doc instead of writing it.
This was referenced Sep 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
run-translation.mjsaccepted whatever the model returned. The drift check only compares hashes, so broken mirrors still passed it. In the two refreshes on 2026-09-23/24 (#14676, #14724), fallback models:<think>…</think>blocks and English meta-prose such as "I'll keep the table header…" and "My plan became clear";esPROVIDER_REFERENCE.mdlost 23 rows, andbsREMOVED_PROVIDERS.mdcollapsed from 67 lines to 32.Both PRs had to be repaired by hand before merge.
Change
validateTranslatedChunk(source, output)(exported, pure):<think>…</think>blocks and a```markdownfence that wraps the whole answer;<think>tag;I'll,I need to,Okay,,My plan, …) that appear more often than in the source.translateBody()retries a rejected chunk up to 3 times. If the chunk still fails, the doc fails and the mirror is not written. The runner already reports failed docs and retries them.Validation
tests/unit/i18n-translation-output-guard.test.ts(9 cases, built from the real refresh-5/6 failures). It failed before the change (no export) and passes after.cinf503,auto/gemini→ dva 502).