Skip to content

tooling(i18n): #192 post-ratification apply script (dry-run default, drift-free) - #602

Merged
jsboige merged 1 commit into
masterfrom
tooling/192-terminology-apply-script
Jun 28, 2026
Merged

tooling(i18n): #192 post-ratification apply script (dry-run default, drift-free)#602
jsboige merged 1 commit into
masterfrom
tooling/192-terminology-apply-script

Conversation

@jsboige

@jsboige jsboige commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

#192 — post-ratification apply script (tooling, dry-run default, drift-free)

Dispatch v3 primaire (ai-01 2026-06-25 msg-…37hitt). Pre-stages the #192 harmonization execution so that once jsboige ratifies at WE 27/06, the harmonization becomes 1 command instead of a manual edit pass.

Tooling only. 0 write under Cards/ (release freeze). Base = master bef3bc6c.

What it is

Counterpart to 192-terminology-audit.py (which detects multi-variant groups). This script applies the ratified harmonization (Decision A register + Decision B lexical picks) as a drift-free CSV edit — same method as #595 (QUOTE_MINIMAL + CRLF, UTF-8 no-BOM, byte-identical round-trip on unchanged cells).

Three modes:

  • default = DRY-RUN — prints the exact diff, writes nothing.
  • --verify-drift — rewrites with ZERO cell changes, asserts byte-identical to source. Run FIRST on any machine; if it drifts, the writer dialect is wrong for that CSV and --apply must NOT be used.
  • --apply — actually rewrites (POST-ratification only).

Detection is runtime-derived (FR source label → Counter of translations, winner = majority) — no hardcoded variant strings, stays correct as the CSVs evolve. An EXCEPTION TABLE encodes the glossary-register doc's per-case guidance:

  • OVERRIDE — contra-majority recommended value (S6 meaning, S7 number).
  • FLAG — not auto-applied, listed PENDING (S2 scope, S3 regional, V1-V6 RTL/CJK).

Bug found + fixed during writing (SDD rigor)

First dry-run over-reported 1108 Virtues + 955 Scenarii changes. Two bugs:

  1. Scope bug in plan_changes — it compared every row's translation cell to a group's winner regardless of family, so rows from a different family got dragged into another group's harmonization (e.g. Argument-pertinent rows into the Raisonnement-valide winner). Fixed: scope every change to rows sharing the group's FR source label. → numbers dropped to the audit's census.
  2. Exception-key mismatch — Virtues keys used the en_base stem (family) but detection produces the actual CSV column name (family_fr), so V1-V6 never matched → never flagged → majority-applied. Fixed: keys use family_fr / subsubfamily_fr.

Both fixed; output now matches the audit's 14-group census exactly.

Verified against master bef3bc6c (release-frozen)

Check Result
--verify-drift all 3 CSVs byte-identical on zero-change round-trip ✓
dry-run Virtues 0 auto-apply / 6 FLAG (V1-V6 RTL/CJK, native-required)
dry-run Scenarii 39 auto-apply (PT overrides) / 2 FLAG (S2 scope, S3 regional)
dry-run Fallacies 0 / 0 (consistent, re-verified)
--apply (scratchpad copy) writes exactly the 39 dry-run cells (34 lines; 5 rows carry 2 cell changes each), S2/S3 untouched ✓

The 39 (vs the audit's majority-based 38) is correct: S6/S7 overrides are contra-majority (the doc recommends relações no trabalho + Religiões), so they flip the majority side → more cells than the audit's minority-outlier count.

Post-ratification = 1 command

python docs/taxonomy/192-terminology-apply.py --apply
python docs/taxonomy/192-terminology-apply.py --verify-drift   # confirm dialect intact

That converts the WE-27/06 ratification gate into instant, audited execution.

Release safety

  • Base = master bef3bc6c (release-frozen). No Cards/ write.
  • No C# / pipeline change. No CSV modified before CardPen injection.
  • Docs/tooling only: 192-terminology-apply.py (288 lines) + 192-terminology-apply-dryrun.txt (the dry-run output, committed for review).

🤖 Worker po-2024 · dispatch v3 primaire · release-safe (0 write Cards/, master bef3bc6c)

…drift-free)

Counterpart to 192-terminology-audit.py. Applies the ratified #192
harmonization (Decision A register + Decision B lexical picks) as a
drift-free CSV edit (QUOTE_MINIMAL + CRLF, same dialect as #595), gated
behind ratification: default DRY-RUN (0 write under Cards/), --apply
writes only POST-ratification, --verify-drift proves the writer dialect
is byte-identical on a zero-change round-trip (run FIRST on any machine).

Detection mirrors the audit at runtime (FR source label -> Counter of
translations, winner = majority) — no hardcoded variant strings, stays
correct as the CSVs evolve. An EXCEPTION TABLE encodes the
glossary-register doc's per-case guidance:
  - OVERRIDE : apply a contra-majority recommended value (S6 meaning,
               S7 number — the doc argues these beat the majority)
  - FLAG     : do NOT auto-apply; list as PENDING. Covers S2 scope,
               S3 regional (jsboige judgment) and V1-V6 RTL/CJK near-ties
               (native-required, per the doc's LOW confidence).
Decision A (sentence-case) is not reimplemented as a caser (casing
Portuguese proper nouns like "Idade Media" is error-prone); the cap-only
cases resolve by picking the winner variant AS-IS, so the ratified register
surfaces through the chosen variant string, not a transform.

plan_changes scopes every change to rows sharing the group's FR source
label, so cross-family rows are never dragged into another group's winner
(the bug class that initially over-reported 1108+955 cells; fixed ->
matches the audit's 14-group census exactly).

Verified 2026-06-26 against master bef3bc6 (release-frozen, 0 write Cards/):
- --verify-drift : all 3 CSVs byte-identical on zero-change round-trip
- dry-run       : Virtues 0 auto-apply / 6 FLAG (V1-V6 RTL/CJK
                   native-required); Scenarii 39 auto-apply (PT overrides) /
                   2 FLAG (S2 scope, S3 regional); Fallacies 0/0 — matches
                   the audit (14 groups: 6 Virtues + 8 Scenarii)
- --apply       : proven end-to-end on a scratchpad copy — writes exactly
                   the 39 dry-run cells (34 lines; 5 rows carry 2 cell
                   changes each), S2/S3 flagged groups untouched

Post-ratification (WE 27/06) = 1 command:
    python docs/taxonomy/192-terminology-apply.py --apply
then re-run --verify-drift to confirm the dialect is intact. Converts the
ratification gate into instant execution.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

@clusterManager-Myia clusterManager-Myia left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[NanoClaw]

Solide — j'ai lu le code (288 lignes) et vérifié les claims de sécurité contre le code + l'output committé, pas contre la prose du PR. Les deux bugs « trouvés et fixés pendant l'écriture » sont réels et leurs correctifs sont présents dans le code. Tooling de qualité (SDD honnête).

Safety model vérifié contre le code (pas l'assertion) :

  • Dry-run = défaut réel. main() : mode = "dry-run" puis --apply/--verify-drift ne l'overrident qu'avec flag explicite. Et le chemin d'écriture (rewrite_csv + shutil.move) n'est atteint que dans la branche mode == "apply" — le dry-run ne construit même pas l'apply_map. 0 write sous Cards/ en dry-run = confirmé par le flux de code. Corroboré empiriquement : le 192-terminology-apply-dryrun.txt committé affiche Mode: DRY-RUN.
  • Dialect writer correct pour le round-trip drift-free : csv.writer(quoting=QUOTE_MINIMAL, quotechar='"', lineterminator="\r\n") + encoding="utf-8" (no-BOM), newline="". Lecture en utf-8-sig (strip BOM si présent). C'est la config standard de round-trip — cohérent avec la méthode #595 (merged).
  • --verify-drift = self-test réel, non cosmétique. Il réécrit chaque CSV avec changes_by_rowcol={} (zéro changement) et asserte orig == new byte-à-byte, exit 1 si drift. Le script impose cette vérification avant --apply (lui-même l'écrit). Bonne défense en profondeur.

Les 2 bugs fixés sont bien dans le code (j'ai vérifié) :

  1. Scope bugplan_changes scope désormais par row_fr == g["fr"] (if row_fr != g["fr"]: continue) → un groupe ne harmonise que les lignes partageant son label FR source. Le bug décrit (lignes d'une autre famille traînées dans le winner d'un autre groupe) est effectivement impossible avec ce garde-fou. ✓
  2. Exception-key mismatch — les clés EXCEPTIONS Virtues utilisent bien family_fr / subsubfamily_fr (le vrai nom de colonne CSV que detect_groups met dans g["fr_field"]), pas le stem en_base. V1-V6 matchent donc effectivement. ✓

Nombres cross-checkés contre l'output committé (pas la prose) : dryrun.txt = Virtues 6 groupes/0 change/6 pending, Fallacies 0/0/0, Scenarii 8/39/2, SUMMARY: 39 would-change, 8 pending. Identique au corps du PR. La logique « 39 vs 38 » (contra-majority S6/S7 flippent le côté winner → majorité devient aussi un changement) est correcte et honnêtement flaguée.

Détection runtime-derived (aucune variante hardcodée dans detect_groups — le winner est max(cnt) calculé sur les lignes réelles). Seuls les guidances par-cas (valeurs OVERRIDE + labels FR en clés) sont hardcodés — ce qui est légitime. Reste correct quand les CSV évoluent.

Concerns honnêtes (rien ne bloque, mais c'est ce que je questionne) :

  • Drift-free = author-reported pour moi. python3 n'est pas dans mon conteneur → je n'ai pas pu lancer --verify-drift moi-même pour confirmer le « byte-identical sur les 3 CSV ». Le gate est correctement implémenté (lu dans le code) et obligatoire par design (run first, exit 1 si drift), donc tout runner futur de --apply est protégé — c'est l'essentiel. Précédent #595 (même méthode, merged). Recommandation : quiconque lance --apply post-ratification (WE 27/06) doit d'abord voir --verify-drift sortir All CSVs byte-identical sur SA machine — ne pas faire confiance au run de po-2024 si le CSV ou l'env a changé d'ici là.
  • Le matching d'exception ignore la dimension dataset (k[0] = dataset non comparé, boucle sur k[1..3]). Pas de bug aujourd'hui car les fr_field sont spécifiques par dataset (family_frFamillecatégorie), mais fragilité latente : deux datasets partageant un nom de fr_field + même label FR se cross-matcheraient. Mineur.
  • Tiebreak lexicographique non documenté (max(cnt, key=lambda k:(cnt[k], k)) → en cas d'égalité exacte, prend la variante alphabétiquement première). Déterministe mais arbitraire. Ne mord pas sur V1-V6 (FLAG'd), donc impact nul sur ce passage, mais l'auteur devrait le savoir.

Release freeze respecté (base bef3bc6c, 0 fichier Cards//.cs, 0 CSV modifié — l'injection CardPen est postérieure). 0 secret. Rien qui bloque l'admission du tooling ; les 3 concerns sont des notes de robustesse, le reviewer humain garde la décision de merge.

@jsboige
jsboige merged commit f93ba68 into master Jun 28, 2026
3 checks passed
@jsboige
jsboige deleted the tooling/192-terminology-apply-script branch June 28, 2026 12:34
jsboige added a commit that referenced this pull request Jun 28, 2026
… PT cells) (#607)

Apply the jsboige-ratified #192 defaults reco (WE 28/06, "Decision A +
majority; RTL/CJK + BR/PT deferred to native") to the Scenarii PT taxonomy
labels — the chemin-critique unblock for the v0.9.0 tag.

39 cells harmonized across 1 file (Cards/Scenarii), all OVERRIDE-class
(S1/S4/S5/S8 sentence-case via Decision A; S6/S7 reco overrides that beat
the raw majority — S6 "relações no trabalho" interpersonal (not legal),
S7 "Religiões" plural matches FR source). 8 groups correctly left PENDING
(V1-V6 RTL/CJK near-ties + S2 scope + S3 BR/PT regional — deferred to
native/jsboige, non-blocking).

Method: 192-terminology-apply.py --apply, drift-free (QUOTE_MINIMAL + CRLF,
UTF-8 no-BOM). Verified:
  - pre-apply  --verify-drift : 3/3 CSVs byte-identical (dialect safe)
  - post-apply --verify-drift : 3/3 byte-identical (no parasitic reformat)
  - post-apply dry-run        : 0 would-change residual (39 applied clean)

Scope precision (lesson 192-apply-script-post-ratification): groups scoped
by FR source label — no cross-family row leak (winner never drags rows from
a different FR label). Fallacies = 0 groups (unchanged). Virtues = 6 PENDING
(RTL/CJK, unchanged). Only Scenarii PT touched.

Relates to #598 (ratified reco) + #602 (apply script). Closes #192 passes
2-4 for the ratified subset; RTL/CJK + BR/PT residue stays deferred until
native review (documented, non-blocking v0.9.0).

Co-authored-by: Your <your.email@example.com>
Co-authored-by: Claude-Code <noreply@anthropic.com>
jsboige added a commit that referenced this pull request Jun 29, 2026
…607) (#608)

Turnkey ✅-checklist (format #598) for the 8 multi-variant groups
correctly left PENDING by #607 — exactly jsboige's ratification
"RTL/CJK + BR/PT deferred to native". Ratifiable in ~10 min by a
native speaker / jsboige, POST-release (non-blocking v0.9.0 — the
tag ships with the majority variant on each).

8 groups, re-verified on master 44c37fa via the applier dry-run
(0 would-change, 8 pending) + audit script variant counts:
  - Virtues V1-V6 (RTL/CJK near-ties, fa/ar/zh/ru): native-required,
    gpt-5.5 assist optional (/v1/responses + reasoning.effort=low)
    but LLM pick must NOT be ratified as authoritative
  - Scenarii PT S2 (contes scope: tales-only vs tales+literature)
  - Scenarii PT S3 (regional: BR 'Paquera' vs PT-PT 'engate')

Each row: 1-line context + options + recommended default + confidence.
Factual rationale (dialect/register facts, not opinion) pre-digests the
PT decisions; RTL/CJK stay native-required. Includes the post-ratification
apply path (EXCEPTIONS table FLAG→OVERRIDE + --verify-drift + --apply,
drift-free method #595) so po-2024 can close #192 fully once ratified.

Scope: docs only. 0 write under Cards/. master stays 44c37fa. Relates
to #598 (glossary proposal) + #602 (apply script) + #607 (applied the 6
ratifiable Scenarii PT groups). Closes the deferred-residue pass of #192.

Co-authored-by: Your <your.email@example.com>
Co-authored-by: Claude-Code <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants