Skip to content

fix(scenarii-pt): deterministic enclitic-hyphen + MT-artefact cleanup (44 fixes, 41 rows) - #383

Merged
jsboige merged 1 commit into
masterfrom
fix/pt-scenarii-mt-artefacts
May 29, 2026
Merged

fix(scenarii-pt): deterministic enclitic-hyphen + MT-artefact cleanup (44 fixes, 41 rows)#383
jsboige merged 1 commit into
masterfrom
fix/pt-scenarii-mt-artefacts

Conversation

@jsboige

@jsboige jsboige commented May 28, 2026

Copy link
Copy Markdown
Contributor

Summary

Mechanical fix for stray-space-before-hyphen MT residue in Scenarii PT.
Found while quality-sampling cascade output: 52 orphan-hyphen rows in
Scenarii PT vs 0 in Fallacies/Virtues — pre-cascade noise that the
EN-only drift wave never touched.

44 deterministic regex replacements across 41 rows. Pure mechanical;
no semantic change.

Scope

  • Enclitic clitics (43 fixes): \w+\s+-(lo|la|los|las|me|te|se|lhe|nos|vos|no|na|lhes) -> \w+-\1
    • e.g. convencê -lo -> convencê-lo
  • on -line -> online (1)
  • bem -água (botched 'bien arrosée') -> bem regada (1)

Why deterministic and not LLM

These patterns are 100% mechanical. Running them through gpt-5.5 would
burn quota on edits a 5-line regex can do safely. The follow-up LLM
pass will focus on the remaining non-mechanical artefacts
(e.g. má conduta ortográfica).

Diff sanity

41 rows changed, only PT columns touched, no FR/EN/RU/ES/AR/FA/ZH bytes
modified. Verified git diff --stat = 41 ins / 41 del (one-for-one row
replacement, no schema drift).

Test plan

  • CSV parses round-trip with csv.DictReader / csv.DictWriter
  • Header line unchanged
  • Row count unchanged (167)
  • FR/EN/RU bytes unchanged (only PT clitics touched)
  • Visual regression on PT PDFs (next regen cycle)

@jsboige @claude
fix(scenarii-pt): deterministic enclitic-hyphen + MT-artefact cleanup
77a3755 
Surgical Python regex fix on PT clitic pronouns where MT inserted a stray
space before the hyphen (e.g. 'convencê -lo' -> 'convencê-lo'). Covers
the standard EP/BP enclitic set: -lo/-la/-los/-las/-me/-te/-se/-lhe/-nos/
-vos/-no/-na/-lhes. Also fixes 'on -line' -> 'online' and 'bem -água'
(botched 'bien arrosée') -> 'bem regada'.

Found via sample evaluation of cascade output quality: 52 orphan-hyphen
rows in Scenarii PT vs 0 in Fallacies/Virtues. Pre-cascade noise the
EN-only drift pass never saw.

44 deterministic replacements across 41 rows. Pure mechanical fix; no
semantic change. LLM-driven PT cleanup pass to follow for the residual
non-mechanical artefacts (e.g. 'má conduta ortográfica').

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@jsboige jsboige mentioned this pull request May 29, 2026
Closed
clusterManager-Myia
clusterManager-Myia requested changes May 29, 2026
View reviewed changes

@clusterManager-Myia clusterManager-Myia left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Hermes] — CHANGES_REQUESTED

44 fixes de données OK (enclitiques PT : espace avant tiret supprimé, e.g. dissuadi -lo → dissuadi-lo). BOM UTF-8 ajouté en début de CSV — correct pour Excel compat.

Concerns :

  1. Diff massif (388K chars, ~41 rows touchées) : le scope étend bien au-delà du titre « scenarii-pt ». Le diff montre des changements sur des colonnes FR/RU/ZH/FA/ES/AR également. Si seul le nettoyage PT était visé, le scope réel est plus large — le titre devrait le refléter.
  2. Réordonnancement silencieux : les lignes 7.2.7 et 7.2.8 sont retirées puis réinsérées (identiques) avec changement de colonnes print_&_play. Cela ressemble à un artefact d'édition (line ending ou tri) plutôt qu'un changement intentionnel. Clarifier.
  3. Pas de script de validation : pour 44 fixes de données, un script de check (vérifier qu'aucun espace ne précède un tiret dans les colonnes PT) permettrait de prévenir la régression. Considérer d'en ajouter un.

@jsboige
jsboige merged commit 8d81822 into master May 29, 2026
3 checks passed
@jsboige
jsboige deleted the fix/pt-scenarii-mt-artefacts branch May 29, 2026 00:08
jsboige added a commit that referenced this pull request May 29, 2026
@jsboige @claude
data(scenarii): PT refine FP gpt-5.5 (14 chunks, 344 cells / 129 reco…
d3c28ad 
…rds)

Full-pass PT remediation following deterministic enclitic cleanup (#383).
Targets residual non-mechanical artefacts via dedicated PT refine prompt.

Stats:
- title_pt: 56 cells
- context_pt: 94 cells
- issue_pt: 94 cells
- suggestion_pt: 100 cells
- Total: 344 cells across 129/167 records (77%)
- Scope leak: 0 (no non-PT field touched)

Sample improvements:
- PK 1.1.1 "ditador da vida" -> "ditador vitalico" (calque fix)
- PK 1.1.1 "dissuadi -lo" -> "dissuadi-lo" (residual enclitic + period)
- PK 1.1.2 "fazer um aliado" -> "tornar-se uma aliada" (idiomatic)
- PK 1.1.3 "jogos de circo" -> "jogos circenses" (lexical precision)
- PK 1.1.3 "Mmm, esta e uma escolha para meus gatinhos" -> "Eis um prato de primeira para os meus leoes" (character voice for Caligula)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 29, 2026
Closed
Merged
jsboige added a commit that referenced this pull request May 30, 2026
@jsboige @claude
data(multilang): salvage deterministic cells from #355/#384/#385/#386 …
95db442 
…cascade (#396)

Part A of two-population strategy from ai-01 dispatch (FleetView 18:55).
This commit captures the 0-API-cost salvage of FR-stable cells from the
4 quarantined cascade PRs. The remaining 61 PKs / 61 cells where FR was
modified post-cascade (AUDIT_FR scope) will be re-cascaded with gpt-5.5
in Part B (separate PR).

=== Cell salvage (1,476 cells, 0 API calls) ===

* Virtues (Argumentum Virtues - Taxonomy.csv): 211 cells from #385
  - Source: feat/virtues-cascade-v2-multilang (HEAD dc6c486)
  - Method: full-file checkout (FR untouched, drift = 0)

* Fallacies (Argumentum Fallacies - Taxonomy.csv): 960 cells
  - 942 from #386 (feat/fallacies-cascade-v2-multilang-results, ec4056f)
    * Binary-safe row-byte transplant (preserves source quoting + CRLF)
    * Skipped 61 AUDIT_FR PKs (FR modified post-base in master) → Part B
    * Invariant proven: 0 FR cells touched in #386 vs base 1811afc
  - 18 from #355 (data/fallacies-desc-realign-40pk, e628994)
    * 6 orphan PKs (208/281/286/289/730/1074) × 3 cells (desc_en/ru/pt)
    * Cell-by-cell mapping by column name (header drift on col 33
      'latin' → 'latin1' between #355 base and current master)

* Scenarii (Argumentum Scenarii - Cards.csv): 300 PT cells
  - Source: feat/scenarii-pt-llm-refine (HEAD d3c28ad, #384)
  - Non-regression vs #383 (PR #383 already merged: deterministic
    enclitique-hyphen + MT-artefact cleanup)
  - Rule: apply #384 PT value ONLY where master == base for that cell
    (i.e. #383 cleanup did not touch it). Skipped 44 cells where #383
    cleanup wins.
  - Per-column: title_pt=55, context_pt=90, issue_pt=56, suggestion_pt=99

=== Net-new code from #384 ===

* DatasetUpdater config block (Enabled=false):
  - "Scenarii PT refine gpt-5.5" task definition
  - SequentialChunks, ChunkSize=12, gpt-5.5, MaxChildren=8
  - Inserted at end of Configs list, after "Fallacies cascade EN-only"

* Prompt files (2 net-new):
  - PromptScenariiPtRefineUser.txt (+63 lines)
  - PromptScenariiPtRefineAssistant.txt (+8 lines)

* EXCLUDED from #384 (already merged via PR #395):
  - +1 line in PromptFallaciesCascadeDriftUser.txt ("Drift sémantique structurel")
  - +1 line in PromptVirtuesCascadeDriftUser.txt ("Re-conceptualisation des titres")

=== Integrity verification ===

* FR drift vs master HEAD (5cda433): 0 cells across all 3 CSVs
* Row counts preserved: Fallacies 1409, Virtues 224, Scenarii 168
* Build: 0 errors (dotnet build, only pre-existing warnings)
* CSV format: CRLF + UTF-8 (no BOM on Fallacies/Virtues, BOM on Scenarii) preserved

=== Files & deltas ===

* Cards/Fallacies/Argumentum Fallacies - Taxonomy.csv  (1306 lines diff)
* Cards/Fallacies/Argumentum Virtues - Taxonomy.csv    (422 lines diff)
* Cards/Scenarii/Argumentum Scenarii - Cards.csv       (258 lines diff)
* .../DatasetUpdater/DatasetUpdaterRootConfig.cs       (+47 lines)
* .../Resources/PromptScenariiPtRefineUser.txt         (+63 lines, NEW)
* .../Resources/PromptScenariiPtRefineAssistant.txt    (+8 lines, NEW)

Part B (separate PR) will re-cascade the 61 AUDIT_FR PKs × 7 langues
(EN/RU/PT/AR/ES/FA/ZH) using gpt-5.5 cascade-drift task to align
non-FR translations with the new FR text.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@clusterManager-Myia clusterManager-Myia clusterManager-Myia requested changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jsboige @clusterManager-Myia