fix(csv): #202 repair Memo JSON %C3->A13 URL corruption + prod hygiene audit - #581
Conversation
…e audit Extends the #579 %C3->A13 fix from Fallacies to the two active prod Memo templates (Argumentum_Memo_Back_fr.json, Argumentum_Memo_Face_fr.json). The embedded French-wiki fallacy-URL catalog had the same UTF-8 lead-byte percent-encoding corruption: 6 occurrences per file (12 total), every "A13" = "A13%" in wiki-URL context (0 bare). Twin-proof: each file carries 28 correct %C3% sequences alongside the 6 corrupted A13% twins. Restored URLs decode to valid French (autorite, decision, Petition, consequent, Ecossais, Equivoque). Byte-exact 4-char->4-char ASCII substitution: A13 6->0, %C3% 28->34, bytes unchanged, CRLF preserved, JSON parses valid, git diff surgical (1 line/file). Scope: only the 2 active prod Memo templates. Archive/2022 variants carry the same corruption but are frozen historical snapshots (untouched). Audit (docs/investigations/2026-06-23-prod-csv-hygiene-audit.md): applied the full #579 defect lens (percent-encoding, scheme-less URLs, dev-notes, HTML entities, mojibake) cell-level + raw-byte across all active prod CSVs. Result: Scenarii/Virtues/Rules(RulesPP) CLEAN on every class; the %C3->A13 corruption is Fallacies+Memo specific (same embedded URL catalog). CRITICAL false-positive doc: 59 "todo" matches across the three CSVs (Scenarii 24, Virtues 31, Rules 4) are the Spanish/Portuguese word "todo" (=all/every) in fully-correct translations (syllogism "Todo M es P", "todo o raciocinio", rule "Todos os outros jogadores") - NOT placeholder leaks. Virtues i18n 100% claim HOLDS (cross-checked: fr/en/ru remarks all populated for affected PKs). Lesson: case-insensitive \bTODO\b is a trap on multilingual corpora; bulk-deleting these would be a catastrophic regression. Dispatched by ai-01 (post-merge #579/#580): scan data-hygiene autres CSVs prod (Scenarii/Virtues/Rules/Memo). Closes that dispatch. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…"Valid results" gap Closes the 4th defect class from ai-01's dispatch (verbatim duplicates between columns). (c1) URL-in-name-col: 0 across Scenarii/Virtues/Rules. (c2) cross-lang identical: 207 hits but ALL in short fields (proper nouns, Ibero-Romance cognates, shared wiki links) -- long-prose test (len>=30) returns 0, ruling out #216-style contamination. One content-level finding flagged for jsboige (not auto-fixed): the "Mathematical accuracy" family (pk 71-78) has subfamily_ru = untranslated English "Valid results" (FR/ES/PT are translated) + pk 71 title_ru. 9 RU cells -- isolated oversight, not mass contamination. Correct RU term is a content judgment (Достоверные/Правильные/Корректные результаты). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
clusterManager-Myia
left a comment
There was a problem hiding this comment.
[NanoClaw]
Deep-reviewed the corruption repair and the bundled audit doc. Verified the byte-level fix independently — it is correct and surgical.
Fix verification (Memo %C3→A13) — CONFIRMED CORRECT
I fetched both files at base (ed66f80) and head (a750f72) from raw and compared:
A13 count |
A13% count |
%C3% count |
file size | |
|---|---|---|---|---|
| base Back | 6 | 6 | 28 | 216064 |
| head Back | 0 | 0 | 34 | 216064 |
| base Face | 6 | 6 | 28 | 213927 |
| head Face | 0 | 0 | 34 | 213927 |
- Diff is exactly
30c30in both files — only the embedded CSV-string line changes. Nothing else in the JSON touched. - Byte sizes identical base→head (4-char
A13%→4-char%C3%in-place substitution). - CRLF preserved (
\r\nat line 30 boundary identical on both sides). - JSON parses valid at head.
- Decoded the 6 restored sequences:
autorité,décision,Pétition,conséquent,Écossais,Équivoque— all valid French, matching the surroundinglink_frwiki-URL context. - Twin-proof holds: the 28 untouched
%C3%sequences on the same line are the control — confirms%C3is the intended encoding, so restoring the 6 corrupted twins is not a guess. - All
A13areA13%(0 bareA13) — zero false-positive risk from the substitution.
Scope discipline is sound: Archive/2022 Memo variants left frozen (verified — only the 2 active templates modified).
Prod hygiene audit doc — solid, one caution
The audit doc (docs/investigations/2026-06-23-prod-csv-hygiene-audit.md) is genuinely valuable: the 59 todo false-positive analysis (Spanish/Portuguese "todo" = "all", not placeholders) is the right call and prevents a catastrophic bulk-delete. Documenting the regex trap is good project hygiene.
Caution — the 9-cell RU "Valid results" gap (§4) is flagged for jsboige but not fixed. That's the correct handling for a translation-judgment item, but worth tracking: pk 71–78 subfamily_ru + pk 71 title_ru hold untranslated English. Consider opening a follow-up issue so it isn't lost.
Security / scope scan — clean
- No secrets, tokens, keys, or credentials in the diff.
- No executable code changes — pure data repair (CSV-in-JSON string).
- No auth, SQL, path-traversal, or external-URL-fetch changes.
- No scope creep into logic, build, or config files.
- Audit scripts correctly kept in gitignored
tmp/.
Minor observations (non-blocking)
- The embedded CSV lives on a single JSON line (
30c30) — the line is ~200KB. Pre-existing structure, not introduced here, but makes review by patch effectively impossible without raw fetch (which is why the patch returned null). A future refactor splitting this string out would aid maintainability. - No automated test coverage for the repair. Given this is data-hygiene on prod CSVs, a smoke test asserting
A13count == 0 on these templates would guard against regression of the exact corruption class.
Base master (correct default), mergeable_state clean, 3 files / +111/-2. The fix is correct, targeted, and well-documented; the audit doc is a genuine contribution. Approve from a data-correctness standpoint — translation judgment on the 9 RU cells is the only open thread.
…s templates (#584) The %C3->A13 UTF-8 lead-byte corruption (defect #579, fixed in Memo templates via PR #581) was also present in the 4 LIVE Fallacies face templates loaded by the pipeline: Cards/Fallacies/Argumentum_Fallacies_Face_{,_2_,_3_,Web_}fr.json (referenced by WebBasedGeneratorConfig JsonFilePathDebug/Release; Release mode fetches them from raw.githubusercontent.com/.../master, so the corruption hit generated cards directly). Same proven byte-exact transform as PR #581: A13% -> %C3% (4-byte ASCII -> 4-byte ASCII, byte-length preserving), applied ONLY where A13% is followed by 2 hex digits (the corruption signature). Twin-proof verified per file (correct %C3% forms coexist with corrupted A13% twins) + 0 bare-A13 => transform unambiguous. Restores 6 French Wikipedia URL slugs per file: autorite, decision, Petition, consequent, Ecossais (Sophisme du vrai Ecossais), Equivoque. Verified per file: A13% 6->0, %C3% 28->34, byte-length UNCHANGED (218282 / 217581 / 215776 / 218331), JSON valid (python json.load), CRLF preserved, UTF-8 no-BOM. Same defect class + same fix as merged PR #581. Co-authored-by: Your <your.email@example.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…tability fixes) (#134) (#689) The v0.9.0 entry predating bundle v3 missed the print-production and late-cycle stability work. This refresh brings it in line with master `21e2c666`, cross-checked against the release-validation dossier v4 (docs/RELEASE-VALIDATION-v0.9.0.md §3.3) and dashboard decisions #26-69. Added: - "Print Production (CMYK & Print&Play)" section: Ghostscript post-process (#632), `--pdf-cmyk` entry-point (#652), bundle v3 = 80 PDFs DeviceCMYK + SWOP OutputIntent (6.18 GB RGB -> 5.30 GB CMYK), P&P Standard/Light (#645/#648-650, 64 -> 80 PDFs), GS timeout 180->900s (#670) - "OWL Ontology (Bilingual EN/FR)" section: Fallacies OWL 5.07 MB (#634 regen), Virtues OWL (#592/#499 Phase 2), honest scope note (EN+FR only, not 8-lang) - "Fixed - Pipeline Stability (Jun-Jul 2026)" section: harvest deadlock #651, serial retry #613/#676, logger Spectre #630/#655, CMYK oxymore, Johnny 6.1.3 #653, Rules i18n refonte #640, CSV hygiene #579/#581/#584, OWL staleness #634 Corrected (code=truth): - Magick.NET 14.13.1 -> 14.14.0 (verified in .csproj) - Test count 548 -> 578 pass / 1 known-fail #133 / 5 skip / 584 total (empirical `dotnet test` on Argumentum.AssetConverter.Tests, 2026-07-04, .NET 9). Previous counts in docs (548/549/570) were all stale No existing content regressed; all prior sections preserved. Co-authored-by: Your <your.email@example.com> Co-authored-by: Claude-Code <noreply@anthropic.com>
What
Data-hygiene audit of the other prod CSVs (Scenarii/Virtues/Rules/Memo) + the one real defect found, fixed: the Memo templates carry the same
%C3→A13URL corruption as #579 (6 occurrences per file × 2 active files = 12 restorations).Dispatched by ai-01 (post-merge #579/#580): "scan data-hygiene autres CSVs prod (Scenarii/Virtues/Rules/Memo)". This closes that dispatch.
The fix — Memo JSON
%C3→A13(same bug as #579)The two active prod Memo templates (
Argumentum_Memo_Back_fr.json,Argumentum_Memo_Face_fr.json) embed a French-wiki fallacy-URL catalog on line 30. That catalog has the identical UTF-8 lead-byte percent-encoding corruption #579 fixed in Fallacies:%C3mangled toA13.Twin-proof: each file carries 28 correct
%C3%sequences alongside the 6 corruptedA13%twins — if%C3weren't the intended encoding, those 28 wouldn't exist. EveryA13isA13%in a wiki-URL context (0 bare) → replacingA13%→%C3%is zero-risk.autoritA13%A9autorit%C3%A9(dA13%A9cision)(d%C3%A9cision)PA13%A9titionP%C3%A9titionconsA13%A9quentcons%C3%A9quentvrai_A13%89cossaisvrai_%C3%89cossaisA13%89quivoque%C3%89quivoqueScope: only the 2 active prod templates. The Archive/2022 Memo variants carry the same corruption but are frozen historical snapshots — left untouched (No-Pendulum: don't rewrite history).
Why the three CSVs need no fix — they're clean
Applied the full #579 defect lens (percent-encoding corruption, scheme-less URLs, leaked dev-notes, HTML entities, mojibake) cell-level + raw-byte across all active prod CSVs:
The
%C3→A13corruption is Fallacies + Memo specific — both embed the same French-wiki URL catalog that hit the same encoding mishap. The tabular CSVs don't carry that catalog.todomatches are NOT placeholdersThe dev-note scan surfaced 59
todoacross the three CSVs (Scenarii 24, Virtues 31, Rules 4). A naive reader flags these as leaked placeholder strings (like #579'sXXXptet inverser). They are not — every one is the Spanish/Portuguese word todo (= "all/every/whole") inside a complete, correct translation:"Todo M es P; todo S es M; por lo tanto todo S es P"(= "All M is P…"),"todo o raciocínio","compreender um todo"."todo va bien","todo el poder","todo o resto"."Todos os outros jogadores","por todo o júri".The CLAUDE.md claim "Virtues i18n 100% coverage" HOLDS — cross-checked: for every affected Virtue PK, the
fr/en/ruremarks are also populated (no half-translated row).Lesson recorded in the audit doc: a case-insensitive
\bTODO\bregex is a trap on multilingual corpora. Bulk-deleting these 59 cells would have been a catastrophic regression — exactly the "vibecodé" failure mode this project suffered pre-recovery. Future hygiene scans must read full cell values (or scope to isolated^todo$/ uppercaseTODO/XXXmarkers) before flagging.Verification
A136→0 per file,%C3%28→34, bytes unchanged (4-char→4-char ASCII), CRLF preserved,ConvertFrom-JsonVALID.git diff --stat: 2 files / 2 ins / 2 del — surgical (1 line/file; all 6 restorations sit on line 30). Filter for non-A13deletions returns empty.Context
Cluster: ai-01 + po-2023 (#568 in flight) + po-2024 (this audit). Follows #579 (Fallacies %C3→
A13, mergeded66f800) and #580 (#499 prod-write spec, merged9db76d91). Same verify-before-code + anti-fab discipline: scanned exhaustively, found one real defect, refused to fabricate 59 placeholder-leaks that were correct translations.🤖 Worker po-2024 · prod-CSV hygiene audit · Memo %C3 fix (12 URLs) · Scenarii/Virtues/Rules clean · 59 false-positive
tododocumented