tools(dnn-i18n): #457 content-type extractor + reimport dry-run (fixture, Option C tooling) - #524
Merged
Merged
Conversation
…ure, Option C tooling) Bricks 1 + 3 of the #457 dispatch (ai-01 msg-…141408). Brick 2 (DatasetUpdater Option C config, gpt-5.5 ui.*/res.*) was ALREADY merged in #487 (Enabled=false) — not rebuilt; this PR adds the missing tooling AROUND it. Brick 1 — extract_dnn_ui_strings.py: reusable content-type -> CSV extractor that codifies the manual PHASE1 audit. ui.* = anchor-based (fails loud if a hardcoded string disappears, never invents rows); res.* = @Resources.* references with DB-only fr flagged honestly. Regex @?Resources. captures both directive (@Resources.X) and nested-expression (@Html.Raw(Resources.X)) forms — recovers RuleMemoInstructions. Brick 3 — reimport_dnn_ui_strings.py: `verify` (key-set HARD diff vs reference) + `reimport` (render canonical payload to stdout, NEVER writes). res.* fr empty-in-extract by design (DB-only values); KEY-SET match is the hard contract. DoD: test_roundtrip.py proves extract -> verify -> reimport on a committed fixture, zero prod mutation (PASS, exit 0). Cross-validated vs REAL prod templates: 10/10 keys match docs/dnn-localization/dnn-ui-strings.csv. Anti-fab: anchor-based ui.* (negative test proven exit 2), no free-text fabrication, honest DB-only flagging. Gate-safe: pure tooling, no prod/DB/CSV/config mutation, does NOT touch dnn-ui-strings.csv (po-2024 lane #490). Stdlib Python only. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Collaborator
|
[NanoClaw] Clean i18n tooling (bricks 1+3) for DNN localization: anchor-based ui.* extractor (anti-fab: fails loud if anchor disappears, never invents rows), res.* reference scanner (DB-only values honestly flagged), CSV reimport dry-run verifier, and a round-trip test with fixture. Stlib-only Python, cross-validated against prod templates (10/10 key match). Gate boundaries explicit: no DB touch, no live reimport, no CSV modification, no DatasetUpdater config change. Subprocess only used to run its own test script. Good README with anti-fabrication guarantees. LGTM. |
jsboige
added a commit
that referenced
this pull request
Jun 17, 2026
…tractor) (#525) Closes the "inventory content-types" idle task (ai-01 msg-…141408). Extends PHASE1-content-audit.md (#461) with a complete localization scope map and feeds the extractor (PR #524, merged). 5 content-types mapped (Argumentum-app scope only; DNN framework out-of-scope): - A custom-template hardcoded UI strings (2 keys) — extractor-done - B @Resources.* keys (8, values DB-only) — extractor-done (keys + DB-only flag) - C 2 static HTML content pages (fallacies.html ~24K chars + MariagePourTous ~13K, standalone FR prose, NOT CSV-driven) — repo-extractable gap, recommend separate document-tier translation lane (not the ui.*/res.* string task) - D 2sxc content items (glossary/FAQ/homepage/App resource VALUES) — DB-only, the bulk - E DNN page settings — DB-only. Both D+E need portal export (jsboige, gated). Findings: 26 2sxc apps installed but only Argumentum is custom (25 stock). The live « Materiel » coquille is NOT in any repo template (negative grep across Argumentum scope) → it's DB-only (content-type D), consistent with the "canonical FR lives in SQL" pattern. No prod mutation, no DB/RDP, no dnn-ui-strings.csv touch. Pure scope-map doc. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
jsboige
added a commit
that referenced
this pull request
Jun 20, 2026
docs(dnn): #457 document-tier translation workflow (Phase 2-4 design) The Phase 1 inventory (457-site-content-type-inventory.md) named WHAT to translate but deferred the HOW for content-type C (the 2 standalone FR HTML pages, ~37k chars of prose). The string-tier toolchain (tools/dnn_i18n/, PRs #524/#487) handles ui.*/res.* keys but does not fit prose. This doc designs the document-tier lane: extraction -> DatasetUpdater -> re-import. Scope: DNNPlatform/fallacies/fallacies.html (81 KB, structured charte) + MariagePourTous.html (38 KB, prose). Design: - Phase 2 CSV schema: segment-oriented (segment_id = file#element-index), source_path + selector for deterministic re-import, inner-text fields per language. NOT the key->value string-tier dialect. - Phase 2 DatasetUpdater task shape: document-tier variant vs string-tier (prose prompt, char-budget chunking, gpt-5.5, reasoning effort low, max_output_tokens >=7000, Enabled=false until GO). - Phase 1.5 extractor: the net-new gating tool (html.parser stdlib, file-specific selectors h2/h3/li vs p/h2/h3, inner-text extract + selector store for re-wrap). Nothing translates until this exists. - Phase 4 re-import: selector-based patch, dry-run verifier (same DoD as string-tier round-trip), localized HTML files per language. - RTL/CJK (ar/fa/zh): reuse PR #359 font stacks at page level (dir=rtl/lang), template variant decision for jsboige. - Sequencing: extractor is critical path. - 4 open questions for jsboige (inner-text vs outer-HTML, one-template vs per-lang, are pages still served, GO to build extractor). Index README #554 updated to reference the new doc in supporting-docs table. Pure design/references -- no prod mutation, no extraction executed, no DatasetUpdater enabled. @ Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
#457 DNN i18n tooling — bricks 1 + 3 (content-type extractor + re-import dry-run verifier) + round-trip DoD test. Dispatched by ai-01 (
msg-…141408, non-gated #457 tooling while #131 exec waits on jsboige's target decision).Key finding that shaped the work
Brick 2 (DatasetUpdater Option C config) was ALREADY merged in #487 (
ca9a8640) — entityDnnUiString,KnownDataSets.DnnUiStrings, prompts, task config (DatasetUpdaterRootConfig.cs:2636-2696,Enabled=false, gpt-5.5). The investigation (SDDD triple-grounding) confirmed the config-only gpt-5.5 path is canonical. So this PR adds the missing tooling around it rather than rebuilding the config:Brick 1 —
extract_dnn_ui_strings.pyParses the Argumentum 2sxc
.cshtmltemplates, emits thednn-ui-strings.csvdialect (key,context,source_file,fr,en,ru,pt,es,ar,fa,zh,notes):ui.*= anchor-based (NOT free-text): each entry declares a verbatim anchor that must exist in the named source. Fails loud (exit 2) if a refactor removes a string — never silently drops a row, never fabricates.res.*=@?Resources.<Key>regex capturing both@Resources.X(directive) and@Html.Raw(Resources.X)(nested expression). The nested form recoversRuleMemoInstructions(the one a naïve@Resources.regex misses).frleft empty + flaggedDB-only(canonical values live in SQL/2sxc App Resources — pattern appris).Brick 3 —
reimport_dnn_ui_strings.pyverify— key-set diff vs reference. KEY-SET match is HARD (exit 1 on divergence).res.*fr empty-in-extract is by-design WARNING.reimport— renders the canonical DNN/2sxc payload to stdout only (never writes — live apply is DB/RDP-gated, jsboige).DoD proof —
test_roundtrip.pyAnti-fabrication (proven, not asserted)
ui.*anchor → exit 2 (verified, not piped).DNNPlatform/Portals/1/2sxc/Argumentum/yields 10/10 keys matchingdocs/dnn-localization/dnn-ui-strings.csv. Only delta =res.*fr (empty-in-extract by design — DB-only).Gate boundaries (HARD)
dnn-ui-strings.csv(po-2024 lane, feat(dnn-i18n): #457 translate DNN UI strings FR -> 7 langs via gpt-5.5 #490).Enabled=false).🤖 Generated with Claude Code