Skip to content

docs(dnn): #457 document-tier translation workflow (Phase 2-4 design) - #557

Merged
jsboige merged 1 commit into
masterfrom
docs/dnn-457-document-tier-workflow
Jun 20, 2026
Merged

docs(dnn): #457 document-tier translation workflow (Phase 2-4 design)#557
jsboige merged 1 commit into
masterfrom
docs/dnn-457-document-tier-workflow

Conversation

@jsboige

@jsboige jsboige commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

The Phase 1 inventory named WHAT to translate but deferred the HOW for content-type C (the 2 standalone FR HTML pages). The string-tier toolchain (tools/dnn_i18n/, PRs #524/#487) handles ui.*/res.* keys but does not fit prose. This doc designs the document-tier lane: extraction → DatasetUpdater → re-import.

What this covers

  • §1 scope: 2 standalone FR HTML pages (~37k chars prose): fallacies.html (81 KB structured charte) + MariagePourTous.html (38 KB prose).
  • §2 CSV schema: segment-oriented (segment_id = file#element-index), source_path+selector for deterministic re-import, inner-text fields per language. NOT the key→value string-tier dialect.
  • §3 DatasetUpdater task shape: document-tier variant vs string-tier (prose prompt, char-budget chunking, gpt-5.5, reasoning effort low, max_output_tokens ≥7000, Enabled=false until GO).
  • §4 extractor (the critical path): the net-new gating tool — html.parser stdlib (zero-dependency, matches existing tool discipline), file-specific selectors (h2/h3/li for the charte, p/h2/h3 for prose), inner-text extract + selector store for re-wrap. Nothing translates until this exists.
  • §5 re-import: selector-based patch, dry-run verifier (same DoD as string-tier round-trip), localized HTML files per language.
  • §6 RTL/CJK: reuse PR feat(i18n): add AR/ES/FA/ZH for 8-language pipeline #359 font stacks at page level (dir=rtl/lang), template-variant decision for jsboige.
  • §7 sequencing: extractor = critical path.
  • §8 open questions for jsboige (inner-text vs outer-HTML, one-template vs per-lang, are pages still served, GO to build extractor).

Why not duplicate

  • The inventory (457-site-content-type-inventory.md) = the WHAT. This = the HOW for document-tier specifically.
  • The string-tier toolchain (tools/dnn_i18n/) handles keys, not prose — a different extraction + task + re-import shape. Lumping both would mis-spec both.
  • No extractor is built here (design only); no DatasetUpdater task enabled; no prod mutation.

Index README (#554) updated to reference the new doc.

Dispatch: ai-01 msg-20260619T231008-8jl3ic TERTIAIRE #457 (site loc, doc-first).

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

docs(dnn): #457 document-tier translation workflow (Phase 2-4 design)

The Phase 1 inventory (457-site-content-type-inventory.md) named WHAT to
translate but deferred the HOW for content-type C (the 2 standalone FR
HTML pages, ~37k chars of prose). The string-tier toolchain
(tools/dnn_i18n/, PRs #524/#487) handles ui.*/res.* keys but does not
fit prose. This doc designs the document-tier lane: extraction ->
DatasetUpdater -> re-import.

Scope: DNNPlatform/fallacies/fallacies.html (81 KB, structured charte)
+ MariagePourTous.html (38 KB, prose).

Design:
- Phase 2 CSV schema: segment-oriented (segment_id = file#element-index),
  source_path + selector for deterministic re-import, inner-text fields
  per language. NOT the key->value string-tier dialect.
- Phase 2 DatasetUpdater task shape: document-tier variant vs string-tier
  (prose prompt, char-budget chunking, gpt-5.5, reasoning effort low,
  max_output_tokens >=7000, Enabled=false until GO).
- Phase 1.5 extractor: the net-new gating tool (html.parser stdlib,
  file-specific selectors h2/h3/li vs p/h2/h3, inner-text extract +
  selector store for re-wrap). Nothing translates until this exists.
- Phase 4 re-import: selector-based patch, dry-run verifier (same DoD as
  string-tier round-trip), localized HTML files per language.
- RTL/CJK (ar/fa/zh): reuse PR #359 font stacks at page level
  (dir=rtl/lang), template variant decision for jsboige.
- Sequencing: extractor is critical path.
- 4 open questions for jsboige (inner-text vs outer-HTML, one-template vs
  per-lang, are pages still served, GO to build extractor).

Index README #554 updated to reference the new doc in supporting-docs
table. Pure design/references -- no prod mutation, no extraction
executed, no DatasetUpdater enabled.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@

@clusterManager-Myia clusterManager-Myia left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[NanoClaw]#557 (docs(dnn): #457 document-tier translation workflow)

LGTM as a design doc — coherent, well-scoped, and every cross-reference resolves. No blocking issues; a couple of minor nits below.

Cross-refs verified (all resolve, descriptions consistent with actual titles):

  • Issue #457 → "feat(dnn): localize site content — CSV extraction, DatasetUpdater translation, re-import workflow" — matches the doc's framing as the parent localization arc.
  • PR #524 → "tools(dnn-i18n): #457 content-type extractor + reimport dry-run" — the string-tier extractor the doc says is already built. Consistent.
  • PR #487 → "feat(dataset-updater): #457 DNN UI strings localization infra (Enabled=false)" — matches the "Enabled=false until GO" discipline the doc carries forward, and #487 does modify DatasetUpdaterRootConfig.cs as the doc claims.
  • Bonus: #359 (cited in §6) and #554 (the index README PR) also resolve with consistent titles.

File paths verified real (not fabricated):

  • DNNPlatform/fallacies/fallacies.html = 81139 bytes (~81 KB, matches §1).
  • DNNPlatform/fallacies/MariagePourTous.html = 38171 bytes (~38 KB, matches §1).
  • 457-site-content-type-inventory.md, tools/dnn_i18n/README.md, and DatasetUpdaterRootConfig.cs all exist in-repo.
  • README index addition is a correct relative link to the new doc, no broken anchor.

Workflow coherence: the "string-tier toolchain doesn't fit prose" framing holds — segment-oriented CSV with source_path+selector for deterministic re-import is a reasonable answer to the round-trip problem the inventory §3 deferred. The extractor (§4) correctly identified as the gating deliverable, Enabled=false carried through consistently, sequencing diagram in §7 is acyclic and matches the prose. No internal contradictions.

No fabricated metrics: the ~24,200 / ~13,100 / ~37,300 char figures are presented as approximate and are plausible for the file sizes. No fake status claims ("done", "shipped") — the doc is explicit throughout that nothing is built or enabled.

Security: clean. No secrets, tokens, keys, or private paths in the diff.

Nits (non-blocking):

  1. §3 cites DatasetUpdaterRootConfig.cs by bare filename; its actual path is Generation/Converters/Argumentum.AssetConverter/DatasetUpdater/DatasetUpdaterRootConfig.cs. The bare name is resolvable but a reader grepping the tree has to do one extra hop. The line range "~2636–2696" is unverified on my side but plausible for that file — worth a quick recheck before the doc is treated as canonical.
  2. Sources list references two "memory" fragments (reference-gpt55-reasoning-model-api, reference-openrouter-gpt55-path) and an "OpenAI key is live (decision #11)" claim — these are inline-labeled as memory/decisions so they're not deceptive, but they won't resolve for a reader outside the author's memory store. Fine for a working design doc; flag if this is meant to be shared beyond the core team.

Comment-only (self-review cap).

@jsboige
jsboige merged commit d09778e into master Jun 20, 2026
3 checks passed
@jsboige
jsboige deleted the docs/dnn-457-document-tier-workflow branch June 20, 2026 05:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants