fix: keep visual whitespace out of authoritative structure

CHANGELOG.d/2.13.1-source-indent-semantics.md

@@ -0,0 +1,5 @@
+## Fix source-only indentation depth
+
+- Keep leading spaces and ` ` available as diagnostics without persisting
+  them as authoritative structure; only declared HTML/CSS/OOXML or list
+  nesting is explicit.

docs/adr/0103-source-whitespace-is-not-authoritative-structure.md

@@ -0,0 +1,38 @@
+# ADR 0103: Source-only whitespace is not authoritative structure
+
+- Status: Accepted
+- Date: 2026-08-20
+- Depends on: [0073](0073-llm-structure-adjudication.md), [0102](0102-semantic-source-unit-boundaries.md)
+
+## Context
+
+Rich-text exports often contain leading spaces or ` ` characters that
+only preserve an editor's visual alignment. Treating every distinct width as
+a nesting level makes unrelated list items appear deeply nested, especially
+when an export mixes tabs, non-breaking spaces, and manual alignment. The
+result is an authoritative structure decision without structural evidence.
+
+## Decision
+
+`Chunk` retains the total source indentation for diagnostics and fallback
+rendering, but exposes declared indentation separately. Only HTML/CSS/OOXML
+declarations and nested list-container depth populate declared indentation.
+Persistence may mark a unit `explicit` only when declared indentation exists.
+Source-only whitespace follows the existing contextual-orchestrator
+adjudication path; if that channel is unavailable, it remains `unresolved`
+with level zero.
+
+## Consequences
+
+- Visual alignment cannot manufacture hierarchy or distort the buyer view.
+- Real CSS, OOXML, and list-container structure remains authoritative.
+- Reprocessing with contextual-orchestrator can resolve source-only cases while
+  preserving the original body and diagnostic width.
+
+## References
+
+World Wide Web Consortium. (n.d.). *HTML Living Standard: The `ol`, `ul`, and
+`li` elements*. WHATWG. https://html.spec.whatwg.org/multipage/grouping-content.html
+
+World Wide Web Consortium. (n.d.). *CSS box model module level 3*. W3C.
+https://www.w3.org/TR/css-box-3/

docs/product-technical-gap-baseline.md

@@ -380,7 +380,24 @@ ADR 0115. PR #311 remains an active stacked delivery candidate; its required
 checks and formal approval must be rechecked at the exact current head before
 any protected merge claim.
 
-## Image locator and buyer table checkpoint: 2026-08-20
+ ## Source-only indentation checkpoint: 2026-08-20
+
+The semantic-unit parser now keeps source leading whitespace and declared
+HTML/CSS/OOXML or list-container indentation as separate evidence. Visual
+alignment alone cannot become an `explicit` hierarchy level; it is sent to
+contextual-orchestrator when available and otherwise remains `unresolved` at
+level zero. This prevents mixed editor exports from manufacturing deep list
+nesting in the buyer view.
+
+- Decision record: ADR 0103
+- Implementation: PR #319, stacked on the adjacent-table correction in PR #317
+- Local evidence: backend `731 passed, 16 skipped`; frontend `134 passed`, lint,
+  build, and Storybook build passed.
+- Integration status: PR #319 is not protected-main truth; exact parent head,
+  formal review, terminal Checks, and post-merge browser evidence remain
+  required.
+
+ ## Image locator and buyer table checkpoint: 2026-08-20
 
 A first bounded private reprocessing run completed five parent-image
 descriptions and five region embeddings, but aggregate inspection found five
@@ -463,6 +480,6 @@ open at `49804b0f`; its current `Analyze (python)` check is queued and its merge
 state is blocked. Earlier SQL/SAST comments are bound to predecessor commits,
 so they are not treated as current-head failures without a fresh matching scan.
 
-The current evidence supports continued review/check processing, not a merge
-claim. Re-read the exact head, current review commit, all required Checks, and
-stack dependency immediately before any protected merge.
+ The current evidence supports continued review/check processing, not a merge
+ claim. Re-read the exact head, current review commit, all required Checks, and
+ stack dependency immediately before any protected merge.

lineageweave/chunking.py

@@ -225,6 +225,9 @@ class Chunk:
         indent_width: source indentation in semantic units, retained as
             structural metadata while presentation whitespace is removed from
             ``text``.
+        declared_indent_width: indentation declared by HTML/CSS/OOXML or a
+            nested list container. Source-only leading spaces are excluded so
+            callers can distinguish authored structure from visual alignment.
     """
 
     text: str
@@ -234,6 +237,7 @@ class Chunk:
     image_data: bytes | None = field(default=None, compare=True)
     style: str | None = None
     indent_width: int = 0
+    declared_indent_width: int = 0
 
 
 def chunk_by_paragraph(text: str) -> list[Chunk]:
@@ -307,7 +311,7 @@ def __init__(self) -> None:
         # ("image", (mime_type, bytes), "", None) -- a single sequence in
         # true document order, so an image's index among its siblings
         # reflects where it actually sat.
-        self._finished: list[tuple[str, object, str, str | None, int]] = []
+        self._finished: list[tuple[str, object, str, str | None, int, int]] = []
 
     def handle_starttag(self, tag: str, attrs: list[tuple[str, str | None]]) -> None:
         """Collect relevant text state when an HTML start tag is encountered."""
@@ -316,7 +320,7 @@ def handle_starttag(self, tag: str, attrs: list[tuple[str, str | None]]) -> None
             if src:
                 decoded = _decode_data_uri_image(src)
                 if decoded is not None:
-                    self._finished.append(("image", decoded, "", None, 0))
+                    self._finished.append(("image", decoded, "", None, 0, 0))
             return
         if tag in {"br", "w:br"} and self._stack:
             self._stack[-1][1].append("\n")
@@ -372,7 +376,14 @@ def _finish_block(
                 indent_width = declared_width + source_indent
                 label = "footnote" if _FOOTNOTE_START.match(text) else tag_name
                 self._finished.append(
-                    ("text", text, label, style, indent_width)
+                    (
+                        "text",
+                        text,
+                        label,
+                        style,
+                        indent_width,
+                        declared_width,
+                    )
                 )
 
     def handle_data(self, data: str) -> None:
@@ -390,7 +401,7 @@ def handle_data(self, data: str) -> None:
         elif text.strip() or had_nbsp:
             self._unscoped_buffer.append(text)
 
-    def finished(self) -> list[tuple[str, object, str, str | None, int]]:
+    def finished(self) -> list[tuple[str, object, str, str | None, int, int]]:
         """Return the normalized records collected from the HTML fragment."""
         while self._stack:
             declared_width = sum(entry[3] for entry in self._stack)
@@ -399,7 +410,9 @@ def finished(self) -> list[tuple[str, object, str, str | None, int]]:
         if not self._finished:
             fallback = normalize_semantic_text("".join(self._unscoped_buffer))
             if fallback:
-                return [("text", fallback, "", None, _source_indent_width(fallback))]
+                return [
+                    ("text", fallback, "", None, _source_indent_width(fallback), 0)
+                ]
         return self._finished
 
 
@@ -508,7 +521,14 @@ def chunk_by_dom(html: str) -> list[Chunk]:
     parser.feed(html)
     entries = parser.finished()
     chunks: list[Chunk] = []
-    for index, (kind, value, tag_name, style, indent_width) in enumerate(entries):
+    for index, (
+        kind,
+        value,
+        tag_name,
+        style,
+        indent_width,
+        declared_indent_width,
+    ) in enumerate(entries):
         if kind == "text":
             chunks.append(
                 Chunk(
@@ -518,6 +538,7 @@ def chunk_by_dom(html: str) -> list[Chunk]:
                     label=tag_name,
                     style=style,
                     indent_width=indent_width,
+                    declared_indent_width=declared_indent_width,
                 )
             )
         else:
@@ -546,6 +567,7 @@ def chunk_by_source_body(body: str) -> list[Chunk]:
             index=index,
             label=label,
             indent_width=indent_width,
+            declared_indent_width=0,
         )
         for index, (text, indent_width, label) in enumerate(_split_plain_text_units(body))
     ]

lineageweave/post_content_persistence.py

@@ -111,17 +111,25 @@ async def persist_post_content(
         if chunk.unit_type != "image" and unit_text
     ]
     explicit_widths = sorted(
-        {int(chunk.indent_width) for chunk in text_chunks if int(chunk.indent_width) > 0}
+        {
+            int(chunk.declared_indent_width)
+            for chunk in text_chunks
+            if int(chunk.declared_indent_width) > 0
+        }
     )
     # CSS/XML indentation values are presentation widths, not semantic depth.
-    # Rank the observed widths instead of dividing by a gcd: 56px and 80px
-    # are two nesting levels even when their pixel-unit gcd is 1.
+    # Rank declared widths instead of dividing by a gcd: 56px and 80px are two
+    # nesting levels even when their pixel-unit gcd is 1. Leading source
+    # whitespace is deliberately excluded: editors use it for visual alignment
+    # and it is not authoritative hierarchy without an orchestrator decision.
     explicit_levels = {width: level for level, width in enumerate(explicit_widths, start=1)}
-    unresolved = [chunk for chunk in text_chunks if int(chunk.indent_width) <= 0]
+    unresolved = [
+        chunk for chunk in text_chunks if int(chunk.declared_indent_width) <= 0
+    ]
     unresolved_indexes = {chunk.index for chunk in unresolved}
     structure_by_index: dict[int, StructureDecision] = {}
     for chunk in text_chunks:
-        width = int(chunk.indent_width)
+        width = int(chunk.declared_indent_width)
         if width > 0:
             structure_by_index[chunk.index] = StructureDecision(
                 unit_index=chunk.index,

tests/test_chunking.py

@@ -126,6 +126,7 @@ def test_chunk_by_dom_keeps_indentation_as_metadata_not_embedding_text() -> None
 
     assert [chunk.text for chunk in chunks] == ["Level one", "Level two"]
     assert [chunk.indent_width for chunk in chunks] == [2, 4]
+    assert [chunk.declared_indent_width for chunk in chunks] == [0, 0]
 
 
 def test_chunk_by_dom_reads_html_and_word_indentation_declarations() -> None:
@@ -138,6 +139,7 @@ def test_chunk_by_dom_reads_html_and_word_indentation_declarations() -> None:
 
     assert [chunk.text for chunk in chunks] == ["HTML", "Word"]
     assert [chunk.indent_width for chunk in chunks] == [4, 4]
+    assert [chunk.declared_indent_width for chunk in chunks] == [4, 4]
 
 
 def test_chunk_by_dom_reads_the_css_margin_shorthand_not_just_margin_left() -> None:

tests/test_post_content_persistence_edges.py

@@ -176,3 +176,27 @@ def test_unavailable_embedding_channel_is_skipped_and_empty_body_is_safe() -> No
         == 0
     )
     assert not any("post_content_embedding" in query for query, _args in conn.fetchvals)
+
+
+def test_source_only_whitespace_is_not_persisted_as_explicit_depth() -> None:
+    """Presentation alignment must not become authoritative hierarchy."""
+    conn = _Connection()
+
+    assert (
+        _persist(
+            conn,
+            "post-4",
+            "<p>&nbsp;&nbsp;First item</p><p>&nbsp;&nbsp;&nbsp;&nbsp;Second item</p>",
+        )
+        == 2
+    )
+
+    structure_rows = [
+        args
+        for query, args in conn.executed
+        if "insert into post_content_unit_structure" in query
+    ]
+    assert [(args[1], args[2]) for args in structure_rows] == [
+        (0, "unresolved"),
+        (0, "unresolved"),
+    ]