Skip to content

fix: keep visual whitespace out of authoritative structure - #319

Merged
seonghobae merged 10 commits into
codex/post-structure-case-fixesfrom
codex/normalize-source-indent-semantics
Aug 21, 2026
Merged

fix: keep visual whitespace out of authoritative structure#319
seonghobae merged 10 commits into
codex/post-structure-case-fixesfrom
codex/normalize-source-indent-semantics

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Buyer-visible gap

Rich-text exports that use   or leading spaces for visual alignment were persisted as progressively deeper explicit nesting. Mixed editor whitespace could therefore distort list hierarchy in the buyer view.

Change

  • Preserve total source indentation for diagnostics.
  • Track declared HTML/CSS/OOXML/list-container indentation separately.
  • Persist explicit structure only from declared indentation; source-only whitespace uses contextual-orchestrator adjudication and otherwise remains unresolved at level zero.
  • Add ADR 0103 and regression coverage.

Exact stack

  • Parent PR: fix(ui): preserve adjacent source table boundaries #317 (codex/post-structure-case-fixes).
  • Exact parent: 1faed2f680fa6d7bbe946a1a884522757d1639d6.
  • Exact current head: e3f00eaae9255f5f56eaa5d93b6fa2ea6ea3e8c5.
  • GitHub compare reports this branch ahead by 7 and behind by 0; the delta is limited to source-indentation semantics, persistence, tests, ADR/changelog, and the current gap baseline.
  • The ordinary parent restack is complete; no force-push or protected-branch bypass was used.

Verification boundary

  • Prior code-equivalent local evidence: 733 passed, 16 skipped, 4 warnings.
  • Frontend lint, TypeScript build, 136 tests, production build, Storybook, actionlint, and git diff --check passed on the predecessor product-equivalent tree.
  • Exact-current-head Tests, Security Scan, and SAST Semgrep are queued and remain non-passing until terminal success.
  • No current-head formal approval exists; the earlier Devin comment predates this restack.
  • No active review thread is unresolved.

No real records, identifiers, or provider credentials are included. Merge only after #317 lands, exact-head Checks pass, and an independent exact-head approval exists.


Open in Devin Review

@coderabbitai

coderabbitai Bot commented Aug 20, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 6d1b5a84-268c-4c10-9901-5ad526272277

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae

Copy link
Copy Markdown
Contributor Author

@opencode-agent review exact current HEAD 9f7574f. Check the declared-vs-source indentation boundary, table/list regressions, and current required Checks. Report actionable findings only; do not approve, bypass, or use stale parent evidence.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Review exact current HEAD 9f7574f. The whitespace-to-semantic-structure boundary is a dependency for adjacent table and visual-region work; required Full test suite and Frontend lint, test, build checks are green. Please publish a formal independent review for this exact SHA.

… into codex/normalize-source-indent-semantics

# Conflicts:
#	docs/product-technical-gap-baseline.md

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

@seonghobae

Copy link
Copy Markdown
Contributor Author

Review exact current HEAD e84994e9145d3f7a1a7849e94c91de02cccf62fc. This whitespace/semantic-structure boundary is the dependency for adjacent-table and visual-region work; local full Python and frontend checks are green. Please publish a formal independent review for this exact SHA and report actionable findings only.

@seonghobae

Copy link
Copy Markdown
Contributor Author

@OpenCode review exact current head 9a459417fc75ce7792e8a8ea0dade1f2c0589660. The prior LOW Strix finding is fixed at the shared persistence boundary: only expected OSError/RuntimeError/ValueError channel failures remain retryable, while unexpected defects propagate to the durable worker. Verification: 738 passed, 16 skipped; focused persistence 12 passed with 100% line and branch coverage; public-callable docstring gate passed; frontend 140 passed plus lint, production build, and Storybook.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Reviewed against exact current head 9e59b704a00fdd6d4f75fa6f548babd5f7fd12:

  • Source-only leading spaces and   remain diagnostics only; declared HTML/CSS/OOXML/list indentation is the only explicit structure persisted.
  • Structure and embedding provider failures remain retryable/unresolved, while unexpected programming defects propagate to the durable worker.
  • Focused verification: 62 passed across chunking, persistence, normalization, embedding, and structure tests.
  • Full backend verification: 738 passed, 16 skipped, 4 warnings.

The PR remains blocked pending independent formal review and terminal Checks; no bypass used.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Correction: the exact current head is 9e59b704a00fdd6d4f75fa6f548babd5dfb7fd12. The verification and review evidence above applies to this full SHA.

@seonghobae

Copy link
Copy Markdown
Contributor Author

The branch has moved to exact current HEAD e3f00eaae9255f5f56eaa5d93b6fa2ea6ea3e8c5 since the prior review request. Please review this exact content-persistence observability head and publish the independent verdict; predecessor evidence does not transfer. No merge is asserted while required Checks and approval remain pending.

Copy link
Copy Markdown
Contributor Author

@opencode-agent Review exact current head e3f00eaae9255f5f56eaa5d93b6fa2ea6ea3e8c5 against exact stacked parent 1faed2f680fa6d7bbe946a1a884522757d1639d6. The ordinary restack is complete and GitHub reports ahead 7 / behind 0. Verify that visual/source whitespace is retained only for diagnostics, explicit hierarchy derives only from declared HTML/CSS/OOXML/list-container structure, contextual-orchestrator adjudication remains bounded, unresolved source-only whitespace stays level zero, persistence and tests cover mixed editors, and no authoritative structure is invented. Publish a durable exact-head receipt and formal verdict or explicit fail-visible evidence. Do not merge, update/restack branches, or transfer predecessor evidence.

@seonghobae
seonghobae enabled auto-merge (squash) August 21, 2026 01:03
@seonghobae seonghobae added the bug Something isn't working label Aug 21, 2026 — with ChatGPT Codex Connector
… into codex/review-pr319

# Conflicts:
#	docs/product-technical-gap-baseline.md
devin-ai-integration[bot]

This comment was marked as resolved.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 0 new potential issues.

Open in Devin Review

* fix: retain partial image regions

* docs: record partial image region gap

* test: cover partial parent vision failure

* fix: safely handle malformed region locator output

* style: use deferred region result annotation

* docs: reconcile partial-region evidence counts

* fix: validate retained visual locator regions (#324)

* fix: reject unbounded visual locator regions

* docs: refresh exact-head product gap audit

* test: cover visual region fallback branches

* fix: fall back from malformed visual locator output

* docs: refresh visual locator exact-head evidence

* test: keep invalid locator fallback coordinate-free

* docs: keep PR 320 status under its checkpoint

* docs: refresh PR 324 exact-head evidence

* fix: safely handle malformed region locator output

* fix: keep internal image instructions out of buyer evidence (#329)

* fix: keep internal image instructions out of buyer evidence

* fix: protect persisted image region captions

* fix: preserve legitimate Korean image captions

* docs: allocate unique buyer image ADR number

* fix: reject invalid vision region response types

* fix: block internal vision destinations

* fix: keep provider failures out of buyer errors

* feat: trace post processing and Valkey sessions (#345)

* feat: trace post processing and valkey sessions

* docs: align orchestrator runtime pin

* docs: codify telemetry boundary

* fix: normalize OTLP trace endpoint

* chore: pin orchestrator telemetry fix

* fix: keep provider details out of ingestion ledger

* fix: propagate W3C trace context to gateway
@seonghobae
seonghobae merged commit a4c23c6 into codex/post-structure-case-fixes Aug 21, 2026
1 check was pending

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 6 new potential issues.

Open in Devin Review

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Narrowed exception set may let rare transport errors escape isolation

The batch failure handling changed from a broad except Exception to except (OSError, RuntimeError, ValueError) in both the structure loop (post_content_persistence.py) and the embedding loop (post_content_persistence.py). This is intentional per ADR 0103/the changelog (AssertionError and other defects should propagate to the durable worker). I verified the common expected transport failures are still isolated: HttpClientError subclasses RuntimeError (http_client.py), TimeoutError/ConnectionError/ssl.SSLError are OSError subclasses, and response validation raises ValueError. The residual edge is that a bare http.client.HTTPException subclass (e.g. IncompleteRead, BadStatusLine) is neither OSError/RuntimeError/ValueError, so a genuinely transient transport hiccup of that kind would now propagate and fail the whole post job rather than leaving the batch retryable. This is a low-probability edge and aligns with the documented decision, so not flagged as a bug — noting for awareness.

(Refers to this code)

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Adjudicated and explicit indent levels use independent numbering

With source-only whitespace now routed to adjudication instead of being treated as explicit, a document can end up with both declared-indent chunks (whose indent_level comes from explicit_levels in post_content_persistence.py) and orchestrator-adjudicated chunks (whose indent_level comes from client.infer). These two numbering schemes are independent and may not be consistently ordered relative to each other, so a mixed document could persist inconsistent nesting levels across the explicit and adjudicated units. This is not introduced as a new defect by this PR (the adjudication path already numbered independently), but the change makes the mixed-source scenario more common. Worth confirming the buyer view tolerates the two schemes coexisting.

(Refers to this code)

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Comment thread backend/app/main.py
Comment on lines +2660 to +2664
except Exception as exc: # noqa: BLE001 - provider boundary is fail-closed.
raise HTTPException(
status.HTTP_503_SERVICE_UNAVAILABLE,
"Post chat is unavailable: contextual-orchestrator returned no complete evidence object",
) from exc

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Duplicate unreachable exception handler

chat_about_post now has two identical except Exception as exc: handlers back to back. The first already catches everything, so the second (main.py) is unreachable dead code, an apparent merge artifact.

Suggested change
except Exception as exc: # noqa: BLE001 - provider boundary is fail-closed.
raise HTTPException(
status.HTTP_503_SERVICE_UNAVAILABLE,
"Post chat is unavailable: contextual-orchestrator returned no complete evidence object",
) from exc
cited_ids = list(answer.cited_post_ids)
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Comment on lines +155 to +162
_INTERNAL_IMAGE_INSTRUCTION = re.compile(
r"^(?:"
r"this post is an image(?:\s*[.!?]|.*(?:ask\s+questions|read\s+its\s+text).*)"
r"|이 글의 이미지입니다(?:\s*[.!?]|.*(?:keyman\s*(?:을|를)\s*추출|질문해.*텍스트를\s*읽으세요|이미지\s*안의\s*텍스트를\s*읽으세요).*)"
r"|(?:this image|이 이미지는).*(?:keyman\s*(?:을|를)\s*추출|ask\s+questions|read\s+its\s+text|질문해.*(?:읽으세요|추출하세요)|텍스트를\s*(?:읽으세요|추출하세요)).*"
r")$",
re.IGNORECASE,
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Unpunctuated internal instruction can leak

_INTERNAL_IMAGE_INSTRUCTION matches this post is an image only when followed by punctuation or an instruction phrase. A caption that is exactly This post is an image with no trailing text passes through unsuppressed. Whether such a bare caption exists in persisted data determines if this matters.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Comment on lines +133 to 154
with traced(
"lineageweave.http.post_json",
{
"http.request.method": "POST",
"server.address": hostname,
"url.path": parsed.path or "/",
"service.peer.name": "contextual-orchestrator",
},
) as span:
inject_trace_context(request_headers)
status, raw = _request(
"POST",
url,
body=json.dumps(request_payload).encode("utf-8"),
headers=request_headers,
timeout=timeout,
)
if span is not None:
span.set_attribute("http.response.status_code", status)
if status >= 400:
raise HttpClientError(f"HTTP {status} from {hostname}")
return _decode_json_object(raw, hostname)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: HTTP error status not recorded on span

In post_json, the status >= 400 raise and the JSON decode run after the traced span closes (http_client.py). The span records only http.response.status_code; 4xx/5xx and decode failures never set span error status. Correctness is unaffected.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Comment on lines +330 to +361
hostname = parsed.hostname
if not hostname:
raise ValueError("vision client URL is missing a hostname")
if parsed.username or parsed.password:
raise ValueError("vision client URL must not contain user credentials")
try:
parsed.port
except ValueError as exc:
raise ValueError("vision client URL has an invalid port") from exc
normalized_hostname = hostname.rstrip(".").casefold()
if (
normalized_hostname in _BLOCKED_VISION_HOSTNAMES
or normalized_hostname.endswith(".localhost")
):
raise ValueError(
"vision client URL points to a private, loopback, link-local, or metadata destination"
)
try:
address = ipaddress.ip_address(hostname)
except ValueError:
address = None
if address is not None and (
address.is_private
or address.is_loopback
or address.is_link_local
or address.is_reserved
or address.is_multicast
or address.is_unspecified
):
raise ValueError(
"vision client URL points to a private, loopback, link-local, or metadata destination"
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Localhost vision endpoint silently disabled

The vision client now rejects loopback/localhost/metadata hosts even with allow_insecure_http=True, and orchestrator_vision_client swallows the ValueError into NullImageContentClient. A dev pointing the orchestrator at 127.0.0.1/localhost loses vision while other channels keep working. The Compose default uses host orchestrator, and ADR 0121 states this is intended.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant