Skip to content

Assemble chunked results, and put the documented commands under test - #36

Merged
ivanvyd merged 2 commits into
mainfrom
fix/complete-results-and-docs-contract
Aug 5, 2026
Merged

Assemble chunked results, and put the documented commands under test#36
ivanvyd merged 2 commits into
mainfrom
fix/complete-results-and-docs-contract

Conversation

@ivanvyd

@ivanvyd ivanvyd commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Two correctness gaps and one class of defect that nothing was catching.

Chunked results are assembled

A result split across chunks came back as chunk zero. It was correctly flagged as truncated — that
much was honest — but the rows were missing, and an incomplete export is the failure a first real
user meets on real data rather than a demo table.

The Genie API has no endpoint taking a chunk index. What it does have is the Statement Execution
response returned verbatim, which pairs next_chunk_index with next_chunk_internal_link
documented as an opaque path to join with the workspace host. That link was already arriving on
every chunked result and being dropped during deserialization. Following it completes the result.

The alternative — Genie's downloads endpoint pair — is a second workflow with its own polling and
expiry, and its response is chunked the same way, so it completes nothing on its own. It also needs
a download_id_signature that docs/planning/genie-api-surface.md did not record; that omission is
corrected here. Reasoning in
ADR 0004.

Every way of failing to complete a result still reports truncation — no link beside the index, a
link resolving off-workspace, a chunk the caller may not read, a repeated link, or the row cap. The
property that mattered before this change is unchanged and is now enforced across five exits
instead of one.

Two defects found while reviewing the first cut

  • A chunk pointing at itself looped forever. MaxResultRows was no backstop, because a
    self-pointing chunk carrying no rows never grows toward it. Guarded the way ListAllAgentsAsync
    already guards its repeated page token. Removing the guard makes the new test hang rather than
    fail — that is how it was confirmed, not by reading it.
  • Link validation was checking the shape of the string. //evil.example.com/x looks like a
    path and resolves to another host, which would have sent the bearer token there. Verified by
    probe, not assumed. Validation now resolves the link against the workspace and compares scheme,
    host and port — one mechanism that cannot be talked around, replacing two heuristics that could.

The documented commands are under test

Every fenced lakespeak … invocation in the repository's markdown is now parsed against the real
System.CommandLine tree. A documented command, subcommand or flag that stops existing fails the
build and names the file it appears in. Program.CreateRootCommand exists to make that tree
reachable from a test; Main is otherwise unchanged.

This has a concrete cause. README.md claimed 89 tests; a real run gives 175. The number
drifted through several releases because prose has no owner and no compiler. It is deleted rather
than corrected, for the same reason.

Verification

  • 223 tests pass across five projects; build clean with warnings-as-errors; dotnet format --verify-no-changes clean; dotnet restore --locked-mode succeeds.
  • Both new behaviours have a regression proof: the assembly test fails without the fix, and the
    loop-guard test hangs without the guard.

Not verified live. Whether a caller may read the remaining chunks of a statement Genie executed
on their behalf is a permission question no documentation settles. If the answer is no, behaviour
is exactly what it was before this change — the first chunk, flagged. compatibility.md,
limitations.md, ROADMAP.md and the ADR all record this as untested rather than implying
otherwise.

ivanvyd added 2 commits August 5, 2026 20:41
A result split across chunks came back as chunk zero, correctly flagged as
truncated but incomplete. The flag was honest; the shortfall was still the
failure people would hit first on real data.

The Statement Execution contract pairs next_chunk_index with
next_chunk_internal_link, documented as a workspace-relative path to be
treated as opaque. Genie returns that response verbatim, so the link was
already arriving and being discarded. Following it completes the result.

Every way of failing to follow it still reports truncated: no link, a link
that is not a workspace path, a chunk the caller may not read, or the
MaxResultRows cap. A partial result is never reported as complete, which is
the property that mattered before this change and still does.

The link is server-supplied and the bearer token rides on every workspace
request, so an absolute or protocol-relative link is refused rather than
followed. It is not logged, since the case worth logging is the one where
an attacker chose it.
Three things the first cut of the chunk walk got wrong or left open.

A chunk that points at itself looped forever. The row cap was no backstop:
a self-pointing chunk carrying no rows never grows toward it. Same failure
ListAllAgentsAsync already guards with its repeated-page-token check, and
now guarded the same way. Removing the guard makes the new test hang rather
than fail, which is how it was confirmed.

Link validation moved from the shape of the string to the resolved host.
`//evil.example.com/x` looks like a path and resolves to another host --
verified, not assumed -- and no amount of prefix-checking is provably free
of the next such quirk. Resolving against the workspace and comparing
scheme, host and port is one mechanism instead of two heuristics, and it
cannot be talked around.

The documented commands now have a test. Every fenced `lakespeak ...` in
the markdown is parsed against the real System.CommandLine tree, so a flag
that stops existing fails the build. Program.CreateRootCommand exists to
make that tree reachable from a test.

The README's "89 tests" is gone rather than corrected. The real number was
175 and nobody noticed it drift, which is the argument against putting a
number in prose at all.
@ivanvyd
ivanvyd merged commit e493425 into main Aug 5, 2026
10 checks passed
@ivanvyd
ivanvyd deleted the fix/complete-results-and-docs-contract branch August 5, 2026 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant