Skip to content
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
748c888
fix(serve): allow bounded reads of large text files
doudouOUC Jul 28, 2026
e784e6d
fix(serve): bound large-text reads by scan cost, not by which knob wa…
doudouOUC Jul 28, 2026
f55c867
refactor(core): thread the descriptor instead of forking text-read he…
doudouOUC Jul 28, 2026
a74eeea
refactor(core): make CoreReadTextFileHandleRequest standalone
doudouOUC Jul 29, 2026
a1b2188
Merge remote-tracking branch 'origin/main' into pr-a-descriptor-threa…
Jul 29, 2026
63822f6
fix(serve): address review feedback on decoded-output cap, lineEnding…
Jul 29, 2026
63d49c4
fix(serve): scope PR to descriptor threading, revert #7947 behavior r…
Jul 29, 2026
bc4d4c6
test(core): port deep-read and encoding-probe tests to the handle pat…
Jul 29, 2026
9c0e08d
test(core): pin handle read-to-EOF and unlimited-limit contracts (#7967)
qwen-code-ci-bot Jul 29, 2026
224db1f
Merge branch 'main' into pr-a-descriptor-threading
wenshao Jul 29, 2026
67ad71f
docs(serve): disclose handle-read behaviour deltas and add boundary t…
Jul 30, 2026
cdbdfde
test(core): add positive handle-read test for Infinity and omitted li…
Jul 30, 2026
1e22035
fix(core): satisfy ReadTextRangeResult type in handle-read test mock …
Jul 30, 2026
dc0604c
fix(core): bound handle reads to stat size and reject Infinity limit …
Jul 30, 2026
88e743d
Merge remote-tracking branch 'origin/main' into pr-a-descriptor-threa…
Jul 30, 2026
20d606b
fix(core): satisfy ReadTextRangeResult type in fileSystemService test…
Jul 30, 2026
d2b36fc
Merge branch 'main' into pr-a-descriptor-threading
qwen-code-dev-bot Jul 30, 2026
b3dafee
refactor(core): reuse detectLineEndingFromContent in whole-file fast …
qwen-code-dev-bot Jul 30, 2026
2558093
fix(core): export ReadTextRangeResult type and cover CRLF detection (…
qwen-code-dev-bot Jul 31, 2026
af5da8e
test(core): cover whole-file lineEnding and truncatedByBytes shape (#…
Jul 31, 2026
6532ddf
Merge branch 'main' into pr-a-descriptor-threading
qwen-code-dev-bot Jul 31, 2026
3c46060
Merge branch 'main' into pr-a-descriptor-threading
qwen-code-dev-bot Jul 31, 2026
7f5e851
Merge branch 'main' into pr-a-descriptor-threading
qwen-code-dev-bot Jul 31, 2026
9e01ff2
Merge branch 'main' into pr-a-descriptor-threading
qwen-code-dev-bot Jul 31, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
186 changes: 186 additions & 0 deletions docs/design/2026-07-29-handle-bound-text-range-reads.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,186 @@
# Handle-Bound Text Range Reads

## Context

PR #7947 let the Serve workspace filesystem return bounded line windows from
text files above `MAX_READ_BYTES` (256 KiB). To keep those reads pinned to one
inode across validation, binary probing, and streaming, it threaded a
caller-owned `FileHandle` down into `readTextRange` as an optional field, and
added a second optional field, `forceStreaming`, to suppress the buffering fast
path that would otherwise defeat the memory bound.

Two optional fields on one entry point produced four combinations, of which one
is meaningful, one is unreachable, and one is unsafe:

| `fileHandle` | `forceStreaming` | Result |
| ------------ | ---------------- | ---------------------------------------------------------------------- |
| unset | unset | ordinary path read |
| unset | set | streams a small file — used by one test |
| set | set | the Serve boundary's read |
| set | unset | buffers the whole file through the handle — **no caller can reach it** |

The unreachable combination carried a dedicated helper, `readFileHandleBuffer`,
with no test coverage. Separately, `readFileWithLineAndLimit` accepted the same
`fileHandle` but could only honor it on its range branch: an unbounded read fell
through to a by-path `readFileWithEncodingInfo`, silently returning bytes from
whatever the path resolved to at that moment rather than from the pinned inode.
PR #7947's follow-up commit guarded that with a runtime `RangeError`, which
documented the trap without removing it.

Encoding detection had forked for the same reason. `detectFileEncoding` takes a
path and opens its own descriptor, so the handle path could not use it; a
private `detectFileHandleEncoding` was added alongside, deriving the encoding
name from `decodeBufferWithEncodingInfoAsync(...).encoding` instead of from
chardet directly. The two disagree when chardet names an encoding `iconv-lite`
cannot load: the path variant returns that name, the handle variant returns
`'utf-8'` and defers to the streaming decoder's `fatal: true` failure. Both
refuse the file, with different messages.

## Goals

- One encoding detector, usable from a path or a borrowed descriptor.
- No mode flags on the range reader; make the unreachable combination
unrepresentable rather than merely unused.
- Make the by-path fallthrough structurally impossible instead of guarded.
- No observable change at the Serve boundary or in the `read_file` tool.

## Non-goals

- Collapsing `decodeBufferWithEncodingInfo` (sync) into its async twin. The sync
variant is a deliberate public-API compatibility shim
([`lazy-first-use-dependencies.md`](./lazy-first-use-dependencies.md)) pinned
by a parity test.
- Any change to what the Serve boundary returns. This is preparation for
byte-cursor paging, not that feature.

## Design

### One detector

`detectFileEncoding(source: string | FileHandle)`. A supplied handle is
_borrowed_: reads use explicit positions so the caller's file position is
untouched, and the `finally` block closes only a descriptor this function
itself opened. `detectFileHandleEncoding` is deleted, and the open-coded
BOM-to-name switch is replaced with the existing `bomEncodingToName`.

This makes the handle path slightly stricter, which is the intended direction:
an encoding `iconv-lite` cannot load now raises
`LargeNonUtf8TextError(detected)` naming that encoding, rather than reaching the
decoder and raising the generic `'invalid-utf8'` variant. The refusal is
unchanged; the message improves. The Serve boundary maps both to `binary_file`,
so nothing downstream moves.

A second, smaller delta comes with the merge: `detectFileEncoding` catches all
errors and falls back to `'utf-8'`, whereas `detectFileHandleEncoding` had no
handler and let an I/O failure propagate. The failure is not lost — a handle bad
enough to fail the 8 KiB probe fails the streaming read immediately after, and a
file that is not really UTF-8 is still refused by the `fatal: true` decoder — so
the error surfaces from a different call rather than disappearing. Accepted for
the single fallback policy; noted because it is a real change in which call
reports the problem.

### Two entry points

```ts
readTextRange(request: ReadTextRangeRequest) // path
readTextRangeFromHandle(fh, request: ReadTextRangeFromHandleRequest)
```

The handle variant always streams — there is no flag, because a caller reaches
for a handle precisely when it needs the read bounded, and the buffering fast
path would read the whole file. Its request type has no `path` (nothing for one
to disambiguate) and makes both byte bounds required rather than optional:
`maxOutputBytes` caps what the read returns, `maxScanBytes` caps what it costs,
and a handle-bound read exists because a security boundary needs both.

`maxScanBytes` stays optional on the path variant, where it defaults to
`Infinity` so the `read_file` tool is unchanged.

Both delegate to the same streaming implementation, which now takes
`source: string | FileHandle` and selects `createReadStream` or
`chunksFromHandle` accordingly. `readFileHandleBuffer` and the branch that
called it are deleted.

### The fallthrough disappears

`readFileWithLineAndLimit` loses `fileHandle`, `forceStreaming`, and
`maxScanBytes` — its single production caller passes none of them.
`StandardFileSystemService.readTextFileFromHandle` now calls
`readTextRangeFromHandle` directly, and the two read paths share a
`toReadTextFileResponse` helper so their metadata shaping cannot drift. With no
`fileHandle` parameter left to ignore, the `RangeError` guard is removed: the
trap it described can no longer be expressed.

`readTextFileFromHandle` stays off the `FileSystemService` interface, so
`AcpFileSystemService` and the typed fallback mock in `filesystem.test.ts` are
untouched.

## Blast radius

- `readTextRange` is not exported from `packages/core/src/index.ts`; only the
two error classes are. The reshaped surface is core-internal.
- `readTextRange` and `readFileWithLineAndLimit` have exactly one production
caller each (`fileUtils.ts`, `fileSystemService.ts`).
- `detectFileEncoding` is public via `export * from './utils/fileUtils.js'`.
Widening a parameter is source-compatible.
- The only cross-package importer of the touched modules is
`packages/cli/src/serve/fs/workspace-file-system.ts`. Its only change is
dropping two arguments the handle path no longer accepts — see below; the
`decodeBufferWithEncodingInfoAsync` import it also carries is untouched.

### `CoreReadTextFileHandleRequest` becomes standalone

It was `Omit<CoreReadTextFileRequest, 'limit' | 'stats' | 'maxOutputBytes'> &
{...}`, which left two fields the handle path never reads:

- **`stats`** was documented as required — "must pass the Stats captured from
that handle" — and nothing downstream read it. The handle path always streams,
so it never needs a size to choose a strategy, and the encoding probe does its
own `fstat`.
- **`path`** became dead once `readTextRangeFromHandle` replaced the
path-plus-handle call: the read is bound to the descriptor, and errors are
labelled with the path by the Serve boundary that owns it.

Neither was caught by the compiler. The ACP `ReadTextFileRequest` this type
derived from permits extra properties, so passing a field the type had removed
raised nothing. That is the argument for declaring the type standalone rather
than deriving it: the `Omit` chain was stripping four of six inherited fields
and quietly re-admitting the rest.

282 production logic lines change in `packages/core`; across `packages/` the
change is net −68 lines.

## Testing

The existing suites are the specification: the whole point is that the Serve
boundary cannot tell. `packages/cli/src/serve/fs/` and the bridge adapter — 222
tests — pass unmodified, as does the full `packages/core` `src/utils` +
`src/services` run (6078 tests).

Three tests in `read-text-range.test.ts` moved to `readTextRangeFromHandle`. Two
used `fileHandle` directly. The third used a _path_ with `forceStreaming: true`
to force streaming on a file too small to leave the fast path, so that it could
exercise the budget-at-EOF boundary; with the flag gone, the handle variant is
the only thing that always streams.

One of the moved tests changed meaning. It previously passed a handle for one
file and a path naming a different file, asserting the handle won — a test for
the confusion the old signature permitted. The handle variant has no `path`, so
that confusion is now unrepresentable and the test would assert nothing. It was
rewritten to cover the property that actually motivated the API: open a handle,
rename another file over the path, and confirm the read still follows the inode.

Two tests in `fileSystemService.test.ts` were deleted rather than repaired. They
mocked `readFileWithLineAndLimit` and asserted the argument object it received;
since `readTextFileFromHandle` no longer calls it, they could only have been
kept by re-pointing them at a new mock, which would again assert only that one
function passes arguments to another. The behaviour they nominally covered is
tested against real files in `read-text-range.test.ts` and at the real boundary
in `workspace-file-system.test.ts`. The argument-validation tests beside them
are kept — they need no mock.

## Follow-up

`chunksFromHandle` gained a `from` parameter that nothing yet passes a non-zero
value for. It is the single seam byte-cursor text paging needs, and that is the
change this refactor exists to enable.
Loading
Loading