refactor(read): one head-window rule for files, stdin and slices

[KuSh]

Follow-up to #4122, addressing both review notes on it: unify the two head-window implementations, and stop the tracking row inflating the emitted size.

One rule for the head window

read_head_lines (chunked, added in #4122) and head_window (a slice over bytes in memory) each carried their own scan for the end of the nth line. They agreed only because a test said so. head_prefix now holds that rule once — where the first n newline-terminated lines end in a slice, and how many terminators are there. head_window applies it to bytes it already has; copy_head_lines applies it to each chunk with a running count, which is what lets it stop at the nth newline instead of draining the source.

copy_head_lines takes any Read and writes each chunk on as it comes, so two things follow from the unification:

• stdin stops early. rtk read - --head-lines N used to read_to_end first. yes | rtk read - --head-lines 3 kept reading until memory ran out; it now prints three lines and exits, like head -n 3. (The hook only rewrites head -n N <file>, never a pipeline, so this path is reached by typing it.)
• a window costs a chunk of memory, not its own length. rtk read /dev/zero --head-lines 1 held 440 MB and climbing after three seconds; it now holds flat — 17.3 MB for the debug build, 7.2 MB for the release build, steady across three seconds. It still blocks, as head -n 1 /dev/zero does, but at bounded memory. The same applies to any file whose first line is enormous — a minified bundle, say — which the hook does rewrite.

Output is unchanged. Windows from files and from stdin were compared byte-for-byte against native head -n N over n in {0, 1, 2, 3, 10, 4999, 100000} on binary data, CRLF text, an unterminated last line, a 200 KB random file and a 5000-line file; exit codes match too (1 on a missing file, SIGPIPE on a closed stdout).

What the equivalence test covers

test_head_windows_match_the_reference pins copy_head_lines and head_window against reference_head_window, a frozen copy of the pre-refactor slice kept in the test module. Both are compared to that third implementation rather than to each other, so a shared rule that drifts is still caught rather than agreeing with itself.

Its inputs deliberately span several READ_CHUNKs, because reading across chunks is the only thing the chunked reader adds: a line twice the chunk size, a newline exactly on the boundary and on either side, 4000 short lines, a CRLF straddling the boundary, an unterminated last line, empty input; each at n in {0, 1, 2, 3, 10, 1000, 4000}. I checked it catches regressions by mutation: stopping after the first chunk, off-by-one in the shared rule, scanning each chunk for n instead of the lines still wanted, and writing the whole chunk instead of its accepted prefix all fail it; a narrow input set that fits in one chunk leaves the first of those green.

test_copy_head_lines_stops_on_a_trickling_endless_source covers what a pipe does and a file does not: five bytes per read, no end, a CRLF split across two reads. It counts the bytes the source hands out, so a reader that pulls one chunk past the nth newline fails it (verified by mutation) — the class this path exists to prevent. read_stdin_head_stops_without_draining_an_endless_producer runs the binary against a producer that prints three lines and keeps the pipe open, under a 20 s deadline, so a draining regression fails without growing memory; the unit tests on an endless FIFO and on the trickling source are bounded the same way.

A closed downstream now ends the process at the chunk it is writing, where the buffered path wrote everything in one go after the reader had finished. Over 25 runs of … --head-lines 5000 | head -1, this branch exited 141 every time, native head 17 times, the buffered path never — closer to head, and enough to make a pipeline under set -o pipefail fail where it used to pass. The producer side changes the same way: seq 100000 | rtk read - --head-lines 5 under pipefail exits 141, as with head, where develop exits 0.

One divergence from head stays, deliberately: a seekable stdin is not rewound. { rtk read - --head-lines 1 >/dev/null; wc -c; } < f leaves 8192 bytes consumed where head -n 1 leaves 2, because the chunk that satisfied the count is not seeked back. The previous behaviour left nothing at all, having drained the stream.

What a window is measured against

rtk read <f> --head-lines N exists because the hook rewrites head -n N <f> to it, and it prints the window that command prints. The rows measured it against the whole file anyway, so RTK took credit for the user's own choice of command — and the labels said cat <f>, which is not what ran. Measured on a 20000-line file:

invocation	before	after
read big.txt --head-lines 5	cat big.txt 27224 / 3 / 100%	head -n 5 big.txt 3 / 3 / 0%
read big.txt --head-lines 5 -n	cat big.txt 77224 / 10 / 100%	head -n 5 big.txt 10 / 10 / 0%
read big.txt --head-lines 5000	cat big.txt 27224 / 5974 / 78.1%	head -n 5000 big.txt 5974 / 5974 / 0%
read big.txt --tail-lines 5	cat big.txt 27224 / 8 / 100%	tail -n 5 big.txt 8 / 8 / 0%
read big.txt (a real cat)	27224 / 27224 / 0%	unchanged
read read.rs --level aggressive	7643 / 1895 / 75.2%	unchanged

Each row carries two numbers, the window and what RTK printed of it; neither is a constant in the code, and every path today prints the window whole, so every such row reads 0%. That is the true figure, not a floor: output is byte-identical to native head -n N at n ∈ {5, 500, 5000, 19999, 99999} and to tail -n N at {5, 5000}, from a file and from stdin, and --head-lines cannot combine with --max-lines, so nothing trims a requested window anywhere.

--line-numbers is measured the same way: it renders the window rather than printing it, but the window is still all the user asked RTK to do, so the row is measured against what was shown. rtk read big.txt --head-lines 5 -n booked 77224 / 10 / 100% on develop and books 10 / 10 / 0% here.

Only a window the user asked for is measured this way. --max-lines, the filter levels and a plain read keep the whole input as their baseline, because there RTK is the one deciding to emit less than was asked for — the --level aggressive row above is untouched. One combination is left over-credited: a filter level together with a window. --level aggressive --head-lines 3 books 99.6% where the filter alone earns 75.4%, and --level aggressive --tail-lines 5 books 100.0%, because there the user and RTK each cut part of the output and there is no single command the row stands in for. The hook never produces either spelling.

This also settles what a pipe is worth: cat big.txt | rtk read - --head-lines 5 books 3 / 3 / 0%, the same as every other spelling of the same window, because that is what the window is worth and not because the producer's size is unknowable.

Emitted size in the tracking row

track_bytes took the emitted side as a &str, so a window was measured after String::from_utf8_lossy had replaced each invalid sequence with the three-byte replacement character. Both sides are byte counts now.

With no window measured against a size on disk any more, a size that lies stops being read as a measurement too. Rows recorded before and after, against a binary built from develop:

invocation	before	after
read /dev/urandom --head-lines 5	295 / 542 / -83.7%	351 / 351 / 0%
read /proc/cpuinfo --head-lines 5	0 / 29 / -29 tokens	29 / 29 / 0%
read /sys/kernel/mm/transparent_hugepage/enabled --head-lines 5	1024 / 6 / 99.4%	6 / 6 / 0%
read <non-UTF-8 file> --tail-lines 1	11 / 8 / 27.3%	3 / 3 / 0%

rtk gain clamps a negative saving out of its headline, so those two never credited RTK there; nothing clamped the over-credited rows, which moved it directly — that is what the window baseline above is for.

🤖 Generated with Claude Code

[rtk-wshm-sync-bot]

wshm · Automated triage by AI

📊 Automated PR Analysis

♻️ Type	refactor
🟡 Risk	medium

Summary

This PR unifies two divergent head-window implementations (for files and stdin) into a single head_prefix/copy_head_lines rule, fixing stdin draining the whole input before applying --head-lines and unbounded memory growth on huge first lines. It also fixes track_bytes calls that were measuring emitted size after lossy UTF-8 conversion inflated invalid bytes, switching to raw byte-length tracking.

Review Checklist

• Tests present
• Breaking change
• Docs updated

Linked issues: #4122

---

Analyzed automatically by wshm · This is an automated analysis, not a human review.