Repository navigation
volumes: read a held volume live in ls/du/cat/get (ENG-7113) - #1113
Merged
Merged
Conversation
While a run or read-write session holds a volume, ls, du, cat and get ask the platform for a live read and go through the read-only SSH view it starts or reuses, running the listing remotely in the same output formats. Otherwise, and while a finished writer still syncs back, they read storage and print why. (ENG-7113)
cabrinha
requested review from
DamianB-BitFlipper,
JannikSt,
JohannesHa,
akirillo,
burnpiro,
eexwhyzee,
kcoopermiller,
kennethnym,
mrmoxon and
willccbb
as code owners
October 26, 2026 17:27
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e87405bce5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
A filename may contain newlines, so a line can never bound a live stat record: the remote loop now prints one NUL-terminated record per path (BusyBox stat has no NUL escape, so the shell prints the terminators). du only matches a file operand exactly when it has no trailing slash, like the storage implementation.
skpulipaka26
approved these changes
Oct 29, 2026
skpulipaka26
approved these changes
Oct 29, 2026
eexwhyzee
approved these changes
Oct 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
ENG-7113: while a run (or a read-write session) holds a volume,
prime volumes ls,du,catandgetread the volume as the writer has it right now, so a run's checkpoints can be listed and downloaded mid-run. Before this change those commands read R2, which catches up only when the run ends.This needs platform#7011. That PR owns the source decision; the CLI only asks for it.
How
ls/du/cat(via_volume_reads) andget(via_transfer) call the transfer route withlive: true.via: sessionwith a read-only live view that mounts the writer's PVC. It starts the view or reuses an existing one, and the CLI waits for it the same waysshdoes.via: r2as before. Anote, for example "the run X is syncing its changes back …", is printed to stderr.ls: onefind … -exec stat -c '%s|%F|%Y|%n'(BusyBox in the session image, which is Alpine). The output is rendered in exactly the storage shapes: plain,-l/-hand-R, with directories asdir/and--output jsonunchanged.runs/.sessions/is hidden, as before.du: one whole-volumefindthat counts files. A live volume has no directory markers.cat: streamscatover ssh to stdout as bytes.get: the existing session transfer path (rsync, else scp), given the view._open_sessionand the wait helpers take auiconsole for this.ssh --read-onlyalso says when it is a live view.ls/du/cat, and thegethelp and README are updated.Compatibility: older backends ignore
liveand keep routing to R2. Avia: sessionread route used to be refused ("a session started on the volume; rerun"). It is now served through that session.Tests
New
tests/test_volume_live_reads.py:alpine:3.22(the volume-ssh image's base) over a temp dir mounted at/volume, in place of ssh. These tests skip without Docker. They cover:lsjson/plain/-l/-Rover a run's checkpoint tree, with internal sync state hidden and notes on stderr onlyls/du/cat, and a file named with a trailing/dusubtree and whole-volume countscat|in a name and newline tailsgetrouting through the live view, withlive=Trueand the route's session reusedlivelivesent only when asked;note,liveViewOf)Existing tests: the routing assertions now include
live: True, and fake routes acceptlive.The 4 failures are
test_put_refuses_a_symlink_source[*]. They fail the same way on a cleanorigin/maincheckout here: the macOS temp path wraps "points to" across a line. They are unrelated to this PR.ruff check,ruff format(changed files) andty check src/are clean.🤖 Generated with Claude Code