Skip to content

fix(file-tools): check remote binary overwrites on the execution target, not the controller - #124106

Merged
kshitijk4poor merged 6 commits into
NousResearch:mainfrom
kshitijk4poor:fix/p1-0926-c12-remote-binary-guard
Sep 26, 2026
Merged

kshitijk4poor merged 6 commits into
NousResearch:mainfrom
kshitijk4poor:fix/p1-0926-c12-remote-binary-guard

Conversation

@kshitijk4poor

Copy link
Copy Markdown

On a remote terminal backend, write_file / patch no longer overwrite an existing binary file that only exists on the remote side. The existing-binary guard now checks the filesystem where the write actually runs, not the controller's disk. Fixes #122662. This salvages #123293 by @JoaoMarcos44, with @Liuzikaii (#122663, the first fix and the issue report) credited as co-author.

Why

  • The guard that refuses to overwrite an existing .sqlite / .pdf / other binary target (_check_binary_document_write) ran Path(...).is_file() on the controller.
  • write_file, patch and V4A run on the task's backend (Docker, SSH, Modal, Vercel sandbox, ...). A file that exists only there looked "new", so its bytes were replaced and the tool reported verified: true.
  • Live-reproduced on main for both a Docker and a Vercel-sandbox environment.

Changes

  • Probe the execution target (@JoaoMarcos44, fix(file-tools): decide binary overwrite checks on the execution target filesystem #123293). A new _target_regular_file_state() asks the task's own file_ops whether a regular file exists at the path the write will use. It returns exists, absent or unavailable.
  • Fail closed. If the backend can't be probed (starting, unreachable, removed), the write is refused with a message saying the file was not modified. A probe where [ -f ] succeeded but the size was unparseable counts as existing.
  • New files are still allowed. Creating a new binary-extension file on the remote backend works as before.

Behaviour notes

  • A binary-extension write may now start the task's sandbox a little earlier: the guard has to reach the target filesystem before later guards (approval, mirror) run. This affects cost and latency only; the sandbox would be created on the next call anyway.
  • When a probe's output is polluted by shell noise, the refusal can say "existing binary file" for what is really a new file. It still refuses, as before; only the wording is off.

Validation

  • tests/tools/test_remote_binary_write_guard.py: 2 test functions, 7 cases, all pass on the rebased head.
    • An existing remote-only binary is refused and left byte-identical through write_file, patch and V4A, including an environment class with no name hint.
    • A new remote binary is created.
    • An unreachable or erroring transport fails closed.
  • With the guard changed to treat missing as existing (file copy, restored after), 6 of the 7 cases fail.
  • tests/tools/test_file_tools.py passes too (48 passed, 2 skipped together).
  • Probe (isolated HOME): an existing remote .sqlite is refused and intact, a new remote file is created, and both host cases behave as on main.

Not included

  • _stale_overwrite_blocker still checks the host (Path(resolved).exists()), so on remote backends a remote-only text file with no read baseline passes the stale-overwrite guard. Same bug class, but a different guard; it's left for a follow-up.

  • Absence isn't enforced at write time (andrexibiza's P2 on fix(file-tools): decide binary overwrite checks on the execution target filesystem #123293). The guard's "absent" answer is a snapshot taken before the write. If another backend process creates the file between the probe and the write, the write can still replace it. This race predates this PR, and closing it needs a backend-enforced create-if-absent write, so it's left for a follow-up.

Credit

@JoaoMarcos44's commit from #123293 is kept with authorship. @Liuzikaii submitted the first fix (#122663) and filed #122662, and is credited with a Co-authored-by trailer.

JoaoMarcos44 and others added 6 commits September 26, 2026 22:03
…et filesystem

_check_binary_document_write stat'ed the controller host only, so a binary
(.sqlite/.pdf/...) that existed solely in the task's execution target
(Docker/SSH/... namespace) was treated as a new file and destroyed by a
plain-text write_file/patch that even reported verified:true (NousResearch#122662).

The existence decision now goes through one tri-state helper
(_target_regular_file_state: exists/absent/unavailable) that asks the LIVE
file-ops layer - the same backend the write executes on. Host-backed envs
keep today's Path.is_file semantics (OSError -> proceed); other backends are
probed via ShellFileOperations._probe_regular_file, whose missing/not_regular
answers prove absence while every other status fails closed with a retry
message. Locality comes from the live environment object
(_file_ops_uses_host_paths), never env_type strings or class-name hint tables
(VercelSandboxEnvironment is unclassified there). The PDF and generic binary
refusal messages are unchanged verbatim; opaque-document and SQLite-sidecar
unconditional refusals are untouched.

_stale_overwrite_blocker keeps its host-only probe on purpose: remote reads
never record a full_write_baseline (version stability requires host metadata),
so converting it would refuse every remote overwrite of a file the task fully
read. Tracked as follow-up.

(cherry picked from commit 8b5eb26)
_probe_regular_file returns "bad_size" only after `[ -f ]` succeeded and
`wc -c` output was unparseable, so a regular file IS present on the
execution target. Treat it as "exists" (the precise existing-binary
refusal) instead of the generic "unavailable" retry message; both refuse,
but the retry hint is wrong for a file that is known to exist. Mirrors
how the other _probe_regular_file callers treat bad_size.

Idea and the original report/Docker reproduction come from NousResearch#122663, the
first submitted fix for this bug.

Fixes NousResearch#122662

Co-authored-by: liuzikaii <2319582736@qq.com>
Keep the two contracts that pin NousResearch#122662: (1) write_file / patch replace /
V4A update against a binary that exists only on an unhinted non-local
backend (VercelSandboxEnvironment, scoped config claiming 'local') is
refused with bytes untouched, for .sqlite and .pdf; (2) a probe transport
failure (raise or error return) fails closed. Creation-allowed, locality
and tilde-fallback cases are covered by the first invariant's setup or by
tests/tools/test_binary_document_write_guard.py, and the salvage shape
caps a stack at two invariant tests.
…ntract

The helper docstring restated the inline why-comments (exact probe string,
backend $HOME tilde fallback, which statuses prove absence). Keep only the
contract callers need: three return values and that anything not proven
absent is "unavailable" and must fail closed. Inline comments stay.
The substitution pairs were built by looping over a one-element tuple (left
over from a multi-pair version), and _bash_safe() only existed to do a
function-local import from a module already imported at the top. Import
_bash_safe_path directly and build the three pairs as a literal list.
The remote guard contract only pinned refusal of an existing target-side
binary. Nothing pinned the other half: a path the backend proves absent must
stay writable, or a guard that over-reports "exists" (e.g. treating a
"missing" probe as present) would block every new .pdf/.sqlite on Docker/SSH
without a red test. Extend the kept T1 test instead of adding a function
(test budget); verified it fails 6/6 under a missing->exists mutation.
@kshitijk4poor
kshitijk4poor enabled auto-merge (rebase) September 26, 2026 16:43
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists tool/file File tools (read, write, patch, search) labels Sep 26, 2026
@kshitijk4poor
kshitijk4poor merged commit 952f65c into NousResearch:main Sep 26, 2026
48 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Medium — degraded but workaround exists tool/file File tools (read, write, patch, search) type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(file-tools): host-only binary guard allows destructive remote overwrites

3 participants