Skip to content

fix(ssh): avoid remote tar metadata failures during file sync - #13955

Closed
zwcf5200 wants to merge 1 commit into
NousResearch:mainfrom
zwcf5200:contrib/ssh-filesync-tar-metadata
Closed

fix(ssh): avoid remote tar metadata failures during file sync#13955
zwcf5200 wants to merge 1 commit into
NousResearch:mainfrom
zwcf5200:contrib/ssh-filesync-tar-metadata

Conversation

@zwcf5200

Copy link
Copy Markdown
Contributor

Summary

Fix SSH bulk file sync so tar extraction does not fail when remote tar tries to restore metadata for parent directories such as /, /home, or ..

The previous bulk upload archived the whole staging root with tar ... -C <staging> . and extracted it at / on the remote host. On macOS, bsdtar can also emit libarchive xattr metadata. In SSH profiles this produced warnings/errors like:

file_sync: sync failed, rolled back state: tar extract over SSH failed
tar: ./home: cannot utime: Operation not permitted
tar: .: cannot utime: Operation not permitted
tar: Ignoring unknown extended header keyword LIBARCHIVE.xattr...

Changes

  • Archive only the explicit file entries using a tar manifest instead of archiving the staging root.
  • Keep the existing symlink staging approach for preserving remote paths.
  • Disable macOS archive metadata with --no-xattrs and COPYFILE_DISABLE=1.
  • Add regression coverage for manifest-based SSH bulk upload.

Related

Related to #10205. That PR scopes extraction to .hermes; this change additionally avoids archiving parent directory entries and suppresses macOS xattr metadata that can break remote extraction.

Testing

  • source venv/bin/activate && scripts/run_tests.sh tests/tools/test_ssh_bulk_upload.py tests/tools/test_file_sync.py
  • source venv/bin/activate && uv run --active ruff check tools/environments/ssh.py tests/tools/test_ssh_bulk_upload.py
  • Manual SSH verification against a remote profile: terminal_tool("pwd") returned /home/ubuntu/workspace with no file_sync: sync failed tar/utime warning.

Upload SSH file-sync archives from an explicit manifest instead of archiving the staging root. This avoids extracting directory entries like / and /home on the remote host, which can fail when tar tries to restore mtime or mode.

Disable local macOS archive metadata with --no-xattrs and COPYFILE_DISABLE=1 so bsdtar does not emit LIBARCHIVE.xattr headers that remote tar reports as warnings.
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists backend/ssh SSH remote execution backend/file-sync File sync across remote backends labels Apr 22, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Related to #10205 (SSH bulk sync extraction scope). This PR additionally avoids archiving parent dir entries and suppresses macOS xattr metadata.

@alt-glitch

Copy link
Copy Markdown
Collaborator

Related to #10205

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend/file-sync File sync across remote backends backend/ssh SSH remote execution P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants