Skip to content

fix(file_ops): preserve native Windows paths for ripgrep - #83363

Closed
Dolverin wants to merge 6 commits into
NousResearch:mainfrom
Dolverin:fix/windows-rg-native-rg-paths
Closed

fix(file_ops): preserve native Windows paths for ripgrep#83363
Dolverin wants to merge 6 commits into
NousResearch:mainfrom
Dolverin:fix/windows-rg-native-rg-paths

Conversation

@Dolverin

@Dolverin Dolverin commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Bug Description

On native Windows with the Git Bash local backend, search_files can fail for absolute paths in two ways:

  • content search: native rg.exe receives an MSYS path such as /c/Users/... and fails with IO error ... (os error 3);
  • file search: the same path shape can degrade to a silent total_count: 0, which looks like a legitimate no-match result.

Regex patterns are affected by the same boundary: backslash escapes such as \d, \w, and \( must not be rewritten as if they were Windows path separators.

Fixes #67629
Fixes #63177

Root Cause

_escape_shell_arg() routes arguments through _bash_safe_path(), which intentionally rewrites C:\... / C:/... to Git Bash form (/c/...) for bash and MSYS-aware tools.

That is correct for bash builtins and the grep/find paths, but not for a Win32 rg.exe: Hermes deliberately sets MSYS_NO_PATHCONV=1 and MSYS2_ARG_CONV_EXCL=* to stop MSYS from mangling native command switches, so no later layer converts /c/... back before native ripgrep receives it.

The same path-oriented rewriting must not be applied to search patterns: a regex is not a path.

Fix

This PR is a current-main salvage/rebase of #67914, preserving the original commits and attribution, plus one focused follow-up for the zero-match probe sibling call sites.

  • Add _native_windows_path_for_exe() for native Windows executables.
  • Add _escape_native_exe_arg() and use it for native rg path arguments.
  • Add _escape_pattern_arg() so search patterns are shell-quoted without MSYS path rewriting.
  • Use native paths for rg --files, content search, and the three zero-match rg probes.
  • Keep grep path arguments on the existing MSYS-aware escaping.
  • Gate native path conversion to the local Windows backend so SSH/Docker/Modal/Daytona targets keep their own POSIX namespaces (/mnt/d/... must not become D:/... on a remote target).
  • Preserve drive-relative inputs such as D:logs/app.txt instead of turning them into drive-rooted paths.
  • Add explicit -e / -- argument boundaries for rg/grep searches and zero-match probes so option-like patterns such as -foo remain data.

Salvage / Attribution

  • Salvages fix(file_ops): keep native Windows paths for rg search (#67629) #67914 by cherry-picking the three original commits onto current main.
  • Preserves Bartok9's commit authorship.
  • Preserves the Co-authored-by: rille111 trailer on the consolidated pattern-handling commit.
  • Adds one separate follow-up commit for _zero_match_probe(), which used the same incorrect path/pattern escaping for native rg.

How to Verify

On native Windows with Git Bash and a Win32 ripgrep build:

  1. Create a fixture outside ignored/commented roots, for example C:\Temp\hermes-rg-live-verify\subdir\sample.txt containing needle-windows-rg-path-fix.
  2. Run file and content searches with all three path spellings:
    • C:\Temp\hermes-rg-live-verify
    • C:/Temp/hermes-rg-live-verify
    • /c/Temp/hermes-rg-live-verify
  3. Confirm both target='files' and target='content' return the sample file instead of os error 3 or silent zero results.
  4. For zero-match steering, verify that a casing miss, a regex-metachar literal, and a hidden-only match each produce the intended warning instead of a bare zero.

Test Plan

  • Regression coverage added for native path conversion, local-vs-remote backend boundaries, pattern preservation, zero-match probes, drive-relative paths, and option-like patterns.
  • Focused tests: scripts/run_tests.sh tests/tools/test_file_operations.py tests/tools/test_local_env_windows_msys.py -k "native_windows or escape_native or search_with_rg_uses or escape_pattern or preserves_regex or native_windows_path_for_exe or zero_match_probe_uses or drive_relative or argument_boundaries" -q → 15 passed.
  • Sabotage run: restoring only the two production files from origin/main makes the new regression tests fail; restoring the fix makes them pass.
  • Relevant local Windows suite: 116 passed / 18 failed on this branch versus 103 passed / 21 failed on an origin/main baseline worktree. The remaining failing test names are pre-existing baseline failures tied to Windows symlink privileges and this host's global-ignore/Temp environment; no new failing test names were introduced.
  • ruff check tools/environments/local.py tools/file_operations.py tests/tools/test_file_operations.py tests/tools/test_local_env_windows_msys.py → passed.
  • git diff --check → passed.
  • Manual native Windows live verification for C:\..., C:/..., and /c/... path forms.

Risk Assessment

Low / Medium — the change is confined to search-command argument construction. Bash/MSYS-aware file operations keep _escape_shell_arg(), grep paths keep MSYS form, and native Windows path conversion is limited to the local backend. CI should cover the non-Windows no-op paths.

@yuzilongleif-collab

Copy link
Copy Markdown
Contributor

I found two argument-boundary cases on exact head 9a398460087b441d5fdfcb381218c5851afcb4e4.

1. Drive-relative paths become drive-rooted

_native_windows_path_for_exe() currently rewrites a drive-relative path:

source = r"D:logs\app.txt"       # relative to D:'s current directory
actual = _native_windows_path_for_exe(source)
assert actual == "D:logs/app.txt"
assert not ntpath.isabs(actual)

Actual: D:/logs/app.txt, which is drive-rooted. This contradicts the helper contract that relative paths are left unchanged. The drive-qualified regex needs to distinguish D:foo from D:/foo / D:\foo.

2. A regex beginning with - is parsed as an rg option

The changed rg command construction still emits the pattern positionally, e.g.:

rg ... '-foo' 'D:/dir with space'

Shell quoting does not terminate option parsing. Independently verified with ripgrep 14.1.0:

printf '%s\n' '-foo' | rg '-foo'       # exit 2: rg: oo: No such file or directory
printf '%s\n' '-foo' | rg -- '-foo'    # exit 0

_search_with_rg() and _zero_match_probe() both need an argument boundary (-- before the pattern, or preferably -e <pattern>). This second case appears adjacent/pre-existing rather than introduced by the Windows path conversion, but it is in the command-building surface being changed here.

I added two minimal probes against the exact head; both fail as described. The main absolute native-path and backslash-regex cases remain unaffected by these findings.

@Dolverin

Copy link
Copy Markdown
Contributor Author

Thanks @yuzilongleif-collab — both boundary cases are addressed in ba5d231aa.

  • Drive-relative input such as D:logs�pp.txt now remains drive-relative; only slash-rooted drive paths normalize to D:/....
  • rg/grep content searches and the zero-match probes now pass patterns via -e and insert -- before the search path, so -foo remains data.
  • Added regression coverage for both cases. Focused tests are 15/15, and a native Windows live search for -foo returns the expected hit without an option-parse error.

Bartok9 and others added 6 commits August 10, 2026 20:46
…67629)

Closes NousResearch#67629

Root cause: search_files path args went through _bash_safe_path which
rewrites D:\... to /d/... for Git Bash, but the Win32 ripgrep binary does
not resolve MSYS drive paths (os error 3).

Fix: add _native_windows_path_for_exe + _escape_native_exe_arg and use
that for rg path args so drive paths stay as D:/... while bash still
launches the pipeline.

Verification: pytest tests/tools/test_local_env_windows_msys.py
tests/tools/test_file_operations.py -k "native_windows or escape_native or search_with_rg_uses"

(cherry picked from commit 0761f4c)
The native drive rewrite in _escape_native_exe_arg applied whenever the
host OS was Windows, regardless of the execution backend. A non-local
backend (SSH, Docker, Modal, Daytona) owns its own path namespace, so a
valid target-side path like /mnt/d/project was reinterpreted as a host
Windows drive form (D:/project), breaking remote rg lookups
(reported on NousResearch#67914).

Restrict the conversion to _IS_WINDOWS AND the local backend via a new
_is_local_backend() helper. Non-local backends keep their own path
semantics. Adds a regression test asserting /mnt/d/... and plain POSIX
paths pass through untouched for non-local backends.

(cherry picked from commit dbce6c7)
Add _escape_pattern_arg and use it for rg + grep patterns so backslash
regex escapes (\w, \d, \() are preserved and not mangled into forward
slashes on Windows. Consolidates the pattern portion of NousResearch#69183 per the

Co-authored-by: rille111 <rille111@users.noreply.github.com>
(cherry picked from commit b012329)
The zero-match near-miss probes are native rg invocations too. Use the same local-Windows native path escaping and quote-only pattern handling as the primary rg search so case/literal/hidden hints do not silently disappear under MSYS_NO_PATHCONV.

Regression coverage captures all three probe commands.
Preserve drive-relative Windows paths by only normalizing slash-rooted drive paths.

Pass search patterns with -e and insert -- before search paths in rg/grep content searches and zero-match probes so option-like patterns remain data.
@Dolverin
Dolverin force-pushed the fix/windows-rg-native-rg-paths branch from ba5d231 to db2d3d3 Compare August 10, 2026 18:47
@yuzilongleif-collab

Copy link
Copy Markdown
Contributor

Verified on current exact head db2d3d34b31c9a144f7c8dd6a25c1876c24a708d.

  • D:logs\app.txt now remains drive-relative as D:logs/app.txt.
  • The rg / grep search and zero-match paths now use -e <pattern> plus -- <path>, so -foo cannot be parsed as an option.
  • Focused local verification: 7 passed across the native-path and option-like-pattern regressions.

Both findings from my prior comment are resolved; I have no remaining concern on those two boundaries.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists tool/file File tools (read, write, patch, search) backend/local Local shell execution platform/windows Native Windows-specific behavior or breakage sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows duplicate This issue or pull request already exists labels Aug 10, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of #67914: it already covers the local native-rg path conversion and protects remote backend path namespaces.

@alt-glitch alt-glitch removed the duplicate This issue or pull request already exists label Aug 10, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Correction: #83363 is the current-main salvage that supersedes closed #67914, carrying its path/pattern consolidation and adding zero-match coverage. It is related prior art, not a duplicate; the stale duplicate disposition has been removed.

@teknium1

Copy link
Copy Markdown
Contributor

Closing as implemented on main: #84378 (merged) covers this fix — a shared native-path escaper (_escape_native_tool_arg) applied to all six rg call sites, the zero-match probes, and the shell linter interpolation, with Windows regression tests. Thank you for diagnosing this — your PR correctly identified the same root cause (native binaries + MSYS_NO_PATHCONV means /c/... is never translated back), and the merged fix lands the same approach across the full call-site set. Sorry we couldn't land this one directly.

@teknium1 teknium1 closed this Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend/local Local shell execution P2 Medium — degraded but workaround exists platform/windows Native Windows-specific behavior or breakage sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows tool/file File tools (read, write, patch, search) type/bug Something isn't working

Projects

None yet

5 participants