Skip to content

fix(search): zero-match probes return the file paths they found, not just counts - #80525

Merged
teknium1 merged 2 commits into
mainfrom
fix/zero-match-probe-paths
Aug 17, 2026
Merged

teknium1 merged 2 commits into
mainfrom
fix/zero-match-probe-paths

Conversation

@teknium1

@teknium1 teknium1 commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

The zero-match search probes now include the file paths they already found instead of hint-only counts — ending the weak-model re-search spiral measured in the A/B eval (qwen3-coder-30b: 3.3 → 9.3 turns on err_case_search because the probe said "2 case-insensitive matches exist" but not where).

Root cause: all three probes (case-insensitive, hidden/gitignored, literal-vs-regex) ran the widened rg --count-matches search, then discarded the per-file paths from its own output and returned only totals.

Closes #80522.

Changes

  • tools/file_operations.py: shared _tally() helper parses path:count lines once; all three probe warnings now carry up to 5 matched paths (+N more beyond that). Fixes the class, not one probe.
  • tests/tools/test_search_zero_match_and_multipath.py: path-presence assertions on all three probes + a cap test (8 files → 5 shown, +3 more).

Validation

check result
targeted tests (real rg) 17/17 pass
sabotage run (old code + new tests) 4 fail, as they must
E2E (real search_tool, temp HERMES_HOME) both probes return paths

Infographic

Search probes now hand over the paths

…just counts

The casing/hidden/literal probes already ran the widened search to produce
their counts, then threw away the paths and returned a hint-only warning.
Strong models pivot in one turn; weak models spiral — the A/B eval measured
qwen3-coder-30b going 3.3 -> 9.3 turns on err_case_search, retrying casing
variants the probe had already resolved.

All three probes (case-insensitive, hidden/gitignored, literal-vs-regex) now
include up to 5 matched paths (+N more) in the warning via a shared tally
helper. Fixes the class, not the site.

Closes #80522
@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/tools Tool registry, model_tools, toolsets tool/file File tools (read, write, patch, search) labels Aug 6, 2026
@github-actions

github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

૮ >ﻌ< ა ci review

ran on 607335f — chore: re-trigger CI

⚠️ Warnings

CI timings · View report · View job

Wall time 6m44s vs 3m29s (+93.3%). 14 job(s) slower, 9 faster, 1 unchanged.

  • Python tests / Run tests slice 2/12: -39.0s
  • Python tests / Run tests slice 1/12: -33.0s
  • Detect affected areas: +33.0s
  • Python tests / Run tests slice 12/12: -26.0s
  • Python tests / Run tests slice 9/12: +18.0s

OSV vulnerability scan · View job

5 known vulnerabilities found in pinned dependencies.

How to fix:

Review the findings in the Security tab. Update the affected dependencies if a patched version is available.

@teknium1
teknium1 merged commit a35625d into main Aug 17, 2026
45 checks passed
@teknium1
teknium1 deleted the fix/zero-match-probe-paths branch August 17, 2026 05:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/tools Tool registry, model_tools, toolsets P3 Low — cosmetic, nice to have tool/file File tools (read, write, patch, search) type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: zero-match casing probe is hint-only — withholds the paths it already found, sends weak models into re-search spirals (measured +6 turns)

2 participants