fix(sdk): improve grep literal guidance and sandbox glob routing

[Mason Daugherty (mdrxy)]

• grep now warns when a no-match pattern looks like a regex, since the tool only matches literal text.
• The grep tool description and parameter docs now state up front that the pattern is literal, with explicit do-nots and a pointer to execute + rg for genuine regex.
• Slash-containing globs (e.g. src/**/*.py) now work in sandbox backends — previously they silently returned zero results because GNU grep --include only matches basenames.
• The grep tool description is now reconciled at request time against the backend's actual execute capability, so the rg fallback line appears only when execute is available.
• Three new retrieval evals verify that a model recovers after a regex-style grep miss.
• Default grep and glob timeouts are halved (grep 30s → 15s, glob 10s → 5s backend / 20s → 10s middleware) so bad patterns and large trees fail faster.

---

The grep tool matches literal text — FilesystemBackend shells out to ripgrep with -F, and the in-memory backends do a plain substring check. Models routinely reach for regex anyway (| alternation, .*, escapes like \.), which is matched verbatim and silently returns "No matches found" even when the content exists. This PR adds two complementary, non-breaking mitigations: a sharper tool description that sets expectations before the call, and a post-call hint that steers the model back to literal searches after a miss.

It also fixes a sandbox backend bug where slash-containing globs silently returned nothing, makes the grep description self-adjust to the backend's execute capability at request time, and halves the default wall-clock timeouts so that a pathological pattern or a huge tree fails faster.

User-facing behavior changes

• The grep tool description now leads with "LITERAL text pattern across files (NOT regex)" and lists explicit do-nots: no | alternation, no .* wildcards, no \. escapes. When the execute tool is active for the backend, it also points to execute with rg for genuine regex needs; that line is dropped automatically when execute is unavailable, so the description never suggests a tool the model can't call.

  • Example: a model that previously called grep(pattern="foo|bar") and got "No matches found" should now read the tool description and run two separate calls — grep(pattern="foo") and grep(pattern="bar") — instead.

• When grep returns no matches and the pattern carries strong regex signals, the tool response now appends a one-line hint:

  Note: grep matches literal text, not regex, so characters like `|`, `.*`, and `\.` are searched verbatim. Search for the literal text you need instead; for `|` alternation, run a separate search per alternative.
  

  The hint stays backend-agnostic (it never names the execute tool, which is not present in every configuration) and only appears on a genuine no-match result. If the pattern happens to match literally (e.g. a file contains a = b|c), or if matches existed but were all removed by read permissions, no hint is shown.

• The glob parameter description on GrepSchema now clarifies that it is an in-tool file filter (not a call to the separate glob tool), spells out the matching semantics (a pattern without / matches the file name at any depth; a pattern containing / matches the search-root-relative path, e.g. src/**/*.py), and notes that brace expansion (e.g. *.{ts,tsx}) is not supported on all backends — the wcmatch-backed backends expand braces, but the Context Hub backend treats them literally, so running a separate search per extension is the reliable choice.

• The output_mode parameter description now spells out the exact output shape for each mode instead of a terse one-liner: files_with_matches returns newline-separated paths, content returns matching lines grouped by file under a <path>: header with indented <line_number>: <line text> lines (no surrounding context), and count returns one <path>: <match_count> line per file.

• Sandbox backends (BaseSandbox) now route slash-containing globs through an in-process Python glob instead of GNU grep --include. Previously, a pattern like src/**/*.py was passed to grep --include=src/**/*.py, which only matches basenames and silently returned zero results. The new _GREP_PATH_GLOB_TEMPLATE resolves the glob relative to the search root, reads matching files directly, and emits the same path\0line_num:text record structure that _parse_grep_output already consumes, so the rest of the pipeline is unchanged. Basename-only globs (no /) continue through GNU grep for speed.

• The grep tool description is now a placeholder that is reconciled at request time. _create_grep_tool provisions the description assuming execute is available (so the rg fallback line is present). _filter_unsupported_tools_and_apply_prompt then checks whether execute is actually active for the backend and swaps in the without-execute variant via _with_filtered_grep_description when it isn't. The static description on self.tools is only provisional until a request runs. The unsupported-tools logic was extracted into _unsupported_tools_and_execution_state and _tool_name helpers to support this.

• Default filesystem tool timeouts are halved:

  • grep sync phase: 30s → 15s
  • grep async wrapper: 65s → 35s (wraps two sync phases plus headroom)
  • glob backend walk: 10s → 5s
  • glob middleware deadline: 20s → 10s

  The layering invariants are preserved: the backend glob budget stays below the middleware deadline (5s < 10s), and the async grep timeout stays above two sync phases (35s > 30s). read_file has no timeout (single direct read, no search or tree walk) and is unchanged.

How regex detection works

Detection is deliberately conservative so common literal code searches don't trigger false hints. The heuristic flags:

• | — alternation, the most common regex reflex
• .* or .+ — wildcard quantifiers
• Escaped metacharacters or character classes: \. , \w, \d, \s, \b, etc.

Bare ., (, ), [, ], ?, ^, $ are intentionally not flagged because they appear routinely in literal code searches:

def __init__(self):    # ( ) : are literal
self.tools              # . is literal
arr[0]                  # [ ] are literal

Motivation

A silent "No matches found" for a regex-style pattern is a common failure mode: the content exists, the model just asked for it the wrong way and gets no signal that the tool doesn't speak regex. The tool description fix addresses the problem before the call by making the literal-only contract unmistakable. The hint addresses it after the call by catching the model when it still reaches for regex, steering it toward separate literal searches instead of burning calls on regex variants. (The rg escape hatch lives in the always-visible tool description rather than the hint, so the hint stays useful even on backends without an execute tool.)

The sandbox glob fix addresses a quieter but equally confusible failure: a model that passes glob="src/**/*.py" to grep on a sandbox backend got an empty result with no indication that the pattern was the problem — the files existed and the pattern was valid, but GNU grep --include only matches basenames. The in-process glob route makes slash-containing patterns work as documented.

The dynamic description swap ensures the rg fallback line never appears for a backend that can't serve execute, preventing the model from being directed to a tool it can't call.

The timeout reduction addresses latency: the original values (30s grep, 20s glob) were chosen independently based on each operation's cost profile. Halving them lets the model receive a "narrow your search" signal sooner, reducing wasted wall-clock time on pathological patterns or huge directory trees.

[codspeed-hq]

Merging this PR will improve performance by 24.37%

⚡ 1 improved benchmark
✅ 20 untouched benchmarks
⏩ 79 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	WallTime	test_filesystem_init	490.4 µs	394.3 µs	+24.37%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing mdrxy/sdk/grep-literal-regex-hint (f5e9de3) with main (20107ed)}

Footnotes

1. 79 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[open-swe]

✅ Open SWE Review: No issues found

Open SWE reviewed this PR and found no potential bugs to report.

Open in Web • View Open SWE trace

[github-actions]

ℹ️ PR scope/file mismatch acknowledged via the allow-scope-mismatch label.

Title scope(s): sdk

Touched package dir(s) not covered by those scopes:

• package label evals from libs/evals/

Remove the label to re-enable the block.

[open-swe]

Open SWE Review found 1 potential issue.

Open in Web • View Open SWE trace

[open-swe]

Open SWE Review found 1 potential issue.

Open in Web • View Open SWE trace

[open-swe]

Open SWE Review found 1 potential issue.

Open in Web • View Open SWE trace

[Mason Daugherty (mdrxy)]

Eval results — grep regex-recovery (baseline tier)

Ran the three grep regex-recovery evals (| alternation, .* wildcard, \. escaped metachar) against four models, 3 trials each. All 36 runs passed — 100% correctness with perfect step and tool-call ratios.

Model	Correctness	Solve Rate	Step Ratio	Tool Call Ratio	Median Duration
claude-sonnet-4-5	1.00 ±0.00	0.568 ±0.028	1.00 ±0.00	1.00 ±0.00	3.72s
gpt-5.5	1.00 ±0.00	0.640 ±0.093	1.00 ±0.00	1.00 ±0.00	3.07s
glm-5p2 (fireworks)	1.00 ±0.00	0.648 ±0.112	1.00 ±0.00	1.08 ±0.14	3.48s
claude-opus-4-8	1.00 ±0.00	0.483 ±0.027	1.00 ±0.00	1.00 ±0.00	4.09s

Every model recovers after a regex-style grep miss, finds the right files, and reports the correct paths. The minor tool-call-ratio variance on glm-5p2 (1.08) reflects an occasional extra call that still succeeds.

Tier reclassification

Moved these three evals from hillclimb → baseline in f5e9de354. They pass deterministically across all tested models with perfect step/tool-call ratios — they behave as regression gates for the literal-grep regex hint feature, not progress signals.