Skip to content

perf(core): Optimize chunk merging and avoid redundant string split in grep tool#1303

Merged
yamadashy merged 1 commit intomainfrom
perf/lazy-load-minimatch
Mar 28, 2026
Merged

perf(core): Optimize chunk merging and avoid redundant string split in grep tool#1303
yamadashy merged 1 commit intomainfrom
perf/lazy-load-minimatch

Conversation

@yamadashy
Copy link
Copy Markdown
Owner

@yamadashy yamadashy commented Mar 26, 2026

Summary

Performance optimizations for string operations in tree-sitter parsing and MCP grep tool.

Changes

  • Optimize mergeAdjacentChunks (parseFile.ts): Replace string += concatenation with array accumulation + join( ) to avoid O(k²) string copying when merging adjacent code chunks.
  • Avoid redundant content.split( ) (grepRepomixOutputTool.ts): Extract searchInLines function that accepts pre-split lines array. performGrepSearch now splits content once and reuses the array for both search and formatting, avoiding a redundant O(n) split on large output files (3-5MB).

Checklist

  • Run npm run test
  • Run npm run lint

@coderabbitai

This comment has been minimized.

@gemini-code-assist

This comment has been minimized.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 26, 2026

⚡ Performance Benchmark

Latest commit:2fac8d8 perf(core): Optimize chunk merging and avoid redundant string split in grep tool
Status:✅ Benchmark complete!
Ubuntu:2.25s (±0.03s) → 2.25s (±0.03s) · +0.00s (+0.0%)
macOS:1.21s (±0.09s) → 1.15s (±0.05s) · -0.05s (-4.5%)
Windows:2.70s (±0.04s) → 2.67s (±0.01s) · -0.03s (-1.2%)
Details
  • Packing the repomix repository with node bin/repomix.cjs
  • Warmup: 2 runs (discarded)
  • Measurement: 10 runs / 20 on macOS (median ± IQR)
  • Workflow run
History

3b22907 perf(core): Remove Zod from startup path and optimize output generation

Ubuntu:2.20s (±0.03s) → 2.19s (±0.01s) · -0.01s (-0.2%)
macOS:1.18s (±0.08s) → 1.24s (±0.17s) · +0.06s (+5.4%)
Windows:2.62s (±0.15s) → 2.59s (±0.05s) · -0.03s (-1.3%)

4b68828 perf(core): Remove Zod from startup path and optimize output generation

Ubuntu:2.26s (±0.01s) → 2.27s (±0.02s) · +0.00s (+0.1%)
macOS:1.21s (±0.07s) → 1.22s (±0.06s) · +0.01s (+1.1%)

1d1de0b perf(core): Remove Zod from startup path and optimize output generation

Ubuntu:2.41s (±0.03s) → 2.42s (±0.06s) · +0.02s (+0.7%)
macOS:1.28s (±0.20s) → 1.23s (±0.13s) · -0.05s (-4.1%)
Windows:3.04s (±0.07s) → 3.04s (±0.07s) · -0.01s (-0.3%)

05712eb perf(core): Remove Zod from startup path and optimize output generation

Ubuntu:2.37s (±0.02s) → 2.39s (±0.06s) · +0.02s (+0.8%)
macOS:1.75s (±0.52s) → 1.57s (±0.56s) · -0.19s (-10.6%)
Windows:3.21s (±0.59s) → 3.26s (±0.33s) · +0.06s (+1.8%)

gemini-code-assist[bot]

This comment was marked as resolved.

@claude

This comment has been minimized.

@codecov
Copy link
Copy Markdown

codecov bot commented Mar 26, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.13%. Comparing base (aeed190) to head (2fac8d8).
⚠️ Report is 10 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1303      +/-   ##
==========================================
+ Coverage   87.10%   87.13%   +0.02%     
==========================================
  Files         115      115              
  Lines        4358     4367       +9     
  Branches     1014     1015       +1     
==========================================
+ Hits         3796     3805       +9     
  Misses        562      562              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages bot commented Mar 26, 2026

Deploying repomix with  Cloudflare Pages  Cloudflare Pages

Latest commit: 2fac8d8
Status: ✅  Deploy successful!
Preview URL: https://9144e5e2.repomix.pages.dev
Branch Preview URL: https://perf-lazy-load-minimatch.repomix.pages.dev

View logs

Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 3 additional findings.

Open in Devin Review

@yamadashy yamadashy marked this pull request as draft March 26, 2026 13:48
@yamadashy yamadashy force-pushed the perf/lazy-load-minimatch branch 3 times, most recently from 4b68828 to 3b22907 Compare March 28, 2026 05:29
@yamadashy yamadashy marked this pull request as ready for review March 28, 2026 05:44
…n grep tool

- Replace string += with array accumulation + join('\n') in mergeAdjacentChunks
  to avoid O(k²) copying when merging adjacent tree-sitter code chunks
- Extract searchInLines from searchInContent in grepRepomixOutputTool so
  performGrepSearch splits content once and reuses the lines array for both
  search and formatting, avoiding a redundant O(n) split on large files
@yamadashy yamadashy force-pushed the perf/lazy-load-minimatch branch from 3b22907 to 2fac8d8 Compare March 28, 2026 06:15
@yamadashy yamadashy changed the title perf(core): Lazy-load minimatch and simplify permission check perf(core): Optimize chunk merging and avoid redundant string split in grep tool Mar 28, 2026
@yamadashy yamadashy merged commit a801b60 into main Mar 28, 2026
61 checks passed
@yamadashy yamadashy deleted the perf/lazy-load-minimatch branch March 28, 2026 06:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant