Speed up mixed-line-ending scans with memchr2

[j178]

Use memchr2 to skip non-line-ending bytes when counting and normalizing line endings, while continuing to treat CRLF as one ending.

Performance

Measured on an Apple M3 Pro with macOS 15.7.7 and rustc 1.97.0. Both origin/master (6c898499) and this change (64daab4d) were built with --profile profiling, then compared with hyperfine using:

prek run mixed-line-ending --all-files

The table uses pooled medians from two order-reversed runs to reduce ordering bias:

Workload	Runs per binary	master	This PR	Change
prek repo: 309 files, 3.97 MB	100	44.65 ms	44.70 ms	no measurable change
20 × 1 MiB LF text files, ~83-byte lines	80	120.05 ms	100.54 ms	16.3% faster (1.19×)
20 × 1 MiB mixed LF/CRLF files, including rewrite	60	287.30 ms	274.51 ms	4.5% faster (1.05×)

The normal repository case remains dominated by process startup, Git enumeration, and file I/O. The improvement becomes visible as the amount of text scanned per invocation grows; rewriting sees a smaller gain because output I/O dominates.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[prek-ci-bot]

📦 Cargo Bloat Comparison

Binary size change: +0.00% (29.2 MiB → 29.2 MiB)

Expand for cargo-bloat output

Head Branch Results

 File  .text     Size             Crate Name
 1.1%   2.3% 332.0KiB        aws_lc_sys aws_lc_0_42_0_aes_gcm_encrypt_avx512
 1.1%   2.3% 332.0KiB        aws_lc_sys aws_lc_0_42_0_aes_gcm_decrypt_avx512
 0.2%   0.5%  65.4KiB              prek <prek::cli::Command as clap_builder::derive::Subcommand>::augment_subcommands
 0.2%   0.4%  55.1KiB              prek <<prek::config::Config as serde_core::de::Deserialize>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::<<serde_saphyr::de::deserializer::YamlDeserializer as serde_core::de::Deserializer>::deserialize_map::MA>
 0.2%   0.3%  49.4KiB annotate_snippets annotate_snippets::renderer::render::render
 0.2%   0.3%  47.0KiB              prek prek::run::{closure#0}
 0.1%   0.3%  38.4KiB              prek prek::cli::run::run::run::{closure#0}
 0.1%   0.2%  34.3KiB              prek <prek::cli::RunOptions as clap_builder::derive::Args>::augment_args
 0.1%   0.2%  30.1KiB     granit_parser <granit_parser::scanner::Scanner<granit_parser::input::str::StrInput>>::fetch_more_tokens
 0.1%   0.2%  28.0KiB        aws_lc_sys aws_lc_0_42_0_edwards25519_scalarmuldouble_alt
 0.1%   0.2%  27.5KiB        aws_lc_sys aws_lc_0_42_0_edwards25519_scalarmuldouble
 0.1%   0.2%  27.2KiB              prek <<prek::config::RemoteHook as serde_core::de::Deserialize>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::<<serde_saphyr::de::deserializer::YamlDeserializer as serde_core::de::Deserializer>::deserialize_map::MA>
 0.1%   0.2%  26.2KiB              prek prek::cli::try_repo::try_repo::{closure#0}
 0.1%   0.2%  24.9KiB               xml <xml::reader::parser::PullParser>::dispatch_token
 0.1%   0.2%  24.6KiB              prek <prek::languages::node::installer::NodeInstaller>::install::{closure#0}
 0.1%   0.2%  24.3KiB              prek <<prek::config::LocalHook as serde_core::de::Deserialize>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::<<serde_saphyr::de::deserializer::YamlDeserializer as serde_core::de::Deserializer>::deserialize_map::MA>
 0.1%   0.2%  24.0KiB     granit_parser <granit_parser::scanner::Scanner<serde_saphyr::de::buffered_input::ReaderInput>>::fetch_more_tokens
 0.1%   0.2%  23.7KiB              prek <<prek::config::UpdateOptions as serde_core::de::Deserialize>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::<<serde_saphyr::de::deserializer::YamlDeserializer as serde_core::de::Deserializer>::deserialize_map::MA>
 0.1%   0.2%  22.8KiB              prek prek::archive::unzip::<fs_err::tokio::file::File, &std::path::PathBuf>::{closure#0}
 0.1%   0.2%  22.3KiB         [Unknown] Lp384_montjscalarmul_alt_p384_montjadd
42.3%  88.4%  12.4MiB                   And 25902 smaller methods. Use -n N to show more.
47.9% 100.0%  14.0MiB                   .text section size, the file size is 29.2MiB

Base Branch Results

 File  .text     Size             Crate Name
 1.1%   2.3% 332.0KiB        aws_lc_sys aws_lc_0_42_0_aes_gcm_encrypt_avx512
 1.1%   2.3% 332.0KiB        aws_lc_sys aws_lc_0_42_0_aes_gcm_decrypt_avx512
 0.2%   0.5%  65.4KiB              prek <prek::cli::Command as clap_builder::derive::Subcommand>::augment_subcommands
 0.2%   0.4%  55.1KiB              prek <<prek::config::Config as serde_core::de::Deserialize>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::<<serde_saphyr::de::deserializer::YamlDeserializer as serde_core::de::Deserializer>::deserialize_map::MA>
 0.2%   0.3%  49.4KiB annotate_snippets annotate_snippets::renderer::render::render
 0.2%   0.3%  46.8KiB              prek prek::run::{closure#0}
 0.1%   0.3%  38.4KiB              prek prek::cli::run::run::run::{closure#0}
 0.1%   0.2%  34.3KiB              prek <prek::cli::RunOptions as clap_builder::derive::Args>::augment_args
 0.1%   0.2%  30.1KiB     granit_parser <granit_parser::scanner::Scanner<granit_parser::input::str::StrInput>>::fetch_more_tokens
 0.1%   0.2%  28.0KiB        aws_lc_sys aws_lc_0_42_0_edwards25519_scalarmuldouble_alt
 0.1%   0.2%  27.5KiB        aws_lc_sys aws_lc_0_42_0_edwards25519_scalarmuldouble
 0.1%   0.2%  27.2KiB              prek <<prek::config::RemoteHook as serde_core::de::Deserialize>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::<<serde_saphyr::de::deserializer::YamlDeserializer as serde_core::de::Deserializer>::deserialize_map::MA>
 0.1%   0.2%  25.7KiB              prek prek::cli::try_repo::try_repo::{closure#0}
 0.1%   0.2%  24.9KiB               xml <xml::reader::parser::PullParser>::dispatch_token
 0.1%   0.2%  24.6KiB              prek <prek::languages::node::installer::NodeInstaller>::install::{closure#0}
 0.1%   0.2%  24.3KiB              prek <<prek::config::LocalHook as serde_core::de::Deserialize>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::<<serde_saphyr::de::deserializer::YamlDeserializer as serde_core::de::Deserializer>::deserialize_map::MA>
 0.1%   0.2%  24.0KiB     granit_parser <granit_parser::scanner::Scanner<serde_saphyr::de::buffered_input::ReaderInput>>::fetch_more_tokens
 0.1%   0.2%  23.7KiB              prek <<prek::config::UpdateOptions as serde_core::de::Deserialize>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::<<serde_saphyr::de::deserializer::YamlDeserializer as serde_core::de::Deserializer>::deserialize_map::MA>
 0.1%   0.2%  23.0KiB              prek prek::archive::unzip::<fs_err::tokio::file::File, &std::path::PathBuf>::{closure#0}
 0.1%   0.2%  22.3KiB         [Unknown] Lp384_montjscalarmul_alt_p384_montjadd
42.3%  88.3%  12.4MiB                   And 25905 smaller methods. Use -n N to show more.
47.9% 100.0%  14.0MiB                   .text section size, the file size is 29.2MiB

[prek-ci-bot]

⚡️ Hyperfine Benchmarks

Summary: 0 regressions, 0 improvements above the 10% threshold.

Environment

• OS: Linux 6.17.0-1020-azure
• CPU: 4 cores
• prek version: prek 0.4.10+29 (559b99f 2026-07-23)
• Rust version: rustc 1.97.1 (8bab26f4f 2026-07-14)
• Hyperfine version: hyperfine 1.20.0

CLI Commands

Benchmarking basic commands in the main repo:

prek --version

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base --version	2.2 ± 0.1	2.1	2.4	1.00
prek-head --version	2.3 ± 0.2	2.1	3.2	1.05 ± 0.10

prek list

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base list	9.6 ± 0.2	9.2	10.0	1.00 ± 0.03
prek-head list	9.5 ± 0.2	9.1	10.4	1.00

prek validate-config .pre-commit-config.yaml

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base validate-config .pre-commit-config.yaml	3.3 ± 0.5	3.1	6.7	1.07 ± 0.16
prek-head validate-config .pre-commit-config.yaml	3.1 ± 0.1	3.0	3.3	1.00

prek sample-config

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base sample-config	2.5 ± 0.1	2.4	3.1	1.02 ± 0.05
prek-head sample-config	2.4 ± 0.0	2.4	2.6	1.00

Cold vs Warm Runs

Comparing first run (cold) vs subsequent runs (warm cache):

prek run --all-files (cold - no cache)

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base run --all-files	67.3 ± 1.4	65.2	70.1	1.01 ± 0.02
prek-head run --all-files	66.9 ± 0.9	65.5	68.4	1.00

prek run --all-files (warm - with cache)

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base run --all-files	66.9 ± 1.1	65.7	70.2	1.00
prek-head run --all-files	67.4 ± 2.2	65.1	74.1	1.01 ± 0.04

Full Hook Suite

Running the builtin hook suite on the benchmark workspace:

prek run --all-files (full builtin hook suite)

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base run --all-files	67.4 ± 1.4	64.5	70.8	1.01 ± 0.03
prek-head run --all-files	66.6 ± 1.2	63.9	70.0	1.00

Individual Hook Performance

Benchmarking each hook individually on the test repo:

prek run trailing-whitespace --all-files

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base run trailing-whitespace --all-files	20.7 ± 0.5	19.9	22.2	1.02 ± 0.03
prek-head run trailing-whitespace --all-files	20.2 ± 0.4	19.6	21.3	1.00

prek run end-of-file-fixer --all-files

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base run end-of-file-fixer --all-files	19.8 ± 0.6	19.0	21.2	1.00
prek-head run end-of-file-fixer --all-files	19.9 ± 0.5	19.2	21.3	1.00 ± 0.04

prek run check-json --all-files

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base run check-json --all-files	7.8 ± 0.2	7.5	8.4	1.02 ± 0.05
prek-head run check-json --all-files	7.7 ± 0.3	7.3	8.6	1.00

prek run check-yaml --all-files

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base run check-yaml --all-files	7.6 ± 0.1	7.4	7.9	1.00
prek-head run check-yaml --all-files	7.6 ± 0.3	7.4	8.8	1.00 ± 0.04

prek run check-toml --all-files

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base run check-toml --all-files	7.8 ± 0.3	7.2	8.8	1.02 ± 0.05
prek-head run check-toml --all-files	7.6 ± 0.2	7.2	8.3	1.00

prek run check-xml --all-files

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base run check-xml --all-files	7.5 ± 0.3	7.2	8.4	1.00
prek-head run check-xml --all-files	7.6 ± 0.5	7.1	8.8	1.01 ± 0.07

prek run detect-private-key --all-files

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base run detect-private-key --all-files	11.0 ± 0.7	9.8	13.2	1.00
prek-head run detect-private-key --all-files	11.2 ± 0.6	10.3	13.0	1.01 ± 0.09

prek run fix-byte-order-marker --all-files

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base run fix-byte-order-marker --all-files	18.1 ± 0.9	16.8	20.2	1.01 ± 0.07
prek-head run fix-byte-order-marker --all-files	17.9 ± 0.9	17.0	21.1	1.00

Installation Performance

Benchmarking hook installation (fast path hooks skip Python setup):

prek install-hooks (cold - no cache)

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base install-hooks	4.5 ± 0.0	4.5	4.6	1.00
prek-head install-hooks	4.5 ± 0.1	4.5	4.6	1.00 ± 0.02

prek install-hooks (warm - with cache)

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base install-hooks	4.6 ± 0.1	4.5	4.7	1.01 ± 0.03
prek-head install-hooks	4.5 ± 0.1	4.4	4.6	1.00

File Filtering/Scoping Performance

Testing different file selection modes:

prek run (staged files only)

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base run	36.3 ± 0.5	35.4	37.6	1.00
prek-head run	36.5 ± 0.6	34.8	37.7	1.01 ± 0.02

prek run --files '*.json' (specific file type)

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base run --files '*.json'	7.9 ± 0.1	7.8	8.2	1.00
prek-head run --files '*.json'	8.0 ± 0.1	7.8	8.2	1.00 ± 0.02

Workspace Discovery & Initialization

Benchmarking hook discovery and initialization overhead:

prek run --dry-run --all-files (measures init overhead)

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base run --dry-run --all-files	6.8 ± 0.1	6.7	7.0	1.01 ± 0.03
prek-head run --dry-run --all-files	6.7 ± 0.2	6.6	7.6	1.00

Meta Hooks Performance

Benchmarking meta hooks separately:

prek run check-hooks-apply --all-files

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base run check-hooks-apply --all-files	11.9 ± 0.2	11.6	12.4	1.08 ± 0.07
prek-head run check-hooks-apply --all-files	11.0 ± 0.7	10.5	12.9	1.00

prek run check-useless-excludes --all-files

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base run check-useless-excludes --all-files	10.7 ± 0.1	10.5	10.9	1.00
prek-head run check-useless-excludes --all-files	10.8 ± 0.3	10.6	11.7	1.01 ± 0.03

prek run identity --all-files

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
prek-base run identity --all-files	10.0 ± 0.2	9.7	10.7	1.01 ± 0.03
prek-head run identity --all-files	9.9 ± 0.1	9.8	10.1	1.00

[codecov]

Codecov Report

❌ Patch coverage is 94.11765% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 93.38%. Comparing base (6c89849) to head (64daab4).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
...ek/src/hooks/pre_commit_hooks/mixed_line_ending.rs	94.11%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2391      +/-   ##
==========================================
- Coverage   93.39%   93.38%   -0.01%     
==========================================
  Files         129      129              
  Lines       27642    27637       -5     
==========================================
- Hits        25816    25810       -6     
- Misses       1826     1827       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.