perf(sort): MSD hybrid radix sort for template-coordinate sorting

[nh13]

Summary

• Replace LSD radix sort (5 full O(n) field passes) with MSD hybrid approach for template-coordinate sorting
• Radix sort by primary field (tid1/tid2/pos1) first — resolves >90% of ordering — then sub-sort only the small equal-value runs by remaining fields
• Reuse bytes_needed_u64 from radix.rs instead of duplicating

Benchmarks

Tested on kapa-umi (89M reads, 4GB):

Metric	4 threads	12 threads
Sort phase speedup	1.9x (490ms→261ms/chunk)	1.7x (1465ms→849ms/chunk)
End-to-end speedup	1.08x (1m52s→1m44s)	1.17x (1m30s→1m17s)

End-to-end gains are diluted by I/O (read, chunk write, merge); the sort phase itself is consistently ~1.8x faster.

Test plan

• cargo ci-test — all 1934 tests pass
• cargo ci-fmt and cargo ci-lint clean
• Benchmarked on kapa-umi with hyperfine and per-chunk profiling
• Output BAM verified by sort command's built-in verification

[codecov]

Codecov Report

❌ Patch coverage is 92.59259% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.13%. Comparing base (0c934bb) to head (c0fb06c).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
src/lib/sort/inline_buffer.rs	92.45%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #191      +/-   ##
==========================================
- Coverage   86.13%   86.13%   -0.01%     
==========================================
  Files         110      110              
  Lines       51974    52019      +45     
==========================================
+ Hits        44768    44805      +37     
- Misses       7206     7214       +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 34cccb49-0c0f-4097-ba45-ca63afb6ea44

📥 Commits

Reviewing files that changed from the base of the PR and between 33d00a8 and c0fb06c.

📒 Files selected for processing (2)

• src/lib/sort/inline_buffer.rs
• src/lib/sort/radix.rs

---

📝 Walkthrough

Walkthrough

Refactored radix_sort_template_refs to an MSD-style hybrid: first radix-sort the slice by TemplateKey.primary, scan to partition contiguous equal-primary runs, then sub-sort each run by remaining fields in precedence (secondary, cb_hash, tertiary, name_hash_upper). Runs of 2–64 elements use insertion_sort_template_refs; larger runs recurse using radix_sort_template_field and sub_sort_runs. Added function-pointer constants, an insertion-sort threshold, and switched u64 byte-counting to the shared bytes_needed_u64 (now pub(crate)). Added two proptest oracle tests comparing against full TemplateKey ordering.

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	Title clearly summarizes the main change: replacing LSD with MSD hybrid radix sort for template-coordinate sorting.
Description check	✅ Passed	Description is directly related to the changeset, covering the algorithm change, benchmarks, and test results.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch perf/nh/msd-radix-template-sort

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (1)

src/lib/sort/inline_buffer.rs (1)

1084-1127: Add an oracle test for the recursive path.

A proptest that compares this sorter against expected.sort_by_key(|r| r.key) on randomized equal-primary-heavy inputs would lock down the new tie-break recursion better than hand-picked cases.

As per coding guidelines, "Use proptest for property-based testing".

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@src/lib/sort/inline_buffer.rs` around lines 1084 - 1127, Add a proptest that
verifies the recursive tie-break path in sub_sort_runs by generating arrays
where many records share the primary run_field value (to force run_len >
SUB_SORT_INSERTION_THRESHOLD and trigger the radix_sort_template_field +
recursive call). The test should build randomized TemplateRecordRef arrays with
identical primary keys and randomized remaining_fields, run the sorter that
ultimately calls sub_sort_runs, and assert the result equals a stable reference
sort (e.g., by mapping to a composite key and using expected.sort_by_key(|r|
r.key)). Ensure the test covers cases where bytes_needed_u64(max_val) > 0 and
remaining_fields.len() > 1 so the branch that calls sub_sort_runs recursively is
exercised.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@src/lib/sort/inline_buffer.rs`:
- Around line 1084-1127: Add a proptest that verifies the recursive tie-break
path in sub_sort_runs by generating arrays where many records share the primary
run_field value (to force run_len > SUB_SORT_INSERTION_THRESHOLD and trigger the
radix_sort_template_field + recursive call). The test should build randomized
TemplateRecordRef arrays with identical primary keys and randomized
remaining_fields, run the sorter that ultimately calls sub_sort_runs, and assert
the result equals a stable reference sort (e.g., by mapping to a composite key
and using expected.sort_by_key(|r| r.key)). Ensure the test covers cases where
bytes_needed_u64(max_val) > 0 and remaining_fields.len() > 1 so the branch that
calls sub_sort_runs recursively is exercised.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 72d30368-6297-4cf3-803a-dbde78dd771c

📥 Commits

Reviewing files that changed from the base of the PR and between 0c934bb and 33d00a8.

📒 Files selected for processing (2)

• src/lib/sort/inline_buffer.rs
• src/lib/sort/radix.rs

[nh13]

@coderabbitai review

[coderabbitai]

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.