Skip to content

Implement FileBufRead for SequentialFileReader#9701

Merged
kskalski merged 1 commit into
anza-xyz:masterfrom
kskalski:ks/file_read_buf_uring
Jan 19, 2026
Merged

Implement FileBufRead for SequentialFileReader#9701
kskalski merged 1 commit into
anza-xyz:masterfrom
kskalski:ks/file_read_buf_uring

Conversation

@kskalski
Copy link
Copy Markdown

@kskalski kskalski commented Dec 22, 2025

Problem

Accounts db uses stack based file reader (heaviest used in generate index) that does small sync reads. They could use io-uring reader to save some cpu on syscalls, but SequentialFileReader's API operates on single file and is initialized with a path.
In order to use io-uring reader for generate index and other accounts-db cases we need to support FileBufRead trait that has set_file(&File) function such that reader will perform reads on File that is kept open before and after reading.

Summary of Changes

  • make reader possible to construct without providing path / file, reader starts as idle
  • add lifetime to the reader, which can be used to ensure that file references added for reading outlive the reader
  • implement FileBufRead trait, which allows activating read for a given &File

Note: this PR is extracted from #6878 and rebased to current code-base. It does include some of the APIs and optimizations (prefetching / reading multiple files, low-level tuning based on profiles where reading was CPU bound) that might not be necessary in near future. They could be undone to minimize the diff of this PR, but the features still seem relevant for longer-term (e.g. reading list of multiple files known ahead of time, minimizing CPU overheads).

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Dec 22, 2025

Codecov Report

❌ Patch coverage is 97.33010% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.5%. Comparing base (729712b) to head (9905887).
⚠️ Report is 33 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff            @@
##           master    #9701    +/-   ##
========================================
  Coverage    82.5%    82.5%            
========================================
  Files         844      844            
  Lines      316758   317014   +256     
========================================
+ Hits       261579   261830   +251     
- Misses      55179    55184     +5     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@kskalski kskalski force-pushed the ks/file_read_buf_uring branch from c065db3 to 9905887 Compare January 14, 2026 03:39
@kskalski kskalski marked this pull request as ready for review January 14, 2026 04:49
Copy link
Copy Markdown

@cpubot cpubot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment thread fs/src/io_uring/sequential_file_reader.rs
Copy link
Copy Markdown

@brooksprumo brooksprumo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. If there are changes for the nits, they can either be handled here or in a follow-up PR.

Comment thread fs/src/io_uring/sequential_file_reader.rs
@kskalski kskalski added this pull request to the merge queue Jan 19, 2026
Merged via the queue into anza-xyz:master with commit 1e0f899 Jan 19, 2026
47 checks passed
@kskalski kskalski deleted the ks/file_read_buf_uring branch January 19, 2026 23:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants