Skip to content

Add FADV_RANDOM to accountsdb appendvec impl#9846

Closed
dachen0 wants to merge 1 commit into
anza-xyz:masterfrom
dachen0:fadvise
Closed

Add FADV_RANDOM to accountsdb appendvec impl#9846
dachen0 wants to merge 1 commit into
anza-xyz:masterfrom
dachen0:fadvise

Conversation

@dachen0
Copy link
Copy Markdown

@dachen0 dachen0 commented Jan 7, 2026

Problem

When running a validator, basically all of memory is used up by kernel page cache trying to guess what we want to cache. By explicitly marking appendvec files as FADV_RANDOM, the kernel should stop trying to cache accounts for us. In the future, we should try to explicitly cache what we need.

Summary of Changes

Mark appendvec files as FADV_RANDOM.
https://www.man7.org/linux/man-pages/man2/posix_fadvise.2.html

@mergify mergify Bot requested a review from a team January 7, 2026 15:46
@dachen0
Copy link
Copy Markdown
Author

dachen0 commented Jan 7, 2026

cc @kskalski

maybe this could also use direct io instead? Vibes wise I think Appendvec would have a lot of randomly sized reads and is not a good candidate for direct io and instead using fadvise is a good middle ground. Let me know your thoughts -- I also don't think this is the only spot we can put a fadvise, I'm not familiar with the accountsdb architecture and maybe the ledger storage can also use some flags like FADV_DONTNEED.
EDIT:
#9701

Looks like accountdb could use direct io if this PR goes through.

@dachen0 dachen0 closed this Jan 7, 2026
@kskalski
Copy link
Copy Markdown

kskalski commented Jan 8, 2026

For serving random access we use regular file io (I believe here

let bytes_read = read_into_buffer(file, self.len(), offset, unsafe {
) and in fact setting DIO option when opening append vec files would allow us to disable kernel caches. That might need adjustments in the way this data is cached by validator (maybe just cache sizing), but I think it's the direction we want to take.

I was planning to use #9701 only for scans first. We should incorporate io-uring on the serving path too, but I was waiting for removal of mmap access mode, so that API changes would be simpler (that will happen once 4.0 is cut soon).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants