Add POSIX madvise support and a closeable MappableByteBuffer API to improve mmap-based blob cache access with explicit prefetching#140371
Merged
ChrisHegarty merged 38 commits intoelastic:mainfrom Jan 15, 2026
Conversation
Add a POSIX C library accessor to NativeAccess and expose the madvise system call through the PosixCLibrary interface. This enables providing memory access advice for blob cache directories in cases where the mmap directory cannot be used.
Collaborator
|
Pinging @elastic/es-core-infra (Team:Core/Infra) |
ChrisHegarty
reviewed
Jan 8, 2026
libs/native/src/main/java/org/elasticsearch/nativeaccess/lib/PosixCLibrary.java
Outdated
Show resolved
Hide resolved
rjernst
reviewed
Jan 8, 2026
libs/native/src/main/java/org/elasticsearch/nativeaccess/PosixNativeAccess.java
Outdated
Show resolved
Hide resolved
libs/native/src/main/java/org/elasticsearch/nativeaccess/jdk/JdkPosixCLibrary.java
Outdated
Show resolved
Hide resolved
libs/native/src/main/java/org/elasticsearch/nativeaccess/NativeAccess.java
Outdated
Show resolved
Hide resolved
Add a POSIX C library accessor to NativeAccess and expose the madvise system call through the PosixCLibrary interface. This enables providing memory access advice for blob cache directories in cases where the mmap directory cannot be used.
…usage in public API
…foreign usage in public API" This reverts commit bdad38c.
ChrisHegarty
reviewed
Jan 9, 2026
libs/native/src/main/java/org/elasticsearch/nativeaccess/jdk/JdkCloseableMappedByteBuffer.java
Outdated
Show resolved
Hide resolved
ChrisHegarty
reviewed
Jan 9, 2026
libs/native/src/main/java/org/elasticsearch/nativeaccess/jdk/JdkCloseableMappedByteBuffer.java
Outdated
Show resolved
Hide resolved
rjernst
reviewed
Jan 14, 2026
libs/native/src/main/java/org/elasticsearch/nativeaccess/jdk/JdkCloseableMappedByteBuffer.java
Outdated
Show resolved
Hide resolved
libs/native/src/main/java/org/elasticsearch/nativeaccess/jdk/JdkPosixCLibrary.java
Show resolved
Hide resolved
libs/native/src/main/java/org/elasticsearch/nativeaccess/jdk/JdkCloseableMappedByteBuffer.java
Outdated
Show resolved
Hide resolved
libs/native/src/main/java/org/elasticsearch/nativeaccess/lib/PosixCLibrary.java
Show resolved
Hide resolved
libs/native/src/test/java/org/elasticsearch/nativeaccess/jdk/JdkPosixCLibraryAccess.java
Outdated
Show resolved
Hide resolved
libs/native/src/test/java/org/elasticsearch/nativeaccess/PosixCLibraryTests.java
Outdated
Show resolved
Hide resolved
ChrisHegarty
changed the title
rjernst
reviewed
Jan 15, 2026
libs/native/src/main/java/org/elasticsearch/nativeaccess/jdk/JdkCloseableMappedByteBuffer.java
Outdated
Show resolved
Hide resolved
rjernst
approved these changes
Jan 15, 2026
Member
rjernst
left a comment
rjernst
left a comment
There was a problem hiding this comment.
I think this will work for now. The layout is a bit complex for how to handle different systems. I think we can simplify it by getting rid of the "jdk" intermediaries since we no longer have jna. We can use MemorySegment in those apis as long as the classes with those method signatures are not exported. I'll play around with that, but this PR looks fine for now.
ChrisHegarty
changed the title
spinscale
pushed a commit
to spinscale/elasticsearch
that referenced
this pull request
Jan 21, 2026
…mprove mmap-based blob cache access with explicit prefetching (elastic#140371) This PR enhances Elasticsearch’s native mmap handling by exposing the POSIX madvise syscall via the NativeAccess interface and integrating it into core I/O paths. It introduces a Closeable MappableByteBuffer API and plugs memory access advice and prefetch support into SharedBytes and frozen index inputs, enabling more efficient control over page cache behaviour.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR enhances Elasticsearch’s native mmap handling by exposing the POSIX madvise syscall via the NativeAccess interface and integrating it into core I/O paths. It introduces a Closeable MappableByteBuffer API and plugs memory access advice and prefetch support into SharedBytes and frozen index inputs, enabling more efficient control over page cache behaviour.