Skip to content

Fix Int4 vector library tests failing on Java 21#144830

Merged
ChrisHegarty merged 8 commits intoelastic:mainfrom
ChrisHegarty:fix/int4-vector-tests-heap-segment
Mar 26, 2026
Merged

Fix Int4 vector library tests failing on Java 21#144830
ChrisHegarty merged 8 commits intoelastic:mainfrom
ChrisHegarty:fix/int4-vector-tests-heap-segment

Conversation

@ChrisHegarty
Copy link
Copy Markdown
Contributor

The bulk test methods in JDKVectorLibraryInt4Tests pass heap-backed MemorySegments (via MemorySegment.ofArray) directly to native downcall handles. On Java 21, heap segments cannot be passed to FFM downcalls because Linker.Option.critical(true) is only available from Java 22+.

Allocate query segments from the arena (off-heap) instead, matching the pattern used by other tests.

relates #144215

The bulk test methods in JDKVectorLibraryInt4Tests pass heap-backed
MemorySegments (via MemorySegment.ofArray) directly to native downcall
handles. On Java 21, heap segments cannot be passed to FFM downcalls
because Linker.Option.critical(true) is only available from Java 22+.

Allocate query segments from the arena (off-heap) instead, matching the
pattern used by other tests.

Introduced by elastic#144215
@ChrisHegarty ChrisHegarty requested a review from ldematte March 24, 2026 08:58
@ChrisHegarty ChrisHegarty requested a review from a team as a code owner March 24, 2026 08:58
@ChrisHegarty ChrisHegarty added >test Issues or PRs that are addressing/adding tests :Search Relevance/Vectors Vector search Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch labels Mar 24, 2026
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-search-relevance (Team:Search Relevance)

Copy link
Copy Markdown
Contributor

@ldematte ldematte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I'm surprised there is no CI failure related to this... is there?
If there are, it's worth closing them with this PR.

@ChrisHegarty ChrisHegarty merged commit fcd191f into elastic:main Mar 26, 2026
36 checks passed
@ChrisHegarty ChrisHegarty deleted the fix/int4-vector-tests-heap-segment branch March 26, 2026 08:37
szybia added a commit to szybia/elasticsearch that referenced this pull request Mar 26, 2026
* upstream/main: (146 commits)
  Revert "[Native] Gradle-related tweaks to improve handling of the simdvec native library (elastic#144539)"
  Fix ArrayIndexOutOfBoundsException in fetch phase with partial results (elastic#144385)
  ESQL: Correctly manage NULL data type for SUM (elastic#144942)
  [ESQL] Fixes GroupedTopNBenchmark not executing (elastic#144944)
  Fix reader context leak when query response serialization fails (elastic#144708)
  Validate individual offset values in BULK_OFFSETS bounds checks (elastic#144643)
  Merge main21 source set into main in simdvec (elastic#144921)
  [TEST] Unmute TsidExtractingIdFieldMapperTests (elastic#144848)
  [Native] Gradle-related tweaks to improve handling of the simdvec native library (elastic#144539)
  Fix `ThreadedActionListenerTests#testRejectionHandling` (elastic#144795)
  Add new DLM Frozen Tier Transition execution plugin and service (elastic#144595)
  Prometheus: execute query_range via parsed EsqlStatement plan (elastic#144416)
  Investigate `testBulkIndexingRequestSplitting` failure (elastic#144766)
  Add test utility for wrapping directories in FilterDirectory layer (elastic#143563)
  Fix ES|QL decay tests with negative scale (elastic#144657)
  Fix circuit breaker leak in percolator query construction (elastic#144827)
  Use XPerFieldDocValuesFormat in AbstractTSDBSyntheticIdCodec (elastic#144744)
  [DOCS] Document how reindex work in CPS (elastic#144016)
  Fix Int4 vector library tests failing on Java 21 (elastic#144830)
  [DiskBBQ] Fix index sorting on flush (elastic#144938)
  ...
seanzatzdev pushed a commit to seanzatzdev/elasticsearch that referenced this pull request Mar 26, 2026
The bulk test methods in JDKVectorLibraryInt4Tests pass heap-backed MemorySegments (via MemorySegment.ofArray) directly to native downcall handles. On Java 21, heap segments cannot be passed to FFM downcalls because Linker.Option.critical(true) is only available from Java 22+.

Allocate query segments from the arena (off-heap) instead, matching the pattern used by other tests.
seanzatzdev pushed a commit to seanzatzdev/elasticsearch that referenced this pull request Mar 27, 2026
The bulk test methods in JDKVectorLibraryInt4Tests pass heap-backed MemorySegments (via MemorySegment.ofArray) directly to native downcall handles. On Java 21, heap segments cannot be passed to FFM downcalls because Linker.Option.critical(true) is only available from Java 22+.

Allocate query segments from the arena (off-heap) instead, matching the pattern used by other tests.
mamazzol pushed a commit to mamazzol/elasticsearch that referenced this pull request Mar 30, 2026
The bulk test methods in JDKVectorLibraryInt4Tests pass heap-backed MemorySegments (via MemorySegment.ofArray) directly to native downcall handles. On Java 21, heap segments cannot be passed to FFM downcalls because Linker.Option.critical(true) is only available from Java 22+.

Allocate query segments from the arena (off-heap) instead, matching the pattern used by other tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Search Relevance/Vectors Vector search Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch >test Issues or PRs that are addressing/adding tests v9.4.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants