-
Notifications
You must be signed in to change notification settings - Fork 1.1k
[FEA] Add Java bindings for AST JIT operations #23117
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
rapids-bot
merged 14 commits into
NVIDIA:main
from
thirtiseven:codex-ansi-jit-java-bindings-main-upmerge
Jul 20, 2026
Merged
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
fdde92b
Expose JIT AST operations in Java
thirtiseven db1e27a
Align Java AST JIT bindings with libcudf APIs
thirtiseven 4680b9a
local review
thirtiseven 6f79a23
local review comments address
thirtiseven 83f500a
Merge branch 'main' into codex-ansi-jit-java-bindings-main-upmerge
thirtiseven 500eae5
Merge remote-tracking branch 'origin/main' into codex-ansi-jit-java-b…
thirtiseven 19f974b
address code rabbit comments
thirtiseven 30cceaf
address comments
thirtiseven 4f2dd1b
address comments
thirtiseven 122192f
address comments
thirtiseven 27d63d8
address comments
thirtiseven 842c4a0
address comments
thirtiseven f869b70
Merge branch 'main' into codex-ansi-jit-java-bindings-main-upmerge
thirtiseven 3aeba52
style
thirtiseven File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| /* | ||
| * SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package ai.rapids.cudf.ast; | ||
|
|
||
| final class AstUtils { | ||
| private AstUtils() { | ||
| } | ||
|
|
||
| static byte checkByte(int value) { | ||
| byte result = (byte) value; | ||
| if (result != value) { | ||
| throw new IllegalArgumentException("value does not fit in a byte: " + value); | ||
| } | ||
| return result; | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| /* | ||
| * SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package ai.rapids.cudf.ast; | ||
|
|
||
| import java.nio.ByteBuffer; | ||
|
|
||
| /** | ||
| * Error handling policy for fallible JIT AST operations. | ||
| * | ||
| * NOTE: This must be kept in sync with `jni_to_jit_error_policy` in CompiledExpression.cpp! | ||
| */ | ||
| public enum JitErrorPolicy { | ||
| /** Propagate an evaluation error to the caller. */ | ||
| PROPAGATE(0), | ||
| /** Produce null for a row where evaluation fails. */ | ||
| NULLIFY(1); | ||
|
|
||
| private final byte nativeId; | ||
|
|
||
| JitErrorPolicy(int nativeId) { | ||
| this.nativeId = AstUtils.checkByte(nativeId); | ||
| } | ||
|
|
||
| int getSerializedSize() { | ||
| return Byte.BYTES; | ||
| } | ||
|
|
||
| void serialize(ByteBuffer bb) { | ||
| bb.put(nativeId); | ||
| } | ||
| } |
125 changes: 125 additions & 0 deletions
125
java/src/main/java/ai/rapids/cudf/ast/JitOperation.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,125 @@ | ||
| /* | ||
| * SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package ai.rapids.cudf.ast; | ||
|
|
||
| import java.nio.ByteBuffer; | ||
| import java.util.Objects; | ||
|
|
||
| /** | ||
| * A libcudf JIT operation. Expressions containing a JIT operation must be evaluated with | ||
| * {@link CompiledExpression#computeColumnJit}. | ||
| * Operator arity, error policy, and target-scale constraints are validated when the expression | ||
| * is compiled. | ||
| */ | ||
| public final class JitOperation extends AstExpression { | ||
| private final JitOperator op; | ||
| private final JitErrorPolicy errorPolicy; | ||
| private final AstExpression[] inputs; | ||
| private final Integer targetScale; | ||
|
|
||
| /** | ||
| * Construct an operation that propagates evaluation errors. | ||
| * | ||
| * @param op operator to apply | ||
| * @param inputs operator inputs | ||
| * @throws NullPointerException if {@code op}, {@code inputs}, or an input is null | ||
| */ | ||
| public JitOperation(JitOperator op, AstExpression... inputs) { | ||
| this(op, JitErrorPolicy.PROPAGATE, null, inputs); | ||
| } | ||
|
|
||
| /** | ||
| * Construct an operation with an explicit error policy. | ||
| * | ||
| * @param op operator to apply | ||
| * @param errorPolicy error handling policy | ||
| * @param inputs operator inputs | ||
| * @throws NullPointerException if any argument or input is null | ||
| */ | ||
| public JitOperation(JitOperator op, JitErrorPolicy errorPolicy, AstExpression... inputs) { | ||
| this(op, errorPolicy, null, inputs); | ||
| } | ||
|
|
||
| /** | ||
| * Construct a target-scale operation that propagates evaluation errors. | ||
| * The target scale is valid only for {@link JitOperator#RESCALE}. | ||
| * | ||
| * @param op operator to apply | ||
| * @param targetScale target fixed-point scale | ||
| * @param inputs operator inputs | ||
| * @throws NullPointerException if {@code op}, {@code inputs}, or an input is null | ||
| */ | ||
| public JitOperation(JitOperator op, int targetScale, AstExpression... inputs) { | ||
| this(op, JitErrorPolicy.PROPAGATE, Integer.valueOf(targetScale), inputs); | ||
| } | ||
|
|
||
| private JitOperation( | ||
| JitOperator op, | ||
| JitErrorPolicy errorPolicy, | ||
| Integer targetScale, | ||
| AstExpression... inputs) { | ||
| this.op = Objects.requireNonNull(op, "op is null"); | ||
| this.errorPolicy = Objects.requireNonNull(errorPolicy, "errorPolicy is null"); | ||
| this.inputs = Objects.requireNonNull(inputs, "inputs is null").clone(); | ||
| this.targetScale = targetScale; | ||
| for (int i = 0; i < this.inputs.length; i++) { | ||
| Objects.requireNonNull(this.inputs[i], "input " + i + " is null"); | ||
| } | ||
| } | ||
|
|
||
| @Override | ||
| int getSerializedSize() { | ||
| int size = ExpressionType.JIT_EXPRESSION.getSerializedSize() + | ||
| op.getSerializedSize() + | ||
| errorPolicy.getSerializedSize() + | ||
| Byte.BYTES + // targetScale present | ||
| Byte.BYTES; // inputs.length | ||
| if (targetScale != null) { | ||
| size += Integer.BYTES; | ||
| } | ||
| for (AstExpression input : inputs) { | ||
| size += input.getSerializedSize(); | ||
| } | ||
| return size; | ||
| } | ||
|
|
||
| @Override | ||
| void serialize(ByteBuffer bb) { | ||
| ExpressionType.JIT_EXPRESSION.serialize(bb); | ||
| op.serialize(bb); | ||
| errorPolicy.serialize(bb); | ||
| bb.put((byte) (targetScale == null ? 0 : 1)); | ||
|
igorpeshansky marked this conversation as resolved.
|
||
| if (targetScale != null) { | ||
| bb.putInt(targetScale); | ||
| } | ||
| bb.put((byte) inputs.length); | ||
| for (AstExpression input : inputs) { | ||
| input.serialize(bb); | ||
| } | ||
| } | ||
|
|
||
| @Override | ||
| public String toString() { | ||
| StringBuilder ret = new StringBuilder(op.toString()); | ||
| if (errorPolicy != JitErrorPolicy.PROPAGATE) { | ||
| ret.append("[").append(errorPolicy).append("]"); | ||
| } | ||
| ret.append("("); | ||
| for (int i = 0; i < inputs.length; i++) { | ||
| if (i > 0) { | ||
| ret.append(", "); | ||
| } | ||
| ret.append(inputs[i]); | ||
| } | ||
| if (targetScale != null) { | ||
| if (inputs.length > 0) { | ||
| ret.append(", "); | ||
| } | ||
| ret.append("scale=").append(targetScale); | ||
| } | ||
| return ret.append(")").toString(); | ||
| } | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.