-
Notifications
You must be signed in to change notification settings - Fork 180
issue #4514 tonumber function as part of roadmap #4287 #4605
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
@penghuo please review |
ppl/src/main/java/org/opensearch/sql/ppl/parser/AstExpressionBuilder.java
Show resolved
Hide resolved
core/src/main/java/org/opensearch/sql/expression/function/PPLFuncImpTable.java
Outdated
Show resolved
Hide resolved
0e85841 to
f070684
Compare
|
@penghuo please review |
core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 3
♻️ Duplicate comments (1)
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java (1)
62-72: Remove unused variable.Line 65 declares
int base = 10;but this variable is never used in the method.Apply this diff:
public Expression implement( RexToLixTranslator translator, RexCall call, List<Expression> translatedOperands) { Expression fieldValue = translatedOperands.get(0); - int base = 10; if (translatedOperands.size() > 1) { Expression baseExpr = translatedOperands.get(1); return Expressions.call(ToNumberFunction.class, "toNumber", fieldValue, baseExpr); } else { return Expressions.call(ToNumberFunction.class, "toNumber", fieldValue); } }
🧹 Nitpick comments (3)
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java (2)
23-57: Well-documented public API.The JavaDoc is comprehensive and clearly explains the function behavior, supported bases, and value ranges. The return type inference (
DOUBLE_FORCE_NULLABLE) and operand metadata (STRING_OR_STRING_INTEGER) are correctly configured.Optional improvement: Consider adding a note in the JavaDoc about the function returning
nullwhen the input string cannot be parsed (e.g., invalid digits for the specified base).
97-99: Add comment explaining empty catch block.The empty catch block at lines 97-99 silently swallows all exceptions. While past review comments indicate this is intentional (return
nullfor invalid inputs), the empty block should include a comment explaining this design decision to improve code maintainability.Apply this diff:
} catch (Exception e) { - + // Return null for malformatted input strings (by design) }As per coding guidelines on error handling and code clarity.
integ-test/src/test/java/org/opensearch/sql/ppl/ConversionFunctionIT.java (1)
73-109: Consider usingassertNullfor better clarity.Lines 83, 96, and 108 verify that the result is
nullusingassertEquals. For better readability and clearer intent, consider usingassertNull.Apply this diff:
@Test public void testOctalWithUnsupportedValue() throws IOException { JSONObject actual = executeQuery( String.format( "source=%s |head 1| eval a = tonumber('20415.442',8) | fields a", TEST_INDEX_ACCOUNT)); verifySchema(actual, schema("a", "double")); - assertEquals(actual.getJSONArray("datarows").getJSONArray(0).get(0), null); + assertNull(actual.getJSONArray("datarows").getJSONArray(0).get(0)); } @Test public void testBinaryWithUnsupportedValue() throws IOException { JSONObject actual = executeQuery( String.format( "source=%s |head 1| eval a = tonumber('1010.11',2) | fields a", TEST_INDEX_ACCOUNT)); verifySchema(actual, schema("a", "double")); - assertEquals(actual.getJSONArray("datarows").getJSONArray(0).get(0), null); + assertNull(actual.getJSONArray("datarows").getJSONArray(0).get(0)); } @Test public void testHexWithUnsupportedValue() throws IOException { JSONObject actual = executeQuery( String.format( "source=%s |head 1| eval a = tonumber('A.B',16) | fields a", TEST_INDEX_ACCOUNT)); verifySchema(actual, schema("a", "double")); - assertEquals(actual.getJSONArray("datarows").getJSONArray(0).get(0), null); + assertNull(actual.getJSONArray("datarows").getJSONArray(0).get(0)); }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (11)
core/src/main/java/org/opensearch/sql/calcite/utils/PPLOperandTypes.java(1 hunks)core/src/main/java/org/opensearch/sql/expression/function/BuiltinFunctionName.java(1 hunks)core/src/main/java/org/opensearch/sql/expression/function/PPLBuiltinOperators.java(2 hunks)core/src/main/java/org/opensearch/sql/expression/function/PPLFuncImpTable.java(2 hunks)core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java(1 hunks)core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java(1 hunks)docs/user/ppl/functions/conversion.rst(1 hunks)integ-test/src/test/java/org/opensearch/sql/ppl/ConversionFunctionIT.java(1 hunks)ppl/src/main/antlr/OpenSearchPPLLexer.g4(1 hunks)ppl/src/main/antlr/OpenSearchPPLParser.g4(2 hunks)ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java(1 hunks)
🧰 Additional context used
📓 Path-based instructions (4)
**/*.java
📄 CodeRabbit inference engine (.rules/REVIEW_GUIDELINES.md)
**/*.java: UsePascalCasefor class names (e.g.,QueryExecutor)
UsecamelCasefor method and variable names (e.g.,executeQuery)
UseUPPER_SNAKE_CASEfor constants (e.g.,MAX_RETRY_COUNT)
Keep methods under 20 lines with single responsibility
All public classes and methods must have proper JavaDoc
Use specific exception types with meaningful messages for error handling
PreferOptional<T>for nullable returns in Java
Avoid unnecessary object creation in loops
UseStringBuilderfor string concatenation in loops
Validate all user inputs, especially queries
Sanitize data before logging to prevent injection attacks
Use try-with-resources for proper resource cleanup in Java
Maintain Java 11 compatibility when possible for OpenSearch 2.x
Document Calcite-specific workarounds in code
Files:
core/src/main/java/org/opensearch/sql/expression/function/PPLBuiltinOperators.javappl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.javacore/src/main/java/org/opensearch/sql/calcite/utils/PPLOperandTypes.javacore/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.javacore/src/main/java/org/opensearch/sql/expression/function/BuiltinFunctionName.javacore/src/main/java/org/opensearch/sql/expression/function/PPLFuncImpTable.javacore/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.javainteg-test/src/test/java/org/opensearch/sql/ppl/ConversionFunctionIT.java
**/*Test.java
📄 CodeRabbit inference engine (.rules/REVIEW_GUIDELINES.md)
**/*Test.java: All new business logic requires unit tests
Name unit tests with*Test.javasuffix in OpenSearch SQL
Files:
ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.javacore/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java
integ-test/**/*IT.java
📄 CodeRabbit inference engine (.rules/REVIEW_GUIDELINES.md)
End-to-end scenarios need integration tests in
integ-test/module
Files:
integ-test/src/test/java/org/opensearch/sql/ppl/ConversionFunctionIT.java
**/*IT.java
📄 CodeRabbit inference engine (.rules/REVIEW_GUIDELINES.md)
Name integration tests with
*IT.javasuffix in OpenSearch SQL
Files:
integ-test/src/test/java/org/opensearch/sql/ppl/ConversionFunctionIT.java
🧠 Learnings (5)
📚 Learning: 2025-12-02T17:27:55.938Z
Learnt from: CR
Repo: opensearch-project/sql PR: 0
File: .rules/REVIEW_GUIDELINES.md:0-0
Timestamp: 2025-12-02T17:27:55.938Z
Learning: Test SQL generation and optimization paths for Calcite integration changes
Applied to files:
ppl/src/main/antlr/OpenSearchPPLLexer.g4ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java
📚 Learning: 2025-12-02T17:27:55.938Z
Learnt from: CR
Repo: opensearch-project/sql PR: 0
File: .rules/REVIEW_GUIDELINES.md:0-0
Timestamp: 2025-12-02T17:27:55.938Z
Learning: Applies to **/*IT.java : Name integration tests with `*IT.java` suffix in OpenSearch SQL
Applied to files:
ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.javainteg-test/src/test/java/org/opensearch/sql/ppl/ConversionFunctionIT.java
📚 Learning: 2025-12-02T17:27:55.938Z
Learnt from: CR
Repo: opensearch-project/sql PR: 0
File: .rules/REVIEW_GUIDELINES.md:0-0
Timestamp: 2025-12-02T17:27:55.938Z
Learning: Applies to **/*.java : All public classes and methods must have proper JavaDoc
Applied to files:
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java
📚 Learning: 2025-12-02T17:27:55.938Z
Learnt from: CR
Repo: opensearch-project/sql PR: 0
File: .rules/REVIEW_GUIDELINES.md:0-0
Timestamp: 2025-12-02T17:27:55.938Z
Learning: Flag missing JavaDoc on public APIs during code review
Applied to files:
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java
📚 Learning: 2025-12-02T17:27:55.938Z
Learnt from: CR
Repo: opensearch-project/sql PR: 0
File: .rules/REVIEW_GUIDELINES.md:0-0
Timestamp: 2025-12-02T17:27:55.938Z
Learning: Applies to **/*Test.java : Name unit tests with `*Test.java` suffix in OpenSearch SQL
Applied to files:
core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java
🧬 Code graph analysis (2)
core/src/main/java/org/opensearch/sql/expression/function/PPLBuiltinOperators.java (1)
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java (1)
ToNumberFunction(41-102)
core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java (2)
core/src/main/java/org/opensearch/sql/calcite/utils/PPLOperandTypes.java (1)
PPLOperandTypes(19-324)core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java (1)
ToNumberFunction(41-102)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (28)
- GitHub Check: build-linux (25, doc)
- GitHub Check: bwc-tests-full-restart (21)
- GitHub Check: build-linux (21, doc)
- GitHub Check: bwc-tests-full-restart (25)
- GitHub Check: build-linux (25, unit)
- GitHub Check: build-linux (21, integration)
- GitHub Check: build-linux (21, unit)
- GitHub Check: build-linux (25, integration)
- GitHub Check: bwc-tests-rolling-upgrade (25)
- GitHub Check: bwc-tests-rolling-upgrade (21)
- GitHub Check: security-it-linux (25)
- GitHub Check: security-it-linux (21)
- GitHub Check: build-windows-macos (macos-14, 25, integration)
- GitHub Check: build-windows-macos (macos-14, 21, doc)
- GitHub Check: build-windows-macos (macos-14, 25, unit)
- GitHub Check: build-windows-macos (macos-14, 25, doc)
- GitHub Check: security-it-windows-macos (windows-latest, 25)
- GitHub Check: security-it-windows-macos (windows-latest, 21)
- GitHub Check: build-windows-macos (macos-14, 21, integration)
- GitHub Check: build-windows-macos (windows-latest, 21, -PbuildPlatform=windows, integration)
- GitHub Check: security-it-windows-macos (macos-14, 25)
- GitHub Check: build-windows-macos (macos-14, 21, unit)
- GitHub Check: CodeQL-Scan (java)
- GitHub Check: build-windows-macos (windows-latest, 25, -PbuildPlatform=windows, integration)
- GitHub Check: build-windows-macos (windows-latest, 21, -PbuildPlatform=windows, unit)
- GitHub Check: build-windows-macos (windows-latest, 25, -PbuildPlatform=windows, unit)
- GitHub Check: security-it-windows-macos (macos-14, 21)
- GitHub Check: test-sql-cli-integration (21)
🔇 Additional comments (10)
ppl/src/main/antlr/OpenSearchPPLLexer.g4 (1)
411-411: LGTM!The TONUMBER token is correctly added to the TEXT FUNCTIONS section and follows the same pattern as other text function tokens.
core/src/main/java/org/opensearch/sql/calcite/utils/PPLOperandTypes.java (1)
108-112: LGTM!The STRING_OR_STRING_INTEGER operand type checker correctly supports both
tonumber(string)andtonumber(string, int)signatures using a CompositeOperandTypeChecker with OR composition. This follows the established pattern in the file.core/src/main/java/org/opensearch/sql/expression/function/BuiltinFunctionName.java (1)
159-159: LGTM!The TONUMBER function name is correctly added to the Text Functions section and follows the established naming pattern.
ppl/src/main/antlr/OpenSearchPPLParser.g4 (1)
1307-1307: LGTM!TONUMBER is correctly added to both
textFunctionName(line 1307) andsearchableKeyWord(line 1513), enabling it to be parsed as both a function name and a keyword. This follows the same pattern as TOSTRING.Also applies to: 1513-1513
core/src/main/java/org/opensearch/sql/expression/function/PPLFuncImpTable.java (1)
216-216: LGTM!The TONUMBER function is correctly imported and registered in the function table. The registration follows the same pattern as other functions like TOSTRING and properly maps to
PPLBuiltinOperators.TONUMBER.Also applies to: 960-960
core/src/main/java/org/opensearch/sql/expression/function/PPLBuiltinOperators.java (1)
70-70: LGTM!The TONUMBER operator is correctly defined by instantiating
ToNumberFunctionand mapping it to the UDF name "TONUMBER". The import is properly placed in alphabetical order, and the pattern matches other similar operators like TOSTRING.The implementation appropriately:
- Returns
DOUBLE_FORCE_NULLABLEto handle both integer and double results- Uses
STRING_OR_STRING_INTEGERoperand metadata for type checking- Handles parse failures by returning null
Also applies to: 416-416
ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java (1)
12-215: Verify that integration tests cover TONUMBER function across real indexed data and query execution scenarios.The unit test class provides excellent coverage of TONUMBER including binary, hex, decimal conversions, NULL handling, and overflow behavior. However, integration tests should be verified to ensure coverage of:
- Real data conversions from indexed fields
- Error handling with actual query execution
- NULL value handling in production scenarios
Confirm that integration tests exist in the integ-test directory and include TONUMBER function coverage alongside this unit test suite.
core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java (2)
18-26: LGTM! Metadata tests are correct.The tests properly verify that the function returns
DOUBLE_FORCE_NULLABLEand acceptsSTRING_OR_STRING_INTEGERoperands.
28-79: Excellent test coverage overall!The test suite comprehensively covers:
- Various bases (2, 8, 10, 16, 36)
- Edge cases (zeros, large numbers, empty fractional parts)
- Error handling (invalid bases, invalid digits)
- Case insensitivity for hexadecimal inputs
- Negative numbers and decimal values
Also applies to: 95-175
integ-test/src/test/java/org/opensearch/sql/ppl/ConversionFunctionIT.java (1)
16-71: Integration tests provide good coverage.The tests properly verify the
tonumberfunction across multiple bases (decimal, hex, binary, octal) and correctly assert the schema as "double", which aligns with theDOUBLE_FORCE_NULLABLEreturn type inference.
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java
Show resolved
Hide resolved
core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
♻️ Duplicate comments (1)
core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java (1)
81-93: Fix or remove misleading/duplicated “Decimal” base‑2/16 tests
testToNumberWithDecimalBase2andtestToNumberWithDecimalBase16don’t include decimal points and effectively duplicatetestToNumberWithBase2andtestToNumberWithBase16, which makes the intent unclear.Either:
- Rename them and keep as is, or
- Better, change the inputs to include
'.'and assertnullto explicitly cover the “decimal not supported for base ≠ 10” behavior, or simply delete them sincetestToNumberInvalidDigitsand the ITs already cover those cases.Example repurpose:
- @Test - void testToNumberWithDecimalBase2() { - assertEquals(2L, ToNumberFunction.toNumber("10", 2)); - assertEquals(1L, ToNumberFunction.toNumber("1", 2)); - assertEquals(3L, ToNumberFunction.toNumber("11", 2)); - } - - @Test - void testToNumberWithDecimalBase16() { - assertEquals(255L, ToNumberFunction.toNumber("FF", 16)); - assertEquals(16L, ToNumberFunction.toNumber("10", 16)); - assertEquals(171L, ToNumberFunction.toNumber("AB", 16)); - } + @Test + void testToNumberWithDecimalBase2() { + assertNull(ToNumberFunction.toNumber("10.1", 2)); + assertNull(ToNumberFunction.toNumber("1.0", 2)); + assertNull(ToNumberFunction.toNumber("11.11", 2)); + } + + @Test + void testToNumberWithDecimalBase16() { + assertNull(ToNumberFunction.toNumber("FF.0", 16)); + assertNull(ToNumberFunction.toNumber("10.8", 16)); + assertNull(ToNumberFunction.toNumber("AB.C", 16)); + }
🧹 Nitpick comments (1)
integ-test/src/test/java/org/opensearch/sql/ppl/ConversionFunctionIT.java (1)
21-109: IT coverage for TONUMBER is solid; consider clearer null assertionsThese tests nicely exercise TONUMBER end‑to‑end for decimal, hex, binary, octal, and invalid/unsupported values with schema verification as
double, which matches the UDF contract.For the “unsupported value” cases (Lines 73–109), consider switching to
assertNull(actual.getJSONArray("datarows").getJSONArray(0).get(0))(or at leastassertEquals(null, …)) so the expected/actual order and the intent (“value should be null”) are more obvious in failure messages.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java(1 hunks)core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java(1 hunks)docs/user/ppl/functions/conversion.rst(1 hunks)integ-test/src/test/java/org/opensearch/sql/ppl/ConversionFunctionIT.java(1 hunks)ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java
- docs/user/ppl/functions/conversion.rst
🧰 Additional context used
📓 Path-based instructions (4)
**/*.java
📄 CodeRabbit inference engine (.rules/REVIEW_GUIDELINES.md)
**/*.java: UsePascalCasefor class names (e.g.,QueryExecutor)
UsecamelCasefor method and variable names (e.g.,executeQuery)
UseUPPER_SNAKE_CASEfor constants (e.g.,MAX_RETRY_COUNT)
Keep methods under 20 lines with single responsibility
All public classes and methods must have proper JavaDoc
Use specific exception types with meaningful messages for error handling
PreferOptional<T>for nullable returns in Java
Avoid unnecessary object creation in loops
UseStringBuilderfor string concatenation in loops
Validate all user inputs, especially queries
Sanitize data before logging to prevent injection attacks
Use try-with-resources for proper resource cleanup in Java
Maintain Java 11 compatibility when possible for OpenSearch 2.x
Document Calcite-specific workarounds in code
Files:
integ-test/src/test/java/org/opensearch/sql/ppl/ConversionFunctionIT.javappl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.javacore/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java
integ-test/**/*IT.java
📄 CodeRabbit inference engine (.rules/REVIEW_GUIDELINES.md)
End-to-end scenarios need integration tests in
integ-test/module
Files:
integ-test/src/test/java/org/opensearch/sql/ppl/ConversionFunctionIT.java
**/*IT.java
📄 CodeRabbit inference engine (.rules/REVIEW_GUIDELINES.md)
Name integration tests with
*IT.javasuffix in OpenSearch SQL
Files:
integ-test/src/test/java/org/opensearch/sql/ppl/ConversionFunctionIT.java
**/*Test.java
📄 CodeRabbit inference engine (.rules/REVIEW_GUIDELINES.md)
**/*Test.java: All new business logic requires unit tests
Name unit tests with*Test.javasuffix in OpenSearch SQL
Files:
ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.javacore/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java
🧠 Learnings (3)
📚 Learning: 2025-12-02T17:27:55.938Z
Learnt from: CR
Repo: opensearch-project/sql PR: 0
File: .rules/REVIEW_GUIDELINES.md:0-0
Timestamp: 2025-12-02T17:27:55.938Z
Learning: Applies to **/*IT.java : Name integration tests with `*IT.java` suffix in OpenSearch SQL
Applied to files:
integ-test/src/test/java/org/opensearch/sql/ppl/ConversionFunctionIT.javappl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java
📚 Learning: 2025-12-02T17:27:55.938Z
Learnt from: CR
Repo: opensearch-project/sql PR: 0
File: .rules/REVIEW_GUIDELINES.md:0-0
Timestamp: 2025-12-02T17:27:55.938Z
Learning: Test SQL generation and optimization paths for Calcite integration changes
Applied to files:
ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java
📚 Learning: 2025-12-02T17:27:55.938Z
Learnt from: CR
Repo: opensearch-project/sql PR: 0
File: .rules/REVIEW_GUIDELINES.md:0-0
Timestamp: 2025-12-02T17:27:55.938Z
Learning: Applies to **/*Test.java : Name unit tests with `*Test.java` suffix in OpenSearch SQL
Applied to files:
core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java
🧬 Code graph analysis (2)
integ-test/src/test/java/org/opensearch/sql/ppl/ConversionFunctionIT.java (1)
integ-test/src/test/java/org/opensearch/sql/sql/IdentifierIT.java (1)
Index(218-262)
core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java (2)
core/src/main/java/org/opensearch/sql/calcite/utils/PPLOperandTypes.java (1)
PPLOperandTypes(19-324)core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java (1)
ToNumberFunction(41-102)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (28)
- GitHub Check: build-linux (21, unit)
- GitHub Check: build-linux (21, doc)
- GitHub Check: build-linux (25, integration)
- GitHub Check: build-linux (25, unit)
- GitHub Check: build-linux (25, doc)
- GitHub Check: build-linux (21, integration)
- GitHub Check: bwc-tests-full-restart (21)
- GitHub Check: bwc-tests-full-restart (25)
- GitHub Check: bwc-tests-rolling-upgrade (21)
- GitHub Check: bwc-tests-rolling-upgrade (25)
- GitHub Check: security-it-linux (25)
- GitHub Check: security-it-linux (21)
- GitHub Check: build-windows-macos (macos-14, 25, unit)
- GitHub Check: build-windows-macos (windows-latest, 21, -PbuildPlatform=windows, unit)
- GitHub Check: build-windows-macos (macos-14, 21, doc)
- GitHub Check: build-windows-macos (macos-14, 25, doc)
- GitHub Check: build-windows-macos (macos-14, 25, integration)
- GitHub Check: build-windows-macos (windows-latest, 21, -PbuildPlatform=windows, integration)
- GitHub Check: build-windows-macos (windows-latest, 25, -PbuildPlatform=windows, integration)
- GitHub Check: security-it-windows-macos (windows-latest, 25)
- GitHub Check: build-windows-macos (macos-14, 21, integration)
- GitHub Check: security-it-windows-macos (macos-14, 21)
- GitHub Check: build-windows-macos (macos-14, 21, unit)
- GitHub Check: security-it-windows-macos (macos-14, 25)
- GitHub Check: build-windows-macos (windows-latest, 25, -PbuildPlatform=windows, unit)
- GitHub Check: security-it-windows-macos (windows-latest, 21)
- GitHub Check: test-sql-cli-integration (21)
- GitHub Check: CodeQL-Scan (java)
🔇 Additional comments (2)
core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java (1)
18-46: Type metadata and base‑10 behavior tests look consistentThese tests exercise the declared return type, operand metadata, and both integer/decimal base‑10 paths in line with the current
ToNumberFunctioncontract. No changes needed here.ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java (1)
18-214: Comprehensive Calcite & Spark SQL coverage for TONUMBERThese tests thoroughly validate TONUMBER through Calcite: they check logical plans, result values (including nulls and long boundary/overflow behavior), and Spark SQL translation for binary, hex, decimal, and invalid inputs. This gives strong confidence in the integration path; no changes needed here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
♻️ Duplicate comments (1)
ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java (1)
12-12: Add explicit NULL input test cases.Based on the past review discussion, NULL cases should be covered. The current tests validate unparseable strings returning null, but don't explicitly test NULL as input. Add test cases for:
tonumber(null)tonumber(null, 10)tonumber('123', null)
🧹 Nitpick comments (2)
ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java (2)
12-214: Add base parameter boundary tests.The PR specifies that base must be between 2 and 36 inclusive. Consider adding tests to verify behavior with invalid base values:
- Base < 2 (e.g., base 0, base 1, negative base)
- Base > 36
- Base at valid boundaries (base 2, base 36)
12-16: Consider adding class-level JavaDoc.While test method names are descriptive, a class-level JavaDoc comment would help document the purpose and scope of these tests, especially since the coding guidelines specify that all public classes should have proper JavaDoc.
Example:
+/** + * Tests for TONUMBER function in PPL-to-Calcite translation. + * Validates logical plan generation, result correctness, and Spark SQL conversion + * across various number bases (binary, hexadecimal, decimal) and edge cases. + */ public class CalcitePPLToNumberFunctionTest extends CalcitePPLAbstractTest {As per coding guidelines, all public classes and methods should have proper JavaDoc.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java(1 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
**/*.java
📄 CodeRabbit inference engine (.rules/REVIEW_GUIDELINES.md)
**/*.java: UsePascalCasefor class names (e.g.,QueryExecutor)
UsecamelCasefor method and variable names (e.g.,executeQuery)
UseUPPER_SNAKE_CASEfor constants (e.g.,MAX_RETRY_COUNT)
Keep methods under 20 lines with single responsibility
All public classes and methods must have proper JavaDoc
Use specific exception types with meaningful messages for error handling
PreferOptional<T>for nullable returns in Java
Avoid unnecessary object creation in loops
UseStringBuilderfor string concatenation in loops
Validate all user inputs, especially queries
Sanitize data before logging to prevent injection attacks
Use try-with-resources for proper resource cleanup in Java
Maintain Java 11 compatibility when possible for OpenSearch 2.x
Document Calcite-specific workarounds in code
Files:
ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java
**/*Test.java
📄 CodeRabbit inference engine (.rules/REVIEW_GUIDELINES.md)
**/*Test.java: All new business logic requires unit tests
Name unit tests with*Test.javasuffix in OpenSearch SQL
Files:
ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java
🧠 Learnings (2)
📚 Learning: 2025-12-02T17:27:55.938Z
Learnt from: CR
Repo: opensearch-project/sql PR: 0
File: .rules/REVIEW_GUIDELINES.md:0-0
Timestamp: 2025-12-02T17:27:55.938Z
Learning: Test SQL generation and optimization paths for Calcite integration changes
Applied to files:
ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java
📚 Learning: 2025-12-02T17:27:55.938Z
Learnt from: CR
Repo: opensearch-project/sql PR: 0
File: .rules/REVIEW_GUIDELINES.md:0-0
Timestamp: 2025-12-02T17:27:55.938Z
Learning: Applies to **/*IT.java : Name integration tests with `*IT.java` suffix in OpenSearch SQL
Applied to files:
ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (28)
- GitHub Check: build-linux (25, doc)
- GitHub Check: build-linux (25, integration)
- GitHub Check: build-linux (21, doc)
- GitHub Check: bwc-tests-full-restart (21)
- GitHub Check: bwc-tests-full-restart (25)
- GitHub Check: build-linux (21, integration)
- GitHub Check: build-linux (25, unit)
- GitHub Check: build-linux (21, unit)
- GitHub Check: bwc-tests-rolling-upgrade (25)
- GitHub Check: bwc-tests-rolling-upgrade (21)
- GitHub Check: security-it-linux (25)
- GitHub Check: security-it-linux (21)
- GitHub Check: security-it-windows-macos (windows-latest, 25)
- GitHub Check: security-it-windows-macos (windows-latest, 21)
- GitHub Check: security-it-windows-macos (macos-14, 25)
- GitHub Check: security-it-windows-macos (macos-14, 21)
- GitHub Check: build-windows-macos (windows-latest, 25, -PbuildPlatform=windows, integration)
- GitHub Check: build-windows-macos (macos-14, 25, integration)
- GitHub Check: build-windows-macos (macos-14, 25, doc)
- GitHub Check: build-windows-macos (macos-14, 21, unit)
- GitHub Check: build-windows-macos (macos-14, 21, doc)
- GitHub Check: build-windows-macos (macos-14, 21, integration)
- GitHub Check: build-windows-macos (windows-latest, 25, -PbuildPlatform=windows, unit)
- GitHub Check: build-windows-macos (macos-14, 25, unit)
- GitHub Check: build-windows-macos (windows-latest, 21, -PbuildPlatform=windows, unit)
- GitHub Check: build-windows-macos (windows-latest, 21, -PbuildPlatform=windows, integration)
- GitHub Check: test-sql-cli-integration (21)
- GitHub Check: CodeQL-Scan (java)
🔇 Additional comments (3)
ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java (3)
14-16: LGTM!Standard test constructor properly initializes with the SCOTT_WITH_TEMPORAL schema.
18-214: LGTM! Consistent test structure.All test methods follow a consistent and thorough pattern:
- Define PPL query
- Build RelNode
- Verify logical plan
- Verify execution result
- Verify Spark SQL translation
This comprehensive approach ensures the TONUMBER function works correctly across the entire Calcite translation pipeline.
Based on learnings, SQL generation and optimization paths for Calcite integration changes should be tested.
127-164: Add documentation explaining the custom overflow behavior in ToNumberFunction tests.The tests validate that hexadecimal values exceeding Long's range produce custom overflow results (e.g.,
'FFFFFFFFFFFFFFFF'(base 16) → -1.0,'-FFFFFFFFFFFFFFFF'(base 16) → 1.0). This differs from standard JavaLong.parseLong()behavior, which throwsNumberFormatExceptionfor out-of-range values.Add inline comments to these test methods explaining why this custom overflow handling exists and what design goal it serves. If this behavior is documented elsewhere (e.g., in ToNumberFunction implementation or design docs), link to it from the tests for clarity.
|
@dai-chen please review. I have made return type double. Current test failures are unrelated . |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
🧹 Nitpick comments (2)
docs/user/ppl/functions/conversion.md (2)
300-332: Use fenced code blocks for consistency.The example code blocks are using indented style instead of fenced style. For consistency with the rest of the document and better syntax highlighting, use fenced code blocks with the
ppllanguage identifier.Apply this pattern to all four examples:
- os> source=people | eval int_value = tonumber('010101',2) | fields int_value | head 1 - fetched rows / total rows = 1/1 - +-----------+ - | int_value | - |-----------| - | 21.0 | - +-----------+ +```ppl +os> source=people | eval int_value = tonumber('010101',2) | fields int_value | head 1 +fetched rows / total rows = 1/1 ++-----------+ +| int_value | +|-----------| +| 21.0 | ++-----------+ +```Based on static analysis hints (markdownlint-cli2).
289-291: Simplify the technical details for user-facing documentation.The IEEE 754 format specifications and hexadecimal representations are overly technical for end-user documentation. Consider simplifying to something like: "Base values can be between 2 and 36. Supports standard numeric ranges for doubles (base 10) and signed 64-bit integers (other bases)."
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
core/src/main/java/org/opensearch/sql/expression/function/BuiltinFunctionName.java(1 hunks)core/src/main/java/org/opensearch/sql/expression/function/PPLBuiltinOperators.java(2 hunks)core/src/main/java/org/opensearch/sql/expression/function/PPLFuncImpTable.java(2 hunks)docs/user/ppl/functions/conversion.md(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.java
📄 CodeRabbit inference engine (.rules/REVIEW_GUIDELINES.md)
**/*.java: UsePascalCasefor class names (e.g.,QueryExecutor)
UsecamelCasefor method and variable names (e.g.,executeQuery)
UseUPPER_SNAKE_CASEfor constants (e.g.,MAX_RETRY_COUNT)
Keep methods under 20 lines with single responsibility
All public classes and methods must have proper JavaDoc
Use specific exception types with meaningful messages for error handling
PreferOptional<T>for nullable returns in Java
Avoid unnecessary object creation in loops
UseStringBuilderfor string concatenation in loops
Validate all user inputs, especially queries
Sanitize data before logging to prevent injection attacks
Use try-with-resources for proper resource cleanup in Java
Maintain Java 11 compatibility when possible for OpenSearch 2.x
Document Calcite-specific workarounds in code
Files:
core/src/main/java/org/opensearch/sql/expression/function/PPLBuiltinOperators.javacore/src/main/java/org/opensearch/sql/expression/function/BuiltinFunctionName.javacore/src/main/java/org/opensearch/sql/expression/function/PPLFuncImpTable.java
⚙️ CodeRabbit configuration file
**/*.java: - Verify Java naming conventions (PascalCase for classes, camelCase for methods/variables)
- Check for proper JavaDoc on public classes and methods
- Flag redundant comments that restate obvious code
- Ensure methods are under 20 lines with single responsibility
- Verify proper error handling with specific exception types
- Check for Optional usage instead of null returns
- Validate proper use of try-with-resources for resource management
Files:
core/src/main/java/org/opensearch/sql/expression/function/PPLBuiltinOperators.javacore/src/main/java/org/opensearch/sql/expression/function/BuiltinFunctionName.javacore/src/main/java/org/opensearch/sql/expression/function/PPLFuncImpTable.java
🧬 Code graph analysis (1)
core/src/main/java/org/opensearch/sql/expression/function/PPLBuiltinOperators.java (1)
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java (1)
ToNumberFunction(41-102)
🪛 markdownlint-cli2 (0.18.1)
docs/user/ppl/functions/conversion.md
300-300: Code block style
Expected: fenced; Actual: indented
(MD046, code-block-style)
310-310: Code block style
Expected: fenced; Actual: indented
(MD046, code-block-style)
321-321: Code block style
Expected: fenced; Actual: indented
(MD046, code-block-style)
332-332: Code block style
Expected: fenced; Actual: indented
(MD046, code-block-style)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (28)
- GitHub Check: security-it-linux (21)
- GitHub Check: security-it-linux (25)
- GitHub Check: build-linux (21, doc)
- GitHub Check: build-linux (21, integration)
- GitHub Check: bwc-tests-full-restart (21)
- GitHub Check: build-linux (25, doc)
- GitHub Check: build-linux (25, integration)
- GitHub Check: build-linux (25, unit)
- GitHub Check: build-linux (21, unit)
- GitHub Check: bwc-tests-rolling-upgrade (21)
- GitHub Check: bwc-tests-full-restart (25)
- GitHub Check: bwc-tests-rolling-upgrade (25)
- GitHub Check: security-it-windows-macos (macos-14, 21)
- GitHub Check: security-it-windows-macos (windows-latest, 21)
- GitHub Check: security-it-windows-macos (macos-14, 25)
- GitHub Check: security-it-windows-macos (windows-latest, 25)
- GitHub Check: build-windows-macos (macos-14, 25, integration)
- GitHub Check: build-windows-macos (macos-14, 25, doc)
- GitHub Check: build-windows-macos (macos-14, 25, unit)
- GitHub Check: build-windows-macos (macos-14, 21, doc)
- GitHub Check: build-windows-macos (windows-latest, 21, -PbuildPlatform=windows, unit)
- GitHub Check: build-windows-macos (windows-latest, 25, -PbuildPlatform=windows, integration)
- GitHub Check: build-windows-macos (macos-14, 21, unit)
- GitHub Check: build-windows-macos (windows-latest, 25, -PbuildPlatform=windows, unit)
- GitHub Check: build-windows-macos (macos-14, 21, integration)
- GitHub Check: build-windows-macos (windows-latest, 21, -PbuildPlatform=windows, integration)
- GitHub Check: CodeQL-Scan (java)
- GitHub Check: test-sql-cli-integration (21)
🔇 Additional comments (5)
core/src/main/java/org/opensearch/sql/expression/function/BuiltinFunctionName.java (1)
163-163: LGTM!The enum constant follows proper naming conventions and is correctly placed in the Text Functions section.
core/src/main/java/org/opensearch/sql/expression/function/PPLBuiltinOperators.java (2)
70-70: LGTM!Import is properly placed and used on line 416.
416-416: The TONUMBER operator declaration is correct; however, verify that the underlying ToNumberFunction implementation aligns with OpenSearch SQL standards.The operator declaration on line 416 follows the established pattern and is syntactically correct. However, the original concern about error handling should be investigated further. Based on OpenSearch SQL documentation, the framework typically uses CAST with explicit null-handling patterns rather than silent error suppression through empty catch blocks. If
ToNumberFunctionimplements silent null return via an empty catch block, this approach may diverge from standard OpenSearch SQL error handling patterns and should be reconsidered in favor of explicit error handling or validation.core/src/main/java/org/opensearch/sql/expression/function/PPLFuncImpTable.java (2)
221-221: LGTM!Import is properly placed and used on line 978.
978-978: LGTM!The TONUMBER operator registration follows the established pattern and is correctly placed alongside related text conversion functions.
|
@dai-chen please review. I have made return type double. Current test failures are unrelated and resolved conflicts |
44fa8c9 to
b34e374
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
♻️ Duplicate comments (1)
core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java (1)
81-93: Fix or remove misleading “Decimal” tests.
testToNumberWithDecimalBase2andtestToNumberWithDecimalBase16don’t use decimal-point inputs and mostly duplicate the earlier base-2/base-16 tests, despite the “Decimal” names.You may want to either:
- Remove these tests as redundant, or
- Change the inputs to actually contain decimal points and assert
nullfor non–base-10 decimals (similar totestToNumberInvalidDigits), and rename if needed.This will make the intent of these tests clearer and avoid duplication.
🧹 Nitpick comments (3)
docs/user/ppl/functions/conversion.md (1)
275-339: Use fenced code blocks and tighten TONUMBER wording.The new TONUMBER section looks good functionally, but there are a couple of polish items:
- Lines 300–338: the examples are indented code blocks, whereas the rest of this doc uses fenced blocks (
ppl /text). markdownlint (MD046) is already flagging this—please convert these to fenced blocks for consistency and to satisfy the linter.- Minor wording nits (optional):
- “converts the value in first argument” → “converts the value in the first argument”
- “The second argument describe” → “The second argument describes”
- Consider clarifying that the documented numeric limits are representable ranges, not hard validation limits enforced by the function.
core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java (1)
142-150: PreferassertNullfor invalid-digit cases (optional).For the invalid-digit scenarios you currently use
assertEquals(null, ...). UsingassertNull(ToNumberFunction.toNumber(...))would better express the intent and reads more clearly, but behavior is already correct.core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java (1)
41-101: Implementation aligns with documented TONUMBER semantics.The UDF wiring, base validation (2–36), operand metadata (
STRING_OR_STRING_INTEGER), andDOUBLE_FORCE_NULLABLEreturn inference all look consistent with the TONUMBER design. The statictoNumbermethods correctly implement:
- Base 10: integer vs decimal via
"."check, usingLong.parseLong/Double.parseDouble.- Non–base-10:
BigInteger(numStr, base).longValue().- Returning
nullon parse failures via the try/catch, matching the user doc’s NULL behavior for unparseable values.If you’d like to tighten things further, you could (optionally) narrow the catch to parsing-related exceptions and/or make the “returns null on invalid numeric text” contract explicit in method-level Javadoc, but the current logic is coherent.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (11)
core/src/main/java/org/opensearch/sql/calcite/utils/PPLOperandTypes.java(1 hunks)core/src/main/java/org/opensearch/sql/expression/function/BuiltinFunctionName.java(1 hunks)core/src/main/java/org/opensearch/sql/expression/function/PPLBuiltinOperators.java(2 hunks)core/src/main/java/org/opensearch/sql/expression/function/PPLFuncImpTable.java(2 hunks)core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java(1 hunks)core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java(1 hunks)docs/user/ppl/functions/conversion.md(1 hunks)integ-test/src/test/java/org/opensearch/sql/ppl/ConversionFunctionIT.java(1 hunks)ppl/src/main/antlr/OpenSearchPPLLexer.g4(1 hunks)ppl/src/main/antlr/OpenSearchPPLParser.g4(2 hunks)ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (7)
- ppl/src/main/antlr/OpenSearchPPLLexer.g4
- core/src/main/java/org/opensearch/sql/calcite/utils/PPLOperandTypes.java
- integ-test/src/test/java/org/opensearch/sql/ppl/ConversionFunctionIT.java
- core/src/main/java/org/opensearch/sql/expression/function/PPLFuncImpTable.java
- ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java
- ppl/src/main/antlr/OpenSearchPPLParser.g4
- core/src/main/java/org/opensearch/sql/expression/function/PPLBuiltinOperators.java
🧰 Additional context used
📓 Path-based instructions (3)
**/*.java
📄 CodeRabbit inference engine (.rules/REVIEW_GUIDELINES.md)
**/*.java: UsePascalCasefor class names (e.g.,QueryExecutor)
UsecamelCasefor method and variable names (e.g.,executeQuery)
UseUPPER_SNAKE_CASEfor constants (e.g.,MAX_RETRY_COUNT)
Keep methods under 20 lines with single responsibility
All public classes and methods must have proper JavaDoc
Use specific exception types with meaningful messages for error handling
PreferOptional<T>for nullable returns in Java
Avoid unnecessary object creation in loops
UseStringBuilderfor string concatenation in loops
Validate all user inputs, especially queries
Sanitize data before logging to prevent injection attacks
Use try-with-resources for proper resource cleanup in Java
Maintain Java 11 compatibility when possible for OpenSearch 2.x
Document Calcite-specific workarounds in code
Files:
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.javacore/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.javacore/src/main/java/org/opensearch/sql/expression/function/BuiltinFunctionName.java
⚙️ CodeRabbit configuration file
**/*.java: - Verify Java naming conventions (PascalCase for classes, camelCase for methods/variables)
- Check for proper JavaDoc on public classes and methods
- Flag redundant comments that restate obvious code
- Ensure methods are under 20 lines with single responsibility
- Verify proper error handling with specific exception types
- Check for Optional usage instead of null returns
- Validate proper use of try-with-resources for resource management
Files:
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.javacore/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.javacore/src/main/java/org/opensearch/sql/expression/function/BuiltinFunctionName.java
**/*Test.java
📄 CodeRabbit inference engine (.rules/REVIEW_GUIDELINES.md)
**/*Test.java: All new business logic requires unit tests
Name unit tests with*Test.javasuffix in OpenSearch SQL
Files:
core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java
**/test/**/*.java
⚙️ CodeRabbit configuration file
**/test/**/*.java: - Verify test coverage for new business logic
- Check test naming follows conventions (*Test.java for unit, *IT.java for integration)
- Ensure tests are independent and don't rely on execution order
- Validate meaningful test data that reflects real-world scenarios
- Check for proper cleanup of test resources
Files:
core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java
🧠 Learnings (4)
📚 Learning: 2025-12-02T17:27:55.938Z
Learnt from: CR
Repo: opensearch-project/sql PR: 0
File: .rules/REVIEW_GUIDELINES.md:0-0
Timestamp: 2025-12-02T17:27:55.938Z
Learning: Applies to **/*.java : All public classes and methods must have proper JavaDoc
Applied to files:
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java
📚 Learning: 2025-12-02T17:27:55.938Z
Learnt from: CR
Repo: opensearch-project/sql PR: 0
File: .rules/REVIEW_GUIDELINES.md:0-0
Timestamp: 2025-12-02T17:27:55.938Z
Learning: Flag missing JavaDoc on public APIs during code review
Applied to files:
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java
📚 Learning: 2025-12-02T17:27:55.938Z
Learnt from: CR
Repo: opensearch-project/sql PR: 0
File: .rules/REVIEW_GUIDELINES.md:0-0
Timestamp: 2025-12-02T17:27:55.938Z
Learning: Applies to **/*Test.java : Name unit tests with `*Test.java` suffix in OpenSearch SQL
Applied to files:
core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java
📚 Learning: 2025-12-02T17:27:55.938Z
Learnt from: CR
Repo: opensearch-project/sql PR: 0
File: .rules/REVIEW_GUIDELINES.md:0-0
Timestamp: 2025-12-02T17:27:55.938Z
Learning: Applies to **/*Test.java : All new business logic requires unit tests
Applied to files:
core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java
🧬 Code graph analysis (1)
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java (2)
core/src/main/java/org/opensearch/sql/calcite/utils/PPLOperandTypes.java (1)
PPLOperandTypes(19-324)core/src/main/java/org/opensearch/sql/expression/function/ImplementorUDF.java (1)
ImplementorUDF(17-40)
🪛 markdownlint-cli2 (0.18.1)
docs/user/ppl/functions/conversion.md
300-300: Code block style
Expected: fenced; Actual: indented
(MD046, code-block-style)
310-310: Code block style
Expected: fenced; Actual: indented
(MD046, code-block-style)
321-321: Code block style
Expected: fenced; Actual: indented
(MD046, code-block-style)
332-332: Code block style
Expected: fenced; Actual: indented
(MD046, code-block-style)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (28)
- GitHub Check: security-it-linux (21)
- GitHub Check: security-it-linux (25)
- GitHub Check: build-linux (25, integration)
- GitHub Check: bwc-tests-full-restart (25)
- GitHub Check: build-linux (21, integration)
- GitHub Check: build-linux (25, doc)
- GitHub Check: bwc-tests-full-restart (21)
- GitHub Check: build-linux (21, unit)
- GitHub Check: build-linux (21, doc)
- GitHub Check: build-linux (25, unit)
- GitHub Check: bwc-tests-rolling-upgrade (25)
- GitHub Check: bwc-tests-rolling-upgrade (21)
- GitHub Check: build-windows-macos (macos-14, 21, doc)
- GitHub Check: build-windows-macos (macos-14, 25, integration)
- GitHub Check: build-windows-macos (macos-14, 25, unit)
- GitHub Check: build-windows-macos (windows-latest, 25, -PbuildPlatform=windows, unit)
- GitHub Check: build-windows-macos (macos-14, 25, doc)
- GitHub Check: build-windows-macos (macos-14, 21, unit)
- GitHub Check: security-it-windows-macos (windows-latest, 21)
- GitHub Check: build-windows-macos (windows-latest, 25, -PbuildPlatform=windows, integration)
- GitHub Check: build-windows-macos (macos-14, 21, integration)
- GitHub Check: build-windows-macos (windows-latest, 21, -PbuildPlatform=windows, unit)
- GitHub Check: security-it-windows-macos (macos-14, 25)
- GitHub Check: build-windows-macos (windows-latest, 21, -PbuildPlatform=windows, integration)
- GitHub Check: security-it-windows-macos (macos-14, 21)
- GitHub Check: security-it-windows-macos (windows-latest, 25)
- GitHub Check: CodeQL-Scan (java)
- GitHub Check: test-sql-cli-integration (21)
🔇 Additional comments (1)
core/src/main/java/org/opensearch/sql/expression/function/BuiltinFunctionName.java (1)
161-167: TONUMBER registration looks consistent.The new
TONUMBER(FunctionName.of("tonumber"))entry is correctly named and grouped with the text functions, matching existing patterns.
dai-chen
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the changes! Please fix the DCO.
e5edbad to
3ffd8eb
Compare
3ffd8eb to
5f963a0
Compare
… part of roadmap opensearch-project#4287 Signed-off-by: Asif Bashar <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
♻️ Duplicate comments (1)
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java (1)
81-101: Return type inconsistency with declared inference.The method returns
Longfor integers (line 91) and non-base-10 numbers (line 95), butgetReturnTypeInference()declaresDOUBLE_FORCE_NULLABLE(line 51). According to past review comments, the decision was made to always returnDouble.This creates type inconsistency that can cause issues in Calcite's type system:
- Line 91: Returns
Long.parseLong(numStr)for base-10 integers- Line 95: Returns
bigInteger.longValue()for non-base-10 numbers- Only line 89 returns
Doublefor base-10 decimalsApply this diff to always return
Double:public static Number toNumber(String numStr, int base) { if (base < 2 || base > 36) { throw new IllegalArgumentException("Base has to be between 2 and 36."); } Number result = null; try { if (base == 10) { if (numStr.contains(".")) { result = Double.parseDouble(numStr); } else { - result = Long.parseLong(numStr); + result = (double) Long.parseLong(numStr); } } else { BigInteger bigInteger = new BigInteger(numStr, base); - result = bigInteger.longValue(); + result = (double) bigInteger.longValue(); } } catch (Exception e) { } return result; }Note: This issue was raised in previous reviews and reportedly addressed in commits 15817ae to 882b488, but the current code still shows the inconsistency.
🧹 Nitpick comments (1)
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java (1)
65-65: Remove unused variable.The variable
baseis declared but never used. The base value is either determined fromtranslatedOperandsor passed as a parameter.Apply this diff:
public Expression implement( RexToLixTranslator translator, RexCall call, List<Expression> translatedOperands) { Expression fieldValue = translatedOperands.get(0); - int base = 10; if (translatedOperands.size() > 1) {
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (11)
core/src/main/java/org/opensearch/sql/calcite/utils/PPLOperandTypes.java(1 hunks)core/src/main/java/org/opensearch/sql/expression/function/BuiltinFunctionName.java(1 hunks)core/src/main/java/org/opensearch/sql/expression/function/PPLBuiltinOperators.java(2 hunks)core/src/main/java/org/opensearch/sql/expression/function/PPLFuncImpTable.java(2 hunks)core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java(1 hunks)core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java(1 hunks)docs/user/ppl/functions/conversion.md(1 hunks)integ-test/src/test/java/org/opensearch/sql/ppl/ConversionFunctionIT.java(1 hunks)ppl/src/main/antlr/OpenSearchPPLLexer.g4(1 hunks)ppl/src/main/antlr/OpenSearchPPLParser.g4(2 hunks)ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (5)
- ppl/src/main/antlr/OpenSearchPPLLexer.g4
- core/src/main/java/org/opensearch/sql/expression/function/PPLFuncImpTable.java
- core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java
- core/src/main/java/org/opensearch/sql/expression/function/BuiltinFunctionName.java
- integ-test/src/test/java/org/opensearch/sql/ppl/ConversionFunctionIT.java
🧰 Additional context used
📓 Path-based instructions (5)
**/*.java
📄 CodeRabbit inference engine (.rules/REVIEW_GUIDELINES.md)
**/*.java: UsePascalCasefor class names (e.g.,QueryExecutor)
UsecamelCasefor method and variable names (e.g.,executeQuery)
UseUPPER_SNAKE_CASEfor constants (e.g.,MAX_RETRY_COUNT)
Keep methods under 20 lines with single responsibility
All public classes and methods must have proper JavaDoc
Use specific exception types with meaningful messages for error handling
PreferOptional<T>for nullable returns in Java
Avoid unnecessary object creation in loops
UseStringBuilderfor string concatenation in loops
Validate all user inputs, especially queries
Sanitize data before logging to prevent injection attacks
Use try-with-resources for proper resource cleanup in Java
Maintain Java 11 compatibility when possible for OpenSearch 2.x
Document Calcite-specific workarounds in code
Files:
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.javacore/src/main/java/org/opensearch/sql/calcite/utils/PPLOperandTypes.javacore/src/main/java/org/opensearch/sql/expression/function/PPLBuiltinOperators.javappl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java
⚙️ CodeRabbit configuration file
**/*.java: - Verify Java naming conventions (PascalCase for classes, camelCase for methods/variables)
- Check for proper JavaDoc on public classes and methods
- Flag redundant comments that restate obvious code
- Ensure methods are under 20 lines with single responsibility
- Verify proper error handling with specific exception types
- Check for Optional usage instead of null returns
- Validate proper use of try-with-resources for resource management
Files:
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.javacore/src/main/java/org/opensearch/sql/calcite/utils/PPLOperandTypes.javacore/src/main/java/org/opensearch/sql/expression/function/PPLBuiltinOperators.javappl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java
**/calcite/**/*.java
⚙️ CodeRabbit configuration file
**/calcite/**/*.java: - Follow existing patterns in CalciteRelNodeVisitor and CalciteRexNodeVisitor
- Verify SQL generation and optimization paths
- Document any Calcite-specific workarounds
- Test compatibility with Calcite version constraints
Files:
core/src/main/java/org/opensearch/sql/calcite/utils/PPLOperandTypes.javappl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java
**/*Test.java
📄 CodeRabbit inference engine (.rules/REVIEW_GUIDELINES.md)
**/*Test.java: All new business logic requires unit tests
Name unit tests with*Test.javasuffix in OpenSearch SQL
Files:
ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java
**/test/**/*.java
⚙️ CodeRabbit configuration file
**/test/**/*.java: - Verify test coverage for new business logic
- Check test naming follows conventions (*Test.java for unit, *IT.java for integration)
- Ensure tests are independent and don't rely on execution order
- Validate meaningful test data that reflects real-world scenarios
- Check for proper cleanup of test resources
Files:
ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java
**/ppl/**/*.java
⚙️ CodeRabbit configuration file
**/ppl/**/*.java: - For PPL parser changes, verify grammar tests with positive/negative cases
- Check AST generation for new syntax
- Ensure corresponding AST builder classes are updated
- Validate edge cases and boundary conditions
Files:
ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java
🧠 Learnings (4)
📚 Learning: 2025-12-02T17:27:55.938Z
Learnt from: CR
Repo: opensearch-project/sql PR: 0
File: .rules/REVIEW_GUIDELINES.md:0-0
Timestamp: 2025-12-02T17:27:55.938Z
Learning: Applies to **/*.java : All public classes and methods must have proper JavaDoc
Applied to files:
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java
📚 Learning: 2025-12-02T17:27:55.938Z
Learnt from: CR
Repo: opensearch-project/sql PR: 0
File: .rules/REVIEW_GUIDELINES.md:0-0
Timestamp: 2025-12-02T17:27:55.938Z
Learning: Flag missing JavaDoc on public APIs during code review
Applied to files:
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java
📚 Learning: 2025-12-02T17:27:55.938Z
Learnt from: CR
Repo: opensearch-project/sql PR: 0
File: .rules/REVIEW_GUIDELINES.md:0-0
Timestamp: 2025-12-02T17:27:55.938Z
Learning: Test SQL generation and optimization paths for Calcite integration changes
Applied to files:
ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java
📚 Learning: 2025-12-02T17:27:55.938Z
Learnt from: CR
Repo: opensearch-project/sql PR: 0
File: .rules/REVIEW_GUIDELINES.md:0-0
Timestamp: 2025-12-02T17:27:55.938Z
Learning: Applies to **/*IT.java : Name integration tests with `*IT.java` suffix in OpenSearch SQL
Applied to files:
ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java
🧬 Code graph analysis (2)
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java (2)
core/src/main/java/org/opensearch/sql/calcite/utils/PPLOperandTypes.java (1)
PPLOperandTypes(19-324)core/src/main/java/org/opensearch/sql/expression/function/ImplementorUDF.java (1)
ImplementorUDF(17-40)
core/src/main/java/org/opensearch/sql/expression/function/PPLBuiltinOperators.java (1)
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java (1)
ToNumberFunction(41-102)
🪛 markdownlint-cli2 (0.18.1)
docs/user/ppl/functions/conversion.md
300-300: Code block style
Expected: fenced; Actual: indented
(MD046, code-block-style)
310-310: Code block style
Expected: fenced; Actual: indented
(MD046, code-block-style)
321-321: Code block style
Expected: fenced; Actual: indented
(MD046, code-block-style)
332-332: Code block style
Expected: fenced; Actual: indented
(MD046, code-block-style)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (57)
- GitHub Check: build-linux (25, integration)
- GitHub Check: build-linux (25, unit)
- GitHub Check: build-linux (21, unit)
- GitHub Check: build-linux (21, doc)
- GitHub Check: bwc-tests-full-restart (25)
- GitHub Check: build-linux (25, doc)
- GitHub Check: build-linux (21, integration)
- GitHub Check: bwc-tests-rolling-upgrade (21)
- GitHub Check: bwc-tests-rolling-upgrade (25)
- GitHub Check: bwc-tests-full-restart (21)
- GitHub Check: security-it-linux (25)
- GitHub Check: security-it-linux (21)
- GitHub Check: security-it-linux (25)
- GitHub Check: security-it-linux (21)
- GitHub Check: build-linux (25, doc)
- GitHub Check: build-linux (21, integration)
- GitHub Check: bwc-tests-full-restart (25)
- GitHub Check: build-linux (21, unit)
- GitHub Check: build-linux (25, integration)
- GitHub Check: build-linux (21, doc)
- GitHub Check: bwc-tests-rolling-upgrade (25)
- GitHub Check: build-linux (25, unit)
- GitHub Check: bwc-tests-rolling-upgrade (21)
- GitHub Check: bwc-tests-full-restart (21)
- GitHub Check: WhiteSource Security Check
- GitHub Check: WhiteSource Security Check
- GitHub Check: build-windows-macos (windows-latest, 25, -PbuildPlatform=windows, unit)
- GitHub Check: build-windows-macos (macos-14, 25, doc)
- GitHub Check: build-windows-macos (windows-latest, 21, -PbuildPlatform=windows, integration)
- GitHub Check: build-windows-macos (macos-14, 25, unit)
- GitHub Check: build-windows-macos (windows-latest, 25, -PbuildPlatform=windows, integration)
- GitHub Check: security-it-windows-macos (windows-latest, 25)
- GitHub Check: build-windows-macos (macos-14, 21, doc)
- GitHub Check: build-windows-macos (macos-14, 21, unit)
- GitHub Check: build-windows-macos (windows-latest, 21, -PbuildPlatform=windows, unit)
- GitHub Check: build-windows-macos (macos-14, 21, integration)
- GitHub Check: build-windows-macos (macos-14, 25, integration)
- GitHub Check: security-it-windows-macos (macos-14, 25)
- GitHub Check: security-it-windows-macos (macos-14, 21)
- GitHub Check: security-it-windows-macos (windows-latest, 21)
- GitHub Check: security-it-windows-macos (windows-latest, 25)
- GitHub Check: security-it-windows-macos (macos-14, 21)
- GitHub Check: security-it-windows-macos (windows-latest, 21)
- GitHub Check: build-windows-macos (windows-latest, 21, -PbuildPlatform=windows, integration)
- GitHub Check: build-windows-macos (windows-latest, 21, -PbuildPlatform=windows, unit)
- GitHub Check: build-windows-macos (macos-14, 25, doc)
- GitHub Check: build-windows-macos (macos-14, 25, unit)
- GitHub Check: build-windows-macos (macos-14, 21, doc)
- GitHub Check: build-windows-macos (macos-14, 21, integration)
- GitHub Check: build-windows-macos (macos-14, 21, unit)
- GitHub Check: build-windows-macos (windows-latest, 25, -PbuildPlatform=windows, unit)
- GitHub Check: build-windows-macos (windows-latest, 25, -PbuildPlatform=windows, integration)
- GitHub Check: build-windows-macos (macos-14, 25, integration)
- GitHub Check: CodeQL-Scan (java)
- GitHub Check: CodeQL-Scan (java)
- GitHub Check: test-sql-cli-integration (21)
- GitHub Check: test-sql-cli-integration (21)
🔇 Additional comments (7)
core/src/main/java/org/opensearch/sql/calcite/utils/PPLOperandTypes.java (1)
108-112: LGTM! Operand metadata correctly supports optional base parameter.The new
STRING_OR_STRING_INTEGERoperand metadata correctly accepts either a single string argument or a string with an integer base, matching the TONUMBER function signature. The implementation follows existing patterns in the file.ppl/src/main/antlr/OpenSearchPPLParser.g4 (2)
1316-1316: LGTM! TONUMBER correctly added as text function.The addition of TONUMBER to
textFunctionNameproperly enables its usage as an eval function, consistent with other conversion functions in the grammar.
1523-1523: LGTM! TONUMBER correctly added as searchable keyword.Adding TONUMBER to
searchableKeyWordallows it to be used as an identifier where needed, following the same pattern as other function names.core/src/main/java/org/opensearch/sql/expression/function/PPLBuiltinOperators.java (2)
70-70: LGTM! Import correctly added.The import of
ToNumberFunctionis properly placed alphabetically within the UDF imports section.
416-416: LGTM! TONUMBER operator correctly registered.The TONUMBER operator instantiation follows the established pattern used by other conversion functions like TOSTRING, properly exposing the UDF with the "TONUMBER" name.
ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java (1)
1-215: Excellent test coverage for TONUMBER function.The test suite comprehensively covers:
- Multiple bases (binary, hex, decimal)
- Valid conversions with expected numeric results
- Invalid inputs returning NULL (unsupported formats like decimals in non-base-10)
- Boundary conditions (min/max values for Long range)
- Overflow behavior for values exceeding Long range
Each test properly verifies the logical plan, execution result, and Spark SQL translation, following established testing patterns.
Based on learnings: Test SQL generation and optimization paths for Calcite integration changes.
core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java (1)
23-40: LGTM! Comprehensive JavaDoc provided.The JavaDoc clearly documents the function usage, parameters, return type, valid base range, and numeric limits for different bases.
As per coding guidelines: All public classes and methods must have proper JavaDoc.
|
@penghuo we need 2 reviewers , please help review. |
…4287 (#4605) Signed-off-by: Asif Bashar <[email protected]> (cherry picked from commit 342a78b) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
… part of roadmap opensearch-project#4287 (opensearch-project#4605) Signed-off-by: Asif Bashar <[email protected]>
…4287 (#4963) * Feature tonumber : issue #4514 tonumber function as part of roadmap #4287 (#4605) Signed-off-by: Asif Bashar <[email protected]> * Enable Calcite in new IT Signed-off-by: Chen Dai <[email protected]> --------- Signed-off-by: Asif Bashar <[email protected]> Signed-off-by: Chen Dai <[email protected]> Co-authored-by: Asif Bashar <[email protected]>
Description
tonumber(, )
This function converts a string to a number.
Usage
You can use this function with the eval and where commands, in the WHERE clause of the from command, and as part of evaluation expressions with other commands.
The argument can be a string or the name of a field that contains a string. If the string contains a decimal point ( . ), then the tonumber function converts it to a double. Otherwise, the function converts the string to an integer.
Be aware that integers are supported differently in different product contexts:
The argument is optional. It defines the base of the number in the argument. It defaults to 10, which corresponds to the decimal system. You can set to a number between 2 and 36, inclusive.
If the tonumber function cannot parse a literal string to a number, the function returns an error.
Basic examples
The following example converts the string values from the store_sales field to numbers, and then stores the numbers in a field named n. This example uses the default base of 10.
... | eval n=tonumber(store_sales)
The following example takes the hexadecimal number and uses a base of 16 to return the number "164".
... | eval n=tonumber("0A4",16)
Resolves #4514
Related Issues
Resolves #4514
Check List
--signoffor-s.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.