Skip to content

Conversation

@himadripal
Copy link
Contributor

@himadripal himadripal commented Feb 25, 2025

Which issue does this PR close?

Few important consideration -

  • Existing string to decimal conversion uses parse_string_to_decimal_native
  • parse_string_to_decimal_native does not have support for e-notation
  • parse_string_to_decimal_native does rounding at scale, not truncate
  • parse_decimal an existing method has e-notation support and use elsewhere
  • add rounding support in parse_decimal
  • moved string to decimal conversion to use parse_decimal to get support for e-notation.

This PR is a 3rd one to break up #6905 , this one add rounding logic to parse_decimal to match the behavior in existing parse_string_to_decimal_native.

Closes #.

Rationale for this change

At present, string to decimal conversion does not support e-notation, in arrow, parse_string_to_decimal_native is called to get generic string to decimal. parse_decimal on the other hand is used from generic parse method and it has e-notation support. This PR is adding rounding and scale 0 handling to match the behavior or parse_string_to_decimal_native method. Then we can replace parse_string_to_decimal_native call with parse_decimal. This way, we will get e-notation support too.

What changes are included in this PR?

Are there any user-facing changes?

add precision and scale in the error message
@github-actions github-actions bot added the arrow Changes to the arrow crate label Feb 25, 2025
@himadripal
Copy link
Contributor Author

#7179 - adds the rounding logic and with that merged, tests will pass. All failed tests are due to parse_decimal truncates instead of round, existing parse_string_to_decimal_native defaults to rounding.

I can make changes to test now and change again when #7179 is merged..

@alamb
Copy link
Contributor

alamb commented Mar 7, 2025

converting to draft as CI is failing -- I am working to clean up the review queue ahead of the release

@himadripal
Copy link
Contributor Author

himadripal commented Mar 7, 2025

converting to draft as CI is failing -- I am working to clean up the review queue ahead of the release

Since this PR is part of one big PR (#6905 ), other dependent part (#7179) needs to be merged first for the tests to be successful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arrow Changes to the arrow crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DataFusion should support casting strings such as "4e7" to decimal

2 participants