Skip to content

Conversation

@findepi
Copy link
Member

@findepi findepi commented Oct 13, 2022

Before the change, the real and double round function would return a bogus value when long Math.round(double) invoked on the rescaled value returns Long.MAX_VALUE.

Fixes #14613

Before the change, the real and double `round` function would return a
bogus value when `long Math.round(double)` invoked on the rescaled value
returns `Long.MAX_VALUE`.
@martint martint merged commit 7e45535 into trinodb:master Oct 13, 2022
@findepi findepi self-assigned this Oct 14, 2022
@findepi findepi deleted the findepi/fix-real-double-round-returning-rescaled-long-max-value-7cde05 branch April 4, 2023 15:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

round(cast(num as double), P) starts returning an unexpected result when P is a large number.

2 participants