Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(NODE-5546): decimal 128 fromString performs inexact rounding #613

Merged
merged 5 commits into from
Aug 24, 2023

Conversation

W-A-James
Copy link
Contributor

@W-A-James W-A-James commented Aug 21, 2023

Description

What is changing?

  • Corrected Decimal128.fromString algorithm to match that in mongo-c-driver
  • Unskipped Decimal128 spec tests
  • Update Decimal128 spec tests
Is there new documentation needed for these changes?
  • No

What is the motivation for this change?

NODE-5546

Release Highlight

Decimal128 constructor now throws when detecting loss of precision

Prior to this release, Decimal128 would round numbers with more than 34 significant digits and lose precision. Now, on detecting loss of precision, Decimal128's constructor and Decimal128.fromString will throw a BSONError. This behaviour should have been the default as the Decimal128 class was always intended to be high-precision floating point value. As such, silently rounding is undesirable behaviour.

// previous behaviour
> new Decimal128('10000000000000000000000000000000001')
new Decimal128("1.000000000000000000000000000000000E+34")

// new behaviour
> new Decimal128('10000000000000000000000000000000001')
Uncaught:
BSONError: "10000000000000000000000000000000001" is not a valid Decimal128 string - inexact rounding
    at invalidErr (/home/wajames/js-bson/lib/bson.cjs:1402:11)
    at Decimal128.fromString (/home/wajames/js-bson/lib/bson.cjs:1555:21)
    at new Decimal128 (/home/wajames/js-bson/lib/bson.cjs:1411:37)

Double check the following

  • Ran npm run check:lint script
  • Self-review completed using the steps outlined here
  • PR title follows the correct format: type(NODE-xxxx)[!]: description
    • Example: feat(NODE-1234)!: rewriting everything in coffeescript
  • Changes are covered by tests
  • New TODOs have a related JIRA ticket

@W-A-James W-A-James changed the title fix(NODE-5546): Fix rounding error fix(NODE-5546): Fix rounding error and add new static constructor Aug 21, 2023
@W-A-James W-A-James changed the title fix(NODE-5546): Fix rounding error and add new static constructor feat(NODE-5546): Fix rounding error and add new static constructor Aug 21, 2023
@W-A-James W-A-James changed the title feat(NODE-5546): Fix rounding error and add new static constructor feat(NODE-5546): Fix fromString implementation to throw on loss of precision Aug 23, 2023
@W-A-James W-A-James marked this pull request as ready for review August 23, 2023 18:35
@W-A-James W-A-James changed the title feat(NODE-5546): Fix fromString implementation to throw on loss of precision fix(NODE-5546): Fix fromString implementation to throw on loss of precision Aug 24, 2023
src/decimal128.ts Outdated Show resolved Hide resolved
@@ -160,6 +160,7 @@ export class Decimal128 extends BSONValue {
static fromString(representation: string): Decimal128 {
// Parse state tracking
let isNegative = false;
let sawSign = false;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎵 I saw the sign
And it opened up my eyes, I saw the sign
Life is demanding without understanding 🎵

@@ -342,10 +346,9 @@ export class Decimal128 extends BSONValue {
// Shift exponent to significand and decrease
lastDigit = lastDigit + 1;

if (lastDigit - firstDigit > MAX_DIGITS) {
if (lastDigit - firstDigit >= MAX_DIGITS) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check this with a test case

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at it, firstDigit is always 0; it doesn't change at all throughout the entire function, so I'm going to go ahead and remove all references to it.

With regards to changing to the gte sign, the cases that trigger this branch are clamped zeros with large positive increments (see spec tests in decimal128-1.json), but any number with a sufficiently large exponent can enter this block.

If we consider a number like 1000000000000000000000000000000000e6112, which has 1 significant digit, 34 digits before the exponent and an exponent that's above max exponent, then what happens when we have a gt sign here is that we end up with the following result:

> Decimal128.fromString('1000000000000000000000000000000000e6112')
new Decimal128("0E+6111")

since we skip the iteration when lastDigit reaches MAX_DIGITS and proceed without error despite the fact that we should have overflowed. This leads us to get to our encoding and produce the incorrect result we see above.

With the gte sign in place, we get the following result, which is what we expect

> Decimal128.fromString('1000000000000000000000000000000000e6112')
Uncaught:
BSONError: "1000000000000000000000000000000000e6112" is not a valid Decimal128 string - overflow
    at invalidErr (/home/wajames/js-bson/lib/bson.cjs:1402:11)
    at Decimal128.fromString (/home/wajames/js-bson/lib/bson.cjs:1540:17)

it seems like we should have spec test that test corner cases like this.

src/decimal128.ts Show resolved Hide resolved
@nbbeeken nbbeeken changed the title fix(NODE-5546): Fix fromString implementation to throw on loss of precision fix(NODE-5546): decimal 128 fromString performs inexact rounding Aug 24, 2023
@nbbeeken nbbeeken self-assigned this Aug 24, 2023
@nbbeeken nbbeeken merged commit 1384cee into main Aug 24, 2023
4 checks passed
@nbbeeken nbbeeken deleted the NODE-5546 branch August 24, 2023 19:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants