You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
// Limit precision to the maximum possible number of significant digits in// an IEEE754 double because we don't need to generate zeros.constint max_double_digits = 767;
if (precision > max_double_digits) precision = max_double_digits;
seem to cause an unexpected truncation in the representation of floating-point numbers.
Ref: https://stackoverflow.com/questions/70122920/maximum-allowed-value-of-the-precision-format-specifier-in-the-fmt-library
The following lines in format-inl.h
seem to cause an unexpected truncation in the representation of floating-point numbers.
Compare the results of this snippet (a)
To this one (b)
It seems to me that the code using
max_double_digits
should take into account the leading zeroes after the decimal point.The text was updated successfully, but these errors were encountered: