'ApprovalTests' fails in Catch 2.10.0

[amitherman95]

Describe the bug
CTest failure

Reproduction steps
just type 'ctest -j 2 --output-on-failure'

Platform information:

• OS: Windows 10
• Compiler+version Mingw32-w64
• Catch version: 2.10.0 latest version

Additional context
CTest output:

19/19 Test #12: ApprovalTests ..........................***Failed  Error regular expression found in output. Regex=[Results differed]  1.88 sec
Running approvals against executable:
  C:/Users/Amit/source/repos/Catch2/build/projects/SelfTest.exe

console.std:
    Results matched

console.sw:
--- console.sw.approved.txt

+++ console.sw.unapproved.txt

-  -0.0 and 0 are within 2.22045e-12% of each other
+  -0.0 and 0 are within 2.22045e-012% of each other
-  0.0 and 2.22507e-308 are within 2.22045e-12% of each other
+  0.0 and 2.22507e-308 are within 2.22045e-012% of each other
-  0.0f and 1.17549e-38 are within 0.00119209% of each other
+  0.0f and 1.17549e-038 are within 0.00119209% of each other

****************************
    Results differed

console.swa4:
    Results matched

junit.sw:
    Results matched


xml.sw:
--- xml.sw.approved.txt

+++ xml.sw.unapproved.txt

-            -0.0 and 0 are within 2.22045e-12% of each other
+            -0.0 and 0 are within 2.22045e-012% of each other
-              0.0 and 2.22507e-308 are within 2.22045e-12% of each other
+              0.0 and 2.22507e-308 are within 2.22045e-012% of each other
-              0.0f and 1.17549e-38 are within 0.00119209% of each other
+              0.0f and 1.17549e-038 are within 0.00119209% of each other

****************************
  Results differed


compact.sw:
--- compact.sw.approved.txt

+++ compact.sw.unapproved.txt

-Matchers.tests.cpp:<line number>: passed: -0., WithinRel(0.) for: -0.0 and 0 are within 2.22045e-12% of each other
-Matchers.tests.cpp:<line number>: passed: v1, WithinRel(v2) for: 0.0 and 2.22507e-308 are within 2.22045e-12% of each other
+Matchers.tests.cpp:<line number>: passed: -0., WithinRel(0.) for: -0.0 and 0 are within 2.22045e-012% of each other
+Matchers.tests.cpp:<line number>: passed: v1, WithinRel(v2) for: 0.0 and 2.22507e-308 are within 2.22045e-012% of each other
-Matchers.tests.cpp:<line number>: passed: v1, WithinRel(v2) for: 0.0f and 1.17549e-38 are within 0.00119209% of each other
+Matchers.tests.cpp:<line number>: passed: v1, WithinRel(v2) for: 0.0f and 1.17549e-038 are within 0.00119209% of each other

****************************
Results differed

If these differences are expected, run approve.py to approve new baselines.


95% tests passed, 1 tests failed out of 19

Total Test time (real) =   1.93 sec

The following tests FAILED:
         12 - ApprovalTests (Failed)
Errors while running CTest

Is this an expected behavior?

[JoeyGrajciar]

@amitherman95 can you reproduce that only on mingw or also on other compiler?

I would not be surprised if it would be Mingw issue similar to #1782 where INFINITY is double instead of float.
Can you test with latest release?

[horenmar]

Judging by the difference in stringification, the underlying library has decided to always print out 3 digits for the exponent of scientific numbers, unlike all other double formatting libraries.

On top of my head I can't say whether that is an allowed implementation, or straight out bug though.

[horenmar]

Judging by this

converts floating-point number to the decimal exponent notation.
For the e conversion style [-]d.ddde±dd is used.
For the E conversion style [-]d.dddE±dd is used.
The exponent contains at least two digits, more digits are used only if necessary. If the value is ​0​, the exponent is also ​0​. Precision specifies the minimum number of digits to appear after the decimal point character. The default precision is 6. In the alternative implementation decimal point character is written even if no digits follow it. For infinity and not-a-number conversion style see notes.

It is a straight out 3rd party bug.

[PureTryOut]

I can reproduce this on Alpine Linux using the latest catch2 (2.12.4 currently). @horenmar do you have a link to an issue for the 3rd party bug?