-
Notifications
You must be signed in to change notification settings - Fork 44
Closed
Description
When experimenting with reenabling dune runtest under Windows (MinGW) in #292 I've noticed that they fail expect tests due to different stddev's being printed.
There's a reported difference with both OCaml 4.14, 5.0, and 5.2:
stats dist:
- num: 100000, avg: -7215552342607541.00, stddev: 2666234426234218496.00, median -16620417636667326, min -4611371852367564818, max 4611613630315464842
+ num: 100000, avg: -7215552342607541.00, stddev: 2666234426234218500.00, median -16620417636667326, min -4611371852367564818, max 4611613630315464842Stddev is computed as follows:
let stddev =
Hashtbl.fold
(fun i res m -> m +. (float_of_int i -. !avg) ** 2. *. float_of_int res)
tbl 0.
|> (fun s -> if !num>0 then s /. float_of_int !num else s)
|> sqrtand printed as:
Printf.bprintf out
" num: %d, avg: %.2f, stddev: %.2f, median %d, min %d, max %d\n"
!num !avg stddev !median min_idx max_idx;I've not dug further into the reason at this point, but suspect that one of the involved floating point operations is causing the difference.
Full details from the 5.2 run:
https://github.com/c-cube/qcheck/actions/runs/9796388729/job/27050729649?pr=292
File "test/core/QCheck2_expect_test.expected", line 1, characters 0-0:
"C:\Program Files\Git\bin\git.exe" --no-pager diff --no-index --color=always -u --ignore-cr-at-eol _build/default/test/core/QCheck2_expect_test.expected _build/default/test/core/QCheck2_expect_test.output
diff --git a/_build/default/test/core/QCheck2_expect_test.expected b/_build/default/test/core/QCheck2_expect_test.output
index bce5b4b..47e0420 100644
--- a/_build/default/test/core/QCheck2_expect_test.expected
+++ b/_build/default/test/core/QCheck2_expect_test.output
@@ -1405,7 +1405,7 @@ stats dist:
+++ Stats for int dist ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
stats dist:
- num: 100000, avg: -7215552342607541.00, stddev: 2666234426234218496.00, median -16620417636667326, min -4611371852367564818, max 4611613630315464842
+ num: 100000, avg: -7215552342607541.00, stddev: 2666234426234218500.00, median -16620417636667326, min -4611371852367564818, max 4611613630315464842
-4611371852367564818..-4150222578233413331: ##################################################### 5003
-4150222578233413330..-3689073304099261843: ####################################################### 5106
-3689073304099261842..-3227924029965110355: ###################################################### 5052
@@ -1430,7 +1430,7 @@ stats dist:
+++ Stats for oneof int dist ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
stats dist:
- num: 1000, avg: 4611686018427388.00, stddev: 2891078433912002560.00, median 0, min -4611686018427387904, max 4611686018427387903
+ num: 1000, avg: 4611686018427388.00, stddev: 2891078433912002600.00, median 0, min -4611686018427387904, max 4611686018427387903
-4611686018427387904..-4150517416584649089: ################# 198
-4150517416584649088..-3689348814741910273: 0
-3689348814741910272..-3228180212899171457: 0
@@ -1459,7 +1459,7 @@ random seed: 153870556
+++ Stats for int_dist_empty_bucket ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
stats dist:
- num: 1000, avg: -38152533987928128.00, stddev: 1828925617669212928.00, median 9, min -4606614955625884935, max 4611686018427387903
+ num: 1000, avg: -38152533987928128.00, stddev: 1828925617669212900.00, median 9, min -4606614955625884935, max 4611686018427387903
-4606614955625884935..-4145699906923221320: ## 27
-4145699906923221319..-3684784858220557704: ## 22
-3684784858220557703..-3223869809517894088: ## 29
File "test/core/QCheck_expect_test.expected", line 1, characters 0-0:
"C:\Program Files\Git\bin\git.exe" --no-pager diff --no-index --color=always -u --ignore-cr-at-eol _build/default/test/core/QCheck_expect_test.expected _build/default/test/core/QCheck_expect_test.output
diff --git a/_build/default/test/core/QCheck_expect_test.expected b/_build/default/test/core/QCheck_expect_test.output
index ed35317..e7f0b76 100644
--- a/_build/default/test/core/QCheck_expect_test.expected
+++ b/_build/default/test/core/QCheck_expect_test.output
@@ -1361,7 +1361,7 @@ stats dist:
+++ Stats for int dist ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
stats dist:
- num: 100000, avg: -7215552342607541.00, stddev: 2666234426234218496.00, median -16620417636667326, min -4611371852367564818, max 4611613630315464842
+ num: 100000, avg: -7215552342607541.00, stddev: 2666234426234218500.00, median -16620417636667326, min -4611371852367564818, max 4611613630315464842
-4611371852367564818..-4150222578233413331: ##################################################### 5003
-4150222578233413330..-3689073304099261843: ####################################################### 5106
-3689073304099261842..-3227924029965110355: ###################################################### 5052
@@ -1386,7 +1386,7 @@ stats dist:
+++ Stats for oneof int dist ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
stats dist:
- num: 1000, avg: 4611686018427388.00, stddev: 2891078433912002560.00, median 0, min -4611686018427387904, max 4611686018427387903
+ num: 1000, avg: 4611686018427388.00, stddev: 2891078433912002600.00, median 0, min -4611686018427387904, max 4611686018427387903
-4611686018427387904..-4150517416584649089: ################# 198
-4150517416584649088..-3689348814741910273: 0
-3689348814741910272..-3228180212899171457: 0
@@ -1415,7 +1415,7 @@ random seed: 153870556
+++ Stats for int_dist_empty_bucket ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
stats dist:
- num: 1000, avg: -38152533987928128.00, stddev: 1828925617669212928.00, median 9, min -4606614955625884935, max 4611686018427387903
+ num: 1000, avg: -38152533987928128.00, stddev: 1828925617669212900.0Metadata
Metadata
Assignees
Labels
No labels