Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fix float64 comparison test failure on archs using FMA
Architectures using FMA optimization yield slightly different results so we cannot assume floating point values will be precisely the same across different architectures. The solution in this change is to check "abs(a-b) < tolerance" instead of comparing the exact values. This will give us confidence that the histogram buckets are near identical. Signed-off-by: Seth Bunce <[email protected]>
- Loading branch information