Skip to content

Commit

Permalink
Fix bin1d_vec to properly account for floating point precision
Browse files Browse the repository at this point in the history
The tolerance for bin size `h` must be based on eps and the _involved_ numbers in calculating it, not the resulting number. The two involved numbers are of the same order of magnitude, and one if it is `a0` (the first bin edge `bin[0] == min(bins)`), for which we already calculate the tolerance (`a0_tol`). So we can use `a0_tol` also as tolerance for `h`.
  • Loading branch information
mherrmann3 committed Feb 5, 2025
1 parent 2feb914 commit e12f6a0
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion csep/utils/calc.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ def bin1d_vec(p, bins, tol=None, right_continuous=False):
h = bins[1] - bins[0]

a0_tol = numpy.abs(a0) * numpy.finfo(numpy.float64).eps
h_tol = numpy.abs(h) * numpy.finfo(numpy.float64).eps
h_tol = a0_tol # must be based on *involved* numbers
p_tol = numpy.abs(p) * numpy.finfo(numpy.float64).eps

# absolute tolerance
Expand Down

0 comments on commit e12f6a0

Please sign in to comment.