Number of instances displayed vs written disagree

[nikohansen]

Like in this figure

which was generated for a paper.

We see 10 steps in the ECDF, but 15 instances written. Why?

See also #1972, #1117.

[brockho]

After quite some experimentation, I found the cause: The number of steps in the ECDFs is not the number of instances, but the number of bootstrapped runs. Because line 50 of genericsettings.py:

simulated_runlength_bootstrap_sample_size = 10 + 990 // (1 + 10 * max((0, in_a_hurry)))  # for tables and plots

evaluates to 10 with the default setting of in_a_hurry, we see only 10 (simulated) runs in the figure. Changing line 50 to, for example, 7 and with pprldmany_target_values=[1e-3] in testbedsettings.py gives the following output with the ALPS data set:

[brockho]

Update: I also found the same line
settings.simulated_runlength_bootstrap_sample_size = 10 + 990 / (1 + 10 * max(0, settings.in_a_hurry))
within config.py...

[nikohansen]

Excellent, thanks, that means we should probably better use the maximum of the number of trials and number of bootstraps. We still need to fix #1972 at some point, I believe I have already some code around.

[nikohansen]

Should be fixed in the development branch.