Format_instance caching leads to wrong data when reading the same file twice

[dwpaley]

This is an odd one.

On branch lazy_HDF5SaclaMPCCD:

from dxtbx.model.experiment_list import ExperimentListFactory

master_h5 = "/net/dials/raid1/dwpaley/dials/build/dials_data/image_examples/SACLA-MPCCD-Phase3-21528-5images.h5"

expts1 = ExperimentListFactory.from_filenames([master_h5])
expts1[0].imageset.get_mask(0)
#expts1[0].imageset.reader().nullify_format_instance() # this would fix it
expts2 = ExperimentListFactory.from_filenames([master_h5])

for e1, e2 in zip(expts1, expts2):
    print(
        e1.imageset.get_beam().get_wavelength() ==
        e2.imageset.get_beam().get_wavelength()
        )

Output:

False
True
True
True
True

In other words: if we construct an experiment list, call the get_mask() method of the imageset of the first experiment, and then construct another experiment list from the same file; then the first beam in the second experiment list is wrong.

A few notes:

• This only happens if FormatHDF5SaclaMPCCD inherits from FormatMultiImageLazy. (i.e. if the beams are not read when constructing the imageset).
• It only happens if we get the mask from the imageset of the first experiment. If we do expts1[1].imageset.get_mask(0), all the beams match.
• Out of the 13 get_xxx methods in expts1[0].imageset, only get_corrected_data, get_raw_data and get_mask cause this problem.
• There have been similar problems in the past; see the two places where nullify_format_instance is called in test_HDF5SaclaMPCCD.py.
• I don't know of a real application where this causes a problem.

For now I'm adding an xfailing test on the aforementioned branch lazy_HDF5SaclaMPCCD.

Thanks @phyy-nx for discussing this at length.