More PY3 support for FormatPY, the xfel legacy image pickle format

[phyy-nx]

Current dxtbx testing for FormatPY uses pickles that were created using py2. While reprocessing old LCLS data I regenerated some image pickles on a py3 build and found I couldn't read them because of encoding problems. This PR fixes that problem using a new function in iotbx/npy.py, image_dict_to_unicode.

Additionally, this makes the understand method of FormatPY more robust. Adds pickle protocol 4 magic bytes check but also tries to de-pickle the file, as the pickle magic bytes aren't guaranteed to fully specify whether a file is a pickle.

Note, the only place image pickles are generated in xfel land is averaging XTC streams with Rayonix data. Everyting else, including xtc_process using Rayonix data, uses cbfs.

[phyy-nx]

This PR is not urgent :) Anyone want to self-assign as a reviewer? Thanks!

[phyy-nx]

Thanks @Anthchirp. I wrote a snippet to look at pickling results on the 6 protocols and then tried it on a couple systems and got slightly different answers. The new version should cover all the bases.

import pickle

with open('dials_regression/image_examples/LCLS_CXI/shot-s00-2011-12-02T21_07Z29.723_00569.pickle', 'rb') as f:
  d = pickle.load(f, encoding='bytes')

def showit(protocol):
  with open('demo_pickle_protocol.pickle', 'wb') as f:
    pickle.dump(d, f, protocol=protocol)

  with open('demo_pickle_protocol.pickle', 'rb') as f:
    print ("protocol", protocol, f.read()[0:4])

for i in range(6):
  showit(i)

Output

02T21_07Z29.723_00569.pickle'>
protocol 0 b'(dp0'
protocol 1 b'}q\x00('
protocol 2 b'\x80\x02}q'
protocol 3 b'\x80\x03}q'
protocol 4 b'\x80\x04\x95\xce'
protocol 5 b'\x80\x05\x95\xce'

[ndevenish]

This fixes python3 reading all pickle images, or pickle images written from python3? Just trying to work out an accurate phrasing for a newsfragments/205.bugfix.

Otherwise don't see why this can't be squashed in.

[phyy-nx]

It fixes pickle images written in py3.