Convert from PyObject to Matrix fails

[lstagner]

Converting a PyObject to a Matrix fails when reading Matrix from this IDL save file using scipy.io.readsav

julia> using PyCall

julia> @pyimport scipy.io as io

julia> a=io.readsav("weight_and_dist.sav")
/usr/lib/python2.7/dist-packages/scipy/io/idl.py:167: UserWarning: warning: empty strings are now set to '' instead of None
  warnings.warn("warning: empty strings are now set to '' instead of None")
Dict{Any,Any} with 8 entries:
  "w_matrix" => PyObject array([[  1.68552301e-101,   2.356978…
  "m02_fbm"  => PyObject rec.array([ (array([[  1.34499655e+10…
  "pitch"    => PyObject array([-0.89999998, -0.70000005, -0.5…
  "mean_fbm" => PyObject array([[  1.31980645e+10,   3.3166579…
  "v17_fbm"  => PyObject rec.array([ (array([[  1.30541168e+10…
  "energy"   => PyObject array([  5.,  15.,  25.,  35.,  45., …
  "m10_fbm"  => PyObject rec.array([ (array([[  1.26698307e+10…
  "p2_fbm"   => PyObject rec.array([ (array([[  1.36183452e+10…

julia> w_matrix = a["w_matrix"]
PyObject array([[  1.68552301e-101,   2.35697831e-066,   3.61510788e-046, ...,
          3.25287399e+003,   6.30708487e+003,   6.26874721e+003],
       [  3.42516635e-103,   9.95975928e-068,   2.55145289e-047, ...,
          2.85362314e+003,   5.95774610e+003,   6.24852521e+003],
       [  6.73923741e-105,   4.07514829e-069,   1.74358145e-048, ...,
          2.47492336e+003,   5.58385846e+003,   6.20714822e+003],
       ..., 
       [  0.00000000e+000,   0.00000000e+000,   0.00000000e+000, ...,
          4.78256276e-049,   6.15881422e-073,   2.18231882e-100],
       [  0.00000000e+000,   0.00000000e+000,   0.00000000e+000, ...,
          2.07619108e-050,   1.37890846e-074,   2.53424699e-102],
       [  0.00000000e+000,   0.00000000e+000,   0.00000000e+000, ...,
          8.63614392e-052,   2.95848174e-076,   2.81982090e-104]])

julia> w = convert(Array{Float64,2},w_matrix)
ERROR: PyError (PyFloat_AsDouble) <type 'exceptions.TypeError'>
TypeError('only length-1 arrays can be converted to Python scalars',)

 in pyerr_check at /home/lstagner/.julia/v0.4/PyCall/src/exception.jl:58
 in convert at /home/lstagner/.julia/v0.4/PyCall/src/conversions.jl:75
 in py2array at /home/lstagner/.julia/v0.4/PyCall/src/conversions.jl:339
 in convert at /home/lstagner/.julia/v0.4/PyCall/src/numpy.jl:430

Relevant mailing list post

[stevengj]

Should be fixed now. The issue was:

• The data is a NumPy array, but it is not stored in the native byte order, and PyArray doesn't currently support arrays in non-native byte order.
• The fallback py2array was being a little too conservative in determining the dimensions of the array because it is not a PyList type. The patch just modifies it to use the NumPy dimensions when they are available.