TypeError for NetCDF float16 output

[duncanwp]

ds = xr.Dataset({"test": np.arange(0, 5, dtype='float16')})
ds.to_netcdf('test.nc')

This fails because the float16 type doesn't exist for NetCDF files, throwing an TypeError: illegal primitive data type.

It might be nice if the xarray netCDF engine promoted this to float32 instead.

Output of xr.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.6.4.final.0 python-bits: 64 OS: Linux OS-release: 3.11.0-26-generic machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_GB.UTF-8 LOCALE: en_GB.UTF-8

xarray: 0.10.0
pandas: 0.22.0
numpy: 1.14.0
scipy: 1.0.0
netCDF4: 1.3.1
h5netcdf: 0.5.0
Nio: None
bottleneck: 1.2.1
cyordereddict: None
dask: 0.16.1
matplotlib: 2.1.2
cartopy: 0.15.1
seaborn: 0.8.1
setuptools: 38.4.0
pip: 9.0.1
conda: None
pytest: None
IPython: 6.2.1
sphinx: None

[shoyer]

It might be nice if the xarray netCDF engine promoted this to float32 instead.

I would generally lean against this type of behavior. Isn't it nice to know that float16 isn't supported rather than being surprised later when you discover your data was automatically upcast?

[shoyer]

I suppose we could do while issuing a warning. Using encoding={'test': {'dtype': np.float32}} in to_netcdf() would be the way to silence the warning.

[jhamman]

Do we have a verdict here? I'm personally fine raising an error (perhaps a better one than the current) unless an encoding/dtype is provided.

[duncanwp]

It's OK - I think @shoyer is right - it shouldn't silently promote the type, and the error message is clear. Thanks

[abinashmk]

It is good that we get a message if the data type is invalid. However, how does specifying the encoding type make it possible to write the data to a NetCDF file? Does it cast to that type? Does specifying the data type in encoding prevent the checks that raise the 'Illegal primitive data type error'?

[dcherian]

Does it cast to that type?

Yes