SZIP/libaec support on Windows

[mkitti]

Solution to issue cannot be found in the documentation.

• I checked the documentation.

Issue

Refiling of conda-forge/h5py-feedstock#63 to here

This is a known issue from #179 (comment) and the residual of #120 .

I'm noting that MSYS2 has a patch that possibly addresses this issue.
https://github.com/msys2/MINGW-packages/blob/master/mingw-w64-hdf5/hdf5-fix-find-szip.patch
https://packages.msys2.org/base/mingw-w64-hdf5

Upstream appears to have fixed this issue.
HDFGroup/hdf5#2492

Installed packages

bzip2              conda-forge/win-64::bzip2-1.0.8-h8ffe710_4
  ca-certificates    conda-forge/win-64::ca-certificates-2022.12.7-h5b45459_0
  cached-property    conda-forge/noarch::cached-property-1.5.2-hd8ed1ab_1
  cached_property    conda-forge/noarch::cached_property-1.5.2-pyha770c72_1
  h5py               conda-forge/win-64::h5py-3.8.0-nompi_py311hfbc32a6_101
  hdf5               conda-forge/win-64::hdf5-1.14.0-nompi_h5af7005_102
  intel-openmp       conda-forge/win-64::intel-openmp-2023.0.0-h57928b3_25922
  krb5               conda-forge/win-64::krb5-1.20.1-heb0366b_0
  libblas            conda-forge/win-64::libblas-3.9.0-16_win64_mkl
  libcblas           conda-forge/win-64::libcblas-3.9.0-16_win64_mkl
  libcurl            conda-forge/win-64::libcurl-7.88.1-h68f0423_0
  libffi             conda-forge/win-64::libffi-3.4.2-h8ffe710_5
  libhwloc           conda-forge/win-64::libhwloc-2.9.0-h51c2c0f_0
  libiconv           conda-forge/win-64::libiconv-1.17-h8ffe710_0
  liblapack          conda-forge/win-64::liblapack-3.9.0-16_win64_mkl
  libsqlite          conda-forge/win-64::libsqlite-3.40.0-hcfcfb64_0
  libssh2            conda-forge/win-64::libssh2-1.10.0-h9a1e1f7_3
  libxml2            conda-forge/win-64::libxml2-2.10.3-hc3477c8_0
  libzlib            conda-forge/win-64::libzlib-1.2.13-hcfcfb64_4
  mkl                conda-forge/win-64::mkl-2022.1.0-h6a75c08_874
  numpy              conda-forge/win-64::numpy-1.24.2-py311h0b4df5a_0
  openssl            conda-forge/win-64::openssl-3.0.8-hcfcfb64_0
  pip                conda-forge/noarch::pip-23.0.1-pyhd8ed1ab_0
  pthreads-win32     conda-forge/win-64::pthreads-win32-2.9.1-hfa6e2cd_3
  python             conda-forge/win-64::python-3.11.0-hcf16a7b_0_cpython
  python_abi         conda-forge/win-64::python_abi-3.11-3_cp311
  setuptools         conda-forge/noarch::setuptools-67.5.1-pyhd8ed1ab_0
  tbb                conda-forge/win-64::tbb-2021.8.0-h91493d7_0
  tk                 conda-forge/win-64::tk-8.6.12-h8ffe710_0
  tzdata             conda-forge/noarch::tzdata-2022g-h191b570_0
  ucrt               conda-forge/win-64::ucrt-10.0.22621.0-h57928b3_0
  vc                 conda-forge/win-64::vc-14.3-hb6edc58_10
  vs2015_runtime     conda-forge/win-64::vs2015_runtime-14.34.31931-h4c5c07a_10
  wheel              conda-forge/noarch::wheel-0.38.4-pyhd8ed1ab_0
  xz                 conda-forge/win-64::xz-5.2.6-h8d14728_0

Environment info

(h5py_hdf5_test) C:\>conda info

     active environment : h5py_hdf5_test
    active env location : C:\Users\_\.julia\conda\3\envs\h5py_hdf5_test
            shell level : 2
       user config file : C:\Users\_\.condarc
 populated config files :
          conda version : 4.10.3
    conda-build version : not installed
         python version : 3.9.5.final.0
       virtual packages : __cuda=11.7=0
                          __win=0=0
                          __archspec=1=x86_64
       base environment : C:\Users\_\.julia\conda\3  (writable)
      conda av data dir : C:\Users\_\.julia\conda\3\etc\conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : C:\Users\_\.julia\conda\3\pkgs
                          C:\Users\_\.conda\pkgs
                          C:\Users\_\AppData\Local\conda\conda\pkgs
       envs directories : C:\Users\_\.julia\conda\3\envs
                          C:\Users\_\.conda\envs
                          C:\Users\_\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/4.10.3 requests/2.25.1 CPython/3.9.5 Windows/10 Windows/10.0.19044
          administrator : False
             netrc file : None
           offline mode : False

[mkitti]

The observation that the libaec builds on Windows appear to be working in zarr-developers/numcodecs#422 suggests the problem is mostly with the cmake build system.

A small patch similar to the MSYS2 patch may be successful. Alternatively, we wait for upstream which should have a patch release near the beginning of May 2023 according to the HDF Group release schedule.

[jakirkham]

Would suggest trying to drop the comment and selector here and bump the build/number

hdf5-feedstock/recipe/meta.yaml

Lines 85 to 86 in 085d052

	# sz support was never working on conda-forge for windows
	- libaec # [unix]

Should add currently there are warnings like this on Windows:

CMake Warning:
-- Build files have been written to: D:/bld/hdf5_1675776169379/work/build
  Manually-specified variables were not used by the project:

    SZIP_INCLUDE_DIR
    SZIP_LIBRARY

So lines like these probably need to be reworked or dropped:

hdf5-feedstock/recipe/bld.bat

Lines 23 to 24 in 085d052

	-D SZIP_LIBRARY:FILEPATH=%LIBRARY_PREFIX%\lib\libsz.lib ^
	-D SZIP_INCLUDE_DIR:PATH=%LIBRARY_PREFIX%\include ^

[mkitti]

Steps to reproduce the problem.

>>> import h5py

>>> with h5py.File("test.h5", "w") as h5f:
...     ds = h5f.create_dataset("test", (1024, 1024), chunks=(16,16), dtype="uint16", compression="szip", compression_opts=('ec',32))
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "C:\Users\_\.julia\conda\3\envs\h5py_hdf5_test\Lib\site-packages\h5py\_hl\group.py", line 183, in create_dataset
    dsid = dataset.make_new_dset(group, shape, dtype, data, name, **kwds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\_\.julia\conda\3\envs\h5py_hdf5_test\Lib\site-packages\h5py\_hl\dataset.py", line 106, in make_new_dset
    dcpl = filters.fill_dcpl(
           ^^^^^^^^^^^^^^^^^^
  File "C:\Users\_\.julia\conda\3\envs\h5py_hdf5_test\Lib\site-packages\h5py\_hl\filters.py", line 185, in fill_dcpl
    raise ValueError('Compression filter "%s" is unavailable' % compression)
ValueError: Compression filter "szip" is unavailable

>>> h5py.h5z.filter_avail(4)
False