Skip to content

Commit

Permalink
Avoid calling non-public h5fd_xxx_init() functions (#967)
Browse files Browse the repository at this point in the history
* Avoid calling non-public h5fd_xxx_init() functions

The h5fd_xxx_init() functions are not part of the HDF5 public API.  A
method of retrieving the hid_t value of drivers that uses
public/supported API calls was suggested in a comment on a related
github issue and is implemented in this commit.  For more details see:

HDFGroup/hdf5#1809 (comment)

* Initialize fapl minimally in Drivers.__init__()

* Disable file locking in Drivers.__init()

Unless the user has already set `HDF5_USE_FILE_LOCKING` in the
environment, set it to `FALSE` to avoid causing problems with mmap'ing.
This needs to be done before the first call to the HDF5 APIs.

* Change how parallel HDF5 suppport is detected

Instead of detecting parallel HDF5 support by whether `h5fd_mpio_init()`
throws an exception, parallel support is detected by finding the
`H5Pset_fapl_mpio` symbol in any loaded library whose name matches
`r"hdf5"i`.  The documentation for `H5Pset_fapl_mpio()` states:

> H5Pset_fapl_mpio() is available only in the parallel HDF5 library

The detection has also been moved into `Drivers.__init__()`
and the inclusion of `mpio.jl` (upon loading of the MPI package) will
only happen if the HDF5 libraries have parallel support.

* Add DRIVERS entry for MPIO lazily in set_driver!

* Replace `contains()` with call to older `match()`

This maintains compatibility with pre-1.5 versions of Julia.

* Improve usage of Libdl

* Use HDF5.API.libhdf5 rather than dllist()
  • Loading branch information
david-macmahon authored Jun 18, 2022
1 parent 3e2a7a9 commit 470eabc
Show file tree
Hide file tree
Showing 2 changed files with 22 additions and 10 deletions.
22 changes: 19 additions & 3 deletions src/drivers/drivers.jl
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ export POSIX
import ..API
import ..HDF5: HDF5, Properties, h5doc

using Libdl: dlopen, dlsym
using Requires: @require


Expand Down Expand Up @@ -85,9 +86,24 @@ function set_driver!(p::Properties, ::POSIX)
end

function __init__()
DRIVERS[API.h5fd_core_init()] = Core
DRIVERS[API.h5fd_sec2_init()] = POSIX
@require MPI="da04e1cc-30fd-572f-bb4f-1f8673147195" include("mpio.jl")
# disable file locking as that can cause problems with mmap'ing
if !haskey(ENV, "HDF5_USE_FILE_LOCKING")
ENV["HDF5_USE_FILE_LOCKING"] = "FALSE"
end

fapl = HDF5.init!(HDF5.FileAccessProperties())
API.h5p_set_fapl_core(fapl, 4096, false)
DRIVERS[API.h5p_get_driver(fapl)] = Core
API.h5p_set_fapl_sec2(fapl)
DRIVERS[API.h5p_get_driver(fapl)] = POSIX
close(fapl)

# Check whether the libhdf5 was compiled with parallel support.
HDF5.HAS_PARALLEL[] = dlopen(HDF5.API.libhdf5) do lib
dlsym(lib, :H5Pset_fapl_mpio; throw_error=false) !== nothing
end

@require MPI="da04e1cc-30fd-572f-bb4f-1f8673147195" (HDF5.has_parallel() && include("mpio.jl"))
end

end # module
10 changes: 3 additions & 7 deletions src/drivers/mpio.jl
Original file line number Diff line number Diff line change
Expand Up @@ -42,13 +42,6 @@ end
MPIO(comm::MPI.Comm; kwargs...) =
MPIO(comm, MPI.Info(;kwargs...))

# Check whether the HDF5 libraries were compiled with parallel support.
try
DRIVERS[API.h5fd_mpio_init()] = MPIO
HDF5.HAS_PARALLEL[] = true
catch e
end

function set_driver!(fapl::Properties, mpio::MPIO)
HDF5.has_parallel() || error(
"HDF5.jl has no parallel support." *
Expand All @@ -60,6 +53,9 @@ function set_driver!(fapl::Properties, mpio::MPIO)
GC.@preserve mpio begin
API.h5p_set_fapl_mpio(fapl, mpi_to_h5(mpio.comm), mpi_to_h5(mpio.info))
end

DRIVERS[API.h5p_get_driver(fapl)] = MPIO
return nothing
end

function get_driver(fapl::Properties, ::Type{MPIO})
Expand Down

0 comments on commit 470eabc

Please sign in to comment.