[jhchouuu]

Problem

import mori fails on any host whose ROCm ships without hipFile:

File ".../site-packages/mori/cpp/__init__.py", line 44, in <module>
ImportError: libhipfile.so.0: cannot open shared object file: No such file or directory

GdsEngine is one optional transfer path inside umbp_common, and umbp_common is linked into libmori_pybinds.so — so target_link_libraries(umbp_common PUBLIC hip::hipfile) (added in #540) became a DT_NEEDED on the Python extension.

Because the dependency is auto-detected (find_package(hipfile QUIET)), whether a wheel carries it depends on what happened to be installed on the build host. A wheel built where hipFile exists cannot be imported anywhere else.

This broke ROCm/ATOM CI: four accuracy jobs died before the first token — import mori.shmem → all 8 DP ranks down → Engine Core Mgr: Received unexpected SHUTDOWN signal. None of those jobs use UMBP at all; rocm/atom-dev:latest runs ROCm 7.2.4, which has no hipFile.

Fix

Build against the headers, resolve the library at runtime.

• New hipfile_dl.{h,cpp}: dlopen("libhipfile.so.0") on first use, dlsym the four entry points GdsEngine calls, cached under std::call_once. Function-pointer types come from <hipfile.h> via decltype, so they cannot drift from the real signatures.
• GdsEngine::Available() reports whether the library loaded. PoolClient registers the engine only when it did; without it RegisterFile returns an invalid ref and the SSD path stays host-staged — exactly what an hipfile-less build already does.
• CMake takes only hip::hipfile's INTERFACE_INCLUDE_DIRECTORIES and links ${CMAKE_DL_LIBS}. GLOBAL is kept so the tests directory can still gate test_gds_engine on the same target; nothing links it.

No capability is lost: GDS is still compiled in whenever the headers are present, and it works wherever the library exists.

Dropping the link alone is not enough. patchelf --remove-needed libhipfile.so.0 on a shipped wheel just moves the failure to undefined symbol: hipFileHandleDeregister — the calls themselves have to go through dlsym. So PUBLIC → PRIVATE or --as-needed would not have fixed this.

Verification

Reproduced and fixed locally, in a container with a stand-in libhipfile installed at build time:

check	before	after
readelf -d libmori_pybinds.so | grep hipfile	NEEDED libhipfile.so.0	none
import mori.shmem, library absent	ImportError	OK
import mori.shmem, library present	OK	OK
GdsEngine symbols compiled in	yes	yes
test_gds_engine	2 passed / 1 skipped (no GPU)	2 passed / 1 skipped

A probe confirms both directions of the runtime decision:

library absent:  Available()=0   RegisterFile -> IsFile=0 handle=(nil)
library present: Available()=1   hipFileHandleRegister(fd=3) called
                 RegisterFile -> IsFile=1 handle=0x7ffff7d6902c

Caveats, stated plainly: the library used in verification was a stand-in that implements the four symbols, so symbol resolution and dispatch are covered but a real GDS disk read is not; and the import check ran in a Python 3.10 build container rather than in rocm/atom-dev itself (different Python and protobuf/gRPC versions). On rocm/atom-dev:latest, libhipfile.so.0 was confirmed to be the only unresolved dependency of the shipped amd_mori_nightly wheel.

Follow-up, not in this PR

libmori_pybinds.so still hard-links gRPC / protobuf / abseil / OpenSSL through umbp_common. Those happen to resolve in ATOM's image today, but the same class of ImportError returns the moment a base image moves to a different protobuf soname. Making import mori independent of UMBP's transitive dependencies — e.g. UMBP as a separately-loaded libmori_umbp.so — would close the category rather than this one instance.

🤖 Generated with Claude Code