Library lookup doesn't work with custom path

[cdlm]

I have hidapi installed via brew under a custom prefix:

$ ls -l /opt/brew/lib/libhidapi*
lrwxr-xr-x 1 damien admin 45 Jun 13 19:25 /opt/brew/lib/libhidapi.0.dylib -> ../Cellar/hidapi/0.10.1/lib/libhidapi.0.dylib
lrwxr-xr-x 1 damien admin 39 Jun 13 19:25 /opt/brew/lib/libhidapi.a -> ../Cellar/hidapi/0.10.1/lib/libhidapi.a
lrwxr-xr-x 1 damien admin 43 Jun 13 19:25 /opt/brew/lib/libhidapi.dylib -> ../Cellar/hidapi/0.10.1/lib/libhidapi.dylib

That breaks qmk because it can't find it.

$ qmk compile
Error: %s: %s ('ImportError', ImportError('Unable to load any of the following libraries:libhidapi-hidraw.so libhidapi-hidraw.so.0 libhidapi-libusb.so libhidapi-libusb.so.0 libhidapi-iohidmanager.so libhidapi-iohidmanager.so.0 libhidapi.dylib hidapi.dll libhidapi-0.dll'))
Traceback (most recent call last):
  File "/opt/brew/Cellar/qmk/0.1.0/libexec/lib/python3.9/site-packages/qmk_cli/script_qmk.py", line 76, in main
    import qmk.cli  # noqa
  File "/Users/damien/Repositories/custom-keyboards/qmk_firmware/lib/python/qmk/cli/__init__.py", line 191, in <module>
    __import__(subcommand)
  File "/Users/damien/Repositories/custom-keyboards/qmk_firmware/lib/python/qmk/cli/console.py", line 9, in <module>
    import hid
  File "/opt/brew/Cellar/qmk/0.1.0/libexec/lib/python3.9/site-packages/hid/__init__.py", line 30, in <module>
    raise ImportError(error)
ImportError: Unable to load any of the following libraries:libhidapi-hidraw.so libhidapi-hidraw.so.0 libhidapi-libusb.so libhidapi-libusb.so.0 libhidapi-iohidmanager.so libhidapi-iohidmanager.so.0 libhidapi.dylib hidapi.dll libhidapi-0.dll

[apmorton]

libhidapi must appear on your system library path - if you don't want to permanently modify your system you can temporarily add `/opt/brew/lib` to LD_LIBRARY_PATH before running qmk. ``` export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/brew/lib qmk compile ```

…

On Mon, Jun 28, 2021, 08:13 Damien Pollet ***@***.***> wrote: I have hidapi installed via brew <https://brew.sh> under a custom prefix: $ ls -l /opt/brew/lib/libhidapi* lrwxr-xr-x 1 damien admin 45 Jun 13 19:25 /opt/brew/lib/libhidapi.0.dylib -> ../Cellar/hidapi/0.10.1/lib/libhidapi.0.dylib lrwxr-xr-x 1 damien admin 39 Jun 13 19:25 /opt/brew/lib/libhidapi.a -> ../Cellar/hidapi/0.10.1/lib/libhidapi.a lrwxr-xr-x 1 damien admin 43 Jun 13 19:25 /opt/brew/lib/libhidapi.dylib -> ../Cellar/hidapi/0.10.1/lib/libhidapi.dylib That breaks qmk <https://qmk.fm> because it can't find it. $ qmk compile Error: %s: %s ('ImportError', ImportError('Unable to load any of the following libraries:libhidapi-hidraw.so libhidapi-hidraw.so.0 libhidapi-libusb.so libhidapi-libusb.so.0 libhidapi-iohidmanager.so libhidapi-iohidmanager.so.0 libhidapi.dylib hidapi.dll libhidapi-0.dll')) Traceback (most recent call last): File "/opt/brew/Cellar/qmk/0.1.0/libexec/lib/python3.9/site-packages/qmk_cli/script_qmk.py", line 76, in main import qmk.cli # noqa File "/Users/damien/Repositories/custom-keyboards/qmk_firmware/lib/python/qmk/cli/__init__.py", line 191, in <module> __import__(subcommand) File "/Users/damien/Repositories/custom-keyboards/qmk_firmware/lib/python/qmk/cli/console.py", line 9, in <module> import hid File "/opt/brew/Cellar/qmk/0.1.0/libexec/lib/python3.9/site-packages/hid/__init__.py", line 30, in <module> raise ImportError(error) ImportError: Unable to load any of the following libraries:libhidapi-hidraw.so libhidapi-hidraw.so.0 libhidapi-libusb.so libhidapi-libusb.so.0 libhidapi-iohidmanager.so libhidapi-iohidmanager.so.0 libhidapi.dylib hidapi.dll libhidapi-0.dll — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#50>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAPRFDAKXJMPDWMZVWOKWDTVBROLANCNFSM47N24SQA> .

[cdlm]

I had tried that before with DYLD_LIBRARY_PATH, and now with LD_LIBRARY_PATH, but that doesn't help (QMK fails to start and demands that I run its install script again). My changes in #51 make it work fine.

[mfncooper]

When setting DYLD_LIBRARY_PATH (or LD_LIBRARY_PATH ) doesn't work, it is quite likely that the issue is SIP (macOS System Integrity Protection). Basically, with SIP enabled, which is the default on modern Macs, those particular variables will never be passed to another process, so setting them is useless. See:

https://developer.apple.com/library/archive/documentation/Security/Conceptual/System_Integrity_Protection_Guide/RuntimeProtections/RuntimeProtections.html

Spawning children processes of processes restricted by System Integrity Protection, such as by launching a helper process in a bundle with NSTask or calling the exec(2) command, resets the Mach special ports of that child process. Any dynamic linker (dyld) environment variables, such as DYLD_LIBRARY_PATH, are purged when launching protected processes.

The path can be set from Python before attempting to load the library, like so:

import os
os.environ['DYLD_LIBRARY_PATH'] = '/opt/homebrew/lib'
import hid

Ugly, but it works.

The alternative seems to be to disable SIP on the machine, which doesn't seem like a good idea at all.

[apmorton]

I'm going to close this as I am unable to reproduce the issue now that I have access to a mac.

On my M2 running Sonoma 14.6.1 with libhidapi installed via brew:

# expected import error by default
$ python3 -c 'import hid; print(hid.hidapi)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/amorton/src/pyhidapi/hid/__init__.py", line 31, in <module>
    raise ImportError(error)
ImportError: Unable to load any of the following libraries:libhidapi-hidraw.so libhidapi-hidraw.so.0 libhidapi-libusb.so libhidapi-libusb.so.0 libhidapi-iohidmanager.so libhidapi-iohidmanager.so.0 libhidapi.dylib hidapi.dll libhidapi-0.dll

# working as expected when DYLD_LIBRARY_PATH is set
$ env DYLD_LIBRARY_PATH=/opt/homebrew/lib python3 -c 'import hid; print(hid.hidapi)'
<CDLL 'libhidapi.dylib', handle 95fbb0c0 at 0x104eeec10>

[123swk123]

@apmorton @cdlm

probably this code change, #66 pull request could help you with this issue

This fix uses PATH environment variable to search for the dll/so files, this way we could place the dll's at custom path and refer it via PATH
I have tested this on Win11 x64 environment.