Skip to content

Add a patch that enable to use libraries exposed via LIBRARY_PATH or #23042

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dagonzalezfo wants to merge 1 commit into from
Closed

Add a patch that enable to use libraries exposed via LIBRARY_PATH or #23042

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion easybuild/easyconfigs/p/Python/Python-3.11.5-GCCcore-13.2.0.eb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,11 @@ toolchainopts = {'pic': True}

source_urls = ['https://www.python.org/ftp/%(namelower)s/%(version)s/']
sources = [SOURCE_TGZ]
checksums = ['a12a0a013a30b846c786c010f2c19dd36b7298d888f7c4bd1581d90ce18b5e58']
patches = ['Python-3.11.5-custom-ctypes.patch']
checksums = [
{'Python-3.11.5.tgz': 'a12a0a013a30b846c786c010f2c19dd36b7298d888f7c4bd1581d90ce18b5e58'},
{'Python-3.11.5-custom-ctypes.patch': 'fd56e81986051f22304dfc88953f5a5aad0c060f8e8649faa3633de644f5a61d'},
]

builddependencies = [
('UnZip', '6.0'),
Expand Down
52 changes: 52 additions & 0 deletions easybuild/easyconfigs/p/Python/Python-3.11.5-custom-ctypes.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
diff -ruN Python-3.11.5.orig/Lib/ctypes/__init__.py Python-3.11.5/Lib/ctypes/__init__.py
--- Python-3.11.5.orig/Lib/ctypes/__init__.py 2025-06-03 17:24:09.817225787 +0200
+++ Python-3.11.5/Lib/ctypes/__init__.py 2025-06-03 17:26:02.851166381 +0200
@@ -13,6 +13,8 @@
from _ctypes import ArgumentError

from struct import calcsize as _calcsize
+from ctypes import util
+import re

if __version__ != _ctypes_version:
raise Exception("Version number mismatch", __version__, _ctypes_version)
@@ -349,6 +351,11 @@
flags |= _FUNCFLAG_USE_ERRNO
if use_last_error:
flags |= _FUNCFLAG_USE_LASTERROR
+ if _os.name == "posix":
+ if name and name.endswith(".so"):
+ s = re.sub(r'lib', '', name)
+ s = re.sub(r'\..*', '', s)
+ self._name=util._findLib_ld(s)
Copy link
Copy Markdown
Contributor

@casparvl casparvl Jun 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand what's happening here. What issue is this actually resolving? Also, invoking _findLib_ld means we need to make absolutely sure that that searches the correct path (though I guess that's ensured by the changes below as well)

Copy link
Copy Markdown
Contributor

@casparvl casparvl Jun 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oooh, wait, this is where you overwrite the name with the full path already? I.e. this, together with the change below, makes sure we get something like /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Blosc2/2.13.2-GCCcore-13.2.0/lib/libblosc2.so back, and then the rest of the CDLL call just acts on that full path?

In other words: this is where we are actually pretending the calling code called the function with the full path, thus avoiding the whole issue?

Copy link
Copy Markdown
Contributor Author

@dagonzalezfo dagonzalezfo Jun 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, by overriding self._name

if _sys.platform.startswith("aix"):
"""When the name contains ".a(" and ends with ")",
e.g., "libFOO.a(libFOO.so)" - this is taken to be an
diff -ruN Python-3.11.5.orig/Lib/ctypes/util.py Python-3.11.5/Lib/ctypes/util.py
--- Python-3.11.5.orig/Lib/ctypes/util.py 2025-06-03 17:24:09.816225753 +0200
+++ Python-3.11.5/Lib/ctypes/util.py 2025-06-03 17:28:52.212009753 +0200
@@ -209,7 +209,7 @@
expr = os.fsencode(expr)

try:
- proc = subprocess.Popen(('/sbin/ldconfig', '-r'),
+ proc = subprocess.Popen((os.environ.get('EPREFIX') + '/sbin/ldconfig', '-r'),
Copy link
Copy Markdown
Contributor

@casparvl casparvl Jun 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is another reason why we should resolve things at the EasyBlock level: there, we can query if EasyBuild was configured with a sysroot build option. If so, we get that value, instead of EPREFIX (which I believe is set by Gentoo-prefix, and thus specific to that OS?). That makes this more generic. It also excludes any possible name collisions (someone could set EPREFIX as an environment for a totally different purpose - unrelated to Gentoo-Prefix, but build_option('sysroot') can only mean one thing: someone wanted to set a sysroot).

stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL)
except OSError: # E.g. command not found
@@ -301,7 +301,14 @@
# See issue #9998 for why this is needed
expr = r'[^\(\)\s]*lib%s\.[^\(\)\s]*' % re.escape(name)
cmd = ['ld', '-t']
- libpath = os.environ.get('LD_LIBRARY_PATH')
+ libpath = []
+ if os.getenv('EPREFIX'):
+ libpath.append(os.getenv('EPREFIX'))
+ if os.getenv('LD_LIBRARY_PATH'):
+ libpath.append(os.getenv('LD_LIBRARY_PATH'))
+ if os.getenv('LIBRARY_PATH'):
+ libpath.append(os.getenv('LIBRARY_PATH'))
+ libpath = ':'.join(libpath)
if libpath:
for d in libpath.split(':'):
cmd.extend(['-L', d])