Only consider explicit dependencies in LLVM easyblock

[Flamefire]

Avoid considering build dependencies for setting options which may lead to missing libraries at runtime.

Note: I changed self.deps to runtime-dependencies only and set it as early as possible. I can't see where this would change current (intended) behavior based on the code in this easyblock and it was likely intended this way anyway.

No other official easyblock uses self.deps so this should be safe.

See also

• add explicit libxml2 and libffi dependencies to LLVM 11.x/12.x/14.x easybuild-easyconfigs#23603
• add sanity check to link system libraries to LLVM easyblock #3877

Closes #3883

[Flamefire]

Test report by @Flamefire

Overview of tested easyconfigs (in order)

• SUCCESS LLVM-12.0.1-GCCcore-10.3.0.eb
• SUCCESS LLVM-12.0.1-GCCcore-11.2.0.eb
• SUCCESS LLVM-14.0.3-GCCcore-11.3.0.eb
• FAIL (build issue) LLVM-18.1.8-GCCcore-13.3.0.eb (partial log available at https://gist.github.com/Flamefire/d3ae2fa4b986d529e412c3ecb2962626)
  [...]
• FAIL (build issue) LLVM-20.1.8-GCCcore-14.3.0.eb (partial log available at https://gist.github.com/Flamefire/70de0a2810e47c16bd4678ffe9f83ea2)

Build succeeded for 45 out of 47 (18 easyconfigs in total)
n1609 - Linux RHEL 8.9 (Ootpa), x86_64, Intel(R) Xeon(R) Platinum 8470 (sapphirerapids), Python 3.9.18
See https://gist.github.com/Flamefire/58c29a66aa346aa25fc7b14fdf0a8e60 for a full test report.

UPDATE
LLVM 18 passes without sanitizer tests: easybuilders/easybuild-easyconfigs#23636
LLVM 20 passes with additional skips: easybuilders/easybuild-easyconfigs#23637

[Thyre]

33aefb3 should probably only be in #3883, right?

[Flamefire]

Given the time required to test this I thought it might be good to combine them here at least for testing and rebase afterwards or keep it.

[Thyre]

Agreed, that makes sense.

[Flamefire]

I rebased this on the other PR so the commit disappears when merged.

Test with both: easybuilders/easybuild-easyconfigs#23637 (comment)

[boegel]

lgtm

[boegel]

tested via easybuilders/easybuild-easyconfigs#23637 (comment)

[Crivella]

Doesen't this risk breaking, or atleast not using a dependency in builds eg on top of EESSI when a dependency might be filtered and than reintroduced by setting EBROOT... through an hook?

[Thyre]

Doesen't this risk breaking, or atleast not using a dependency in builds eg on top of EESSI when a dependency might be filtered and than reintroduced by setting EBROOT... through an hook?

Yes.
self.dependencies() ignores filtered dependencies.
The documentation of this function states:

Returns an array of parsed dependencies (after filtering, if requested)

But that is incorrect, as one cannot choose that filtered deps are still returned. This seems to be quite a few years old.
Should we add an optional parameter to self.dependencies() to also allow reporting filtered deps? all_depenencies() also removes hidden ones, so that won't help either.

[Thyre]

The problem with also reporting filtered deps is that there might be a legitimate reason why one doesn't want a certain dependency. We might still enable some feature based on that dep though, if we just look at the filtered deps...

Do we need a combination of both (i.e. check for filtered dep and EBROOT...)?

[Flamefire]

Do we have an example of this being an issue? Might be worth checking why/how we can run into this in a reasonable use case. Maybe there are other solutions then

[Crivella]

https://github.com/EESSI/software-layer-scripts/blob/99c82b562e243cc6f566e72fbcbbe1a23d568d0c/eb_hooks.py#L816-L834

and

easybuild-easyblocks/easybuild/easyblocks/l/llvm.py

Line 903 in 2c400f0

self.general_opts['LLVM_ENABLE_ZLIB'] = 'ON' if 'zlib' in self.deps else 'OFF'

Will probably not work have not tested it yet

[Crivella]

I think the easy solution would be to keep using get_software_root directly and ensure that the dependencies are properly placed when needed.

This is the approach that is taken in all other easyblocks.

[Flamefire]

This is the approach that is taken in all other easyblocks.

We are not consistent here. Many easyblocks use the explicit dependencies, e.g.:

• https://github.com/Flamefire/easybuild-easyblocks/blob/3ac81b4e27612392a6521e60888ee3be5261719f/easybuild/easyblocks/s/slepc.py#L107
• https://github.com/Flamefire/easybuild-easyblocks/blob/3ac81b4e27612392a6521e60888ee3be5261719f/easybuild/easyblocks/f/flexiblas.py#L73
• https://github.com/Flamefire/easybuild-easyblocks/blob/3ac81b4e27612392a6521e60888ee3be5261719f/easybuild/easyblocks/t/trilinos.py#L206
• https://github.com/Flamefire/easybuild-easyblocks/blob/3ac81b4e27612392a6521e60888ee3be5261719f/easybuild/easyblocks/c/cmake.py#L108

So I guess we need a more general solution how to deal with filtered deps. Not sure what the use case is for them but I guess we need something for "replaced" dependencies. I.e. it looks like an "abuse" of this feature in EESSI: Remove a dependency but pretend it is still there. I'd say we need a cleaner solution for that. Isn't that what "external modules" would do too?