Skip to content

take into list of crates for easyconfigs using Cargo-based easyblock when determining checksums for patches#21416

Closed
boegel wants to merge 1 commit intoeasybuilders:developfrom
boegel:fix_tests_checksums_cargo
Closed

take into list of crates for easyconfigs using Cargo-based easyblock when determining checksums for patches#21416
boegel wants to merge 1 commit intoeasybuilders:developfrom
boegel:fix_tests_checksums_cargo

Conversation

@boegel
Copy link
Member

@boegel boegel commented Sep 16, 2024

for for failing CI:

ERROR: test__parse_easyconfig_DeltaLake-0.15.1-gfbf-2023a.eb (test.easyconfigs.easyconfigs.EasyConfigTest)
Test for easyconfig DeltaLake-0.15.1-gfbf-2023a.eb
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.6.15/x64/lib/python3.6/site-packages/easybuild/tools/filetools.py", line 1274, in verify_checksum
    checksum = checksum[filename]
KeyError: 'DeltaLake-0.15.1_remove-obsolete-pyarrow-hotfix.patch'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/easybuild-easyconfigs/easybuild-easyconfigs/test/easyconfigs/easyconfigs.py", line 1693, in innertest
    template_easyconfig_test(self, spec_path)
  File "/home/runner/work/easybuild-easyconfigs/easybuild-easyconfigs/test/easyconfigs/easyconfigs.py", line 1533, in template_easyconfig_test
    if not verify_checksum(patch_full, checksum):
  File "/opt/hostedtoolcache/Python/3.6.15/x64/lib/python3.6/site-packages/easybuild/tools/filetools.py", line 1276, in verify_checksum
    raise EasyBuildError("Missing checksum for %s in %s", filename, checksum)
easybuild.tools.build_log.EasyBuildError: "Missing checksum for DeltaLake-0.15.1_remove-obsolete-pyarrow-hotfix.patch in {'addr2line-0.21.0.tar.gz': '8a30b2e23b9e17a9f90641c7ab1549cd9b44f296d3ccbf309d2863cfe398a0cb'}"

This problem arose due to the change in easybuilders/easybuild-easyblocks#3448, where self.cfg = self.cfg.copy() was added to the Cargo constructor before self.cfg['sources'] is being changed...

The checksum for DeltaLake-0.15.1_remove-obsolete-pyarrow-hotfix.patch is already there in DeltaLake-0.15.1-gfbf-2023a.eb

@boegel boegel added this to the release after 4.9.3 milestone Sep 16, 2024
# if easyblock derives from Cargo easyblock we need to take into account list of crates,
# which get added to list of sources, to correctly determine number of checksums for sources (incl. crates)
if isinstance(app, Cargo):
src_cnt += len(ec.get('crates', []))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we use the actual list of sources as will be done by the checksums step?
I.e. app.cfg['sources'] as discussed in the other PR/issue

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not since the change in easybuilders/easybuild-easyblocks#3448, because ec['sources'] no longer contains also the crates since that change, that's exactly why this fix is required

Copy link
Contributor

@Flamefire Flamefire Sep 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since that change app.cfg['sources'] should be populated with the crates while ec['sources'] should not. Or did I miss anything? See the comment

x = app_class(EasyConfig(file_content))
x.cfg <-- always do this

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created an alternative PR: #21419

Taking the sources, patches and checksums from app.cfg there and in a single place.

@Flamefire
Copy link
Contributor

where self.cfg = self.cfg.copy() was added to the Cargo constructor before self.cfg['sources'] is being changed...

Just for clarification: Actual issue is not the change of of self.cfg but the implicit change of the passed in ec instance we use here for the test. See the comment.

@boegel boegel deleted the fix_tests_checksums_cargo branch September 16, 2024 18:43
@boegel boegel modified the milestones: release after 4.9.4, 5.0.0 Mar 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants