use python script to setup all codes

[superstar54]

It takes 1-2 sec to set up a code, and we have 14 codes (pw, dos, pp, ph, etc.), so around 20 sec in total. Which is not good, and it takes too long to load the container.

I looked into the issue and found that most of the time is used to set up the verdi command, profile and database. Thus it's would be good to combine all code set up as one.

Since the code setup is run in a different thread, thus, using the InstalledCode directly will fail when storing the code, as I tested in #695. This PR uses the python -c command to run in the subprocess.

The total time to set up all 14 codes is now reduced to 3-5 seconds. Most importantly, setting more codes will not increase the time too much. This is a temporary solution, but it is better than no solution. We can use the InstalledCode in the future when we fix the thread problem.

[superstar54]

Hi @AndresOrtegaGuerrero , you can test this by deleting some of the codes (not used yet by QEApp), and install them again.

#!/bin/bash
verdi code delete "epw-7.2@localhost" "neb-7.2@localhost" "open_grid-7.2@localhost" "ph-7.2@localhost" "pw2gw-7.2@localhost" "pw2wannier90-7.2@localhost" "q2r-7.2@localhost"
time python -m aiidalab_qe install-qe

[AndresOrtegaGuerrero]

Should it be load_code(f"{code_name}-{QE_VERSION}@{computer_name}") . ?

[superstar54]

Yes, you are right.

[AndresOrtegaGuerrero]

Generate the python string to setup an AiiDA code for a given computer. Tries to load an existing code and if not existent, generates Python code to create and store a new code setup.

[AndresOrtegaGuerrero]

Should this PR be dependent on the aiida-core version?

[AndresOrtegaGuerrero]

This was my test

Codes are setup!

real 0m7.560s
user 0m12.340s
sys 0m0.465s

[superstar54]

Hi @AndresOrtegaGuerrero , thanks for the review. I made the changes as you suggested.

Should this PR be dependent on the aiida-core version?

It could be if the Python API of AiiDA changes, e.g., InstalledCode, but for the moment, it should be safe.

[superstar54]

This was my test
Codes are setup!
real 0m7.560s
user 0m12.340s
sys 0m0.465s

Thanks for the test. You can try to run this in the main branch, and check the time difference.

[AndresOrtegaGuerrero]

LGTM, my test show less time