Error for the command : engine = pypesto.engine.MultiProcessEngine()

[chloweckel]

What did you expect to happen?
I want to use this line : engine = pypesto.engine.MultiProcessEngine() I'd like to use this line in my parameter optimization code.

result = optimize.minimize(problem=problem,
optimizer=optimizer,
engine=engine,
options=opt_options,
n_starts=10)

I have already installed amici and it works with this line : engine = pypesto.engine.SingleCoreEngine().
For installing amici, I used one virtual environment with all the packages.

What has happened instead?

When I used the line, pypesto.engine.MultiProcessEngine(), I obtain this error :
ERROR: (“module ‘amici.amici’ has no attribute ‘writeSolverSettingsToHDF5’”, 'Pickling the AmiciObjective requires an AMICI installation with HDF5 support.')
The problem should come from the way I have installed hdf5.

To Reproduce
I saw that someone seemed to have the same problem as me and had solved it : #2187 .

I downloaded hdf5-1.14.5

cd hdf5-1.14.5
make
sudo make install
h5cc -showconfig

result : showconfig :
Parallel HDF5: no
Parallel Filtered Dataset Writes: no
Large Parallel I/O: no
High-level library: yes
Build HDF5 Tests: yes
Build HDF5 Tools: yes
Threadsafety: yes
Default API mapping: v18
With deprecated public symbols: yes
I/O filters (external): deflate(zlib),szip(encoder)
MPE: no
Direct VFD: no
Mirror VFD: no
(Read-Only) S3 VFD: yes
(Read-Only) HDFS VFD: no
Packages w/ extra debug output: none
API tracing: no
Using memory checker: no
Memory allocation sanity checks: no
Function stack tracing: no
Use file locking: best-effort
Strict file format checks: no
Optimization instrumentation: no

sudo apt install libatlas-base-dev swig python3-dev
sudo apt install libhdf5-serial-dev
sudo apt install libboost-chrono-dev libboost-math-dev libboost-serialization-dev

INSTALL="~/hdf5-1.14.5bin/install-sh -c" ~/hdf5-1.14.5/configure --prefix=/home/cweckel/hdf5_lib --enable-hl –enable-cxx

message :
configure: WARNING: you should use --build, --host, --target
configure: WARNING: invalid host type: –enable-cxx
checking for a BSD-compatible install... ~/hdf5-1.14.5bin/install-sh -c
checking whether build environment is sane... yes
checking for a race-free mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
configure: error: source directory already configured; run "make distclean" there first

install --verbose amici

Using pip 24.0 from /home/cweckel/PYPESTO_copy/lib/python3.12/site-packages/pip (python 3.12)
Collecting amici
Using cached amici-0.29.0-cp312-cp312-linux_x86_64.whl
Requirement already satisfied: cmake-build-extension==0.6.0 in ./PYPESTO_copy/lib/python3.12/site-packages (from amici) (0.6.0)
Requirement already satisfied: sympy>=1.12.1 in ./PYPESTO_copy/lib/python3.12/site-packages (from amici) (1.13.3)
Requirement already satisfied: numpy>=1.21.4 in ./PYPESTO_copy/lib/python3.12/site-packages (from amici) (1.26.4)
Requirement already satisfied: python-libsbml in ./PYPESTO_copy/lib/python3.12/site-packages (from amici) (5.20.4)
Requirement already satisfied: pandas>=2.0.2 in ./PYPESTO_copy/lib/python3.12/site-packages (from amici) (2.2.3)
Requirement already satisfied: pyarrow in ./PYPESTO_copy/lib/python3.12/site-packages (from amici) (17.0.0)
Requirement already satisfied: toposort in ./PYPESTO_copy/lib/python3.12/site-packages (from amici) (1.10)
Requirement already satisfied: setuptools>=48 in ./PYPESTO_copy/lib/python3.12/site-packages (from amici) (75.2.0)
Requirement already satisfied: mpmath in ./PYPESTO_copy/lib/python3.12/site-packages (from amici) (1.3.0)
Requirement already satisfied: cmake in ./PYPESTO_copy/lib/python3.12/site-packages (from cmake-build-extension==0.6.0->amici) (3.30.4)
Requirement already satisfied: ninja in ./PYPESTO_copy/lib/python3.12/site-packages (from cmake-build-extension==0.6.0->amici) (1.11.1.1)
Requirement already satisfied: GitPython in ./PYPESTO_copy/lib/python3.12/site-packages (from cmake-build-extension==0.6.0->amici) (3.1.43)
Requirement already satisfied: setuptools-scm in ./PYPESTO_copy/lib/python3.12/site-packages (from cmake-build-extension==0.6.0->amici) (8.1.0)
Requirement already satisfied: python-dateutil>=2.8.2 in ./PYPESTO_copy/lib/python3.12/site-packages (from pandas>=2.0.2->amici) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in ./PYPESTO_copy/lib/python3.12/site-packages (from pandas>=2.0.2->amici) (2024.2)
Requirement already satisfied: tzdata>=2022.7 in ./PYPESTO_copy/lib/python3.12/site-packages (from pandas>=2.0.2->amici) (2024.2)
Requirement already satisfied: six>=1.5 in ./PYPESTO_copy/lib/python3.12/site-packages (from python-dateutil>=2.8.2->pandas>=2.0.2->amici) (1.16.0)
Requirement already satisfied: gitdb<5,>=4.0.1 in ./PYPESTO_copy/lib/python3.12/site-packages (from GitPython->cmake-build-extension==0.6.0->amici) (4.0.11)
Requirement already satisfied: packaging>=20 in ./PYPESTO_copy/lib/python3.12/site-packages (from setuptools-scm->cmake-build-extension==0.6.0->amici) (24.1)
Requirement already satisfied: smmap<6,>=3.0.1 in ./PYPESTO_copy/lib/python3.12/site-packages (from gitdb<5,>=4.0.1->GitPython->cmake-build-extension==0.6.0->amici) (5.0.1)
Installing collected packages: amici
changing mode of /home/cweckel/PYPESTO_copy/bin/amici_import_petab to 775
changing mode of /home/cweckel/PYPESTO_copy/bin/amici_import_petab.py to 775
Successfully installed amici-0.29.0

AMICI version and system environment

• OS and version: Ubuntu/Linux
• AMICI interface: Python
• AMICI version:
• Additional information: VS code

Thank you very much for your help !

[dweindl]

I downloaded hdf5-1.14.5

Do you need this specific self-built hdf5 version? If not, sudo apt install libhdf5-serial-dev should be sufficient and simplify the rest. It needs to be installed before installing amici.

Using cached amici-0.29.0-cp312-cp312-linux_x86_64.whl

The problem might be that HDF5 was not installed during the first attempt to install amici, and that once HDF5 was available, the package wasn't rebuilt to actually use HDF5.
Please re-try the installation with pip install --verbose --no-cache-dir amici, save the full output, re-run your pypesto script, and if things still don't work, post the full installation log and the output of python -m amici.