[WIP] Reduce BLAS dependency

[hhorii]

Summary

Work around memory allocation errors in OpenBLAS when multi-threads run

Details and comments

OpenBLAS sometimes fail test cases of Aer when multiple threads run.

$ python -m unittest test.terra.backends.aer_simulator.test_noise.TestNoise 
....BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
Segmentation fault

This error can be worked around by setting OPENBLAS_NUM_THREADS=1 :

$ OPENBLAS_NUM_THREADS=1 python -m unittest test.terra.backends.aer_simulator.test_noise.TestNoise 
.......................................................
----------------------------------------------------------------------
Ran 55 tests in 4.844s

OK

In CI, stestr runs multiple threads or multiple processes and then encounters unsuitability:

{0} test.terra.backends.aer_simulator.test_noise.TestNoise.test_kraus_gate_noise_on_QFT_cache_blocking_2___statevector____Thrust__ [] ... inprogress

This PR sets OPENBLAS_NUM_THREADS=1 when AerBackend is loaded.

[jakelishman]

Is this something we are capable of detecting within the library? It feels like some bug in our defaults that this can happen.

[hhorii]

The official FAQ said:

If your application is already multi-threaded, it will conflict with OpenBLAS multi-threading.

Maybe openblas_set_num_threads(1) is necessary if Aer uses OpenBLAS.

Another solution may be not to use BLAS. Currently, only matrix.hpp uses it to support matrix operators. Aer uses them only for small size of matrices. I guess that few performance impact from reduction of BLAS.

BTW, the current CI errors are from recent terra release. CI does not download binary from pypi, and then fails to compile qiskit-terra because of lack of rust compiler.

[mtreinish]

We need to update our wheel jobs to use manylinux2014 so the terra binaries are compatible with the build env

[jakelishman]

Maybe openblas_set_num_threads(1) is necessary if Aer uses OpenBLAS

Yeah, I was thinking something along these lines - we know if we're spawning threads/processes already, and we should know whether we're linked against OpenBLAS or something else, so we should make sure we're not conflicting with their assumptions.

We should set up our defaults to "just work" for the most part - if we're having trouble with a simple python -munittest TestNoise, it looks like a behaviour problem on our side.

[mtreinish]

I fixed this for real in #1498 (and a corresponding fix for the stable branch in #1499)

Suggested change

# manylinux-x86_64-image = "manylinux2010"

[mtreinish]

I'm not sure this fixes the underlying issue. For me locally when I run with main I'm able to still get a segfault when I run with OPENBLAS_NUM_THREADS=1:

OPENBLAS_NUM_THREADS=1 .tox/py310/bin/python -m unittest test.terra.backends.aer_simulator.test_noise.TestNoise 
.............OpenBLAS : Program will terminate because you tried to allocate too many memory regions.
...OpenBLAS : Program will terminate because you tried to allocate too many memory regions.
...<frozen importlib._bootstrap>:914: ImportWarning: _SixMetaPathImporter.find_spec() not found; falling back to find_module()
....................................
----------------------------------------------------------------------
Ran 55 tests in 2.141s

OK
zsh: segmentation fault (core dumped)  OPENBLAS_NUM_THREADS=1 .tox/py310/bin/python -m unittest 

I also tried with this PR applied and I also get a segfault in openblas although a without the allocation error message. It might be something with my local openblas install though. But I think we should avoid setting things via env variables via python like this if there is a bug in how we're using openblas I think we should fix that in the source.

[hhorii]

OK. I believe that we need more time to resolve OpenBLAS issue and this PR should not block 0.10.4 release.

[hhorii]

matrix multiplication is frequently used in matrix_product_state method. Here are performance comparisons (QV) of matrix_product_state with 0.10.4 and 168f7b7 (Intel(R) Xeon(R) Gold 6140 CPU, 18cores x 2HT, Ubuntu 18.0.4, gcc 8.4.0). Probably, for low-qubit circuits, library calls to OpenBLAS library were bottlenecks.

Another comparisons on Macbook Pro(Intel Core i7, 6core x 2HT, macOS BigSur).

Aer does not use matrix multiplication heavily, in general.

[hhorii]

In statevector method, matrix multiplication is used in fusion, mainly. Here are performance comparisons (QV) of statevector with 0.10.4 and 168f7b7 (Intel(R) Xeon(R) Gold 6140 CPU, 18cores x 2HT, Ubuntu 18.0.4, gcc 8.4.0).

[hhorii]

qiskit-aer/src/simulators/matrix_product_state/matrix_product_state.hpp

Lines 1055 to 1064 in a21260c

	#pragma omp parallel if (BaseState::threads_ > 1) num_threads(BaseState::threads_)
	{
	MPS temp;
	#pragma omp for
	for (int_t i=0; i<static_cast<int_t>(shots); i++) {
	temp.initialize(qreg_);
	auto single_result = temp.apply_measure_internal(sorted_qubits, rnds_list[i]);
	all_samples[i] = single_result;
	}
	}

The above OMP loop produces SEGV in test.terra.backends.aer_simulator.test_noise.TestNoise.test_readout_noise_5___matrix_product_state____CPU__. By taking OMP pragma, no error is thrown.
If a bug exists in it, we need to fix it instead of work-around by reducing use of BLAS.

[hhorii]

#1555 will resolve this BLAS issue