-
Notifications
You must be signed in to change notification settings - Fork 97
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PyLith and GPUs and accelerators #726
Comments
#! /home/ronze/pylith/bin/python3
if __name__ == '__main__':
import sys
import os
sys.path.insert(0, os.path.abspath('config'))
import configure
configure_options = [
'--LIBS=-lz',
'--download-chaco=1',
'--download-f2cblaslapack=1',
'--download-viennacl',
'--download-ml',
'--prefix=/opt/pylith/dist',
'--with-64-bit-points=1',
'--with-c2html=0',
'--with-clanguage=C',
'--with-debugging=0',
'--with-fc=0',
'--with-hdf5=1',
'--with-hwloc=0',
'--with-large-file-io=1',
'--with-lgrind=0',
'--with-mpicompilers=1',
'--with-shared-libraries=1',
'--with-ssl=0',
'--with-x=0',
'--with-zlib=1',
'CFLAGS=-g -O2',
'CPPFLAGS=-I/opt/pylith/dist/include -I/opt/pylith/dist/include',
'CXXFLAGS=-g -O2 -DMPICH_IGNORE_CXX_SEEK',
'FCFLAGS=',
'LDFLAGS=-L/opt/pylith/dist/lib -L/opt/pylith/dist/lib64 -L/opt/pylith/dist/lib',
'PETSC_ARCH=arch-pylith',
'PETSC_DIR=/opt/pylith/build/cig/petsc-pylith',
]
configure.petsc_configure(configure_options) |
It would help us if you could provide more context. What are your specific reasons for wanting to build PyLith with support for OpenCL? In general, this is not just a matter of building PETSc with OpenCL support. |
The justifications for OpenCL have only diminished. It cannot vectorize code for CPUs, so the performance is terrible there. It used to have decent GPU performance, but Nvidia and AMD and Intel decided to pursue their own libraries instead of an open standard and it is no longer competitive there. |
Thanks for reply The reason I choose opencl is because our HPC uses a variety of acceleration cards, both GPU and DCU(a accelerator card MADE IN CHINA), which supports HIP, OpenCl and their own protocols. So I chose opencl, the most common protocol. Moreover, if opencl shows poor results, I may use other accelerators.The main challenge I’m currently facing is getting the hang of using an accelerator card with PyLith. Thanks again for your reply. |
If you're having trouble, maybe you can teach me CUDA or HIP acceleration in PyLith. |
PyLith does not directly use accelerators (this is in the plan for version 4 .0). Right now, linear algebra only uses accelerators, but this is not a large part of the runtime. Most of the runtime is in the formation of the Jacobian and calculation of the residual. You can try using the GPU with CUDA or HIP by setting the linear algebra types (-dm_vec_type cuda -dm_mat_type cuda), however you will get communication on each Newton iterate because we are forming the values on the CPU. |
Thanks for your reply, I used the method you introduced. -- Initializing timedependent problem with quasistatic formulation.
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Unknown type. Check for miss-spelling or missing package: https://petsc.org/release/install/install/#external-packages
[0]PETSC ERROR: Unknown vector type: cuda Does this method require petpack to download additional packages? [CUDA](https://developer.nvidia.com/cuda-toolkit) A parallel computing platform and application programming interface model created by NVIDIA.
[ViennaCL](http://viennacl.sourceforge.net/) Linear algebra library providing matrix and vector operations using OpenMP, CUDA, and OpenCL. If the method need to additional package,it's inevitable to recompiling petsc. |
It means you need to reconfigure PETSc and turn on CUDA if you want to use that. There are instructions here: https://petsc.org/main/install/install/#installing-petsc-to-use-gpus-and-accelerators A word of warning: This is likely to make very little difference in runtime. Hype has consumed computing with respect to GPUs. The determinant of PyLith runtime is almost always bandwidth, not compute. A high-end CPU and GPU have a very small bandwidth gap, and in order to drive a GPU near peak bandwdith you need 10s of millions of unknowns per GPU. Moreover scalable preconditioners for GPUs are still a research topic rather than settled practice. |
OK,thank you for your reply. |
PETSc supports OpenCL.
According to
Run configure with$--download-viennacl$ ; check config/examples/arch-ci-linux-viennacl.py for example usage.
I think this requires recompiling petsc.Then I found out that Pylith allows for recompiling PETSc, so I followed the tutorial at
to recompile it, but it failed.
The error message is
I have imported the pylith environment and changed the default python environment, which was originally
Now as
Perhaps you can guide a tutorial on pylith installing petsc external package in detail
The text was updated successfully, but these errors were encountered: