-
Notifications
You must be signed in to change notification settings - Fork 13
Home
ROOT library is used as format for input/ouput data. GRANDLIB provides a python binding of GULL et TURTLE from C language, but proper installation is not guaranteed on all operating systems. So to prevent any problem installing these three libraries we provide an docker image where ROOT is installed and the compilation of GULL and TURTLE is correct. GRANDLIB docker images are built from Ubuntu 20.04 with ROOT given by official rootprojet.
Conda is an alternative to Docker but it is currently only available for linux, see here for more information.
Note : not yet available, only with demo mode
You can use GRANDLIB release in Docker image, available on DockerHub grandlib/release. A basic tutorial to use docker for GRANDLIB is available. A specific documentation for GRANDLIB users is available here (need to update).
You can use Jupyter NoteBook inside docker with your host web navigator. Just :
- run docker with
-p 8888:8888
option - go to directory where grandlib release is installed :
/opt/grandlib/grand
- used a specific alias
grand_jupyter
to start jupyter notebook - copy/paste in your host web navigator the last URL given by jupyter in standard output
Example:
$ docker run -p 8888:8888 -it --rm grandlib/release
root@58788b7dc04e:/home# cd /opt/grandlib/grand/
root@58788b7dc04e:/opt/grandlib/grand# grand_jupyter
[I 14:17:35.829 NotebookApp] Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret
[I 14:17:36.012 NotebookApp] Serving notebooks from local directory: /opt/grandlib/grand
[I 14:17:36.012 NotebookApp] Jupyter Notebook 6.4.11 is running at:
[I 14:17:36.012 NotebookApp] http://58788b7dc04e:8888/?token=37f74751591b90d46de412a7565f23f1d6ddb1373519b4b0
[I 14:17:36.012 NotebookApp] or http://127.0.0.1:8888/?token=37f74751591b90d46de412a7565f23f1d6ddb1373519b4b0
[I 14:17:36.012 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 14:17:36.015 NotebookApp]
To access the notebook, open this file in a browser:
file:///root/.local/share/jupyter/runtime/nbserver-13-open.html
Or copy and paste one of these URLs:
http://58788b7dc04e:8888/?token=37f74751591b90d46de412a7565f23f1d6ddb1373519b4b0
or http://127.0.0.1:8888/?token=37f74751591b90d46de412a7565f23f1d6ddb1373519b4b0
If your script used some plots, you must enable correctly graphics in docker, you can review the following tutorials:
- Graphic configuration for Docker under Linux
- Graphic configuration for Docker under Windows 10
- Graphic configuration for Docker under MacOS
For production in computer center, most of them use Apptainer (or Singularity same CLI as Apptainer) instead Docker. So you can create from Docker image a Apptainer image with this command:
apptainer build grandlib_xxxx.sif docker://grandlib/xxx:yyy
or at CCIN2P3 used an existing image here
$THRONG_DIR/apptainer
Use Apptainer on an interactive node
apptainer shell --bind /pbs,/sps/grand /path/to/grandlib_xxxx.sif
Use Apptainer on a computing node
You can find an example of Apptainer/Singularity use with batch system in CCIN2P3 documentation.
You can use GRANDLIB development environment docker image, available on DockerHub grandlib/dev. This docker image is different of release image, it doesn't contain any GRANDLIB code only the development environment. A basic tutorial to use docker for GRANDLIB is available.
- How define your token github for continuous integration (CI)
The github tokens can be generated with different rights, you must select workflow in setting
than developer setting (last line left column)/Personal access tokens/generate new token.
Otherwise the CI isn't executed, quality report isn't generated and isn't sent to SonarQube server.
- How define user and mail in your commit and push
You must define the email associated with your git account before pushing. In the root git package, write these git commands
$ git config --local user.email <mail defined in github>
$ git config --local user.name <user name defined in github>
If you have only one git account, you can use --global
option instead --local
.
A docker image with GRANDLIB environment is available without source of GRANDLIB. You can clone GRANDLIB package inside container or outside and add a link between container/local direction with option -v
to access to your data or sources, see basic tutorial to use docker for GRANDLIB.
Don't forget to initialize grand library before use it with script setup.sh in env directory
$ git clone https://github.com/grand-mother/grand.git
$ docker run -it --rm -v $PWD:/home grandlib/dev
root@58788b7dc04e:/home# ls grand
bin COPYING.LESSER docs env examples grand lib LICENSE log.txt quality
README.rst src tests wheel
root@dc6f5b78ef35:/home# cd grand
root@dc6f5b78ef35:/home/grand# source env/setup.sh
var GRAND_ROOT=/home/grand
==============================
update PATH with grand quality
==============================
update PYTHONPATH with grand source
==============================
Install external lib gull and turtle
====================================
CLONE turtle
MAKE libturtle.so
CLONE gull
MAKE libgull.so
INSTALL turtle.h
INSTALL gull.h
PYTHON _core.abi3.so
root@dc6f5b78ef35:/home/grand#
Like for release docker image case, you can use Jupyter notebook with dev docker image, you must just initialize correctly the grand environment before launch Jupyter with your local version of grandlib package.
Example with a shared volume, so we have a grand package in current directory
$ ls grand
bin COPYING.LESSER docs env examples grand lib LICENSE log.txt quality README.rst src tests wheel
$ docker run -p 8888:8888 -it --rm -v $PWD:/home grandlib/dev
root@58788b7dc04e:/home# ls grand
bin COPYING.LESSER docs env examples grand lib LICENSE log.txt quality README.rst src tests wheel
Now initialize it and launch jupyter
root@dc6f5b78ef35:/home# cd grand
root@dc6f5b78ef35:/home/grand# . env/setup.sh
...
etc
root@dc6f5b78ef35:/home/grand# grand_jupyter
[I 14:17:35.829 NotebookApp] Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret
[I 14:17:36.012 NotebookApp] Serving notebooks from local directory: /opt/grandlib/grand
[I 14:17:36.012 NotebookApp] Jupyter Notebook 6.4.11 is running at:
[I 14:17:36.012 NotebookApp] http://58788b7dc04e:8888/?token=37f74751591b90d46de412a7565f23f1d6ddb1373519b4b0
[I 14:17:36.012 NotebookApp] or http://127.0.0.1:8888/?token=37f74751591b90d46de412a7565f23f1d6ddb1373519b4b0
[I 14:17:36.012 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 14:17:36.015 NotebookApp]
To access the notebook, open this file in a browser:
file:///root/.local/share/jupyter/runtime/nbserver-13-open.html
Or copy and paste one of these URLs:
http://58788b7dc04e:8888/?token=37f74751591b90d46de412a7565f23f1d6ddb1373519b4b0
or http://127.0.0.1:8888/?token=37f74751591b90d46de412a7565f23f1d6ddb1373519b4b0
See dedicated page to use Eclipse to develop GRANDLIB
With professionnal version of Pycharm on your computer you can use docker to define python interpreter, see tutorial
To use ROOT library a path mapping is necessary, in docker image ROOT is installed in /opt/root/lib directory, you must provide it to PyCharm like this:
- Create on your local disk a empty directory my/path/mapping/root
- In your project PyCharm : File>Settings>Project>Python interpreter>Path Mappings
- add "Local Path" = my/path/mapping/root and "Remote path"=/opt/root/lib
- In your project PyCharm : File>Settings>Project>Project Structure> Add Content Root (upper right)
- In Select Content Root Directory, choice my/path/mapping/root
Remarks :
- Pycharm use word Root but it's ROOT library of CERN :-)
- Last Pycharm version may present a bug when configuring python interpreter in Docker. If building the docker image in Pycharm fails with message like
Introspection completed with error: com.github.dockerjava.api.exception.ConflictException: Status 409: {"message":"Container c4ae9500e054ab074ac9b7c9260ce925d3d5d743f719a11f7076a0a3a470c294 is not running"}
then you can use the following workaround : Open the Help menu, then Find Action, search for Registry and uncheck python.use.targets.api (see: https://youtrack.jetbrains.com/issue/PY-55444)
to do
The quality of the code is applicable in priority on the new development.
Quality tools are available in package in quality directory:
- pylint (static analysis, python rule and PEP8)
- coverage.py (dynamic analysis of code from test case)
- mypy (static analysis for optional typing)
-
black (code formatter)
- To disable black on a mathematical formula for example, you can use the directives: "#fmt: off" and "#fmt: on"
see readme.md. In Docker image quality tools are already installed.
All developers have a personal project in GRAND SonarQube server associated to last branch pushed on github to improve the code quality of its own development.
4 levels are defined (low, medium, hight, prod), the difference are mainly on number of pylint issues threshold. You can see quality gate definition on GRAND Quality Gate page, the actual quality gate applied is tagged DEFAULT.
Actually, you can generate API documentation with sphinx-apidoc
In grandlib/dev container you can create HTML API documentation with this script
create_doc_html.sh
in directory
grand/docs/apidoc-only/sphinx
of grand git package, HTML result in
grand/docs/apidoc-only/sphinx/build/html
You can open it with your web browser.
:param name_param : <[Unit]> short parameter description
:type name_param : <type>
Example
:param fr_max: [Hz] The maximal frequency of the bandpass filter
:type fr_max: float
Naming/use convention git branch:
- master : last stable version used for release, protected branch developer can't commit directly
- dev : most stable branch with new features foo_feat incomming
- dev_foo_feat : developer branch in charge of development of new feature foo_feat
For each new feature:
- Create a git branch from dev branch
- Develop feature in this branch
- Develop test associated to the feature
- Apply code formatter black
- Check code quality with local tools and SonarQube server
- Update your branch with current dev branch and solve conflict
- Create a Pull Request in GitHub
- Code review and check by other developer before merging in dev branch
Bug:
- Each bug must be identified by a issue in GitHub with clean explanation to reproduce it
- Solve bug "foo" in dedicaded branch dev_bug_foo
- Follow same development cycle as feature