Initial pull request for PyTrilinos recipe by wfspotz · Pull Request #3361 · conda-forge/staged-recipes

wfspotz · 2017-07-25T17:41:08Z

No description provided.

conda-forge-linter · 2017-07-25T17:41:14Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/pytrilinos) and found it was in an excellent condition.

isuruf · 2017-07-26T03:33:37Z

This has both pytrilinos and trilinos libraries. Is it possible to split them into 2 packages, one with all the C/C++ libraries and the other with the python libraries?

wfspotz · 2017-07-26T15:24:42Z

Trilinos is a suite of packages — currently about 54 — and PyTrilinos is one of them. You can enable or disable any package, subject to dependencies. So you build the whole thing, and PyTrilinos is either enabled or disabled. The build I am proposing here enables only the packages that has python libraries. It is not designed to be split apart. We can call it “trilinos" rather than “pytrilinos”. I don’t know if that helps.

wfspotz · 2017-07-26T15:49:44Z

@isuruf Is your question in response to the timeout on Travis?

isuruf · 2017-07-26T15:51:34Z

timeout can be fixed by removing numpy x.x and using just numpy 1.7 in build and numpy>=1.7 in run. See conda-forge/conda-forge.github.io#415

isuruf · 2017-07-26T15:57:03Z

I suggested splitting because the C/C++ libraries will be tied to python and pull in python to an environment when all you want is the C/C++ library. Also if you are working in a python3 environment and only want the C/C++ library, you can't install it because pytrilinos is python2 only.

isuruf · 2017-07-26T15:58:10Z

That said, for the first iteration, we can have it as it is.

isuruf · 2017-07-26T16:00:19Z

+  url: http://trilinos.csbsju.edu/download/files/trilinos-{{ version }}-Source.tar.gz
+  sha256: {{ sha256 }}
+  patches:
+    - patch.txt


Can you change the file extension to .patch? Also can you explain in a comment above why this patch is needed?

The patch is needed because my build system was calling a script I had written with the shebang line

#! ${PYTHON_EXECUTABLE}

which expands out to the 256+ character path for python, but the Linux environment for conda-forge limits the shebang line to 80 characters. The fix was to run the script by specifying the python executable on the command line rather than depending on the shebang. This fix required modifying several CMakeLists.txt files.

isuruf · 2017-07-26T16:00:52Z

+
+build:
+  number: 0
+  skip: true  # [win]


Change this to win or py3k

Easy enough...

wfspotz · 2017-07-26T18:18:17Z

On Linux, the recipe succeeds until it gets to the testing portion of build.sh, where I call ctest and all the tests fail. I am trying to figure out how to debug this. What I would like to do is something like

ctest
RESULT=$?
if [ $RESULT -ne 0 ]; then
    cat Testing/Temporary/LastTest.log
    exit $RESULT
fi

but I am afraid that the script will end when I get to ctest and the RESULT=$? line will never be executed.

isuruf · 2017-07-27T17:16:39Z

+  -D BUILD_SHARED_LIBS:BOOL=ON \
+  -D TPL_ENABLE_MPI:BOOL=ON \
+  -D MPI_BASE_DIR:PATH=$PREFIX \
+  -D MPI_EXEC:FILEPATH=$PREFIX/bin/mpiexec \


Try adding, -D MPI_EXEC_PRE_NUMPROCS_FLAGS=--allow-run-as-root

So how will this help?

Tests fail with the message below because we are running as root in the docker container,

-------------------------------------------------------------------------- mpiexec has detected an attempt to run as root. Running at root is *strongly* discouraged as any mistake (e.g., in defining TMPDIR) or bug can result in catastrophic damage to the OS file system, leaving your system in an unusable state. You can override this protection by adding the --allow-run-as-root option to your cmd line. However, we reiterate our strong advice against doing so - please do so at your own risk. --------------------------------------------------------------------------

Also add -D MPI_EXEC_MAX_NUMPROCS=$CPU_COUNT to limit the number of processes that are used in travis-ci.

wfspotz · 2017-08-02T19:16:39Z

@isuruf Any idea what

--------------------------------------------------------------------------
The value of the MCA parameter "plm_rsh_agent" was set to a path
that could not be found:

  plm_rsh_agent: ssh : rsh

Please either unset the parameter, or check that the path is correct
--------------------------------------------------------------------------

from CircleCI is referring to?

isuruf · 2017-08-03T06:13:46Z

Docker image doesn't have ssh installed, this suggests a workaround, open-mpi/ompi#1838 (comment)

wfspotz · 2017-08-08T19:27:23Z

@isuruf I'm stumped. I can't figure out how to get around the MCA parameter error. The suggested workaround doesn't help.

Build script for Mac OS X and Linux that builds PyTrilinos and all Trilinos packages supported by PyTrilinos.

Three changes: (1) skip for win OR python 3; (2) refer to new patch filename; (3) change from numpy x.x to numpy 1.7 in build section and >=1.7 in run section.

This change to calling ctest will output test results when there is a failure, to help with debugging.

The CPU limit is preventing tests I want to run and debug from running. Note that all of these MPI tests are relatively small on a small number of MPI nodes. Oversubscribing -- say, a 4 processor job on 2 physical procesors -- is inefficient, but still tests what I want tested.

isuruf · 2017-08-09T03:32:02Z

I rebased and pushed a fix

wfspotz · 2017-08-11T14:21:47Z

@isuruf Progress! The pre-install tests I run in build.sh (i.e., via ctest) are now working, and our error is now with the post-install tests implemented by your new run_test.py:

A requested component was not found, or was unable to be opened.  This
means that this component is either not installed or is unable to be
used on your system (e.g., sometimes this means that shared libraries
that the component requires are unable to be found/loaded).  Note that
Open MPI stopped checking at the first component that it did not find.

Host:      Traviss-Mac-9.local
Framework: pml
Component: isolated

This is true on both Circle-CI and Travis-CI. The os.environ["OMPI_MCA_pml"]="isolated" command doesn't appear to be having the intended effect.

wfspotz · 2017-08-23T17:34:27Z

@isuruf How does run_test.py get run? Is there a different way to set the MCA option?

This reverts commit 3f85d37.

wfspotz · 2017-08-24T17:01:01Z

@isuruf So what is happening with Travis-CI now? The job log is empty.

isuruf · 2017-08-25T15:47:59Z

@wfspotz, thanks for your contribution. A feedstock (https://github.com/conda-forge/pytrilinos-feedstock) will be created soon.

wfspotz · 2017-09-03T00:07:32Z

The feedstock page says that the Linux build is failing. I finally got it to report the output of the build phase to me. Everything looks okay, but at the very end, I see this:

anaconda_upload is not set.  Not uploading wheels: []
+ test -f /home/circleci/test/build_artefacts/conda-forge-build-done
+ exit 1
Exited with code 1

isuruf reviewed Jul 26, 2017

View reviewed changes

isuruf reviewed Jul 27, 2017

View reviewed changes

wfspotz and others added 16 commits August 9, 2017 07:57

Create meta.yaml

beeda3f

Create build.sh

684084e

Build script for Mac OS X and Linux that builds PyTrilinos and all Trilinos packages supported by PyTrilinos.

Create patch.txt

d26a04e

Update to build and run against numpy x.x

0f8f57b

Rename patch.txt to shebang.patch

323c187

Make fixes to meta.yaml

d1ccbb1

Three changes: (1) skip for win OR python 3; (2) refer to new patch filename; (3) change from numpy x.x to numpy 1.7 in build section and >=1.7 in run section.

ctest --output-on-failure

e3db820

This change to calling ctest will output test results when there is a failure, to help with debugging.

Update meta.yaml

9f7226f

More verbose ctest output for debugging

91581e9

Added --allow-run-as-root to build.sh

150be8c

Fixing -D MPI_EXEC_PRE_NUMPROCS_FLAGS=--allow-run-as-root

2f78f93

Added -D MPI_EXEC_MAX_NUMPROCS=$CPU_COUNT to build.sh

6182218

Checking to see if segfault is raised on a single processor

d9f7a53

Fix seg fault in Tpetra tests with patch

5a1377f

Added MPI exec flag for Linux

cd80e0f

isuruf added 2 commits August 9, 2017 07:57

quotes

c6d29f1

semi-colons to separate args

bc7dd92

isuruf added 2 commits August 9, 2017 09:19

Set OMPI_MCA_pml before importing PyTrilinos

3f85d37

Install ssh on the docker container

767390b

wfspotz added 11 commits August 23, 2017 09:41

Create patch.txt

ee0cdd9

Rename patch.txt to shebang.patch

33d21ea

More verbose ctest output for debugging

3642de5

Added --allow-run-as-root to build.sh

467ab53

Fixing -D MPI_EXEC_PRE_NUMPROCS_FLAGS=--allow-run-as-root

0601250

Checking to see if segfault is raised on a single processor

d07a1d6

Fix seg fault in Tpetra tests with patch

911d6b7

Try setting OpenMPI MCS parameter a different way

f009115

Check uname on CircleCI

b4850ef

Add ssh to the run environment

336b6eb

Fix run_test.py to not modify os.environ when sys.platform is darwin

aba2b75

isuruf added 2 commits August 23, 2017 21:10

Revert "Set OMPI_MCA_pml before importing PyTrilinos"

c4c5412

This reverts commit 3f85d37.

Fix requirements

b6cdaf3

isuruf merged commit 5ea4756 into conda-forge:master Aug 25, 2017

carterbox mentioned this pull request Oct 3, 2023

Rebuild for libboost 1.82 conda-forge/trilinos-feedstock#43

Closed

Uh oh!

Conversation

wfspotz commented Jul 25, 2017

Uh oh!

conda-forge-linter commented Jul 25, 2017

Uh oh!

isuruf commented Jul 26, 2017

Uh oh!

wfspotz commented Jul 26, 2017 via email • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wfspotz commented Jul 26, 2017

Uh oh!

isuruf commented Jul 26, 2017

Uh oh!

isuruf commented Jul 26, 2017

Uh oh!

isuruf commented Jul 26, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wfspotz commented Jul 26, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wfspotz commented Aug 2, 2017

Uh oh!

isuruf commented Aug 3, 2017

Uh oh!

wfspotz commented Aug 8, 2017

Uh oh!

isuruf commented Aug 9, 2017

Uh oh!

wfspotz commented Aug 11, 2017

Uh oh!

wfspotz commented Aug 23, 2017

Uh oh!

wfspotz commented Aug 24, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

isuruf commented Aug 25, 2017

Uh oh!

wfspotz commented Sep 3, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

wfspotz commented Jul 26, 2017 via email •

edited

Loading

wfspotz commented Aug 24, 2017 •

edited

Loading