Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sisl in chaquopy #901

Open
isa5456 opened this issue Jul 5, 2023 · 19 comments
Open

sisl in chaquopy #901

isa5456 opened this issue Jul 5, 2023 · 19 comments
Labels
packages

Comments

@isa5456
Copy link

isa5456 commented Jul 5, 2023

Hi everyone!
I am developing an application with android studio, and I am having so many problems when I try to integrate the python library "sisl" into my project using Chaquopy.
Is it possible? If not, do you know another way to do it?
I really appreciate any help you can provide.
@mhsmith
Copy link
Member

mhsmith commented Jul 5, 2023 •
edited
Loading

This package contains native components, so it would have to be built into a wheel file.

The current version also depends on netCDF4 (#823) and scipy>=1.5 (#648), so those should be built first, unless there's an acceptable older version that doesn't need them.

Then, if you'd like to try building this package yourself, follow the instructions here. And if you're successful, please make a pull request so we can add the package to the public repository.

If anyone else wants this package too, let us know by clicking the thumbs-up button above.

@isa5456
Copy link
Author

isa5456 commented Jul 6, 2023 •
edited by mhsmith
Loading

Thank you for answering me!
Then, I have also a doubt about Scipy (because I try to download a scipy>=1.5, but I get an error):
imagen
imagen

"Failed to install scipy==1.5.3 from https://files.pythonhosted.org/packages/93/63/4a566494594a13697c5d5d8a754d6e329d018ddf881520775e0229fa29ef/scipy-1.5.3.tar.gz#sha256=ddae76784574cc4c172f3d5edd7308be16078dd3b977e8746860c76c195fa707"

so first ¿Is it possible to install scipy >=1.5?
Thank you in advance!

@mhsmith
Copy link
Member

mhsmith commented Jul 6, 2023

As #648 says, newer versions of SciPy are currently blocked on getting an Android Fortran compiler. Older versions of sisl may be able to work with older versions of SciPy, but I haven't checked.

@isa5456
Copy link
Author

isa5456 commented Jul 6, 2023

And it will be available SciPy >=1.5 soon?

@mhsmith
Copy link
Member

mhsmith commented Jul 6, 2023

Probably not for at least another 6 months, sorry.

@alunram
Copy link

alunram commented Jul 6, 2023

And it will be available SciPy >=1.5 soon?

it'd be really interesting

@isa5456
Copy link
Author

isa5456 commented Jul 11, 2023

I have another question, but I am not sure if you can help me or not:
I have the next error
imagen
But I don't understand why there is a requirement that needs sisl==0.1.dev0, and I am not sure if there's something I can do
Thank you in advance.

@mhsmith
Copy link
Member

mhsmith commented Jul 11, 2023

Please post the entire build log, and as text, not an image.

@isa5456
Copy link
Author

isa5456 commented Jul 12, 2023

Please post the entire build log, and as text, not an image.

ERROR: Could not find a version that satisfies the requirement sisl==0.1.dev0 (from versions: 0.6.1, 0.6.2, 0.6.3, 0.7.4.dev0, 0.7.4, 0.7.5, 0.7.6, 0.7.7, 0.7.8, 0.7.9, 0.7.10, 0.7.13, 0.8.0, 0.8.1, 0.8.2, 0.8.3, 0.8.4, 0.8.5, 0.9.0, 0.9.1, 0.9.2, 0.9.4, 0.9.5, 0.9.6, 0.9.7, 0.9.8, 0.10.0, 0.11.0, 0.12.0, 0.12.1rc4, 0.12.1, 0.12.2, 0.13.0rc2, 0.13.0)
ERROR: No matching distribution found for sisl==0.1.dev0
Could not find a version that satisfies the requirement sisl==0.1.dev0 (from versions: 0.6.1, 0.6.2, 0.6.3, 0.7.4.dev0, 0.7.4, 0.7.5, 0.7.6, 0.7.7, 0.7.8, 0.7.9, 0.7.10, 0.7.13, 0.8.0, 0.8.1, 0.8.2, 0.8.3, 0.8.4, 0.8.5, 0.9.0, 0.9.1, 0.9.2, 0.9.4, 0.9.5, 0.9.6, 0.9.7, 0.9.8, 0.10.0, 0.11.0, 0.12.0, 0.12.1rc4, 0.12.1, 0.12.2, 0.13.0rc2, 0.13.0)

No matching distribution found for sisl==0.1.dev0

Chaquopy: Exit status 1

Task :app:generateDebugPythonRequirements FAILED
Execution failed for task ':app:generateDebugPythonRequirements'.
Process 'command 'python3.8'' finished with non-zero exit value 1

@mhsmith
Copy link
Member

mhsmith commented Jul 12, 2023

I said the ENTIRE build log. To display it in Android Studio, click the 'Build: failed' caption to the left of the message.

@isa5456
Copy link
Author

isa5456 commented Jul 12, 2023 •
edited
Loading

I said the ENTIRE build log. To display it in Android Studio, click the 'Build: failed' caption to the left of the message:

Executing tasks: [:app:assembleDebug] in project /home/isabel/AndroidStudioProjects/GreetingCard2

Task :app:createDebugVariantModel UP-TO-DATE
Task :app:preBuild UP-TO-DATE
Task :app:preDebugBuild UP-TO-DATE
Task :app:mergeDebugNativeDebugMetadata NO-SOURCE
Task :app:checkDebugAarMetadata UP-TO-DATE
Task :app:generateDebugResValues UP-TO-DATE
Task :app:mapDebugSourceSetPaths UP-TO-DATE
Task :app:generateDebugResources UP-TO-DATE
Task :app:mergeDebugResources UP-TO-DATE
Task :app:packageDebugResources UP-TO-DATE
Task :app:parseDebugLocalResources UP-TO-DATE
Task :app:createDebugCompatibleScreenManifests UP-TO-DATE
Task :app:extractDeepLinksDebug UP-TO-DATE
Task :app:processDebugMainManifest UP-TO-DATE
Task :app:processDebugManifest UP-TO-DATE
Task :app:processDebugManifestForPackage UP-TO-DATE
Task :app:processDebugResources UP-TO-DATE
Task :app:compileDebugKotlin UP-TO-DATE
Task :app:extractPythonBuildPackages UP-TO-DATE

Task :app:generateDebugPythonRequirements
Chaquopy: Installing for arm64-v8a
Looking in indexes: https://pypi.org/simple, https://chaquo.com/pypi-7.0, https://chaquo.com/pypi-13.1
Processing ./sisl
Installing build dependencies: started
Installing build dependencies: still running...
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Installing backend dependencies: started
Installing backend dependencies: still running...
Installing backend dependencies: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'done'
Collecting numpy
Using cached https://chaquo.com/pypi-7.0/numpy/numpy-1.19.5-0-cp38-cp38-android_21_arm64_v8a.whl (3.8 MB)
Collecting scipy
Using cached https://chaquo.com/pypi-7.0/scipy/scipy-1.4.1-0-cp38-cp38-android_21_arm64_v8a.whl (15.8 MB)
Collecting requests==2.24.0
Using cached requests-2.24.0-py2.py3-none-any.whl (61 kB)
Processing ./cftime-1.6.2-cp311-cp311-win_amd64.whl
Processing ./netCDF4-1.6.4-cp311-cp311-win_amd64.whl
Collecting pyparsing>=1.5.7
Using cached pyparsing-3.1.0-py3-none-any.whl (102 kB)
Collecting chaquopy-openblas>=0.2.20
Using cached https://chaquo.com/pypi-7.0/chaquopy-openblas/chaquopy_openblas-0.2.20-5-py3-none-android_21_arm64_v8a.whl (4.3 MB)
Collecting chaquopy-libgfortran>=4.9
Using cached https://chaquo.com/pypi-7.0/chaquopy-libgfortran/chaquopy_libgfortran-4.9-0-py3-none-android_21_arm64_v8a.whl (495 kB)
Collecting chaquopy-libcxx>=7000
Using cached https://chaquo.com/pypi-7.0/chaquopy-libcxx/chaquopy_libcxx-11000-0-py3-none-android_21_arm64_v8a.whl (308 kB)
Collecting idna<3,>=2.5
Using cached idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting chardet<4,>=3.0.2
Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached urllib3-1.25.11-py2.py3-none-any.whl (127 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2023.5.7-py3-none-any.whl (156 kB)
Building wheels for collected packages: sisl
Building wheel for sisl (PEP 517): started
Building wheel for sisl (PEP 517): still running...
Building wheel for sisl (PEP 517): finished with status 'done'
Created wheel for sisl: filename=sisl-0.1.dev0-cp38-cp38-manylinux_2_31_x86_64.whl size=4545244 sha256=b4c9486c1e9c8ac8a48ad5ee07e1df59426395f9eecdd2d61e49c896cafdca15
Stored in directory: /tmp/pip-ephem-wheel-cache-cdz72grd/wheels/a2/c0/c0/f1656a11e8fc706fc91d04718822ff251878dcec8bd43a6339
Successfully built sisl
Installing collected packages: chaquopy-libgfortran, chaquopy-openblas, numpy, chaquopy-libcxx, scipy, idna, chardet, urllib3, certifi, requests, cftime, netCDF4, pyparsing, sisl
Successfully installed certifi-2023.5.7 cftime-1.6.2 chaquopy-libcxx-11000 chaquopy-libgfortran-4.9 chaquopy-openblas-0.2.20 chardet-3.0.4 idna-2.10 netCDF4-1.6.4 numpy-1.19.5 pyparsing-3.1.0 requests-2.24.0 scipy-1.4.1 sisl-0.1.dev0 urllib3-1.25.11
Chaquopy: Installing for armeabi-v7a
Looking in indexes: https://pypi.org/simple, https://chaquo.com/pypi-7.0, https://chaquo.com/pypi-13.1
Collecting cftime==1.6.2
Using cached cftime-1.6.2.tar.gz (52 kB)
Installing build dependencies: started
Installing build dependencies: still running...
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'done'
Collecting chaquopy-libcxx==11000
Using cached https://chaquo.com/pypi-7.0/chaquopy-libcxx/chaquopy_libcxx-11000-0-py3-none-android_16_armeabi_v7a.whl (237 kB)
Collecting chaquopy-libgfortran==4.9
Using cached https://chaquo.com/pypi-7.0/chaquopy-libgfortran/chaquopy_libgfortran-4.9-0-py3-none-android_16_armeabi_v7a.whl (394 kB)
Collecting chaquopy-openblas==0.2.20
Using cached https://chaquo.com/pypi-7.0/chaquopy-openblas/chaquopy_openblas-0.2.20-5-py3-none-android_16_armeabi_v7a.whl (4.0 MB)
Collecting netCDF4==1.6.4
Using cached netCDF4-1.6.4.tar.gz (778 kB)
Installing build dependencies: started
Installing build dependencies: still running...
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Installing backend dependencies: started
Installing backend dependencies: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'done'
Collecting numpy==1.19.5
Using cached https://chaquo.com/pypi-7.0/numpy/numpy-1.19.5-0-cp38-cp38-android_16_armeabi_v7a.whl (3.7 MB)
Collecting scipy==1.4.1
Using cached https://chaquo.com/pypi-7.0/scipy/scipy-1.4.1-0-cp38-cp38-android_16_armeabi_v7a.whl (15.0 MB)
ERROR: Could not find a version that satisfies the requirement sisl==0.1.dev0 (from versions: 0.6.1, 0.6.2, 0.6.3, 0.7.4.dev0, 0.7.4, 0.7.5, 0.7.6, 0.7.7, 0.7.8, 0.7.9, 0.7.10, 0.7.13, 0.8.0, 0.8.1, 0.8.2, 0.8.3, 0.8.4, 0.8.5, 0.9.0, 0.9.1, 0.9.2, 0.9.4, 0.9.5, 0.9.6, 0.9.7, 0.9.8, 0.10.0, 0.11.0, 0.12.0, 0.12.1rc4, 0.12.1, 0.12.2, 0.13.0rc2, 0.13.0)
ERROR: No matching distribution found for sisl==0.1.dev0
Chaquopy: Exit status 1

Task :app:generateDebugPythonRequirements FAILED
Task :app:mergeDebugPythonSources UP-TO-DATE
Task :app:javaPreCompileDebug UP-TO-DATE
Task :app:mergeDebugShaders UP-TO-DATE
Task :app:compileDebugShaders NO-SOURCE
Task :app:generateDebugAssets UP-TO-DATE
Task :app:generateDebugPythonAppAssets UP-TO-DATE
Task :app:generateDebugPythonMiscAssets UP-TO-DATE
Task :app:processDebugJavaRes NO-SOURCE
Task :app:mergeDebugJavaResource UP-TO-DATE
Task :app:checkDebugDuplicateClasses UP-TO-DATE
Task :app:desugarDebugFileDependencies UP-TO-DATE
Task :app:mergeExtDexDebug UP-TO-DATE
Task :app:mergeLibDexDebug UP-TO-DATE
Task :app:generateDebugPythonJniLibs UP-TO-DATE
Task :app:mergeDebugJniLibFolders UP-TO-DATE
Task :app:mergeDebugNativeLibs UP-TO-DATE
Task :app:stripDebugDebugSymbols UP-TO-DATE
Task :app:validateSigningDebug UP-TO-DATE
Task :app:writeDebugAppMetadata UP-TO-DATE
Task :app:writeDebugSigningConfigVersions UP-TO-DATE

FAILURE: Build failed with an exception.

  • What went wrong:
    Execution failed for task ':app:generateDebugPythonRequirements'.

Process 'command 'python3.8'' finished with non-zero exit value 1

To view full details in Android Studio:

  • Click the 'Build: failed' caption to the left of this message.

  • Then scroll up to see the full output.

  • Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.

BUILD FAILED in 11m 21s
33 actionable tasks: 1 executed, 32 up-to-date

This is all from trying to install a local sisl package (with some modification)

@mhsmith
Copy link
Member

mhsmith commented Jul 12, 2023 •
edited
Loading

It looks like you've added the sisl source code location to the pip block in your build.gradle file. This only works for pure-Python packages. For packages with native components, you'll need to follow the instructions in my comment above.

But there's no point in even trying to build sisl until the netCDF4 and SciPy requirements have been solved first. The netCDF4 work in #823 is close to completion, but SciPy will be much more difficult, so I suggest you check whether there's an older sisl version which is compatible with our current SciPy version, 1.4.1.

@isa5456
Copy link
Author

isa5456 commented Jul 18, 2023 •
edited by mhsmith
Loading

Thank you for the answer!. Now I am trying to build sisl package myself. Here are some questions related to it:
The .zip for the ABI I want to build for has to be unzipped in the directory?
The package is in python but has code in fortran and c, so do I need build.sh script?
Is it easier to do if I have a setup.py?
When is necessary a patches subdirectory containing patch files.
I have this error:
image
any idea?
Thank you in advance!

@mhsmith
Copy link
Member

mhsmith commented Jul 18, 2023

Our build system doesn't currently support Fortran (#648 (comment)), so you won't be able to build any package that contains Fortran code. Sorry about that. We'll be working on the Fortran issue in a few months time.

@isa5456
Copy link
Author

isa5456 commented Aug 21, 2023

One last question, as I've read in the SISL documentation, it can be compiled without Fortran using pip. Is this a possibility, or is it simply not feasible due to its Fortran code?"
Finally, Does anyone know other ways to use fortran code in an Android ?
Thank you so much!!!!!

@mhsmith
Copy link
Member

mhsmith commented Aug 21, 2023 •
edited
Loading

It looks like we might have a way forward with Fortran and SciPy. Please subscribe to #648, and I'll post an update there in a few days.

as I've read in the SISL documentation, it can be compiled without Fortran using pip

Where does it say that?

@isa5456
Copy link
Author

isa5456 commented Aug 22, 2023

It looks like we might have a way forward with Fortran and SciPy. Please subscribe to #648, and I'll post an update there in a few days.
Thank you !

as I've read in the SISL documentation, it can be compiled without Fortran using pip

Where does it say that?
image

Link: https://zerothi.github.io/sisl/installation.html
So that's why I'm wondering if there would be any way to install sisl without encountering issues with the Fortran code

@mhsmith
Copy link
Member

mhsmith commented Jun 20, 2024

[From #1185 by @loreoregi]

I'm encountering an issue where CMake cannot find the Python_NumPy_INCLUDE_DIRS when attempting to build the sisl package for Chaquopy with Python 3.9. This issue prevents the build process from completing successfully. I am following the steps provided in the Chaquopy server/pypi README. I'm using Chaquopy 15.1.0, sisl 0.14.3, and CMake 3.22.1.

CMake Error at /home/loreoregi/chaquopy/server/pypi/packages/sisl/build/0.14.3/cp39-cp39-android_21_x86_64/env/lib/python3.9/site-packages/scikit_build_core/resources/find_python/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find Python (missing: Python_NumPy_INCLUDE_DIRS Interpreter
  Development.Module NumPy) (found version "3.9.18")
Call Stack (most recent call first):
  /home/loreoregi/chaquopy/server/pypi/packages/sisl/build/0.14.3/cp39-cp39-android_21_x86_64/env/lib/python3.9/site-packages/scikit_build_core/resources/find_python/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
  /home/loreoregi/chaquopy/server/pypi/packages/sisl/build/0.14.3/cp39-cp39-android_21_x86_64/env/lib/python3.9/site-packages/scikit_build_core/resources/find_python/FindPython.cmake:622 (find_package_handle_standard_args)
  CMakeLists.txt:39 (find_package)

I have tried manually setting the Python_NumPy_INCLUDE_DIRS in build-wheel.py and CMakeLists.txt. verifying the existence of NumPy include directories, ensuring paths are correctly set in the toolchain file, and I still get the same error. I also tried running CMake commands directly, which succeed but differ from the behavior of build-wheel.py.

I would appreciate any guidance or suggestions on how to resolve this issue. Thank you!

@mhsmith mhsmith mentioned this issue Jun 20, 2024
Closed
@mhsmith
Copy link
Member

mhsmith commented Jun 20, 2024

Sorry, I don't remember the details of how CMake finds NumPy include directories. You may find some ideas by searching the existing recipes for packages whose meta.yaml files mention both CMake and NumPy, though this may be an area which has changed over the last few years.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
packages
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mhsmith @isa5456 @alunram