-
Notifications
You must be signed in to change notification settings - Fork 171
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
General issues with converting from conda to pixi #1058
Comments
Thank you @vigneshmanick, this is really useful information, I'll take a more in depth look later and probably move some items into issues or PR's! |
@ruben-arts thanks for the response, i have some further points regarding the issue. Would you prefer that i update the issue description here or create another ? |
Please update it here, I don't think we've gotten around creating individual issues out of them yet :) Thanks for all the testing! |
Have updated the description and might add more points as and when i face them. |
The ability to add [system-requirements] from the pixi command line would be very useful. As far as I can tell it must be manually added to pixi.toml. This makes automation clunky. Everything else can be added with pixi - except this. |
Talking about requirements.txt, it would be useful to be able to pixi add requirements.txt or pixi init --import requirements.txt |
### Summary This PR changes how pixi is added to `PATH`. The value is prepended so that pixi takes precedence. This is similar to conda so that they libraries/tools installed via pixi take precedence. part of #1058 I would also like to discuss the lines above regarding adding to `bash_profile`, why not just add to `.bashrc` since it will be sourced by `.bash_profile` anyway. This will also allow the usage on both login and non login shells.
I have a problem related to this issue: PATH is modified by the Pixi install script only in |
I investigated it a bit more, and it turns out that it is annoyingly non-obvious how to properly add something to PATH so that it works in all typical circumstances. Basically we want the PATH to be updated in both interactive and non-interactive shells, we should add the appropriate commands:
In other shells, such as However, the problem with adding it to PATH at the beginning of |
for bash you could use |
I agree, however it would be better for the Pixi installer to update |
Hello Pixi devs,
I have listed some issues that i have faced when testing moving to pixi from conda.
Installation
In corporate environments the
$HOME
folder is not necessarily in the local machine but rather another mounted NFS drive which is usually restricted in space and also slow. For development in such environments the tools are installed in another folder that is available in the local machine. (e.g/scr/
).Same applies for the
PIXI_CACHE_DIR
folder. This should also not default to$HOME
.Changing the installation locaion via the
PIXI_HOME
env variable is supported, but this information is not readily apparent in the documentation.Suggestions:
Post installation changes
The update to the
PATH
variable should be changed since it has consequences onpixi global install <tool>
. If a tools is already available on the machine and if pixi global install is done for the same, then it doesn't work because of how the variable is set currently. Exampleexport PATH=$PATH:/scratch/user/tools/pixi/bin
. Since the pixi values are appended to thePATH
the tools from pixi are masked by the tools in OS. Conda prepends toPATH
.Suggestion: It should be changed to
export PATH=/scratch/user/tools/pixi/bin:$PATH
The pixi activation is added to
bash_profile
and not.bashrc
, this will result in non login shells not having pixi avaialble. Generallybash_profile
will already source thebashrc
file in one form or other (see sampel below). Conda updates the.bashrc
file if i am not mistaken.Suggestion: Add the changes similar to conda to
.bashrc
instead of in thebash_profile
which will make it avialable for both login and non login shells.The auto complete magic
echo 'eval "$(pixi completion --shell bash)"' >> ~/.bashrc
is appended to the end of the.bashrc
file. This is fine for cases where the file doesn't contain conda activation block but for cases where a conda activation block is present, the modifications should be above this section since conda recommends it's data should be at the end of the file.Suggestion:
It would be preferable if pixi related changes are made above the following lines to reduce friction in moving from conda
Maybe instead of putting the values (e.g cache, pixi_home etc.) in the bashrc, add these directly the the global pixi configuration?. This will help in all the global settings to be consolidated to a single file and also easier to modify. Correct me if i am wronng, currently the global configuration file should be created manually?
Conda offers a command to clean the cache and other temporary data, pixi should offer something similar (e.g
pixi clean cache
)Converting
environment.yml
to pixi environmentIssues i faced when importing an existing conda env into pixi project via
pixi init --import environment.yml
On my work linux machine i got the following error.
I searched the issues and found that the following needs to be set which is not readily apparent in the documentation.
For why the version in my case is a lower value, we use RHEL-7.9 and this is generally the standard in HPC, Uni / Research / enterprise setting. Soon this will be EOL and upgrade most probably will be to RHEL-8.9 which has the version
RHEL 8.9 | 4.18.0-513.5.1.el8_9
.I don't fully understand why the
__linux
version should be that high since the conda packages are built usually via the conda compiler tools and manylinux image which has very low os requirements (think it's using the centos7 image). Atleast none of the packages that i use have the high OS version as shown in the error by PIXI.Suggestion: It would be good if the above error is removed and the required block directly added to the
pixi.toml
Dependencies
I use a mix of both conda and pypi dependencies and and the pip config does not use the pypi registry but a custom self hosted registry e.g
global.index-url='https://nexus.somename.net/repository/pypi/simple'
Some other dependencies are downloaded from custom registries that are specified using the
--extra-index-url
option in pip. For example thevtk
package has anosmesa
variant for windows is available only via kitware registry and can be installed via pip using the followingrequirements.txt
.how do i specify for example, global.index-url='https://nexus.somename.net/repository/pypi/simple' and
--extra-index-url
in the dependencies in pixi?Connectivity issues
During installation i got the following error, i realized that this was due to the drive that had the
cache
folder running full.Suggestion: The error above could include the diskspace information too?
The most important of all, to connect to internet the machines use proxy and also a custom ca-certificate. This is configured on per application basis and for conda this is defined in the
.condarc
file via theproxy
and viaconda config --set ssl_verify <pathToYourFile>.crt
settings.Users have no possiblity to update these settings globally since admin/root rights are generally not provided for these machines. As of now, pixi install works only when i am not in the office network/machines that have custom certificates already added to the root certificate store. The errors for this case have been detailed in Pixi add not working behind proxy and firewall #1035.
Suggestion: It would be good if pixi also would provide similar setting for proxy and custom ca certifacte
On another machine the
anaconda.org
was part ofno_proxy
env variable and due to that the install failed withcannot resolved url
.Suggestion: Check and warn the users if they have set the these and other required urls
no_proxy
.Thanks for your awesome tool and your quick response. If i have made any mistakes feel free to correct me.
Update: 29.03.2024 updated description with further points
Update: 31.03.2024 Added suggestion for cleaning up cache
The text was updated successfully, but these errors were encountered: