provide nsls2-analysis-lite conda package #14
Comments
|
pasted below where I started to get not 100% Do you really want all the details?dxfile-0.5 | | 0%
imagecodecs-2020.5.3 | ############################################################################# | 100%
matplotlib-3.2.2 | ############################################################################# | 100%
ipympl-0.1.1 | | 0%
matplotlib-base-3.2. | ############################################################################# | 100%
mkl_fft-1.3.0 | ############################################################################# | 100%
mkl_random-1.1.0 | ############################################################################# | 100%
numpy-1.19.2 | ############################################################################# | 100%
asdf-2.7.3 | | 0%
asteval-0.9.23 | | 0%
caproto-0.8.0 | | 0%
edffile-5.0.0 | | 0%
event-model-1.17.1 | | 0%
fabio-0.11.0 | | 0%
grid-strategy-0.0.1 | | 0%
igor-0.3 | | 0%
imageio-2.9.0 | ############################################################################# | 100%
modestimage-0.2 | | 0%
msgpack-numpy-0.4.7. | ############################################################################# | 100%
netcdf4-1.4.2 | ############################################################################# | 100%
numba-0.53.0 | ############################################################################# | 100%
numcodecs-0.7.3 | ############################################################################# | 100%
numexpr-2.7.3 | ############################################################################# | 100%
pandas-1.2.3 | ############################################################################# | 100%
periodictable-1.5.3 | | 0%
pyepics-3.4.3 | | 0%
pyerfa-1.7.2 | ############################################################################# | 100%
pyfftw-0.12.0 | | 0%
pyqtgraph-0.11.0 | ############################################################################# | 100%
pystackreg-0.2.5 | | 0%
pytest-arraydiff-0.3 | ############################################################################# | 100%
pytest-doctestplus-0 | ############################################################################# | 100%
pywavelets-1.1.1 | ############################################################################# | 100%
pyxdameraulevenshtei | | 0%
pyzbar-0.1.8 | | 0%
scipy-1.6.1 | ############################################################################# | 100%
spefile-1.6 | | 0%
tifffile-2020.10.1 | | 0%
uncertainties-3.1.5 | | 0%
vispy-0.6.5 | | 0%
xray-vision-0.0.9 | | 0%
xraylib-4.0.0 | | 0%
arvpyf-0.4.3 | | 0%
astropy-4.2 | ############################################################################# | 100%
bumps-0.8.0 | | 0%
dask-2021.3.0 | ############################################################################# | 100%
dxchange-0.1.6 | | 0%
edrixs-0.0.5 | | 0%
lmfit-1.0.1 | | 0%
mendeleev-0.6.1 | | 0%
napari-svg-0.1.4 | | 0%
nexusformat-0.6.0 | | 0%
nsls2-detector-handl | | 0%
oscars-2.1.8 | | 0%
peakutils-1.3.3 | | 0%
pims-0.5 | ############################################################################# | 100%
py4xs-2021.2.1.0 | | 0%
pycentroids-0.2.0rc2 | | 0%
pytest-astropy-0.8.0 | ############################################################################# | 100%
rixs-0.0.3 | | 0%
scikit-image-0.17.2 | ############################################################################# | 100%
scikit-learn-0.24.1 | ############################################################################# | 100%
silx-0.15.0 | | 0%
srw-2019.03.01 | | 0%
srwpy-0.0.4 | | 0%
suitcase-csv-0.2.1 | | 0%
suitcase-json-metada | | 0%
suitcase-jsonl-0.2.0 | | 0%
suitcase-mongo-0.3.0 | | 0%
suitcase-msgpack-0.2 | | 0%
suitcase-specfile-0. | | 0%
suitcase-tiff-0.2.1 | | 0%
xarray-0.17.0 | ############################################################################# | 100%
xas-0.0.1 | | 0%
xraydb-4.4.5 | | 0%
zarr-2.6.1 | ############################################################################# | 100%
area-detector-handle | | 0%
bluesky-live-0.0.7 | | 0%
eiger-io-2.0.4 | | 0%
gwcs-0.16.1 | | 0%
intake-0.6.0 | | 0%
isstools-3.0.0 | | 0%
lixtools-2021.2.21.1 | | 0%
napari-0.4.2 | | 0%
nexpy-0.13.2 | | 0%
pyfai-0.20.0 | | 0%
sasmodels-1.0.4 | | 0%
tomopy-1.1.2 | | 0%
databroker-1.2.2 | | 0%
photutils-1.0.2 | | 0%
sasview-5.0.3 | | 0%
scikit-beam-0.0.24 | | 0%
xraylarch-0.9.50 | | 0%
bluesky-1.6.7 | | 0%
chxtools-3.0.2 | | 0%
csxtools-0.1.16 | | 0%
databroker-pack-0.3. | | 0%
legacy-suitcase-0.7. | | 0%
pyxrf-1.0.4 | | 0%
sixtools-0.0.3 | | 0%
xpdconf-0.4.5 | | 0%
bluesky-kafka-0.3.0 | | 0%
shed-0.7.5 | | 0%
xpdsim-0.4.2 | | 0%
xpdview-0.5.1 | | 0%
analysis-2021C1.2 | | 0%
nslsii-0.1.2 | | 0%
pdfstream-0.4.5 | | 0%
xpdtools-0.8.2 | | 0%
hxntools-0.5.1 | | 0%
xpdan-0.8.2 | | 0%
nsls2-analysis-2021C | | 0%
InvalidArchiveError("Error with archive /nsls2/users/abarbour/.conda/pkgs/botocore-1.20.29-pyhd3eb1b0_14do24g_3/pkg-botocore-1.20.29-pyhd3eb1b0_1.tar.zst. You probably need to delete and re-download or re-create this file. Message from libarchive was:\n\nFailed to create dir 'info/licenses'")
InvalidArchiveError("Error with archive /nsls2/users/abarbour/.conda/pkgs/sphinx-3.5.2-pyhd3eb1b0_0kmk08alo/pkg-sphinx-3.5.2-pyhd3eb1b0_0.tar.zst. You probably need to delete and re-download or re-create this file. Message from libarchive was:\n\nFailed to create dir 'info/licenses'")
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
CondaHTTPError: HTTP 404 NOT FOUND for url <https://conda.anaconda.org/nsls2forge/noarch/tifffile-2020.10.1-pyhe636693_0.tar.bz2>
Elapsed: 00:00.377675
CF-RAY: 64efa1ddba1c616a-ORD
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded
[Errno 122] Disk quota exceeded |
|
@ambarb, I think the installation of the required packages in an environment in a Docker container and the packing it with conda-pack will help to eliminate the downloaded package cache on the target machine. Do you want to try that? |
|
ahh. ok. sounds like there is a plan to deal with this. thanks @mrakitin for the suggestion. my immediate need has passed so I don't need to try this with docker on a beamline machine right this minute because priorities changed. But I will message in the program support channel if later next week it looks like we don't have a replacement option. |
|
I normally run the Docker container on my laptop - it does not have to be on the beamline. We can even automate it so that it could generate the artifacts in the cloud. I developed this simple tool to automatically generate a building script for different environments based on the requirements: https://github.com/mrakitin/conda-pack-template/. That can be extended too to support |
|
Right, but I want to interact with real data, and sometimes I want to try out a package on real data. So the laptop really isn't an option for me. For instance, the CSX flat field correction makes pixels with NaNs. I tried scipy and sk-image libraries to calculate things like center of mass, but none of these libraries handles NaN so I needed to install photoutils in order to make progress. If we need to do more sophisticated masking as a final solution and not rely on photoutilis, fine, but I don't want to be figuring that out when my objective is to analyze data. Plus, I have a Mac, and several things (even using docker) fail to work so it is easier just to do everything in one place. I've found that being able to make my own conda environments are also key in troubleshooting how/when things have broken and the fastest way to get them working again. |
|
Sorry for the confusion. Docker on a laptop or anywhere else than on NSLS-II controls inventory can be used to just generate those conda envs. Then they can be transferred, unpacked, and used for analysis with real data on beamline machines or other places that have access to the data. That's to basically save space in our limited |
conda create --clone _the_2021_official_package --name myenv_custom_envwill not work because creating this conda environment on
/nsls2/users/my_user_nameexceeds allocated disk spaceNot sure if this is the appropriate place for discussing which packages should be part the lite version, but this makes the system less flexible than what we had original
The text was updated successfully, but these errors were encountered: