Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

identify whether or not a file is zipped in remote accessing of files #52

Closed
imedan opened this issue Dec 18, 2023 · 0 comments · Fixed by #53
Closed

identify whether or not a file is zipped in remote accessing of files #52

imedan opened this issue Dec 18, 2023 · 0 comments · Fixed by #53
Assignees
Labels

Comments

@imedan
Copy link

imedan commented Dec 18, 2023

I've run into an issue with accessing astra files remotely, where I get an error that the file cannot be found:

In [1]: import sdss_access

In [2]: sdss_access.__version__
Out[2]: '3.0.3'

In [3]: import tree

In [4]: tree.__version__
Out[4]: '4.0.5'

In [5]: from sdss_access import Access, RsyncAccess

In [6]: access = RsyncAccess(release='ipl3')

In [7]: access.remote()

In [8]: access.add('mwmAllStar', v_astra='0.5.0')

In [9]: access.set_stream()
[ERROR]: Traceback (most recent call last):
  File "/Users/imedan/anaconda3/envs/kdwarf_params/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3508, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-9-f43dbf15e80a>", line 1, in <module>
    access.set_stream()
  File "/Users/imedan/anaconda3/envs/kdwarf_params/lib/python3.8/site-packages/sdss_access/sync/baseaccess.py", line 101, in set_stream
    self.set_stream_task(task)
  File "/Users/imedan/anaconda3/envs/kdwarf_params/lib/python3.8/site-packages/sdss_access/sync/rsync.py", line 60, in set_stream_task
    out = self.get_task_out(task=task)
  File "/Users/imedan/anaconda3/envs/kdwarf_params/lib/python3.8/site-packages/sdss_access/sync/rsync.py", line 37, in get_task_out
    raise AccessError("Return code %r\n%s" % (status, err))
sdss_access.path.path.AccessError: Return code 23
b'rsync: link_stat "spectro/astra/0.5.0/summary/mwmAllStar-0.5.0.fits" (in ipl-3) failed: No such file or directory (2)\nrsync error: some files could not be transferred (code 23) at /AppleInternal/Library/BuildRoots/a0876c02-1788-11ed-b9c4-96898e02b808/Library/Caches/com.apple.xbs/Sources/rsync/rsync/main.c(1404) [receiver=2.6.9]\n'

This does not seem to happen when accessing the file at Utah:

from sdss_access.path import Path
p=Path()
p.full('mwmAllStar', v_astra='0.5.0')
'/uufs/chpc.utah.edu/common/home/sdss50/sdsswork/mwm/spectro/astra/0.5.0/summary/mwmAllStar-0.5.0.fits.gz'

p.exists('mwmAllStar', v_astra='0.5.0')
True 

This issue then seems to be a bug where sdss_access is not correctly utilizing the feature to identify whether a file is zipped or not with remote access.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants