Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Users see FileNotFoundError tracebacks #110

Open
douglatornell opened this issue Dec 1, 2023 · 0 comments
Open

Users see FileNotFoundError tracebacks #110

douglatornell opened this issue Dec 1, 2023 · 0 comments
Assignees
Labels
bug Something isn't working
Milestone

Comments

@douglatornell
Copy link
Member

FileNotFoundError tracebacks are not caught by the code. Instead execution ends ungracefully and gives the user a traceback. There are at least 2 situations in which this happens:

  1. When reading the first and last dataset files to calculate the set of variables to drop; traceback:
Traceback (most recent call last):
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/xarray/backends/file_manager.py", line 211, in _acquire_with_cache_info
    file = self._cache[self._key]
           ~~~~~~~~~~~^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/xarray/backends/lru_cache.py", line 56, in __getitem__
    value = self._cache[key]
            ~~~~~~~~~~~^^^^^
KeyError: [<class 'netCDF4._netCDF4.Dataset'>, ('/results2/SalishSea/month-avg.202111/SalishSeaCast_1m_biol_T_20231001_20231031.nc',), 'r', (('clobber', True), ('diskless', False), ('format', 'NETCDF4'), ('persist', False)), 'b54a73f0-b76e-44f2-bf7e-ba47f19e3334']

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/doug/conda_envs/reshapr-dev/bin/reshapr", line 8, in <module>
    sys.exit(reshapr())
             ^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/doug/warehouse/MOAD/Reshapr/reshapr/cli/extract.py", line 61, in extract
    reshapr.core.extract.cli_extract(config_file, start_date, end_date)
  File "/media/doug/warehouse/MOAD/Reshapr/reshapr/core/extract.py", line 106, in cli_extract
    with open_dataset(ds_paths, chunk_size, config) as ds:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/doug/warehouse/MOAD/Reshapr/reshapr/core/extract.py", line 466, in open_dataset
    with xarray.open_dataset(ds_path, chunks=chunk_size) as ds:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/xarray/backends/api.py", line 572, in open_dataset
    backend_ds = backend.open_dataset(
                 ^^^^^^^^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", line 593, in open_dataset
    store = NetCDF4DataStore.open(
            ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", line 400, in open
    return cls(manager, group=group, mode=mode, lock=lock, autoclose=autoclose)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", line 347, in __init__
    self.format = self.ds.data_model
                  ^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", line 409, in ds
    return self._acquire()
           ^^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", line 403, in _acquire
    with self._manager.acquire_context(needs_lock) as root:
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/xarray/backends/file_manager.py", line 199, in acquire_context
    file, cached = self._acquire_with_cache_info(needs_lock)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/xarray/backends/file_manager.py", line 217, in _acquire_with_cache_info
    file = self._opener(*self._args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "src/netCDF4/_netCDF4.pyx", line 2469, in netCDF4._netCDF4.Dataset.__init__
  File "src/netCDF4/_netCDF4.pyx", line 2028, in netCDF4._netCDF4._ensure_nc_success
FileNotFoundError: [Errno 2] No such file or directory: '/results2/SalishSea/month-avg.202111/SalishSeaCast_1m_biol_T_20231001_20231031.nc'
  1. When there is a missing file in the collection of dataset file paths that xarray.open_mfdataset() is operating on; traceback:
    Screenshot 2023-11-28 at 1 29 48 PM
@douglatornell douglatornell added the bug Something isn't working label Dec 1, 2023
@douglatornell douglatornell added this to the v23.2 milestone Dec 1, 2023
@douglatornell douglatornell modified the milestones: v23.2, v24.0 Dec 22, 2023
@douglatornell douglatornell modified the milestones: v24.1, v25.1 Dec 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant