Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Export of database from .h5 to .h5 on Linux is buggy #110

Open
eneelo opened this issue Jul 13, 2023 · 1 comment
Open

Export of database from .h5 to .h5 on Linux is buggy #110

eneelo opened this issue Jul 13, 2023 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@eneelo
Copy link
Collaborator

eneelo commented Jul 13, 2023

Describe the bug
On Linux, h5-file exported on Linux is corrupted when database is imported from .h5

To Reproduce
Steps to reproduce the behavior:

  1. Import .h5 file produced by SIMA to qats.TsDB
  2. Export database to new .h5 file.
  3. Import .h5 file exported above.
  4. Try to extract one or more time series from the database. This will produce an error like "KeyError: 'Unable to open object (component not found)'" (error message possibly depends on whether the new .h5 file is imported on Linux or in Windows).

Expected behavior
The recipe above should not produce an error.

Screenshots
HOLD

Desktop (please complete the following information):
HOLD

Additional context
HOLD

@eneelo eneelo added the bug Something isn't working label Jul 13, 2023
@eneelo eneelo self-assigned this Jul 13, 2023
@eneelo
Copy link
Collaborator Author

eneelo commented Jul 13, 2023

Inconsistencies found in qats during debugging so far:

  • qats.io.sima_h5.read_names() inserts a backslash ('\') as separator between datagroups and between datagroup and dataset.
  • qats.tsdb.TsDB._make_export_friendly_names() replaces the OS depedendent separator (which is '/' on Linux) by an underscore ('_'), meaning that the backslash inserted by sima_h5.read_names() is not replaced by an underscore.

It is unclear whether this is what causes the bug or if there are additional issues. Suggested steps to fix/identify the issue:

  • Fix inconsistency described above (e.g., by using system separator to separate datagroups/-sets at import).
  • Test import/export/import on Linux. If this doesn't fix the problem, further debugging is required.
  • Consider adding a import/export/import of .h5 as a unit test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant