Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dataset Upload: Improve heuristic to detect mag/layer-only dir #6382

Merged
merged 3 commits into from
Aug 9, 2022

Conversation

fm3
Copy link
Member

@fm3 fm3 commented Aug 8, 2022

When users upload a wkw dataset with only a mag directory or only a layer directory (and no datasource-properties.json), there is a heuristic in place to fix the directory structure (i.e. create the mag parent or layer parent dir). Based on this, the explore route can later infer a datasource-properties.json.
However, the heuristic had false positives on numerical-only layer names. This PR replaces the heuristic with a new one that searches for "header.wkw" files and then looks at how deep in the directory tree they occur.

Steps to test:

  • Upload a normal wkw dataset, should still work
  • Upload just the content of a layer directory, should be wrapped in "color" directory (see binaryData dir)
  • Upload just the content of a mag directory, should be wrapped in "color/1" directory (see binaryData dir)
  • Note that those repaired datasets will not be imported immediately. Rather, you need to go to the settings page first and import, to make them usable (i.e. create a datasource-properties.json file. This did not change in this PR)

Issues:


@fm3 fm3 requested a review from jstriebel August 8, 2022 10:00
@fm3 fm3 self-assigned this Aug 8, 2022
Copy link
Contributor

@jstriebel jstriebel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, didn't test it though

@fm3 fm3 enabled auto-merge (squash) August 9, 2022 12:56
@fm3 fm3 merged commit dc17fbd into master Aug 9, 2022
@fm3 fm3 deleted the mag-list-detection branch August 9, 2022 13:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

wkw datasets with numerical layer name strings are not loaded correctly
2 participants