Skip to content

Retire wk-connect #7032

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 30 commits into from
May 31, 2023
Merged

Retire wk-connect #7032

merged 30 commits into from
May 31, 2023

Conversation

hotzenklotz
Copy link
Member

@hotzenklotz hotzenklotz commented Apr 27, 2023

(merge this only after the remaining connect datastores have been shutdown)

PR removes wk-connect integration from WK frontend. Neuroglancer Precomputed, Zarr, and N5 dataset can now be streamed/loaded natively from the cloud by WEBKNOSSOS instead.

Changes:

  • Remove forms/UI for Precomupted and BossDB dataset imports. Use "Add remote Dataset" instead
  • Remove mentions to wk-connect and BossDB in docs
  • Remove wk-connect from Docker file
  • Merged all the backend changes from remove isConnector property for datastores #7031

Steps to test:

  • Data loading for regular datasets should still work as usual

Issues:


(Please delete unneeded items, merge only when none are left open)

@hotzenklotz hotzenklotz changed the title Remove wkconnect Retire wk-connect Apr 27, 2023
@hotzenklotz hotzenklotz self-assigned this Apr 27, 2023
hotzenklotz and others added 9 commits April 28, 2023 10:17
…ove_wkconnect

* 'master' of github.com:scalableminds/webknossos:
  Update docker compose commands + dev install readme (#7002)
  Add segment groups (#6966)
  Add screenshot nightly test for wkorg (#7030)
  Workaround for WebGL crash for datasets with many segmentation layers (#6995)
  Fix download of public annotation, include access ctx in user cache key (#7025)
  Fix that changing a segment color could lead to a crash (#7000)
  Add more error chaining to annotation download (#7023)
  Guard against NaNs in shader (#7018)
  Store editable mappings in multiple fossildb columns+keys (#6903)
  Context action to move tree to group (#7005)
  Release 23.05.0 (#7014)
  Remove vault cache when reloading dataset (#7007)
  Fix viewing of public datasets (#7010)
  Update screenshots scalebar positioning (#7003)
  Update team members (#6999)
…s into remove_wkconnect

* 'remove_wkconnect' of github.com:scalableminds/webknossos:
  update snapshots
…os into remove_wkconnect

* 'no-more-wkconnect' of github.com:scalableminds/webknossos:
  Fix exception in applying UpdateTreeGroupVisibility skeleton action (#7037)
  Fix organization storage layouting (#7034)
  adapt schema version numbers
  adapt schema number
  changelog + migration guide
  delete datastore in migration. add changelog
  snapshots, test db
  remove isConnector property for datastores
@hotzenklotz hotzenklotz marked this pull request as ready for review May 4, 2023 07:29
@hotzenklotz hotzenklotz requested a review from philippotto May 4, 2023 07:29
@fm3 fm3 requested a review from frcroth May 4, 2023 07:31
@fm3
Copy link
Member

fm3 commented May 4, 2023

@hotzenklotz or anyone who is familiar with this, could you double-check that the webknossos-libs can still talk to webknossos on this branch? There are some occurences of the isConnector property in the python client, but maybe that is not a problem, I’m not sure if it is used https://github.com/scalableminds/webknossos-libs/search?q=isconnector

Copy link
Member

@philippotto philippotto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweet red lines 👍

docs/export.md Outdated
@@ -4,7 +4,7 @@ We believe in using the best tool for the job and WEBKNOSSOS makes it easy to do

## Data Export through Zarr Streaming

Any dataset and volume annotation in WEBKNOSSOS can also be accessed by third-party software through the Zarr streaming interface. In other words, WEBKNOSSOS is a fully, capable server for streaming data according to the Zarr spec to any client and can be viewed as an alternative to cloud storage options such as S3/Google Cloud.
Any dataset and volume annotation in WEBKNOSSOS can also be accessed by third-party software through the Zarr streaming interface. In other words, WEBKNOSSOS is a fully, capable server for streaming data according to the Zarr spec to any client and can be viewed as an alternative to cloud storage options such as AWS S3/Google Cloud.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Any dataset and volume annotation in WEBKNOSSOS can also be accessed by third-party software through the Zarr streaming interface. In other words, WEBKNOSSOS is a fully, capable server for streaming data according to the Zarr spec to any client and can be viewed as an alternative to cloud storage options such as AWS S3/Google Cloud.
Any dataset and volume annotation in WEBKNOSSOS can also be accessed by third-party software through the Zarr streaming interface. In other words, WEBKNOSSOS is a fully, capable server for streaming data according to the Zarr spec to any client and can be viewed as an alternative to cloud storage options such as AWS S3 or Google Cloud.

hotzenklotz and others added 4 commits May 5, 2023 11:08
Co-authored-by: frcroth <[email protected]>
Co-authored-by: Philipp Otto <[email protected]>
…ove_wkconnect

* 'master' of github.com:scalableminds/webknossos: (33 commits)
  Fix bug in fallback rendering when layer has missing mags (#7082)
  Fix duplicate on volumes with no fallback layer (#7085)
  fix scrolling in organization switcher in terms-of-services modal (#7083)
  Auto-Select via SAM (#7051)
  Build STL in chunks when exporting them (#7074)
  Improve performance for large (oversegmentation) meshes (#7077)
  Fix display of used storage in power plan (#7057)
  Fix user limits for invites (#7078)
  adds fileSize to voxelytics workflow list (#7071)
  Improve error logging for bucket requests (#7053)
  Fix zarr streaming datasource-properties.json generation for non-wkw/zarr datasets (#7065)
  Min length for layer names is one (#7064)
  Include voxelytics workflow name in tab title (#7070)
  Fix local to global layer index look up (#7066)
  Merge editable mappings (#7026)
  store correct artifacts for wkorg nightly (#7060)
  Team edit modal (#7043)
  Fix voxel offset in chunk name for unsharded neuroglancer precomputed datasets (#7062)
  Fix offset when loading non-aligned buckets for zarr/n5/precomputed (#7058)
  fixes wallTimes query for workflow reports (#7059)
  ...
@hotzenklotz
Copy link
Member Author

@fm3 Can you please refresh the snapshots once more to make the CI happy. Thanks

hotzenklotz and others added 5 commits May 17, 2023 15:27
…s into remove_wkconnect

* 'remove_wkconnect' of github.com:scalableminds/webknossos:
  Update docs/index.md
  Update docs/export.md
  Update MIGRATIONS.unreleased.md
  Update CHANGELOG.unreleased.md
  refresh snapshots
@hotzenklotz hotzenklotz enabled auto-merge (squash) May 31, 2023 08:22
@hotzenklotz hotzenklotz merged commit ce7cc1d into master May 31, 2023
@hotzenklotz hotzenklotz deleted the remove_wkconnect branch May 31, 2023 08:51
hotzenklotz added a commit that referenced this pull request May 31, 2023
…ty-list-drawings

* 'master' of github.com:scalableminds/webknossos:
  Retire wk-connect (#7032)
  Release 23.06.0 (#7106)
@normanrz normanrz mentioned this pull request May 31, 2023
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants