Release 23.05.1

CHANGELOG.released.md

@@ -7,6 +7,43 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Calendar Versioning](http://calver.org/) `0Y.0M.MICRO`.
 For upgrade instructions, please check the [migration guide](MIGRATIONS.released.md).
 
+## [23.05.1](https://github.com/scalableminds/webknossos/releases/tag/23.05.1) - 2023-05-02
+[Commits](https://github.com/scalableminds/webknossos/compare/23.05.0...23.05.1)
+
+### Fixed
+- Fixed rendering issues on some affected systems that led to "black holes". [#7018](https://github.com/scalableminds/webknossos/pull/7018)
+- Added a workaround for a WebGL crash which could appear when a dataset contained many segmentation layers. [#6995](https://github.com/scalableminds/webknossos/pull/6995)
+
+## [23.05.0](https://github.com/scalableminds/webknossos/releases/tag/23.05.0) - 2023-04-25
+[Commits](https://github.com/scalableminds/webknossos/compare/23.04.2...23.05.0)
+
+### Highlights
+- Segments can now be removed from the segment list via the context menu. [#6944](https://github.com/scalableminds/webknossos/pull/6944)
+- Added support for viewing neuroglancer precomputed segmentations using "compressed segmentation" compression. [#6947](https://github.com/scalableminds/webknossos/pull/6947)
+
+### Added
+- WEBKNOSSOS screenshots now include a logo in the lower left corner. Users with a paid license can disable this feature in the viewport options in the settings tab on the left side of the screen. [#6910](https://github.com/scalableminds/webknossos/pull/6910)
+- Added support for datasets with mixed magnifications (e.g., one layer contains mag 2-2-2 and another contains 2-2-1). [#6943](https://github.com/scalableminds/webknossos/pull/6943)
+- Added rendering precomputed meshes with level of detail depending on the zoom of the 3D viewport. This feature only works with version 3 mesh files. [#6909](https://github.com/scalableminds/webknossos/pull/6909)
+- Editing the meta data of segments (e.g., the name) is now undoable. [#6944](https://github.com/scalableminds/webknossos/pull/6944)
+- Added more icons and small redesigns for various pages in line with the new branding. [#6938](https://github.com/scalableminds/webknossos/pull/6938)
+- Added more graphics and branding to job notification emails. [#6994](https://github.com/scalableminds/webknossos/pull/6994)
+- Added action button in Team Admin page that links to the User Admin page to edit team members there. [#6958](https://github.com/scalableminds/webknossos/pull/6999)
+
+### Changed
+- Moved the view mode selection in the toolbar next to the position field. [#6949](https://github.com/scalableminds/webknossos/pull/6949)
+- Redesigned welcome toast for new, annonymous users with new branding. [#6961](https://github.com/scalableminds/webknossos/pull/6961)
+- When saving annotations, the URL of the webknossos instance is stored in the resulting NML file. [#6964](https://github.com/scalableminds/webknossos/pull/6964)
+
+### Fixed
+- Fixed unintended dependencies between segments of different volume layers which used the same segment id. Now, using the same segment id for segments in different volume layers should work without any problems. [#6960](https://github.com/scalableminds/webknossos/pull/6960)
+- Fixed incorrect initial tab when clicking "Show Annotations" for a user in the user list. Also, the datasets tab was removed from that page as it was the same as the datasets table from the main dashboard. [#6957](https://github.com/scalableminds/webknossos/pull/6957)
+- Fixed that unsaved changes were shown when opening an annotation, although there weren't any. [#6972](https://github.com/scalableminds/webknossos/pull/6972)
+- Fixed misleading email about successful dataset upload, which was in some cases sent even for unusable datasets. [#6977](https://github.com/scalableminds/webknossos/pull/6977)
+- Fixed upload of skeleton annotations with no trees, only bounding boxes, being incorrectly rejected. [#6985](https://github.com/scalableminds/webknossos/pull/6985)
+- Fixed that Google Cloud Storage URLs with bucket names containing underscores could not be parsed. [#6998](https://github.com/scalableminds/webknossos/pull/6998)
+- Fixed regression that caused public datasets to crash when not being logged in. [#7010](https://github.com/scalableminds/webknossos/pull/7010)
+
 ## [23.04.2](https://github.com/scalableminds/webknossos/releases/tag/23.04.2) - 2023-04-14
 [Commits](https://github.com/scalableminds/webknossos/compare/23.04.1...23.04.2)
 

CHANGELOG.unreleased.md

@@ -8,32 +8,13 @@ and this project adheres to [Calendar Versioning](http://calver.org/) `0Y.0M.MIC
 For upgrade instructions, please check the [migration guide](MIGRATIONS.released.md).
 
 ## Unreleased
-[Commits](https://github.com/scalableminds/webknossos/compare/23.04.2...HEAD)
+[Commits](https://github.com/scalableminds/webknossos/compare/23.05.1...HEAD)
 
 ### Added
-- WEBKNOSSOS screenshots now include a logo in the lower left corner. Users with a paid license can disable this feature in the viewport options in the settings tab on the left side of the screen. [#6910](https://github.com/scalableminds/webknossos/pull/6910)
-- Added support for datasets with mixed magnifications (e.g., one layer contains mag 2-2-2 and another contains 2-2-1). [#6943](https://github.com/scalableminds/webknossos/pull/6943)
-- Added rendering precomputed meshes with level of detail depending on the zoom of the 3D viewport. This feature only works with version 3 mesh files. [#6909](https://github.com/scalableminds/webknossos/pull/6909)
-- Segments can now be removed from the segment list via the context menu. [#6944](https://github.com/scalableminds/webknossos/pull/6944)
-- Editing the meta data of segments (e.g., the name) is now undoable. [#6944](https://github.com/scalableminds/webknossos/pull/6944)
-- Added more icons and small redesigns for various pages in line with the new branding. [#6938](https://github.com/scalableminds/webknossos/pull/6938)
-- Added support for viewing neuroglancer precomputed segmentations using "compressed segmentation" compression. [#6947](https://github.com/scalableminds/webknossos/pull/6947)
-- Added more graphics and branding to job notification emails. [#6994](https://github.com/scalableminds/webknossos/pull/6994)
-- Added action button in Team Admin page that links to the User Admin page to edit team members there. [#6958](https://github.com/scalableminds/webknossos/pull/6999)
 
 ### Changed
-- Moved the view mode selection in the toolbar next to the position field. [#6949](https://github.com/scalableminds/webknossos/pull/6949)
-- Redesigned welcome toast for new, annonymous users with new branding. [#6961](https://github.com/scalableminds/webknossos/pull/6961)
-- When saving annotations, the URL of the webknossos instance is stored in the resulting NML file. [#6964](https://github.com/scalableminds/webknossos/pull/6964)
 
 ### Fixed
-- Fixed unintended dependencies between segments of different volume layers which used the same segment id. Now, using the same segment id for segments in different volume layers should work without any problems. [#6960](https://github.com/scalableminds/webknossos/pull/6960)
-- Fixed incorrect initial tab when clicking "Show Annotations" for a user in the user list. Also, the datasets tab was removed from that page as it was the same as the datasets table from the main dashboard. [#6957](https://github.com/scalableminds/webknossos/pull/6957)
-- Fixed that unsaved changes were shown when opening an annotation, although there weren't any. [#6972](https://github.com/scalableminds/webknossos/pull/6972)
-- Fixed misleading email about successful dataset upload, which was in some cases sent even for unusable datasets. [#6977](https://github.com/scalableminds/webknossos/pull/6977)
-- Fixed upload of skeleton annotations with no trees, only bounding boxes, being incorrectly rejected. [#6985](https://github.com/scalableminds/webknossos/pull/6985)
-- Fixed that Google Cloud Storage URLs with bucket names containing underscores could not be parsed. [#6998](https://github.com/scalableminds/webknossos/pull/6998)
-- Fixed regression that caused public datasets to crash when not being logged in. [#7010](https://github.com/scalableminds/webknossos/pull/7010)
 
 ### Removed
 

MIGRATIONS.released.md

@@ -6,6 +6,19 @@ See `MIGRATIONS.unreleased.md` for the changes which are not yet part of an offi
 This project adheres to [Calendar Versioning](http://calver.org/) `0Y.0M.MICRO`.
 User-facing changes are documented in the [changelog](CHANGELOG.released.md).
 
+## [23.05.1](https://github.com/scalableminds/webknossos/releases/tag/23.05.1) - 2023-05-02
+[Commits](https://github.com/scalableminds/webknossos/compare/23.05.0...23.05.1)
+
+### Postgres Evolutions:
+None.
+
+## [23.05.0](https://github.com/scalableminds/webknossos/releases/tag/23.05.0) - 2023-04-25
+[Commits](https://github.com/scalableminds/webknossos/compare/23.04.2...23.05.0)
+- The config key features.isDemoInstance was renamed to features.isWkorgInstance (only needs to be adapted for the main wkorg instance). [#6941](https://github.com/scalableminds/webknossos/pull/6941/files)
+
+### Postgres Evolutions:
+None.
+
 ## [23.04.2](https://github.com/scalableminds/webknossos/releases/tag/23.04.2) - 2023-04-14
 [Commits](https://github.com/scalableminds/webknossos/compare/23.04.1...23.04.2)
 
@@ -22,7 +35,7 @@ None.
 [Commits](https://github.com/scalableminds/webknossos/compare/23.03.1...23.04.0)
 
 ### Postgres Evolutions:
-None.
+- [101-coordinate-transformations.sql](conf/evolutions/101-coordinate-transformations.sql)
 
 ## [23.03.1](https://github.com/scalableminds/webknossos/releases/tag/23.03.1) - 2023-03-14
 [Commits](https://github.com/scalableminds/webknossos/compare/23.03.0...23.03.1)

MIGRATIONS.unreleased.md

@@ -6,7 +6,6 @@ This project adheres to [Calendar Versioning](http://calver.org/) `0Y.0M.MICRO`.
 User-facing changes are documented in the [changelog](CHANGELOG.released.md).
 
 ## Unreleased
-[Commits](https://github.com/scalableminds/webknossos/compare/23.04.2...HEAD)
-- The config key features.isDemoInstance was renamed to features.isWkorgInstance (only needs to be adapted for the main wkorg instance). [#6941](https://github.com/scalableminds/webknossos/pull/6941/files)
+[Commits](https://github.com/scalableminds/webknossos/compare/23.05.1...HEAD)
 
 ### Postgres Evolutions:

frontend/javascripts/oxalis/model/bucket_data_handling/data_rendering_logic.tsx

@@ -200,6 +200,7 @@ export function calculateTextureSizeAndCountForLayer(
 function buildTextureInformationMap<
   Layer extends {
     elementClass: ElementClass;
+    category: "color" | "segmentation";
   },
 >(
   layers: Array<Layer>,
@@ -238,6 +239,7 @@ function getSmallestCommonBucketCapacity<
 function getRenderSupportedLayerCount<
   Layer extends {
     elementClass: ElementClass;
+    category: "color" | "segmentation";
   },
 >(
   specs: GpuSpecs,
@@ -262,15 +264,23 @@ function getRenderSupportedLayerCount<
     (specs.maxTextureCount - textureCountForSegmentation - lookupTextureCount) /
       maximumTextureCountForLayer,
   );
+
+  // Without any GPU restrictions, WK would be able to render all color layers
+  // plus one segmentation layer. Use that as the upper layer count limit to avoid
+  // compiling too complex shaders.
+  const maximumLayerCount =
+    Array.from(textureInformationPerLayer.keys()).filter((l) => l.category === "color").length +
+    (hasSegmentation ? 1 : 0);
   return {
-    maximumLayerCountToRender,
+    maximumLayerCountToRender: Math.min(maximumLayerCountToRender, maximumLayerCount),
     maximumTextureCountForLayer,
   };
 }
 
 export function computeDataTexturesSetup<
   Layer extends {
     elementClass: ElementClass;
+    category: "color" | "segmentation";
   },
 >(
   specs: GpuSpecs,

frontend/javascripts/oxalis/shaders/texture_access.glsl.ts

@@ -202,7 +202,10 @@ export const getColorForCoords: ShaderModule = {
         offsetInBucket = mod(coords, bucketWidth);
         vec3 offsetInBucketUVW = transDim(offsetInBucket);
         if (offsetInBucketUVW.x < 0.01 || offsetInBucketUVW.y < 0.01
-            || offsetInBucketUVW.x >= 31. || offsetInBucketUVW.y >= 31.) {
+            || offsetInBucketUVW.x >= 31. || offsetInBucketUVW.y >= 31.
+            || isNan(offsetInBucketUVW.x) || isNan(offsetInBucketUVW.y)
+            || isNan(offsetInBucketUVW.z)
+          ) {
           beSafe = true;
         }
       }

frontend/javascripts/test/model/binary/data_rendering_logic.spec.ts

@@ -1,3 +1,4 @@
+import _ from "lodash";
 import {
   calculateTextureSizeAndCountForLayer,
   computeDataTexturesSetup,
@@ -25,6 +26,34 @@ const grayscaleElementClass = "uint8";
 const volumeByteCount = 4;
 const volumeElementClass = "uint32";
 
+/*
+ * The current rendering logic in WK only allows
+ * as many layers as necessary. This is done to avoid
+ * that the shaders are compiled for N layers even though
+ * N layers will never be rendered at the same time (because
+ * only one segmentation layer can be rendered at a time).
+ * For that reason, testing the specs has to be done with a
+ * sufficiently large amount of layers. To achieve this,
+ * the helper function createLayers is used.
+ */
+
+const createGrayscaleLayer = () => ({
+  byteCount: grayscaleByteCount,
+  elementClass: grayscaleElementClass,
+  category: "color",
+});
+const createVolumeLayer = () => ({
+  byteCount: volumeByteCount,
+  elementClass: volumeElementClass,
+  category: "segmentation",
+});
+
+function createLayers(grayscaleCount: number, volumeCount: number) {
+  const grayscaleLayers = _.range(0, grayscaleCount).map(() => createGrayscaleLayer());
+  const volumeLayers = _.range(0, volumeCount).map(() => createVolumeLayer());
+  return grayscaleLayers.concat(volumeLayers);
+}
+
 test("calculateTextureSizeAndCountForLayer: grayscale data + minSpecs", (t) => {
   const { textureSize, textureCount } = calculateTextureSizeAndCountForLayer(
     minSpecs,
@@ -90,24 +119,8 @@ test("calculateTextureSizeAndCountForLayer: color data + betterSpecs", (t) => {
   t.is(textureSize, midSpecs.supportedTextureSize);
   t.is(textureCount, 1);
 });
-const grayscaleLayer1 = {
-  byteCount: grayscaleByteCount,
-  elementClass: grayscaleElementClass,
-};
-const grayscaleLayer2 = {
-  byteCount: grayscaleByteCount,
-  elementClass: grayscaleElementClass,
-};
-const grayscaleLayer3 = {
-  byteCount: grayscaleByteCount,
-  elementClass: grayscaleElementClass,
-};
-const volumeLayer1 = {
-  byteCount: volumeByteCount,
-  elementClass: volumeElementClass,
-};
 
-type Layer = typeof grayscaleLayer1;
+type Layer = ReturnType<typeof createGrayscaleLayer>;
 
 const getByteCount = (layer: Layer) => layer.byteCount;
 
@@ -123,7 +136,7 @@ function computeDataTexturesSetupCurried(spec: typeof minSpecs, hasSegmentation:
   return (layers: Layer[]) =>
     computeDataTexturesSetup(
       spec,
-      layers as { elementClass: ElementClass }[],
+      layers as { elementClass: ElementClass; category: "color" | "segmentation" }[],
       getByteCount as any,
       hasSegmentation,
       DEFAULT_REQUIRED_BUCKET_CAPACITY,
@@ -137,59 +150,37 @@ test("Basic support (no segmentation): all specs", (t) => {
     [midSpecs, 15],
     [betterSpecs, 31],
   ];
+  const hundredGrayscaleLayers = createLayers(100, 0);
   for (const [spec, expectedLayerCount] of specs) {
     const computeDataTexturesSetupPartial = computeDataTexturesSetupCurried(spec, false);
-    testSupportFlags(t, computeDataTexturesSetupPartial([grayscaleLayer1]), expectedLayerCount);
     testSupportFlags(
       t,
-      computeDataTexturesSetupPartial([grayscaleLayer1, grayscaleLayer2]),
+      computeDataTexturesSetupPartial(hundredGrayscaleLayers),
       expectedLayerCount,
     );
     testSupportFlags(
       t,
-      computeDataTexturesSetupPartial([grayscaleLayer1, grayscaleLayer2, grayscaleLayer3]),
+      computeDataTexturesSetupPartial(hundredGrayscaleLayers),
+      expectedLayerCount,
+    );
+    testSupportFlags(
+      t,
+      computeDataTexturesSetupPartial(hundredGrayscaleLayers),
       expectedLayerCount,
     );
   }
 });
 
 test("Basic support + volume: min specs", (t) => {
   const computeDataTexturesSetupPartial = computeDataTexturesSetupCurried(minSpecs, true);
-  testSupportFlags(t, computeDataTexturesSetupPartial([grayscaleLayer1, grayscaleLayer2]), 4);
-  testSupportFlags(t, computeDataTexturesSetupPartial([grayscaleLayer1, volumeLayer1]), 1);
-  testSupportFlags(
-    t,
-    computeDataTexturesSetupPartial([grayscaleLayer1, grayscaleLayer2, volumeLayer1]),
-    1,
-  );
-  testSupportFlags(
-    t,
-    computeDataTexturesSetupPartial([
-      grayscaleLayer1,
-      grayscaleLayer2,
-      grayscaleLayer3,
-      volumeLayer1,
-    ]),
-    1,
-  );
+  testSupportFlags(t, computeDataTexturesSetupPartial(createLayers(10, 0)), 4);
+  testSupportFlags(t, computeDataTexturesSetupPartial(createLayers(10, 1)), 1);
+  testSupportFlags(t, computeDataTexturesSetupPartial(createLayers(10, 1)), 1);
+  testSupportFlags(t, computeDataTexturesSetupPartial(createLayers(10, 1)), 1);
 });
 
 test("Basic support + volume: mid specs", (t) => {
   const computeDataTexturesSetupPartial = computeDataTexturesSetupCurried(midSpecs, true);
-  testSupportFlags(t, computeDataTexturesSetupPartial([grayscaleLayer1, volumeLayer1]), 12);
-  testSupportFlags(
-    t,
-    computeDataTexturesSetupPartial([grayscaleLayer1, grayscaleLayer2, volumeLayer1]),
-    12,
-  );
-  testSupportFlags(
-    t,
-    computeDataTexturesSetupPartial([
-      grayscaleLayer1,
-      grayscaleLayer2,
-      grayscaleLayer3,
-      volumeLayer1,
-    ]),
-    12,
-  );
+  testSupportFlags(t, computeDataTexturesSetupPartial(createLayers(20, 1)), 12);
+  testSupportFlags(t, computeDataTexturesSetupPartial(createLayers(5, 1)), 6);
 });