Add LOD mesh support for frontend

[MichaelBuessemeyer]

URL of deployed dev instance (used for testing):

• https://___.webknossos.xyz

Steps to test:

• test local -> add the mesh file with the new format and old format into the l4_sample datasets' segmentation folder l4_sample/segmentation/meshes/.... The meshfiles are available in the following slack thread: https://scm.slack.com/archives/C5AKLAV0B/p1674140733622169?thread_ts=1674138004.502719&cid=C5AKLAV0B
• View the l4_sample dataset
• load a mesh with the new meshfile format (precomputed)
• load a mesh with the old meshfile format (precomputed)
• try reloading the meshes
• hide the meshes
• download the meshes
• zoom in and out. all meshes should be always visible.
• delete the meshes
• and more mesh interactions that you can think of :)

Issues:

• fixes #6671
• fixes Meshfile loading code should use Long for byte offsets #6907

---

(Please delete unneeded items, merge only when none are left open)

• Updated changelog
• Removed dev-only changes like prints and application.conf edits
• Considered common edge cases

[philippotto]

Great stuff 👍 Only have some smaller remarks mostly. One cool thing would be progressive rendering per segment (or per segment chunk), but I'm not sure whether the current scene graph hierarchy is suitable for that. Let me know what you think!

[MichaelBuessemeyer]

I think I covered all open comments, right? (besides the testing of the constants that determine the current LOD).

But a review of the backend part is still missing, right?

[fm3]

But a review of the backend part is still missing, right?

The backend changes look more complex than they are. I only renamed a few things (with the help of the IDE), added the one factor to fix the mag scaling (which we tested) and changed the signatures of some things from Int to Long (which we tested). Since backend reviewers are a scarce resource currently, I’d say it’s fine to just ship it.

[philippotto]

Awesome! Thanks for implementing the custom raycaster. This should be a good boost for performance :)

Some comments:

• for mesh files that don't have multiple lods, the meshes are disappearing when zooming out. in general, the lod code should take into account how many LODs exist (currently, it always assumes 3, but it could be 1 or 5 or 10 ...)
• I'd rename the raycaster module and class to something like VisibilityAwareRaycaster
• Regarding the generalization of the lod selection, I'll come back to you. However, what I'd do in either way is to adapt the logic so that there is an array of zoom thresholds which is iterated then. Something like this:

const LOD_THRESHOLDS = [0.7, 3];

...
let currentIndex = 0;
while (scale > LOD_THRESHOLDS[currentIndex] && currentIndex < availableLodCount) {
  currentIndex++;
  if (currentIndex === LOD_THRESHOLDS.length) {
    // generate next thresholds as necessary
    LOD_THRESHOLDS.push(2 * LOD_THRESHOLDS[currentIndex - 1])
  }
}
return currentIndex;

(not tested, but I hope the gist is comprehensible)

[MichaelBuessemeyer]

for mesh files that don't have multiple lods, the meshes are disappearing when zooming out. in general, the lod code should take into account how many LODs exist (currently, it always assumes 3, but it could be 1 or 5 or 10 ...)

But mesh files that do not have version 3 still work and do not disappear, right? Were you able to trigger some kind of bug like this? I assume you mean mesh version 3 files that only have one or two LODs, right?

Regarding the generalization of the lod selection, I'll come back to you. However, what I'd do in either way is to adapt the logic so that there is an array of zoom thresholds which is iterated then. Something like this:

Thanks for coming up with this heuristic suggestion 🎉

I changed your suggested code a little. Now the custom LOD class maintains a list of thresholds and automatically adds new thresholds if a mesh with a LOD is added that is not supported by the current threshold list.

I also tested the mechanism of adding thresholds depending on the LOD count of the meshes. I simply changed this.lodThresholds = [0.7, 3]; to this.lodThresholds = [0.7]; and it worked without any issues. The only side effect was that the switching between different LODs was done on smaller levels of scale and thus visible to the eye. But as this was just a test and because we actually use 3 for the 2nd threshold, I think is fine.

One bug I can imagine is that if a mesh is loaded that supports 3 LODs and then a mesh with 5 LODs is loaded, the first mesh will be invisible on very small scales / far zoomed out. But this is a pretty hypothetical scenario right now in my opinion. So I wouldn't tackle this right now.
But this should be easy to solve by remembering the max LOD for each mesh and ensuring that a meshes max LOD is rendered at least :) (Although this could dip the performance quite a bit with a lot of meshes)

[philippotto]

for mesh files that don't have multiple lods, the meshes are disappearing when zooming out. in general, the lod code should take into account how many LODs exist (currently, it always assumes 3, but it could be 1 or 5 or 10 ...)

But mesh files that do not have version 3 still work and do not disappear, right? Were you able to trigger some kind of bug like this? I assume you mean mesh version 3 files that only have one or two LODs, right?

I could reproduce the error here with meshfile_v3 (which doesn't have lods apparently). But now it works due to your code change :)

Regarding the generalization of the lod selection, I'll come back to you. However, what I'd do in either way is to adapt the logic so that there is an array of zoom thresholds which is iterated then. Something like this:

Thanks for coming up with this heuristic suggestion 🎉

I changed your suggested code a little. Now the custom LOD class maintains a list of thresholds and automatically adds new thresholds if a mesh with a LOD is added that is not supported by the current threshold list.

Awesome!

I also tested the mechanism of adding thresholds depending on the LOD count of the meshes. I simply changed this.lodThresholds = [0.7, 3]; to this.lodThresholds = [0.7]; and it worked without any issues. The only side effect was that the switching between different LODs was done on smaller levels of scale and thus visible to the eye. But as this was just a test and because we actually use 3 for the 2nd threshold, I think is fine.

Yes, sounds okay to me 👍

One bug I can imagine is that if a mesh is loaded that supports 3 LODs and then a mesh with 5 LODs is loaded, the first mesh will be invisible on very small scales / far zoomed out. But this is a pretty hypothetical scenario right now in my opinion. So I wouldn't tackle this right now. But this should be easy to solve by remembering the max LOD for each mesh and ensuring that a meshes max LOD is rendered at least :) (Although this could dip the performance quite a bit with a lot of meshes)

I think it's a fair assumption for now that all meshes share the same LOD count. Maybe we need to rework the LOD hierarchy in the future, anyway (e.g., to allow that one mesh is rendered in LOD 0 and another one in LOD 3), then this might be solved in the same breath, but let's take it step by step.

[philippotto]

Awesome! Let's ship it, I'd say :)