Always aim to compute the full (ad-hoc) mesh

[philippotto]

This PR removes the hard mesh chunk limit and instead introduces a soft throttle mechanism to avoid putting full load for the mesh generation for a single segment (e.g., a huge segment could take forever to load and should not cause a tremendous load on client + backend).

URL of deployed dev instance (used for testing):

• https://fullmesh.webknossos.xyz/

Steps to test:

• load some ad-hoc meshes for big segments
• I tested it with this segment: https://fullmesh.webknossos.xyz/datasets/sample_organization/l4_sample/view#3584,3789,1369,0,1.3
• I also set a breakpoint at the the sleep statement to ensure that it was triggered

Issues:

• fixes Improve isosurface loading for rendering a single cell #4835

---

(Please delete unneeded items, merge only when none are left open)

• Updated (unreleased) changelog
• Updated (unreleased) migration guide if applicable
• Updated documentation if applicable
• Adapted wk-connect if datastore API changes
• Adapted wk-libs python client if relevant API parts change
• Needs datastore update after deployment
• Ready for review

[daniel-wer]

LGTM. Let's see whether we can tune these values in the future to speed up the ad-hoc meshing. The current config is rather conservative, but I can see why.

Maybe it could work better to let the server decide when to throttle and by how much. For example, the server could have an ad-hoc meshing worker queue and depending on the number of users using the feature in parallel, the requests would be faster or slower. This way, users could enjoy maximum speed if the server load is low.

[philippotto]

Maybe it could work better to let the server decide when to throttle and by how much. For example, the server could have an ad-hoc meshing worker queue and depending on the number of users using the feature in parallel, the requests would be faster or slower. This way, users could enjoy maximum speed if the server load is low.

Yes, this is probably a good idea. The back-end might even just block a request for the time until the pool has a free slot. That way, the front-end wouldn't even need to accept another return value. Let's keep this in mind for the future :)
I'm curious to see how this PR will change things.