Transactional saving of volume annotations

[philippotto]

Description:

• Introduces sensible transaction points for volume annotations (--> also means less versions which will be helpful in the version restore view 🎉). This is done by waiting for small time windows (1 second) in which no volume data was mutated and no dirty volume buckets need to be downloaded. This is similar to how the PushQueue operated, anyway, but it chunked buckets into 32 batches later which destroyed any transactional semantics. Reusing the state machine of the undo mechanism would be nicer, since it is strictly user-interaction based. However, the implementation of that would be far more complicated.
• Improved speed of bucket compression by using larger payloads for the webworkers (an isolated benchmark with reasonable inputs showed that the bucket compression is now 5 to 10 times faster).
• Removes AsyncTaskQueue as its sequentiality and failure-mechanism aren't needed anymore. This abstraction was originally introduced to deal with failing network requests. However, that part is not handled by the AsyncTaskQueue anymore.
• The error handling in the PushQueue was hardened and unified with the error handling in the sagas (by escalating potential exceptions to the sagas).
• Fixes a small bug in the save-saga which could lead to an accumulating backlog of save queue items that would grow faster than they would be saved. This could happen when brushing a high volume of data and only relying on the auto-save mechanism. That mechanism did only send 1 request every 30 seconds (even when multiple requests would be necessary to completely save the state of that time).
• The save button now also warns when the PushQueue is starving (i.e., constantly unable to bundle up a transaction, because a bucket was edited again). Previously, this was achieved by providing a max_time parameter to _.debounce. However, since we want sensible transactions, this is not an option anymore.
• Move some modules from libs to libs/async.
• Lower saving batch constants for volume tracings, because the backend needs more time to process the update actions (see discussion here)
• Improve progress indicator during saving (the tooltip for the save button will also show when buckets need to be downloaded during saving, because this can take a noticable amount of time).

URL of deployed dev instance (used for testing):

• https://___.webknossos.xyz

Steps to test:

• do a volume annotation (with and without fallback)
• set the network speed to very slow
• reload the tab during save (large volume changes are necessary to cause multiple requests)
• ensure that the volume data looks consistent (zoom in and out to check all mags)
• additionally/alternatively:
  • brush a large area
  • save
  • look at the version restore view. there should be one version (maybe other versions for the new segment id and for updateTracing, but not more)
  • compare the above to master.webknossos.xyz where lots of versions will be created. previewing one version and zooming to different mags will show hard-cuts in the brushed data.

TODOs:

• test volume speed on dev instance (which will use prod mode) to ensure that it didn't regress
• verify that new save requests don't get too large. since updateBucket actions are way larger than skeleton update actions, some tuning might be necessary here. <-- no problem, since the compression does a good job of keeping the size small

Issues:

• hopefully fixes #7256
• contributes to Versions of volume data should be created at "transaction" points #6495
• to close the issue fully, the version history view should show the transactions (the back-end needs to store these first, though)

---

(Please delete unneeded items, merge only when none are left open)

• Updated changelog
• Updated migration guide if applicable
• Updated documentation if applicable
• Adapted wk-libs python client if relevant API parts change
• Removed dev-only changes like prints and application.conf edits
• Considered common edge cases
• Needs datastore update after deployment

[philippotto]

todo: revert

[philippotto]

@normanrz Are you open for reviewing this PR? I'll do some additional testing on a dev instance, but the code is ready for review. It might make sense to have a call about this PR and do a review with 4 eyes, since the code is mission-critical for volume saving. Let me know if/when you have time for that 🤙

[normanrz]

This code is a piece of art! Thanks for writing all the descriptive comments. I think I understand what is roughly going on and look forward to the walkthrough.

[normanrz]

Why is the queue a Set?

[philippotto]

The order of the buckets doesn't matter within a transaction, which is why this is a set. However, the mismatch is obviously still an eye sore. I renamed it to pendingBuckets now. The class PushQueue might also be ripe for a new name, but then the PullQueue loses its "name-sibling" 😅 Also, I'm not sure how to call it. BucketTransactionBundler? Might be okay to keep the PushQueue as is.