Threaded inference

[talmo]

Description

Basic threaded inference implementation. With this change, SLEAP object construction (and tracking) happen in a separate thread than neural network inference. This should unblock the GPU while we're building PredictedInstances.

Quick tests show that this super simple implementation increases overall inference speed by ~20% in cases where there are many instances/tracks.

It's not pretty, but I think we'll need to wait to de-duplicate the logic until we refactor the inference module into submodules or something.

Types of changes

• Bugfix
• New feature
• Refactor / Code style update (no logical changes)
• Build / CI changes
• Documentation Update
• Other (explain)

Does this address any currently open issues?

[list open issues here]

Outside contributors checklist

• Review the guidelines for contributing to this repository
• Read and sign the CLA and add yourself to the authors list
• Make sure you are making a pull request against the develop branch (not main). Also you should start your branch off develop
• Add tests that prove your fix is effective or that your feature works
• Add necessary documentation (if appropriate)

Thank you for contributing to SLEAP!

❤️

[codecov]

Codecov Report

Merging #1203 (522222a) into develop (788179d) will decrease coverage by 0.10%.
The diff coverage is 97.12%.

@@             Coverage Diff             @@
##           develop    #1203      +/-   ##
===========================================
- Coverage    72.00%   71.91%   -0.10%     
===========================================
  Files          132      132              
  Lines        23328    23403      +75     
===========================================
+ Hits         16798    16830      +32     
- Misses        6530     6573      +43     

Impacted Files	Coverage Δ
sleap/nn/inference.py	81.54% <97.12%> (+0.94%)	⬆️
sleap/nn/evals.py	55.65% <0.00%> (-18.70%)	⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more