Changed spawn() and the rerun script to call into rerun_bindings (12x startup time improvement)

[abey79]

What

Python package can include so-called "scripts" definitiosn, which are recognised by pip, pipx, etc. When they exist, a stub script is created upon package installation which calls into the defined python function. We use this to map rerun to launch the viewer from the binary embedded in our wheels.

So far, this was done as follows (from pyproject.toml):

[project.scripts]
rerun = "rerun.__main__:main"

The drawback of this is that the entire rerun SDK package was loaded (including numpy, Pillow, etc.), incurring a ~300+ms startup time.

This PR changes it to:

[project.scripts]
rerun = "rerun_bindings:main"

This drastically reduces the loading time, as the rerun python package is no longer loaded. The signature of our binding's main function needed to be changed: it now internally reads from sys.argv instead of accepting an argument.

This PR also changes spawn() to call directly into the bindings, with the same loading time benefit.

Note: This PR improves startup time when running rerun in a venv (with the SDK installed), or when installed via pipx. The startup time of python -m rerun is not improved, as this still involves loading the rerun python package.

Benchmark

❯ cat `which rerun`
   1   │ #!/Users/hhip/src/rerun/venv/bin/python
   2   │ # -*- coding: utf-8 -*-
   3   │ import re
   4   │ import sys
   5   │ from rerun_bindings import main
   6   │ if __name__ == '__main__':
   7   │     sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
   8   │     sys.exit(main())

❯ cat `which rerun_0.9.1`
   1   │ #!/Users/hhip/.local/pipx/venvs/rerun-sdk-0-9-1/bin/python
   2   │ # -*- coding: utf-8 -*-
   3   │ import re
   4   │ import sys
   5   │ from rerun.__main__ import main
   6   │ if __name__ == '__main__':
   7   │     sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
   8   │     sys.exit(main())

❯ hyperfine --warmup 5  "rerun --version" "rerun_0.9.1 --version"
Benchmark 1: rerun --version
  Time (mean ± σ):      34.8 ms ±   1.7 ms    [User: 21.8 ms, System: 11.1 ms]
  Range (min … max):    32.5 ms …  41.5 ms    82 runs
 
Benchmark 2: rerun_0.9.1 --version
  Time (mean ± σ):     422.1 ms ±  52.2 ms    [User: 716.1 ms, System: 1971.6 ms]
  Range (min … max):   333.0 ms … 494.0 ms    10 runs
 
Summary
  rerun --version ran
   12.14 ± 1.62 times faster than rerun_0.9.1 --version

Checklist

• I have read and agree to Contributor Guide and the Code of Conduct
• I've included a screenshot or gif (if applicable)
• I have tested demo.rerun.io (if applicable)
• The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG

• PR Build Summary
• Docs preview
• Examples preview
• Recent benchmark results
• Wasm size tracking

[emilk]

Awesome! Have you tested that spawn still works?

[teh-cmc]

Awesome! Have you tested that spawn still works?

Speaking of spawn, why does spawn still use python -m rerun rather than rerun? It won't benefit from the speed up if I understand correctly?

[abey79]

Awesome! Have you tested that spawn still works?

Yes.

Speaking of spawn, why does spawn still use python -m rerun rather than rerun? It won't benefit from the speed up if I understand correctly?

Excellent point. Let me take a look at that.

[abey79]

@teh-cmc Easy enough!