Replace run_all.py and nox with better alternatives
#6057
Labels
Comments
abey79
added
enhancement
9 tasks
abey79
added a commit
that referenced
this issue
Apr 22, 2024
### What This PR does the following: - Convert all Python examples to proper python packages, replacing the `requirements.txt` file with a `pyproject.toml` file. - Applies same conversion to `tests/python/test_api`. - Add WIP support for running example with pixi - Add a `all_examples` meta-project. - Fix `dev-tools build-examples` for the new example format. - Add a `dev-tools build-examples install` command to `pip install` the examples for the given channel (prerequisite to `build-examples rrd`). - Fix CI for the new example format. - Updates pixi to 0.19.1 in CI. #### pixi support This WIP pixi support allows running examples in the `examples` environment: ``` pixi run -e examples clock ``` Note that by default pixi builds the environment with a shim/empty/fake rerun package (from `examples/python/_empty_rerun_sdk`), which is a work-around to allow manual installation of rerun in the environment. As a result, at first (and whenever a rebuild is needed), this must be run: ``` pixi run -e examples py-build ``` #### `all_examples` This is a meta-package that dynamically depends on all examples. The original hope was to be able to depend on it from pixi, avoiding the need to list example explicitly. However, this is current impossible for a number a reason: - doing so means that examples would not be installed as editable, causing issue with `__file__` relative paths (#6054) - doing so we leak full path in the lock file - pixi currently has an issue where indirect dependency with git dependencies are broken `all_examples` also has a very basic CLI tool that allows listing all examples in a format suitable for pixi: `all_examples list`. Also, it should be relatively easy to cleanly reimplement the functionality of `run_all.py`, which is currently broken: - #6057 As it stands, it allows for efficiently checking dependency conflicts, especially with uv: ```shell cd /tmp uv venv source .venv/bin/activate uv pip install -e path/to/rerun/examples/python/all_examples # ok?? ``` #### TODO - [x] convert a few example to python package - [x] initial version of a `all_examples` meta-package - [x] fix re_build_example - [x] **RE-ENABLE LINK CHECKS!!!!** - ~~properly working pixi setup~~ - ~~python version matrix testing setup~~ ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested the web demo (if applicable): * Using examples from latest `main` build: [rerun.io/viewer](https://rerun.io/viewer/pr/5966?manifest_url=https://app.rerun.io/version/main/examples_manifest.json) * Using full set of examples from `nightly` build: [rerun.io/viewer](https://rerun.io/viewer/pr/5966?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json) * [x] The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG * [x] If applicable, add a new check to the [release checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)! - [PR Build Summary](https://build.rerun.io/pr/5966) - [Recent benchmark results](https://build.rerun.io/graphs/crates.html) - [Wasm size tracking](https://build.rerun.io/graphs/sizes.html) To run all checks from `main`, comment on the PR with `@rerun-bot full-check`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
#5966 updated the way our examples are packaged, basically breaking
run_all.pyand the nox setup. This should be fixed and reimplemented in a better way.For
run_all, the functionality would be much better implemented inall_examples. Most of the listing logic is already there.For nox (aka run all vs. all version of python), it's probably possible to use pixi for this.
The text was updated successfully, but these errors were encountered: