Codegen/IDL 3: introduce `re_types` #2369

teh-cmc · 2023-06-11T19:31:32Z

This implements the re_types crate itself, without any Rerun-specific definitions yet.

It also includes a few standard IDL definitions that aren't specific to Rerun (e.g. Arrow/Python/Rust attributes).

⚠️ This requires both black and ruff to be in $PATH, but only for contributors,not end users.
Even for contributors, black and ruff won't be needed unless they edit some of the .fbs files... and even then, this won't crash if they are missing, it will just fail to pass the CI!

Codegen/IDL PR series:

Excerpt from crate-level docs

The standard Rerun data types, component types, and archetypes.

This crate contains both the IDL definitions for Rerun types (flatbuffers) as well as the code
generated from those using re_types_builder.

Organization

definitions/ contains IDL definitions for all Rerun types (data, components, archetypes).
src/ contains the code generated for Rust.
rerun_py/rerun2/ (at the root of this workspace) contains the code generated for Python.

While most of the code in this crate is auto-generated, some manual extensions are littered
throughout: look for files ending in _ext.rs or _ext.py (also see the "Extensions" section
of this document).

Build cache

Updating either the source code of the code generator itself (re_types_builder) or any of the
.fbs files should re-trigger the code generation process the next time re_types is built.
Manual extension files will be left untouched.

Caching is controlled by a versioning hash that is stored in store_hash.txt.
If you suspect something is wrong with the caching mechanism and that your changes aren't taken
into account when they should, try and remove source_hash.txt.
If that fixes the issue, you've found a bug.

How-to: add a new datatype/component/archetype

Create the appropriate .fbs file in the appropriate place, and make sure it gets included in
some way (most likely indirectly) by archetypes.fbs, which is the main entrypoint for
codegen.
Generally, the easiest thing to do is to add your new type to one of the centralized manifests,
e.g. for a new component, include it into components.fbs.

Your file should get picked up automatically by the code generator.
Once the code for your new component has been generated, implement whatever extensions you need
and make sure to tests any custom constructors you add.

How-to: remove an existing datatype/component/archetype

Simply get rid of the type in question and rebuild re_types to trigger codegen.

Beware though: if you remove a whole definition file re-running codegen will not remove the
associated generated files, you'll have to do that yourself.

Extensions

Rust

Generated Rust code can be manually extended by adding sibling files with the _ext.rs
prefix. E.g. to extend vec2d.rs, create a vec2d_ext.rs.

Trigger the codegen (e.g. by removing source_hash.txt) to generate the right mod clauses
automatically.

The simplest way to get started is to look at any of the existing examples.

Python

Generated Python code can be manually extended by adding a sibling file with the _ext.py
prefix. E.g. to extend vec2d.py, create a vec2d_ext.py.

This sibling file needs to implement an extension class that is mixed in with the
auto-generated class.
The simplest way to get started is to look at any of the existing examples.

crates/re_types/build.rs

crates/re_types/definitions/arrow/attributes.fbs

crates/re_types/definitions/fbs/attributes.fbs

crates/re_types/definitions/python/attributes.fbs

Wumpf

👌

crates/re_types/Cargo.toml

Wumpf · 2023-06-13T14:39:38Z

crates/re_types/build.rs

+    // In the very unlikely chance that the user doesn't have the `fmt` component installed,
+    // there's still no good reason to fail the build.
+    //
+    // The CI will catch the unformatted file at PR time and complain appropriately anyhow.


not sure I agree on these here. This means everything works out locally fine and then you fail CI and it will look like the codegenerator messed up.
Also it opens up to the CI job not having e.g. ruff installed

This means everything works out locally fine and then you fail CI and it will look like the codegenerator messed up.

Which is not really different from our current workflow? You don't run your formatter, CI punishes you.

Sure it's generated code causing the issue but:

This only concerns core contributors specifically working on IDL defintions; surely they understand what's going on when there are this deep into the stack.

The generated appears has just normal unstaged code in your git status, so you should definitely have a look at it before committing it

Also it opens up to the CI job not having e.g. ruff installed

The CI never does codegen!

crates/re_types/definitions/arrow/attributes.fbs

crates/re_types/definitions/fbs/attributes.fbs

crates/re_types/definitions/fbs/scalars.fbs

Wumpf · 2023-06-13T14:49:24Z

oh actually there might be a proper issue: Rust analyzer triggered a cranky build that changed source_hash. But probably the one here is just not the correct one beacuse of your splitting effort?

teh-cmc · 2023-06-13T15:42:14Z

oh actually there might be a proper issue: Rust analyzer triggered a cranky build that changed source_hash. But probably the one here is just not the correct one beacuse of your splitting effort?

Yes! hashes are not to be trusted until PR 6 😄

teh-cmc · 2023-06-14T09:05:01Z

-> 9385a7a

Adds a bunch of build tools to help with dependency hashing when managing implicit build graphs via `build.rs`. Prerequisites for `re_types_builder` & `re_types`. --- Codegen/IDL PR series: - #2362 - #2363 - #2369 - #2370 - #2374 - #2375 - #2410 - #2432

`re_types_builder` provides the build-time tools to parse and inspect flatbuffers schemas, and to generate Python & Rust code from those definitions. You can safely skip `crates/re_types_builder/definitions/reflection.fbs` & `crates/re_types_builder/src/reflection.rs` which are just the auto-generated flatbuffers code to provide the runtime reflection tools (minus 3.2k LoCs, yay!). This is by far the biggest PR of the bunch. It's also the least important one in some ways, considering none of this code sees the light of runtime. The easiest way to make sense of `codegen/python.rs` and `codegen/rust.rs` is to check out the output code: - #2374 - #2375 ⚠️ This now requires `flatc` to be in $PATH, but only for contributors, not end users. Even for contributors, `flatc` won't be needed unless they edit some of the .fbs files. --- Codegen/IDL PR series: - #2362 - #2363 - #2369 - #2370 - #2374 - #2375 - #2410 - #2432 --- _Excerpt from the crate-level docs_ ### Organization The code generation process happens in 4 phases. #### 1. Generate binary reflection data from flatbuffers definitions. All this does is invoke the flatbuffers compiler (`flatc`) with the right flags in order to generate the binary dumps. Look for `compile_binary_schemas` in the code. #### 2. Run the semantic pass. The semantic pass transforms the low-level raw reflection data generated by the first phase into higher level objects that are much easier to inspect/manipulate and overall friendler to work with. Look for `objects.rs`. #### 3. Fill the Arrow registry. The Arrow registry keeps track of all type definitions and maps them to Arrow datatypes. Look for `arrow_registry.rs`. #### 4. Run the actual codegen pass for a given language. We currently have two different codegen passes implemented at the moment: Python & Rust. Codegen passes use the semantic objects from phase two and the registry from phase three in order to generate user-facing code for Rerun's SDKs. These passes are intentionally implemented using a very low-tech no-frills approach (stitch strings together, make liberal use of `unimplemented`, etc) that keep them flexible in the face of ever changing needs in the generated code. Look for `codegen/python.rs` and `codegen/rust.rs`. ### Error handling Keep in mind: this is all _build-time_ code that will never see the light of runtime. There is therefore no need for fancy error handling in this crate: all errors are fatal to the build anyway. Make sure to crash as soon as possible when something goes wrong and to attach all the appropriate/available context using `anyhow`'s `with_context` (e.g. always include the fully-qualified name of the faulty type/field) and you're good to go. ### Testing Same comment as with error handling: this code becomes irrelevant at runtime, and so testing it brings very little value. Make sure to test the behavior of its output though: `re_types`! --------- Co-authored-by: Andreas Reich <[email protected]>

Nothing but definitions! And I say "a" `Points2D` rather than "the" `Points2D` because it's not the point of this PR to nail down the perfect `Points2D` archetype (yet); but rather get close enough and stress-test the whole system in the process. --- Codegen/IDL PR series: - #2362 - #2363 - #2369 - #2370 - #2374 - #2375 - #2410 - #2432 --------- Co-authored-by: Andreas Reich <[email protected]>

The auto-generated Python code, and nothing else. Hand written tests and extensions (including the actual body of the mixins!) will make an appearance in follow-up PRs. --- Codegen/IDL PR series: - #2362 - #2363 - #2369 - #2370 - #2374 - #2375 - #2410 - #2432 --------- Co-authored-by: Andreas Reich <[email protected]>

The auto-generated Rust code, and nothing else. Hand written tests and extensions will make an appearance in follow-up PRs. --- Codegen/IDL PR series: - #2362 - #2363 - #2369 - #2370 - #2374 - #2375 - #2410 - #2432 --------- Co-authored-by: Andreas Reich <[email protected]>

…#2410) Implements all the necessary Python extensions so that the UX of the `Points2D` archetype is --dare I say-- _exquisite_. Also adds a whole bunch of tests for all of these extensions because, well.. Python... Might be easier to review commit by commit 🤷 --- Codegen/IDL PR series: - #2362 - #2363 - #2369 - #2370 - #2374 - #2375 - #2410 - #2432 --------- Co-authored-by: Andreas Reich <[email protected]>

…2432) - First commit bring back all existing extensions (and maybe add one or two extremely trivial ones) - Second commit adds a regression test, which is much simpler than Python since we have the type system to rely for 95% of the stuff. Closes #2241 --- Codegen/IDL PR series: - #2362 - #2363 - #2369 - #2370 - #2374 - #2375 - #2410 - #2432 --------- Co-authored-by: Andreas Reich <[email protected]>

@kpreid

…nts - 2023-07-27 (#2842) ## [0.8.0](v0.7.0...v0.8.0) - Infrastructure investments and more transform improvements - 2023-07-27 [Rerun](https://www.rerun.io/) is an easy-to-use visualization toolbox for computer vision and robotics. * Python: `pip install rerun-sdk` * Rust: `cargo add rerun` and `cargo install rerun-cli` * Online demo: <https://demo.rerun.io/version/0.8.0/> ### Overview & Highlights - `log_pinhole` is now easier to use in simple cases and supports non-RDF camera coordinates. [#2614](#2614) - You only need to set focal length and optional principal point instead of setting the full 3x3 matrix. - There is also a new argument: `camera_xyz` for setting the coordinate system. The default is RDF (the old default). This affects the visible camera frustum, how rays are projected when hovering a 2D image, and how depth clouds are projected. - The visualizer can now show coordinate arrows for all affine transforms within the view. [#2577](#2577) - Linestrips and oriented bounding boxes can now be logged via batch APIs in python. - See: `log_linestrips_2d`, `log_linestrips_3d`, [#2822](#2822) and `log_obbs` [#2823](#2823) - Rust users that build their own Viewer applications can now add fully custom Space Views. Find more information [here](https://www.rerun.io/docs/howto/extend-ui#custom-space-views-classes). - New optional `flush_timeout` specifies how long Rerun will wait if a TCP stream is disconnected during a flush. [#2821](#2821) - In Rust, `RecordingStream::connect` now requires `flush_timeout` specified as an `Option<Duration>`. - To keep default behavior, this can be specified using the `rerun::default_flush_time()` helper. - In Python `flush_init_sec` is now an optional argument to `rr.connect()` - In Rust, the `RecordingStream` now offers a stateful time API, similar to the Python APIs. [#2506](#2506) - You can now call `set_time_sequence`, `set_time_seconds`, and `set_time_nanos` directly on the `RecordingStream`, which will set the time for all subsequent logs using that stream. - This can be used as an alternative to the the previous `MsgSender::with_time` APIs. - The Rerun SDK now defaults to 8ms long microbatches instead of 50ms. This makes the default behavior more suitable for use-cases like real-time video feeds. [#2220](#2220) - Check out [the microbatching docs](https://www.rerun.io/docs/reference/sdk-micro-batching) for more information on fine-tuning the micro-batching behavior. - The web viewer now incremental loads `.rrd` files when streaming over HTTP. [#2412](#2412) ![Open Photogrammetry Preview](https://static.rerun.io/9fa26e73a197690e0403cd35f29e31c2941dea36_release_080_photogrammetry_full.png) ### Ongoing Refactors - There have been a number of significant internal changes going on during this release with little visible impact. This work will land across future releases, but is highlighted here since much of it is visible through the changelog. - The layout of the Viewer is now controlled by a Blueprint datastore. In the future this will allow for direct API control of the layout and configuration of the Viewer. A very early prototype of this functionality is available via the `rerun.experimental` module in Python. - An entirely new code-generation framework has been brought online for Rust, Python and C++. This will eventually enable new object-centric APIs with a more scalable, consistent, and ergonomic experience. - Bringup of C++ support is now underway and will eventually become our third officially supported SDK language. ### Known Regressions - Due to the Blueprint storage migration, blueprint persistence on web is currently broken. Will be resolved in: [#2579](#2579) ### In Detail #### 🐍 Python SDK - Clean up warnings printed when `rr.init` hasn't been called [#2209](#2209) - Normalise Python typing syntax to 3.8+ [#2361](#2361) - Simpler, sturdier stateful time tracking in both SDKs [#2506](#2506) - Fix not taking np.array for single colors [#2569](#2569) - Add a basic pyright config [#2610](#2610) - Improve `log_pinhole` and support non-RDF pinholes [#2614](#2614) - Expose batch APIs for linestrips [#2822](#2822) - Expose batch APIs for oriented bounding boxes [#2823](#2823) #### 🦀 Rust SDK - Add example for adding custom Space Views [#2328](#2328) - Simpler, sturdier stateful time tracking in both SDKs [#2506](#2506) - Automagic flush when `take()`ing a `MemorySinkStorage` [#2632](#2632) - Logging SDK: Log warnings if user data is dropped [#2630](#2630) - Add support for `RecordingStream::serve` [#2815](#2815) #### 🌁 Viewer Improvements - Better handle scroll-to-zoom in 3D views [#1764](#1764) - Add command to screenshot the application [#2293](#2293) - Show layout in blueprint tree view [#2465](#2465) - Double-click to select entity [#2504](#2504) - Add Rerun.io link/text in top bar [#2540](#2540) - New auto-layout of space views [#2558](#2558) - Add 'Dump datastore' command to palette [#2564](#2564) - Support any `dtype` for depth images [#2602](#2602) - Change "Save Selection" command to Cmd+Alt+S [#2631](#2631) - Consistent transform visualization for all entities with transforms [#2577](#2577) - Improve `log_pinhole` and support non-RDF pinholes [#2614](#2614) #### 🚀 Performance Improvements - Flush the batches every 8ms instead of 50 ms [#2220](#2220) - Replace `image` crate jpeg decoder with zune-jpeg [#2376](#2376) - Stream `.rrd` files when loading via http [#2412](#2412) #### 🪳 Bug Fixes - Fix deadlock when misusing the Caches [#2318](#2318) - Fix unstable order/flickering of "shown in" space view list on selection [#2327](#2327) - Fix transforms not applied to connections from transform context [#2407](#2407) - Fix texture clamping and color gradient selection being displayed incorrectly [#2394](#2394) - Fix projected ray length [#2482](#2482) - Tweak the depth bias multiplier for WebGL [#2491](#2491) - Clip image zoom rectangle [#2505](#2505) - Fix missing feature flags for benchmarks [#2515](#2515) - `run_all.py` script fixes [#2519](#2519) - Update egui_tiles with fix for drag-and-drop-panic [#2555](#2555) - Convert objectron proto.py back to using typing.List [#2559](#2559) - Exclude from `objectron/proto/objectron/proto.py` from `just py-format` [#2562](#2562) - Fix pinhole visualization not working with camera extrinsics & intrinsics on the same path [#2568](#2568) - Fix: always auto-layout spaceviews until the user interveens [#2583](#2583) - Fix freeze/crash when logging large times [#2588](#2588) - Update egui_tiles to fix crash [#2598](#2598) - Fix clicking object with single instance (of every component) selecting instance instead of entity [#2573](#2573) - Cleanup internal data-structures when process has been forked [#2676](#2676) - Fix shutdown race-condition by introducing a flush_timeout before dropping data [#2821](#2821) - Fix ui-scale based point/line sizes incorrectly scaled when zooming based on horizontal dimension [#2805](#2805) - Fix visibility toggle for maximized Space Views [#2806](#2806) - Fix loading file via CLI [#2807](#2807) - Fix disconnected space APIs in Python SDK [#2832](#2832) - Avoid unwrap when generating authkey [#2804](#2804) #### 🧑‍🏫 Examples - Add example template [#2392](#2392) - Show hidden url search param in `app.rerun.io` [#2455](#2455) - Minimal example of running an intel realsense depth sensor live [#2541](#2541) - Add a simple example to display Open Photogrammetry Format datasets [#2512](#2512) - Move `examples/api_demo` -> `tests/test_api` [#2585](#2585) #### 📚 Docs - Docs: link to `rr.save` and suggest `rerun` instead of `python -m rerun` [#2586](#2586) - Update docs about transforms [#2496](#2496) - Fixup remaining usages of log_rigid3 in docs [#2831](#2831) #### 🎨 Renderer Improvements - Expose type erased draw data that can be consumed directly [#2300](#2300) - Use less `mut` when using `RenderContext` [#2312](#2312) #### 🧑‍💻 Dev-experience - Better error messages in build.rs [#2173](#2173) - Recommend sccache in CONTRIBUTING.md [#2245](#2245) - introduce `re_tracing` [#2283](#2283) - lint: standardize formatting of let-else-return statements [#2297](#2297) - Centralized build tools in `re_build_tools` [#2331](#2331) - Lint for explicit quotes [#2332](#2332) - Added example screenshot instructions in `just upload --help` [#2454](#2454) - Added support for puling image from an URL to `upload_image.py` [#2462](#2462) - `setup_dev.sh` now installs pngcrush [#2470](#2470) - Added docs/code-examples to the directories checked by py-lint and py-format [#2476](#2476) - Link to demo in PR + check checkboxes [#2543](#2543) - Add script to find external issues we haven't commented on [#2532](#2532) - Move CI-related scripts to its own folder [#2561](#2561) - Render PR description as template [#2563](#2563) - Add basic testing automation against all version of Python using nox [#2536](#2536) - Run clippy on public API too [#2596](#2596) - Bump all `py-lint`-related package versions [#2600](#2600) - Crates publishing script [#2604](#2604) - Fix rust docs deploy [#2615](#2615) - Add support for .gitignore to scripts/lint.py [#2666](#2666) #### 🗣 Refactors - Refactor space-view dependencies: - Move spatial space view to its own crate [#2286](#2286) - Separate crate for bar chart space view [#2322](#2322) - Separate crate for time series space view [#2324](#2324) - Separate crate for tensor space view [#2334](#2334) - Separate viewport related files out to a new re_viewport crate [#2251](#2251) - Remove timepanel dependency from viewport [#2256](#2256) - New trait system for SpaceViews: - Initial Space View trait & port of text space views to the new Space View trait system [#2281](#2281) - Extend/iterate on SpaceViewClass framework with SceneContext & port SpatialSpaceView scene parts [#2304](#2304) - Finalize move of SpatialSpaceView to SpaceViewClass trait framework [#2311](#2311) - Typename cleanup in SpaceViewClass framework [#2321](#2321) - Automatic fallback for unrecognized Space View Class, start removing old ViewCategory [#2357](#2357) - Rename ScenePart -> ViewPartSystem + related renamings [#2674](#2674) - Dynamically registered space view (part/context) systems [#2688](#2688) - Viewer's command queue is now a channel, allowing to queue commands without mutable access [#2339](#2339) - Break up app.rs into parts [#2303](#2303) - Break out `re_log_types::component_types` as `re_components` [#2258](#2258) - Introduce StoreHub and rename Recording->Store [#2301](#2301) - Move StoreHub out of the Viewer during Update [#2330](#2330) - Expand CommandSender to support SystemCommand [#2344](#2344) - Use `camino` crate for UTF8 paths in `re_types_builder` [#2637](#2637) - Separate 2d & 3d spaceview classes, removal of `ViewCategory`, `SpaceViewClass` driven spawn heuristics [#2716](#2716) - Move object property heuristics to heuristics.rs [#2764](#2764) #### 📦 Dependencies - Version `rand` & friends at workspace level [#2508](#2508) - Update to PyO3 0.19 [#2350](#2350) - Pin `half` to `2.2.1` [#2587](#2587) #### 📘 Blueprint Changes - Drive blueprints off of a DataStore [#2010](#2010) - Split SpaceView -> SpaceViewState + SpaceViewBlueprint [#2188](#2188) - Split the Blueprint into AppBlueprint and ViewportBlueprint [#2358](#2358) - Swap the naming of Viewport and ViewportBlueprint [#2595](#2595) - Basic persistence for blueprints [#2578](#2578) #### 🏭 New Codegen Framework - Codegen/IDL 1: add more build tools [#2362](#2362) - Codegen/IDL 2: introduce `re_types_builder` [#2363](#2363) - Codegen/IDL 3: introduce `re_types` [#2369](#2369) - Codegen/IDL 4: definitions for a `Points2D` archetype [#2370](#2370) - Codegen/IDL 5: auto-generated Python code for `Points2D` [#2374](#2374) - Codegen/IDL 7: handwritten Python tests and extensions for `Points2D` [#2410](#2410) - Codegen/IDL 6: auto-generated Rust code for `Points2D` [#2375](#2375) - Codegen/IDL 8: handwritten Rust tests and extensions for `Points2D` [#2432](#2432) - Codegen'd Rust/Arrow 1: upgrading to actual `TokenStream`s [#2484](#2484) - Codegen'd Rust/Arrow 2: matching legacy definitions [#2485](#2485) - Codegen'd Rust/Arrow 3: misc fixes & improvements [#2487](#2487) - Codegen'd Rust/Arrow 4: out-of-sync definitions CI detection [#2545](#2545) - Codegen'd Rust/Arrow 5: doc, definitions and regression tests for combinatorial affixes [#2546](#2546) - Codegen'd Rust/Arrow 6: serialization [#2549](#2549) - Codegen'd Rust/Arrow 7: deserialization [#2554](#2554) - Codegen'd Rust/Arrow 8: carry extension metadata across transparency layers [#2570](#2570) - Codegen'd Rust/Arrow 9: Rust backport! [#2571](#2571) - End-to-end cross-language roundtrip tests for our archetypes [#2601](#2601) - Automatically derive `Debug` and `Clone` in Rust backend [#2613](#2613) - Generating (de)serialization code for dense unions in Rust backend [#2626](#2626) - Fix `FixedSizeList` deserialization edge-case + trivial optimizations [#2673](#2673) - Make `Datatype` & `Component` both inherit from `Loggable` [#2677](#2677) - Roundtrip-able `Transform3D`s [#2669](#2669) - Don't inline recursive datatypes in Rust backend [#2760](#2760) - Automatically derive `tuple_struct` attr and trivial `From` impls where possible [#2772](#2772) - Introduce roundtrip-able `Points3D` archetype (py + rs) [#2774](#2774) - Add `fmt::Debug` implementations to various types. [#2784](#2784) (thanks [@kpreid](https://github.com/kpreid)!) - Isolate testing types in Rust backend [#2810](#2810) - Fix out-of-sync codegen hash [#2567](#2567) - Python backport: add `log_any()` [#2581](#2581) - Integrate unit examples into codegen stack [#2590](#2590) - Disable codegen on windows [#2592](#2592) - Python codegen: big cleaning and paving the way towards transforms [#2603](#2603) - Automatically assume arrow transparency for components [#2608](#2608) - Fix wrong path being `rerun_if_changed()` in `compute_dir_hash` [#2612](#2612) - Support transparency at the semantic layer [#2611](#2611) - Don't use builtin `required` anymore, introduce `nullable` instead [#2619](#2619) - Rust codegen: generate proper docstrings [#2668](#2668) - Support nullable Arrow unions using virtual union arms [#2708](#2708) - Introduce support for querying Archetypes [#2743](#2743) - Introduce legacy shims and migrate DataCell to re_types::Component [#2752](#2752) #### 🌊 Starting work on C++ - Seed of C and C++ SDKs [#2594](#2594) - Move C++ SDK to own folder [#2624](#2624) - C++ codegen [#2678](#2678) - C++ codegen for reporting arrow data type for structs [#2756](#2756) - Don't inline recursive datatypes in C++ backend [#2765](#2765) - C++ codegen to_arrow_data_type for unions [#2766](#2766) - C++ codegen arrow serialize non-union components/datatypes without nested rerun types [#2820](#2820) - C++ codegen of structs and unions [#2707](#2707) - Fix cpp formatter differences [#2773](#2773) #### 🤷‍♂️ Other - test_api: set different app_id based on what test is run [#2599](#2599) - Introduce `rerun compare` to check whether 2 rrd files are functionally equivalent [#2597](#2597) - Remove `files.exclude` in vscode settings [#2621](#2621) - Support feature-gated rust attributes [#2813](#2813) ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested [demo.rerun.io](https://demo.rerun.io/pr/2842) (if applicable) - [PR Build Summary](https://build.rerun.io/pr/2842) - [Docs preview](https://rerun.io/preview/pr%3Arelease-0.8/docs) - [Examples preview](https://rerun.io/preview/pr%3Arelease-0.8/examples)

teh-cmc changed the base branch from main to cmc/hope_2/re_types_builder June 11, 2023 19:32

teh-cmc added the codegen/idl label Jun 11, 2023

teh-cmc force-pushed the cmc/hope_3/re_types branch 2 times, most recently from f8ed633 to 19ee454 Compare June 11, 2023 19:56

teh-cmc changed the title ~~Codegen/IDL 3: introduce re_types & IDL definitions for Points2D~~ Codegen/IDL 3: introduce re_types Jun 11, 2023

This was referenced Jun 11, 2023

Codegen/IDL 1: add more build tools #2362

Merged

Codegen/IDL 2: introduce re_types_builder #2363

Merged

teh-cmc commented Jun 11, 2023

View reviewed changes

teh-cmc force-pushed the cmc/hope_3/re_types branch from ccfa036 to 82df4e4 Compare June 11, 2023 20:35

teh-cmc marked this pull request as ready for review June 11, 2023 21:32

This was referenced Jun 11, 2023

Codegen/IDL 4: definitions for a Points2D archetype #2370

Merged

Codegen/IDL 5: auto-generated Python code for Points2D #2374

Merged

Codegen/IDL 6: auto-generated Rust code for Points2D #2375

Merged

teh-cmc force-pushed the cmc/hope_2/re_types_builder branch from 6b3d2ac to 42eb789 Compare June 12, 2023 08:44

teh-cmc force-pushed the cmc/hope_3/re_types branch 2 times, most recently from 74e410c to b50b5f2 Compare June 12, 2023 09:13

teh-cmc mentioned this pull request Jun 13, 2023

Codegen/IDL 7: handwritten Python tests and extensions for Points2D #2410

Merged

teh-cmc force-pushed the cmc/hope_3/re_types branch 2 times, most recently from 67180bf to 911010e Compare June 13, 2023 10:22

Wumpf approved these changes Jun 13, 2023

View reviewed changes

teh-cmc force-pushed the cmc/hope_2/re_types_builder branch from 28df80b to 3112d07 Compare June 14, 2023 08:49

teh-cmc force-pushed the cmc/hope_3/re_types branch from 911010e to 9385a7a Compare June 14, 2023 09:04

teh-cmc mentioned this pull request Jun 14, 2023

Codegen/IDL 8: handwritten Rust tests and extensions for Points2D #2432

Merged

Base automatically changed from cmc/hope_2/re_types_builder to main June 14, 2023 14:22

teh-cmc force-pushed the cmc/hope_3/re_types branch from 9385a7a to 5a15c6f Compare June 14, 2023 14:27

teh-cmc force-pushed the cmc/hope_3/re_types branch from 5a15c6f to 4d02e6d Compare June 14, 2023 14:31

teh-cmc and others added 18 commits June 14, 2023 16:32

more build tools

8e2744b

self-review

326c195

addressing PR comments

40f4059

introduce re_types_builder

0a7f445

generate reflection code

bf37e50

unindent 0.1 everywhere

a4e8daa

self-review

cf290a7

adhering to py38+ style guide

fb1d9e6

vscode flatbuffer things. Fix comment typo

d8c094e

turn the inner from_similar into _from_similar to appease linters

7ae14c3

generate __all__ everywhere to make python tools behave

0bc204c

make sure __all__ manifests are lexically sorted

6db76c7

adressing PR comments

8d69d7e

introduce re_types

dabd85b

self-review

b5fd082

python shall output to rerun_py/rerun_sdk/rerun2

f630ea3

addressing PR comments

fb86466

git lost it

8c78c01

teh-cmc force-pushed the cmc/hope_3/re_types branch from 4d02e6d to 8c78c01 Compare June 14, 2023 14:33

teh-cmc merged commit 412002b into main Jun 14, 2023

teh-cmc deleted the cmc/hope_3/re_types branch June 14, 2023 14:35

jleibs mentioned this pull request Jul 27, 2023

Release 0.8 - Infrastructure investments and more transform improvements - 2023-07-27 #2842

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Codegen/IDL 3: introduce `re_types` #2369

Codegen/IDL 3: introduce `re_types` #2369

teh-cmc commented Jun 11, 2023 •

edited

Loading

Wumpf left a comment

Wumpf Jun 13, 2023

teh-cmc Jun 14, 2023

Wumpf commented Jun 13, 2023

teh-cmc commented Jun 13, 2023

teh-cmc commented Jun 14, 2023

Codegen/IDL 3: introduce re_types #2369

Codegen/IDL 3: introduce re_types #2369

Conversation

teh-cmc commented Jun 11, 2023 • edited Loading

Organization

Build cache

How-to: add a new datatype/component/archetype

How-to: remove an existing datatype/component/archetype

Extensions

Rust

Python

Wumpf left a comment

Choose a reason for hiding this comment

Wumpf Jun 13, 2023

Choose a reason for hiding this comment

teh-cmc Jun 14, 2023

Choose a reason for hiding this comment

Wumpf commented Jun 13, 2023

teh-cmc commented Jun 13, 2023

teh-cmc commented Jun 14, 2023

Codegen/IDL 3: introduce `re_types` #2369

Codegen/IDL 3: introduce `re_types` #2369

teh-cmc commented Jun 11, 2023 •

edited

Loading