Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

re_datastore: arrow table debug-view #404

Closed
Tracked by #2695
emilk opened this issue Nov 28, 2022 · 5 comments · Fixed by #7226
Closed
Tracked by #2695

re_datastore: arrow table debug-view #404

emilk opened this issue Nov 28, 2022 · 5 comments · Fixed by #7226
Assignees
Labels
🏹 arrow concerning arrow ⛃ re_datastore affects the datastore itself ui concerns graphical user interface

Comments

@emilk
Copy link
Member

emilk commented Nov 28, 2022

A birds-eye view of our inner table/query engine.

This is a nice panel for looking into the main table(s) and seeing their columns etc. Good for debugging, but also shows the user data in its most naked form.

This is in aid of #393

@emilk emilk added the 🏹 arrow concerning arrow label Nov 28, 2022
@Wumpf Wumpf added the ui concerns graphical user interface label Nov 28, 2022
@emilk emilk assigned emilk and unassigned emilk Nov 29, 2022
@teh-cmc teh-cmc added the ⛃ re_datastore affects the datastore itself label Dec 2, 2022
@teh-cmc
Copy link
Member

teh-cmc commented Dec 2, 2022

adding the notes I took while writing the MVP:

Data store GUI browser

  • the store currently provides a very thorough Display implementation that makes it manageable to keep track of what's going on internally
  • it'd be even better to have something similar but as an interactive UI panel, akin to a SQL browser

@teh-cmc teh-cmc changed the title Arrow table debug-view re_datastore: arrow table debug-view Dec 2, 2022
@jondo2010
Copy link
Contributor

My initial "imagine what this would look like" is an interactive/collapsible version of the stdout debug output from the store.

@emilk
Copy link
Member Author

emilk commented Jun 27, 2023

This can now be implemented as a separate space view class. However, it makes very little sense unless we also have the UI to select/switch between different space view classes.

@emilk
Copy link
Member Author

emilk commented Jun 28, 2023

Some inspiration:

  • examples/rust/custom_space_view
  • examples/rust/extend_viewer_ui/src/main.rs
  • Together with egui_extras::TableBuilder (see usage in the current code)

@abey79
Copy link
Member

abey79 commented Jun 29, 2023

A very short, immediately useful thing could be a Dump datastore command in the Command palette that would eprintln! the datastore.

I'm currently sticking this in App::ui():

        if let Some(ctx) = store_context {
            if let Some(recording) = ctx.recording {
                eprintln!("{}", recording.entity_db.data_store);
            }
        }

teh-cmc added a commit that referenced this issue Jun 29, 2023
Credits to @abey79 for the great idea, see
#404 (comment).

Turns out I currently need this right now in order to debug some
backported stuff, so here we go.

Far from the greatest debugger ever made, but infinitely better than the
status quo.

---

<!-- This line will get updated when the PR build summary job finishes.
-->
PR Build Summary: https://build.rerun.io/pr/2564

<!-- pr-link-docs:start -->
Docs preview: https://rerun.io/preview/3ea3463/docs
Examples preview: https://rerun.io/preview/3ea3463/examples
<!-- pr-link-docs:end -->
@abey79 abey79 mentioned this issue Aug 27, 2024
6 tasks
@abey79 abey79 self-assigned this Aug 27, 2024
abey79 added a commit that referenced this issue Aug 27, 2024
### What

- Closes #404

This PR introduce a new, low-level chunk store browser crate
`re_chunk_store_ui` with the following features:
- browse both the recording and blueprint store
- chunk store metadata display
- chunk list view
  - info and config
  - query: all chunks/latest-at-relevent chunks/range-relevant chunks
  - filter: by entity and/or component
  - sort by: chunk id, entity, row count, any timeline
  - copy entire list to clipboard
- chunk view
  - chunk metadata (including underlying arrow metadata)
  - sort by: row id, any timeline
  - copy chunk to clipboard

The feature is available from the rerun menu and the `ctrl-shift-D`
shortcut.

Note: it would be very easy to drop the current `re_viewer_context`
dependency and make a stand-alone chunk store viewer binary that only
depends on `re_ui` and the lower-level store/type crates.

<img width="1656" alt="image"
src="https://github.com/user-attachments/assets/68c6485e-0919-4375-bb0a-772c32898305">
<img width="1656" alt="image"
src="https://github.com/user-attachments/assets/15164965-abf8-4d67-b141-a95b848b2704">



### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested the web demo (if applicable):
* Using examples from latest `main` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/7226?manifest_url=https://app.rerun.io/version/main/examples_manifest.json)
* Using full set of examples from `nightly` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/7226?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG
* [x] If applicable, add a new check to the [release
checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)!
* [x] If have noted any breaking changes to the log API in
`CHANGELOG.md` and the migration guide

- [PR Build Summary](https://build.rerun.io/pr/7226)
- [Recent benchmark results](https://build.rerun.io/graphs/crates.html)
- [Wasm size tracking](https://build.rerun.io/graphs/sizes.html)

To run all checks from `main`, comment on the PR with `@rerun-bot
full-check`.

---------

Co-authored-by: Clement Rey <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏹 arrow concerning arrow ⛃ re_datastore affects the datastore itself ui concerns graphical user interface
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants