Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

All files Rerun can open should support command line, drag, and open from menu #2229

Closed
nikolausWest opened this issue May 26, 2023 · 8 comments · Fixed by #3131
Closed

All files Rerun can open should support command line, drag, and open from menu #2229

nikolausWest opened this issue May 26, 2023 · 8 comments · Fixed by #3131
Assignees
Labels
enhancement New feature or request 🏎️ Quick Issue Can be fixed in a few hours or less 📺 re_viewer affects re_viewer itself ui concerns graphical user interface

Comments

@nikolausWest
Copy link
Member

Since 0.6, Rerun supports opening png images, meshes and rrd files form the command line, but only rrd files can be opened through the Rerun menu -> open flow or by dragging and dropping into the viewport. All files that Rerun can open should be accessible through the same methods.

@nikolausWest nikolausWest added enhancement New feature or request 📺 re_viewer affects re_viewer itself 🏎️ Quick Issue Can be fixed in a few hours or less labels May 26, 2023
@nikolausWest nikolausWest added the ui concerns graphical user interface label Jun 21, 2023
@emilk emilk added this to the 0.9 milestone Aug 17, 2023
@emilk
Copy link
Member

emilk commented Aug 17, 2023

This affects the text on the welcome screen:

image

abey79 added a commit that referenced this issue Aug 17, 2023
### What

This adds a MVP Welcome Screen that shows up instead of the current
"Loading" screen in certain circumstances.

In general terms:
- The Welcome Screen is drawn in the Viewport's place, ie. with the
panels visible.
- The Welcome Screen is never displayed when a recording is available.
- The current implementation of the Welcome Screen is very basic—mostly
links to webpages.
- The current "loading screen" still exists and is "sometimes"
displayed.

The tricky thing is when to show the Welcome Screen vs. the legacy
loading screen. This PR includes an heuristic (implemented in
`App::handle_default_blueprint()`). The main determinant is the nature
of the receiver (file, tcp, web socket, etc.) which we use as a proxy
for the workflow in which the viewer is being used. The TCP receiver is
a bit tricky, as it's used by both the Python SDK (we don't want the
Welcome Screen to distractingly flash at spawn, before data arrives) and
when running manually (`$ rerun`). To address this, this PR introduces a
new `--skip-welcome-screen` CLI option, used by the Python SDK. This
situation is still not entirely perfect though: #3018.

Implementation details:
- Technically, the Welcome Screen is triggered when the app ID is set to
`StoreHub::welcome_screen_app_id()`. A corresponding blank blueprint is
created at startup (by `StoreHub`) to make the UI happy.
`App::handle_default_blueprint()` basically sets that app ID to trigger
the Welcome Screen.
- Likewise, an empty recording is _always_ set as active whenever the
app ID is set but no recording is available. This empty recording isn't
in the list of available recordings
(`ViewerContext::alternate_recordings`). This make the UI happy.
- The status string and source originally displayed in the legacy
loading screen are _also_ displayed on the Welcome Screen in some
circumstances (i.e for "infinite" data sources, tcp, ws, etc.)


Expect these follow-up PRs:
- display the status string in the menu bar instead (as per Mårten
designs)
- address #2229 
- address #3018 
- fix UI of streams with empty recording
- add `external_link` icon wherever we have external links

Fixes #2513 

<img width="1456" alt="image"
src="https://github.com/rerun-io/rerun/assets/49431240/d35a3f02-571d-4fc0-ae7f-67ce8b9d416d">

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested [demo.rerun.io](https://demo.rerun.io/pr/2982) (if
applicable)

- [PR Build Summary](https://build.rerun.io/pr/2982)
- [Docs
preview](https://rerun.io/preview/pr%3Aantoine%2Fwelcome-screen-v0/docs)
- [Examples
preview](https://rerun.io/preview/pr%3Aantoine%2Fwelcome-screen-v0/examples)

---------

Co-authored-by: Emil Ernerfeldt <[email protected]>
@abey79
Copy link
Member

abey79 commented Aug 17, 2023

I'm getting that when dropping files on demo.rerun.io:

image

edit: apparently not on newer builds 🤔

@abey79
Copy link
Member

abey79 commented Aug 17, 2023

Also, preview_files_being_dropped() appears to be inoperative on wasm builds. (that's the black overlay)

@abey79
Copy link
Member

abey79 commented Aug 18, 2023

Currently, CLI loading of files uses SmartChannels, where as Open/file drop use SystemCommand::LoadRrd. The latter is bad, because it hogs the SystemCommand thread and doesn't "stream" the RRD into a recording–it loads all of it at once. This could be bad for big files.

It appears that the way forward is:

  • reimplement SystemCommand::LoadRrd by creating a smart channel (App will need some kind of Arc<HashSet<Receiver<_>>>)
  • add new system command for other types of files (jpg, etc.), reusing the existing support in loading.rs
  • it'll probably be cleaner to move the load_rrd_file_to_channel from rerun/src/run.rs to re_viewer/src/loading.rs?
  • re-implement the file arguments to CLI by using the new SystemCommand mechanism, which will allow to still have a Tcp server when launching a viewer with file arguments (basically it'll decouple the two features).

Thanks to @jleibs for helping me gain clarity on this.

@abey79
Copy link
Member

abey79 commented Aug 21, 2023

@emilk
Copy link
Member

emilk commented Aug 25, 2023

Me, @abey79 and @jleibs discussed this at length with the following conclusions:

We need something like this:

enum DataSource {
    /// A remote RRD file, served over http.
    RrdHttpUrl(String),

    /// A path to a local file.
    #[cfg(not(target_arch = "wasm32"))]
    FilePath(std::path::PathBuf),

    /// Used mostly on Web where there are no paths
    FileContents(FileContents),

    /// A remote Rerun server.
    WebSocketAddr(String),
}

which replaces ArgumentCategory in run.rs. There will be a SystemCommand to load it.

We then have a function like so:

fn stream_data_source(ds: DataSource) -> Result<smart_channel::Receiver<LogMsg>> {}

that will be called by run.rs but also by app.rs.

The other change is that App stores a RecieverSet which is something like struct RecieverSet(Vec<Receiver<LogMsg>>) which handles pruning closed channels.

@teh-cmc
Copy link
Member

teh-cmc commented Aug 26, 2023

The issue title mentions CLI, drag-and-drop and open-from-menu; but what about URLs?

E.g. it's possible to do https://app.rerun.io/?url=https://somewebsitething.com/data.rrd today, and it should be equally possible to do https://app.rerun.io/?url=https://somewebsitething.com/image.jpeg.

@abey79
Copy link
Member

abey79 commented Aug 26, 2023

Opening from URL is indeed also required in the short-term for the welcome screen's example launcher (#3096).

emilk added a commit that referenced this issue Aug 28, 2023
### What
* Part of #2229

Supports mixing of:
* .rrd over HTTP
* File paths
* WebSockets urls

in `rerun` command-line arguments. All will be streamed to the viewer in
parallel.

`File->Open` and drag-and-dropping of files will use the same code path,
and now supports meshes and images on native.

### Saving for later PR:
* Drag-and-drop of images and meshes on web
* Loading images and meshes over HTTP

### How
Created a new `re_data_source` crate with an `enum DataSource` plus all
the functions to stream data from all these sources.

I also removed `RemoteViewerApp` which I originally created to have a
custom GUI for selecting what server to connect to, but that GUI has
been disabled since forever, so high time to remove it.

On web, care is taken to wake up the ui thread when loading data from
web-sockets or HTTP.
On native, we just spin up another thread that notices new messages and
wakes up the ui.

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested [demo.rerun.io](https://demo.rerun.io/pr/3116) (if
applicable)

- [PR Build Summary](https://build.rerun.io/pr/3116)
- [Docs
preview](https://rerun.io/preview/cfd5d11b8862d09158c46d5f3407d4931b8747e3/docs)
<!--DOCS-PREVIEW-->
- [Examples
preview](https://rerun.io/preview/cfd5d11b8862d09158c46d5f3407d4931b8747e3/examples)
<!--EXAMPLES-PREVIEW--><!--EXAMPLES-PREVIEW--><!--EXAMPLES-PREVIEW--><!--EXAMPLES-PREVIEW--><!--EXAMPLES-PREVIEW--><!--EXAMPLES-PREVIEW--><!--EXAMPLES-PREVIEW--><!--EXAMPLES-PREVIEW--><!--EXAMPLES-PREVIEW--><!--EXAMPLES-PREVIEW--><!--EXAMPLES-PREVIEW--><!--EXAMPLES-PREVIEW--><!--EXAMPLES-PREVIEW--><!--EXAMPLES-PREVIEW-->
- [Recent benchmark results](https://ref.rerun.io/dev/bench/)
- [Wasm size tracking](https://ref.rerun.io/dev/sizes/)

---------

Co-authored-by: Antoine Beyeler <[email protected]>
emilk added a commit that referenced this issue Aug 29, 2023
* Closes #2229

### What
Drag-dropping images and meshes now works on web, as does File->Open.

I also managed to get rid of the use of `MsgSender` to create a
`LogMsg`, simplifying the dependencies a bit (if not the code).

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested [demo.rerun.io](https://demo.rerun.io/pr/3131) (if
applicable)

- [PR Build Summary](https://build.rerun.io/pr/3131)
- [Docs
preview](https://rerun.io/preview/551ba6c6df4dd6d2d13760f51942e83bff3fe4ce/docs)
<!--DOCS-PREVIEW-->
- [Examples
preview](https://rerun.io/preview/551ba6c6df4dd6d2d13760f51942e83bff3fe4ce/examples)
<!--EXAMPLES-PREVIEW--><!--EXAMPLES-PREVIEW-->
- [Recent benchmark results](https://ref.rerun.io/dev/bench/)
- [Wasm size tracking](https://ref.rerun.io/dev/sizes/)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request 🏎️ Quick Issue Can be fixed in a few hours or less 📺 re_viewer affects re_viewer itself ui concerns graphical user interface
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants