Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add x-goog-meta-uncompressed-size to uploaded artifacts #2422

Open
jleibs opened this issue Jun 13, 2023 · 0 comments
Open

Add x-goog-meta-uncompressed-size to uploaded artifacts #2422

jleibs opened this issue Jun 13, 2023 · 0 comments
Labels
🕸️ web regarding running the viewer in a browser

Comments

@jleibs
Copy link
Member

jleibs commented Jun 13, 2023

The default gcloud upload behavior is to gzip everything and then serve it with Content-Encoding: gzip

Because of the way that content-decoding works in the browser, progress bars typically have access to the incremental bytes read out of the stream and never actually see the gzipped payload. Unfortunately the specification is also that Content-Length should be the transfer size, rather than the uncompressed content-size.

However, google lets us add our own headers, so we should be able to do something along the lines of:

gsutil setmeta -h "x-goog-meta-uncompressed-size:$(stat -c%s web_viewer/re_viewer_bg.wasm)" gs://rerun-web-viewer/commit/${{env.SHORT_SHA}}/re_viewer_bg.wasm

See: #2421 for additional context.

Also, we might want to disable the gzipping behavior for uploading rrd files since the browser will end up paying an extra, gratuitous decompression step.

@jleibs jleibs added the 🕸️ web regarding running the viewer in a browser label Jun 13, 2023
jleibs added a commit that referenced this issue Jun 15, 2023
…wasm is loading (#2421)

### What

Resolves:
 - #2261

According to the new design, here is how the loading state should work:
1. HTML and CSS loads. It renders the skeleton content without any text.
1. WASM loads. It renders the exact same skeleton content without any
text. The user doesn’t notice that the rendering method switched from
HTML => WASM.
1. If loading takes long (more than 3 seconds) text fades in and
provides additional information.
1. When the App UI is ready, the skeleton content fades away (200ms
duration) and the app fades in.

The first deployed test of this ended up with 300% progress during the
download step, so I've added some extra handling for progress related to
compressed streams.At the moment we default to an adhoc compression
ratio, but once we land:
 - #2422
 then the progress should be completely accurate again.
 
In order to make all the waiting variants more consistent, this also
changes the RRD loading page.

The normal waiting page looks like:

![image](https://github.com/rerun-io/rerun/assets/3312232/a8eb2473-7b7c-433a-a0b2-09c6d209c44e)

Before:

![image](https://github.com/rerun-io/rerun/assets/3312232/adcf27dc-5c59-4642-8fda-18d0de544442)

After:

![image](https://github.com/rerun-io/rerun/assets/3312232/cd4104db-be28-4c32-8171-2b9a9408cf23)


This PR does not handle progress indicator during the loading of the RRD
itself. The hooks necessary to get access to progress feedback depend on
essentially the same work already happening as part of:
#2412 and just directly loading
the rrd file is generally going to be better than showing a progress
indicator. It could be worth revisiting this for things like blueprints
with large static data.

This needs to be tested manually on our major OS / browser combinations:

Builds to test:
 - https://app.rerun.io/commit/e15ebb5
 - https://demo.rerun.io/commit/e15ebb5
 
If your internet connection is too fast, use developer tools to throttle
connection and enable cache.

![image](https://github.com/rerun-io/rerun/assets/3312232/8ba99668-5441-4652-b1f3-9c82c2b6cbb3)

- Linux:
  - [x] Chrome
  - [x] Firefox
- Mac:
  - [x] Chrome
  - [x] Firefox
  - [x] Safari
- Windows:
  - [x] Chrome
  - [x] Firefox
  - [x] Edge 



### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)

<!-- This line will get updated when the PR build summary job finishes.
-->
PR Build Summary: https://build.rerun.io/pr/2421

<!-- pr-link-docs:start -->
Docs preview: https://rerun.io/preview/934925b/docs
Examples preview: https://rerun.io/preview/934925b/examples
<!-- pr-link-docs:end -->
emilk pushed a commit that referenced this issue Jun 15, 2023
…wasm is loading (#2421)

Resolves:
 - #2261

According to the new design, here is how the loading state should work:
1. HTML and CSS loads. It renders the skeleton content without any text.
1. WASM loads. It renders the exact same skeleton content without any
text. The user doesn’t notice that the rendering method switched from
HTML => WASM.
1. If loading takes long (more than 3 seconds) text fades in and
provides additional information.
1. When the App UI is ready, the skeleton content fades away (200ms
duration) and the app fades in.

The first deployed test of this ended up with 300% progress during the
download step, so I've added some extra handling for progress related to
compressed streams.At the moment we default to an adhoc compression
ratio, but once we land:
 - #2422
 then the progress should be completely accurate again.

In order to make all the waiting variants more consistent, this also
changes the RRD loading page.

The normal waiting page looks like:

![image](https://github.com/rerun-io/rerun/assets/3312232/a8eb2473-7b7c-433a-a0b2-09c6d209c44e)

Before:

![image](https://github.com/rerun-io/rerun/assets/3312232/adcf27dc-5c59-4642-8fda-18d0de544442)

After:

![image](https://github.com/rerun-io/rerun/assets/3312232/cd4104db-be28-4c32-8171-2b9a9408cf23)

This PR does not handle progress indicator during the loading of the RRD
itself. The hooks necessary to get access to progress feedback depend on
essentially the same work already happening as part of:
#2412 and just directly loading
the rrd file is generally going to be better than showing a progress
indicator. It could be worth revisiting this for things like blueprints
with large static data.

This needs to be tested manually on our major OS / browser combinations:

Builds to test:
 - https://app.rerun.io/commit/e15ebb5
 - https://demo.rerun.io/commit/e15ebb5

If your internet connection is too fast, use developer tools to throttle
connection and enable cache.

![image](https://github.com/rerun-io/rerun/assets/3312232/8ba99668-5441-4652-b1f3-9c82c2b6cbb3)

- Linux:
  - [x] Chrome
  - [x] Firefox
- Mac:
  - [x] Chrome
  - [x] Firefox
  - [x] Safari
- Windows:
  - [x] Chrome
  - [x] Firefox
  - [x] Edge

* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)

<!-- This line will get updated when the PR build summary job finishes.
-->
PR Build Summary: https://build.rerun.io/pr/2421

<!-- pr-link-docs:start -->
Docs preview: https://rerun.io/preview/934925b/docs
Examples preview: https://rerun.io/preview/934925b/examples
<!-- pr-link-docs:end -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🕸️ web regarding running the viewer in a browser
Projects
None yet
Development

No branches or pull requests

1 participant