Skip to content

web: cache the daemon runtime so returning visitors skip the download - #59

Merged
jamaljsr merged 4 commits into
perf/web-wallet-startupfrom
perf/web-runtime-cache
Aug 6, 2026
Merged

web: cache the daemon runtime so returning visitors skip the download#59
jamaljsr merged 4 commits into
perf/web-wallet-startupfrom
perf/web-runtime-cache

Conversation

@Roasbeef

Copy link
Copy Markdown
Member

In this PR, we cache the daemon's wasm module in Cache Storage, so a returning
visitor reads it off disk instead of pulling 19 MB over the network again. On a
50 Mbps link this takes the runtime load from 3,250 ms to 68 ms, and the whole
boot from 4,654 ms to 476 ms.

This is stacked on #58, which added the instrumentation the numbers below come
from. Review that one first.

The browser will not cache a module this large

The startup metric that dominates cold load is wasmCompileInstantiate, so the
obvious read is that we're compile-bound. We aren't. Serving the same module off
localhost with the bytes already local:

Phase Duration
WebAssembly.compileStreaming 127 ms
WebAssembly.instantiate 7 ms
fused instantiateStreaming 138 ms

Compilation is cheap because V8 compiles Go wasm lazily, deferring function
bodies until they're first called. Instantiation is cheap too, despite the module
carrying 100,000 data segments and 67 MB of initialized data. So ~3.4s of a ~3.6s
wasm load is just moving bytes.

That would be a first-visit cost if the browser kept them, and it doesn't.
Fetching the module repeatedly in one context and reading Resource Timing:

Load 19 MB module transferSize 1 MiB control transferSize
1 19,957,857 1,048,876
2 19,957,857 0
3 19,957,857 0

Same origin, same Cache-Control: public,max-age=31536000,immutable, CDN
reporting a hit. The 1 MiB control asset caches on the first fetch; the module
never does. Chrome simply won't store an HTTP cache entry that big, so every load
pays full transfer forever. That's also why a reload measured no faster than a
cold load (3,761 ms vs 3,571 ms) before this change.

What we do about it

Cache Storage has no such ceiling, so we keep the bytes ourselves and the wasm
load becomes a disk read. Worth noting what isn't possible: caching the
compiled module. A WebAssembly.Module survives structuredClone, but
IndexedDB rejects it outright with "A WebAssembly.Module can not be serialized
for storage", so bytes are the only durable form.

The write isn't awaited. We clone the response, hand the original to
instantiateStreaming, and let the put run alongside it, so filling the cache
doesn't slow down the load that fills it.

One invariant matters for correctness: the cache always holds decompressed wasm,
whatever encoding it arrived in. On the HTTP-decompression path the transport
already inflated the body, so the clone is what we want. On the buffered path (a
host serving a plain .gz as application/gzip) we inflate it ourselves and
store those bytes instead. Without that split, the warm path would read gzip out
of the cache, fail to instantiate, and fall back to the network on every load.

Cached bytes that don't instantiate get evicted and treated as a miss, since
otherwise a truncated entry would wedge the wallet for good with nothing to clear
it.

Releases don't accumulate copies

Since we cut new runtime releases periodically, the cache can't grow without
bound. Asset URLs already carry the version
(<base>/<version>/wavewalletdk.wasm.gz), so a release changes the key rather
than overwriting the old entry, and we prune on every store. A wallet that's been
through several upgrades holds one runtime, not one per release it has ever seen.
The bucket name carries a separate schema version so a future change to what we
store can abandon the old buckets wholesale.

Verified in a browser rather than just in unit tests: seed the cache with a
previous release plus a bucket from an older schema, then let an upgrade load
happen.

before upgrade load: {"wavelength-runtime-v1":[".../v0.0.9/wavewalletdk.wasm.gz"],
                      "wavelength-runtime-v0":[".../v0.0.9/wavewalletdk.wasm.gz"]}
after upgrade load:  {"wavelength-runtime-v1":[".../v0.1.0/wavewalletdk.wasm.gz"]}

None of this is load-bearing. Cache Storage is missing outside a secure context,
throws on property access in some privacy modes, and rejects writes once an
origin is over quota. Every one of those degrades to "no cache", which is the
behavior we had before.

Measured

Against the demo on a throttled 50 Mbps link, cold then reload in the same
context:

=== cold, first visit ===       runtime usable in 4654 ms
  wasmCompileInstantiate   3213 ms  {"path":"gzip","streaming":true,"decompression":"http"}
  wasmTotal                3250 ms

=== warm, returning visitor === runtime usable in  476 ms
  wasmCacheRead              22 ms  {"path":"gzip","bytes":129352347}
  wasmCompileInstantiate     45 ms  {"path":"gzip","streaming":false,"source":"cache"}
  wasmTotal                  68 ms

The cached entry is 129,352,347 bytes, i.e. the decompressed module, which is the
invariant above holding.

Also worth flagging for whoever picks up startup work next: with the load path
fixed, createRpc at ~6.4s is now the slowest thing a new user waits for,
against ~1.2s for unlockRpc on nominally the same key derivation. That gap is
where the next win is, not here.

Testing

53 unit tests pass in packages/web, covering the cache module directly (schema
cleanup, pruning, quota rejection, unenumerable caches) and the loader paths
(cache hit skips the network, cold load stores, buffered path stores inflated
bytes, corrupt entry is evicted and refetched). pnpm build and pnpm typecheck
are clean, and the Playwright smoke test passes against a worker built with the
change.

See each commit message for a detailed description w.r.t the incremental changes.

@jamaljsr jamaljsr left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I measured this one before reading it, so the numbers below are mine.

What I tested:

  • Reproduced the described win on a throttled 50 Mbps link, cold then reload in the same browser context. Cold wasmTotal 3258 ms, warm 95 ms (wasmCacheRead 38 ms plus a 56 ms compile reporting source: "cache"), runtime usable 603 ms against 4814 ms cold. Cold is unchanged from the base, so the unawaited write isn't costing the load that fills the cache.
  • Confirmed the decompressed-bytes invariant exactly: the cache reported bytes: 129328511, byte-for-byte the size of wavewalletdk.wasm, against 19,929,133 for the .gz. It is definitively not storing the compressed form.
  • Reproduced the release-pruning demo in a browser. Seeding a previous release plus a bucket from an older schema, then letting an upgrade load happen, leaves exactly one entry under wavelength-runtime-v1.
  • Counted runtime fetches per page load across four loads, on a correctly configured host and on a half-configured one. That's finding 2 below.
  • 53 unit tests pass in packages/web, and pnpm build and pnpm typecheck are clean.

The mechanism works and the numbers hold up. Findings 1 and 2 are about what happens on host configurations other than the demo's.

Comment thread packages/web/src/runtime-cache.ts
Comment thread packages/web/src/runtime.ts Outdated
Comment thread apps/docs/src/content/docs/web/runtime/data-and-persistence.mdx
Comment thread packages/web/src/runtime.ts Outdated
@jamaljsr
jamaljsr force-pushed the perf/web-runtime-cache branch from 1536ab9 to e58276a Compare August 5, 2026 19:08
@claude

claude Bot commented Aug 5, 2026

Copy link
Copy Markdown

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

@jamaljsr
jamaljsr force-pushed the perf/web-runtime-cache branch from e58276a to e9ca77f Compare August 5, 2026 20:34
@claude

claude Bot commented Aug 5, 2026

Copy link
Copy Markdown

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

In this commit, we add the storage layer that lets a returning visitor
skip re-downloading the wasm module. No caller uses it yet; wiring the
load paths up comes next.

The module is around 20 MB compressed, and that turns out to be large
enough that Chrome refuses to keep it in the HTTP cache. Reading
Resource Timing across repeated loads of one deployment shows
transferSize staying at the full body size on every single load, while a
1 MiB asset served with byte-identical headers drops to 0 after the
first fetch. The headers aren't the problem: this reproduces with
Cache-Control: public,max-age=31536000,immutable and a CDN cache hit.
The browser just won't store an entry that big, so every load pays the
full transfer.

That transfer is essentially the whole startup cost. Serving the same
module from localhost with the bytes already local, compileStreaming
takes 127 ms and instantiate 7 ms, because V8 compiles Go wasm lazily.
So ~3.4s of a ~3.6s wasm load is network, and it repeats forever.

Cache Storage has no such size ceiling, so we keep the bytes ourselves.
Worth noting that caching the *compiled* module isn't an option: a
WebAssembly.Module survives structuredClone, but IndexedDB rejects it
with "A WebAssembly.Module can not be serialized for storage", so bytes
are the only durable form.

Since we periodically cut new runtime releases, the cache has to not
grow without bound. Asset URLs already carry the runtime version
(<base>/<version>/wavewalletdk.wasm.gz), so a release changes the key
rather than overwriting the old entry. We prune on every store, which
leaves a wallet that's been through several upgrades holding one
runtime instead of one per release it has ever seen. The bucket name
carries a separate schema version so a change to what we store can
abandon the old buckets wholesale.

None of this is load-bearing. Cache Storage is missing outside a secure
context, throws on property access in some privacy modes, and rejects
writes once an origin is over quota, so every operation degrades to "no
cache" rather than failing the wallet.
@jamaljsr
jamaljsr force-pushed the perf/web-runtime-cache branch from e9ca77f to 65be143 Compare August 6, 2026 04:50
@claude

claude Bot commented Aug 6, 2026

Copy link
Copy Markdown

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

In this commit, we teach the main-thread loader to check Cache Storage
before reaching for the network, and to stash what it downloads on the
way past. A returning visitor now pays a disk read instead of a 20 MB
transfer.

The write isn't awaited. Filling the cache shouldn't slow down the load
that fills it, so we clone the response, hand the original to
instantiateStreaming, and let the put run alongside it.

One invariant is worth calling out: the cache always holds decompressed
wasm, whatever encoding it arrived in. On the HTTP-decompression path the
transport already inflated the body, so the clone is what we want. On the
buffered path (a host serving a plain .gz as application/gzip) we inflate
it ourselves, so we store those bytes rather than the response. Without
that, the warm path would read gzip out of the cache, fail to
instantiate, and fall back to the network on every single load.

Speaking of which, cached bytes that don't instantiate are evicted and
treated as a miss. A truncated or otherwise broken entry would otherwise
wedge the wallet for good, since nothing else would ever clear it.

We do the same on the raw path, so a self-host serving uncompressed wasm
gets the benefit too, and not just deployments on the compressed one.
In this commit, we mirror the cache-first load into the worker, which is
where it actually matters: the daemon runs in a Web Worker by default, so
the main-thread path we just wired up is the fallback rather than the
common case.

The worker ships as a standalone file that the consumer's bundler emits
from new URL(), so it can't import from the package and the helpers are
inlined here instead. That's the same reason it already mirrors the asset
names literally rather than pulling in RUNTIME_ASSETS. Keep the two in
sync; the comment on each side says so.

Behavior matches runtime.ts exactly, down to storing inflated bytes on
the buffered path and evicting cached bytes that fail to instantiate.
In this commit, we document the Cache Storage bucket, since it shows up
as ~130 MB of origin storage and consumers will want to know what it is
before they find it in devtools. The important points for someone sizing
storage: it holds no wallet state, it keeps one runtime at a time, and it
prunes on the first load after a runtime version bump.

We also note on the hosting page that long-lived cache headers are still
worth setting but won't get you browser caching for a module this size,
which is the whole reason the SDK keeps its own copy. While there, we
call out serving the .gz as application/wasm with Content-Encoding: gzip
to get the streaming compile path, which wasn't written down anywhere.
@jamaljsr
jamaljsr force-pushed the perf/web-runtime-cache branch from 65be143 to f3a4a92 Compare August 6, 2026 05:56
@claude

claude Bot commented Aug 6, 2026

Copy link
Copy Markdown

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

@jamaljsr
jamaljsr merged commit 2a06c8d into main Aug 6, 2026
4 checks passed
@jamaljsr
jamaljsr deleted the perf/web-runtime-cache branch August 6, 2026 14:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants