SWs & Webpack

[KyleAMathews]

Hey, question for the community here — this isn't an issue for offline-plugin but I'm asking here as y'all are the ones most likely it seems to have faced the same problem (and hopefully found some good solutions).

I've integrated offline-plugin into the next version of Gatsby and it works great. A problem I'm seeing though is that when I deploy a new version of a site, Webpack doesn't seem to maintain the same module ids for all modules. So after deploying, I load the site and the chunks that weren't changed are loaded from the service worker and the new chunks are loaded from the server but almost always there'll be a JS error when Webpack can't find a module id to call.

So perhaps I'm just doing this wrong — is the solution just to ensure that you load all chunks from the old SW until a new one is loaded and then switch over en masse?

[NekR]

(text may have a lot of type because I wasn't re-reading/reviewing it)

I actually been thinking about this issue for a while. I'm not sure with which case exactly you have this problem (because there are few ways to get into it), but let me try to cover and answer all them. So..

There are couple of ways with offline-plugin to get into situation when old and new modules have conflicting module IDs, though, that's mostly related to webpack itself. So the solution on webpack level would be to use records, aka recordsPath option in webpack config. This option saves state between compilations and assigns permanent IDs to modules. Other solution on webpack level might be to use named modules (I don't remember exact name of plugin which does that).

Things in offline-plugin causing this problem:

1. You are using updateStrategy: 'changed' (by default in v4 but not v3) which re-downloads only changed files based on hashed generated by webpack. The problem here is webpack doesn't always update chunk's hash when module IDs changed in it. Example: file added into chunk 2 caused module revalidation which changed IDs in chunks 1 and 2, but contents of the chunk 1 didn't change so new hash isn't generated.
   Solutions: 1) use recordsPath in webpack config, or 2) do not user updateStrategy: 'changed', or 3) generate file hashes on offline-plugin level instead of relying on webpack's generated hashes, this is already done for html-webpack-plugin because of how it works.

2. You are using responseStrategy: 'network-first' (from v4) which may cause a situation where some chunks are loaded from the network and has new code and some other from the cache which has old code. The most right solution here is to not use responseStrategy: 'network-first' option or have all your assets unique named, e.g. with a hash in the name (which destroys the purpose of network-first though). recordsPath webpack option may help here, but I still assume there could be some conflicts between modules/chunks anyway.

3. You are using caches.additional / caches.optional options in offline-plugin and aren't unique naming your assets. This is essentially the same problem as with responseStrategy: 'network-first', but in this case SW may cache new chunks along side with old ones which may cause such a problem.
   Solutions: 1) correctly name your assets (e.g. hash or version in its names), or 2) do not user caches.additional / caches.optional options. recordsPath has only potential fix effect as in previous section.

4. You are applying ServiceWorker update once it's available but not reloading the pages. This causes pages to use new cache with the new code while they where used with the old one initially. So if you apply ServiceWorker update and then request some chunk from your server -- it will be loaded with the new code.
   Solutions: 1) Reload pages immediately after applying an update, or 2) Blocking all pages and telling the user to hit reload manually, or 3) do not apply an update immediately, but rather display an UI telling to the user that new update is really. If user wants -- apply update and reload the pages. 4) use recordsPath option which has approximately the same effect as in 2 and 3 sections.

@KyleAMathews I hope it helps. Please let me know if does or not and which exact case is yours here. Thanks! 👌

[KyleAMathews]

@NekR this is really really helpful!! Will be digging through this tomorrow and will get back to you. And recordsPath might be the answer especially as I didn't know it existed (so am not using it) 😆

[nodkz]

Awesome answer! Thanks for info.

[NekR]

@nodkz Would be good to transfer it somehow to the docs :-)
@KyleAMathews I'm glad it's helpful!

[KyleAMathews]

So to close this out — the problem seems 100% caused by not having recordsPath set. I've rebuilt sites a dozen times or so and everything seems rock solid now. Thanks again! I'll let you close it if you want or perhaps leave it open for transferring to docs.

[mciastek]

I noticed the same problem as @KyleAMathews mentioned. I use CommonsChunk plugin with OccurrenceOrder plugin. When new version of app is built and pushed to server, at the first load there is a blank page, which means that there was some problem with chunk loading. I tried to use recordsPath option but it doesn't seem to work with Webpack 2. Do you think that generating chunk manifest file will help with resolving chunk load during Service Worker update?

[NekR]

@mciastek it's really hard to help in this situation without seeing project's code. But it's for sure that recordsPath works in webpack 2.

Do you think that generating chunk manifest file will help with resolving chunk load during Service Worker update?

No, it's unrelated and won't help. offline-plugin takes care of it on its own. The problem what you may have is that chunk hashes aren't updated when they should. See this issue: #129 (and the one it refers too).

Basically, right now offline-plugin relyes on webpack's generated hashes which as stated in that issue is kind of a wrong deciton. In future offline-plugin will be calculating chunk hashes on its own, but right now your may try to use WebpackMd5Hash plugin to solve the issue (you still need to use recordsPath though).

[jampy]

Thanks a lot for the summary - very helpful!

What if I can't use recordsPath? webpack-stable-module-id-and-hash seems interesting, but I haven't tried it. Plugins like these all seem to have some issues... :-/

[KyleAMathews]

@jampy will be trying https://github.com/zhenyong/webpack-stable-module-id-and-hash with GatsbyJS soon! Not very happy with recordsPath requiring you to check in the records.json file. Confusing and noisy.

[NekR]

webpack-stable-module-id-and-hash indeed seems interesting and probably the only good way to fix all that without recordsPath. There are NamedModulesPlugin and HashedModulesPlugin built-in webpack 2 which may help too (they don't modify/fix caches though).

TBH, I think webpack doing bad job here with module ids and hashes. It should be fixed on core level and everything here are just hacks :-(

[jampy]

FYI, I checked the code of webpack-stable-module-id-and-hash and found some issues.

See my pull request: zhenyong/webpack-stable-module-id-and-hash#1

[NekR]

@jampy yeah, great PR 👍

[GGAlanSmithee]

@NekR is this issue still relevant, since offline-plugin now generates its own hashes?

offline-plugin generates its own hashes in since 4.5 and doesn't rely on webpack's generated hashes.

[NekR]

@GGAlanSmithee I think it makes sense to keep this open to pull the information from to the docs someday, but thanks for reminding me about this issue :-)

offline-plugin indeed now generates its own caches for assets and redownloads them only if they truly changed (bytes changed). This doesn't solve the problem that webpack sometimes changes bundles' content when it shouldn't.

[NekR]

Closing. If any one is wishing to transfer some bits of this discussion to docs -- feel free to do so.