loadable not working in SSR with webpack module federation - #640
Comments
|
Hey @ajayjaggi 👋, |
|
Hi. i saw this PR https://github.com/gregberge/loadable-components/pull/638/files and used these fixes to run loadable in my local along with webpack 5. But further in SSR, wrapping components with loadable breaks the flow...Error is published above. |
|
Hello @ajayjaggi, did you try to use correct key for chunkExtractor.js? example return {
- cleanFilename,
+ filename: resolvedFilename,
... |
|
Thanks for this.I used the wrong key. |
|
Did you use webpack |
|
yes for both the websites ie Edge and Demand i have used LoadablePlugin in the webpack config for client side. |
|
https://github.com/ajayjaggi/MicroForntEnd-Basic-Structure - this is the PR for project. |
|
It may take a while before it will start working properly, but 🤞 |
|
Hi @StephaneRob / @theKashey ,can we connect over a call to discuss the issue ? We can try to find the solution together. |
|
@ScriptedAlchemy is the boss in MF lands. We all need his help, not my help. |
|
Each host has a name. Webpack usually appends a name to the remote containers. So they get their own name. Webpack accesses the runtime via window.app1.get. I'm not sure if jsonp matters because they all get namespaced. It would have to be tested but I use loadable and do not have problems. However I'm using MF not standard entry-point runtimes which execute differently. |
|
Quoting from a comment I left on recently merged PR: If you have the remotes on the page immediately. Like hardcoded. All remote code should transport down in a single RTT. If you want to SSR the chunks a remote requires: loadable needs a new way to map names of remotes to federated chunk maps. The only Id it will have is ./someExposedModule - and we would need to go over the remotes stats and find what the browser side script is for that module. Same problem exists in next js. The loadable method straight crashes. You could also use partial hydration and simply hydrate markup when visible. I don't foresee this being hard to accomplish. Especially since I can extend MF apis at will. We could even change the getter on the remote so whenever a server gets code from the remote, it's container will report what request was asked for from the remote. It actually could be done without and need for Babel at all, or even loadable HOC. The only part we would need to adapt is ChunkExtractor to read from the second scope the remote containers are accessing directly. This mechanism would work exactly like react-universal does. Push right into a map, but we will be doing this inside the webpack runtime directly. MF should not be a complex problem to render correctly on. You don't need to use loadable as a wrapper 👆 we only need to flush chunks into it from webpack runtime. |
|
@theKashey how can I import and push extra chunk names into your context. How can we have loadable read the remote containers chunks. I can have webpack infer this from the plugin. What we could do is provide a loadableFederationPlugin which passes the object to MF but we can read the info. Then we would have the paths to all remotes and can require the chunk stats upfront. I could write to a map you expose directly in webpack. As the runtime attaches, it sends context to the host. The "hard" part on the webpack side is emitting the stats and reducing it down to simple maps. If I'm able to push chunk maps and the module as it's required at runtime. This would make it much simpler. Then I wouldn't need to provide stats upfront but could push them into loadable as the remote attaches on its own. |
|
Okay, I'm on my iPad to cant really code much. Here's what the startup code will look like. (Check module federation examples/startup-code) to see it configured in webpack. This entrypoint would be part of webpack-imported, and its added to webpack with something like SingleEntryPlugin or EntryPlugin (or mutation of options, like u do in next) We can also use normal require and stuff in here, its just a entrypoint. As i use federated import(scope/request), RemoteModule will perform global.[scope].get(request) Since we own the getter, its going to push into a global map. We can then read config from MF plugin to get paths to remote container if we need. ImportedFederatedPlugin({ Now you've got the options passed to MF. So we can infer name, and know how to find the container to get the exported client chunks off it. You can also use, in the server. So chunkExtractor would go: remoteChunks = webpack_require(require.resolve(remote)).ClientChunks Alternatively, we could attach to scope during initialization Inside loadable, you'd pretty much want to prevent the typical babel transform. Instead we use webpack directly. We can use require if needed for server transform, then webpack will hoist to app boot time, i think. Lastly, we could use the existing HOC to push ones actually rendered, depending on if the require function pull the chunks right away. Ideally if we can preloadAll or something it might be better. There's likely some details to work out but this will work. |
|
hi @ScriptedAlchemy, i was very curious about finding the solution to the problem. If we could talk and you could guide me through, i would be very happy to contribute. |
|
There's other issues that block webpack support besides this :/ But I'll push a branch somewhere soon that you can look at |
|
Hi @ScriptedAlchemy , If you could share the branch it would be great help |
|
I'm building most of this capability into the module federation dashboard. I'll backport so capabilities to standalone |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
|
|
webpack 5 support was already PR-ed - #676, I am waiting for a little free time to get it merged. |
|
Amazing, I should have poked around the branches a bit more, I didn't see it linked here 🤷. Thanks again and thank you for all the fantastic work thats gone into this project!! |
Any progress? |
|
Any progress on this? Seems to break razzle builds (hash mismatch on production build) |
|
Watching this issue as well. I also created a reproduction repo. README has instructions on how to run locally. A very simple setup where remote exposes
@ajayjaggi curious if you found a workaround? |
|
Any progress? I get the same error |
|
I believe I found a working approach, going to compile a solution soon. Not sure if it's a right place for it, but during loadable constructor calls (aka loadable(...)), it's possible to collect a global object with [chunkName]: importAsync calls. Then loadableReady will wait for Promise.all([...]), for each external chunk listed in __LOADABLE_REQUIRED_CHUNKS___ext.namedChunks |
|
Okay, got it working both server- and client-side. Got a question on which strategy do we take server-side? |
|
@theKashey @ScriptedAlchemy Uploaded a PR with loadableReady turned into an async bootstrap. A lot of things still need to be tested, but it works in my setup. Also I did not actually "build" loadable yet. Just modified it inside node_modules. Then accurately copied the solution to the repository. Will soon check if the built version is working properly. I also didn't manage to collect all the needed chunks for wmf bootstrap + shared + loadable. Aka if I wrap ./bootstrap with loadable it throws on shared not available for eager consumption. If I don't wrap it, bootstrap chunk is not registered with loadable. That has been reported before #690 |
|
There is a little unknown moment around the way loadable names chunks - it has to be deterministic and the names has to be unique.
|
That can be listed in caution for wmf. chunks should probably include [remoteName] section in webpack. We can also try binding resolution to manifest files generated by webpack, if the problem arises. I’ll try production builds, to see if the mapping is done right. Added to todo list.
Don't see a problem here, we render the them all together, so they should be properly flushed.
I only request chunks gathered during SSR and passed to named chunks, so it shouldn't be a problem. Worth checking if that is true thou. Added to todo list. |
|
Lol, found the problem with shared modules, its kinda laughable. WMF requires bootstrap to load eager dependencies, such as React. And loadable must wrap bootstrap, to gather chunks, which requires react. On top of that, loadable is only meant to be used inside components, so webpack bootstrap pretty much breaks the concept. --- edit So far, I don't see how loadable can consume WMF`s import('./bootstrap'). That could become a show-stopper. If that's what you mean under not working federated chunks flushing, then I pretty much understand it now |
|
We currently have two blockers, each with it's own proposed solution: Problem 1. WMF eager static imports are generating chunks that aren't flushed by loadable. Solution: Webpack uses async import boundaries to take its time and preload all the eager static imports. Meaning the dependencies are kept somewhere. WMF turns them into Promise.all like this: Maybe we could patch webpack to add eager import dependencies to compilation.stats? If it is not already possible by mapping chunk children to origin[].moduleId starting with "webpack/sharing/consume", like: Problem 2. Top level async boundary - bootstrap is not flushed. Loadable is meant to process react components, there were no need to support regular imports. So we can't just Solution: basically "bootstrap" or top level async boundary is a synchronous pattern, there should be no "conditional" rendering to it compared to loadable components, thus we can just tell loadable to treat that chunk as one of those always required dependencies. To do so, I suggest introducing So far all of this seems doable. --- edit 1 Spent whole day, trying to build some dependency graph for shared eager module from compilation.stats.chunk.origins, then from raw compilation.stats.modules, but I believe that is impossible, unless, well parsing module "webpack/sharing/consume/**" nested reason->children fields to find dependencies. |
|
I parse the MF plugin options, so i know everything about that plugin and if its shared or not. I track what's loaded per request, so i dont load anything additional other than the execution tree. Preloading only 3-4 files that were actually used. For vendors, in webpack they have a shared module type, so i can look at the stats of graph and find all the ids, chunks they live in, how its referenced. This is a large portion of the magic that makes it work in next.js - might be some applications here, at least conceptually. The module id is a placeholder id, so you have to look past that key and know what's the chunk associated with it. Async import bootstrap also solved many issues, but you'd still want to flush chunks out of SSR for maximum performance https://gist.github.com/ScriptedAlchemy/7c1c7b25665524fbb0dfb4f06db7ebff |
|
These shared modules, you'd want to know what came from where. I do this by tracking the initialization scopes which include what, and from who. |
|
That's actually great news, meaning I'm on the right track. Or even on the right paved road!
Got that part from webpack stats as well, it has all the info about consumes and provides
That's the magic done by loadable, don't really plan to take that bread, loadable tracks rendered components very well, only need to figure out how to let it track async initialization.
That's what I spent whole yesterday doing, and only by the end of the day discovered there is a "reason" section inside webpack/sharing/consume "modules", that tells who uses what.
Yep, that's the info I also got from the webpack.stats |
|
Parsing reasons is pretty common. Another thing to watch for is if it's shared code, you want to put it in the right parents in the map. So like react, I'd push its chunk into the array of files needed at entry point level. Otherwise things could double load. |
Did not reach that part yet. I planned to extract shared provides and shared consumes by chunks, and put them into loadable-stats. After that find a way to require shared without duplication. That approach will also simplify mf orchestration, since I can join multiple loadable-stats and dedupe shared even further. |
|
Okay, been off the radar for a week doing other things, but now I'm back. Finished working on Problem 1.: extracting WMF shared consumes and attaching them to chunks. Now chunks section is extended with sharedConsumes: That in turn are listed in sharedModules section. All the data is extracted purely from stats.json via loadable/webpack-plugin (Updated PR: #885) The next step is Problem 2: Telling loadable to extract async modules between entrypoint and App. As soon as I started thinking about that problem I've hit a multiple choice decision. Prerequisites:
So, there are at least 3 possible ways to proceed: Choice 1: Using webpack stats, I can find all the async chunks between entrypoint and every loadable. Assuming the application only has one path from entrypoint, that would allow gathering these async chunks automatically with no extra boiler code; Choice 2: I could add loadable.bootstrap(() => import('./bootstrap')), this way we know exactly which path to take, or do we? Choice 3: We could have a loadable wrapper at application root, this way we would definitely know which chunk contains the root of application. This solution does not involve much changing to loadable mechanics, since ChunkExtractor can already work well with react components I don’t like choice 1 as it becomes uncontrollable and unpredictable. Choice 3 has a rather bad limitation. So I favor the 2nd choice. We could also combine 2 and 1. Aka use bootstrap to tie down entrypoint with application root. But build the path from entrypoint to the root via webpack stats, following available client-bundle bootstraps. |
|
Did some analyzing of @loadable/babel-plugin contents and to be honest, I'm lost. From the way it is currently implemented, I am not sure what needs to be done next. If we agree choice 2 is is the optimal way, we just need to add new loadable.bootstrap function. That function would have to be transformed towards the same path loadable components operate. Thing is, I don't need to actually change the async chunk with a component, not even sure I need to meddle with the chunk name. But I do need to somehow mark that import or bind it to the the react app part. And that binding has to work even if there are no loadable components on the other side. Tinkering with babel-plugin AST is complicated enough as it is, not to go around experimenting, trying to figure out what has to be done. One more thing: I don't want to turn loadable-components into imported-components, so these async boundary chunks must be applied during render, not import. Or as other choices state, we can use static chunk tree analysis instead. |
Covering edge cases, I got the following results:
Going deeper into the current goals for choice 2 solution:
So from what I see, we can do something like: LoadableBootstrap__src_components_index_jsx for path-related keys and "LoadableBootstrap_some_key__src_components_index_jsx" for keys. Alternatively, I'm not sure if we need to rename chunks at all. Why can't we just gather them during thenable call with their given names/keys. It's not like we actually need to do the loadable magic for sync/async call later on, they are only needed for the flushing. I'll spend some more time clearing out why's and how's chunkName replacement works in loadable/babel-plugin and then try to implement that bootstrap utility. |
|
Got some great news:
What's left to do:
--- edit 1 Solution b. seems most obvious, as it does not obstruct the normal chunk generation flow and uses same mechanics applied by other parts of loadable. Guess I'll start from here. --- edit 2 |
|
Finally finished the async bootstrap chunk registration, It was harder than I thought and required me to do a little hack with unused Now switched to enhancing ChunkExtractor, to make it properly use loadable stats for registering both bootstrap- and wmf shared chunks. @theKashey ever thought about rewriting loadable-stats, so that it only contained fields required by loadable? We could reduce the load on ChunkExtractor and remove a lot of unnecessary data in the stats file without breaking backwards compatibility (except for those, who process stats manually outside ChunkExtractor). If somebody needs some extra data in loadable-stats, they should opt-in for them via configuration. Also quite literally holding myself back from rewriting ChunkExtractor towards respect for SingleResponsibility. |
I also hold the same idea for a long time - #778 (comment) It was "reduced" multiple times:
And increased as well: But we are subject for hurim law and should just create another version of Chunk Extractor with the another "API"/"Goal" set initially ( well, https://github.com/theKashey/webpack-imported )
And probably the same can and should be applied to the loadable-component themselves. I don't feel that the current model is actually clicking with Federation and one might need to find another model. No changes to the MF is required. "Loadable" should just discover federated modules(you did it) and then discover "Loadable" inside them. Repeat until success. |
It feels overwhelmingly wrong to create another plugin, just to allow loadable to simplify stats. It also feels wrong to create 2 stats files by loadable. So what I had in mind, was major release, with a new flag that allows for minimum stats, eventually switching that flag defaulting to minimal stats. For now, It feels I'm violating pioneers rule, by adding 2 new methods to already overbloated ChunkExtractor class, I'm leaving something in worse state then it was. Guess that's what open source is like, your every step feels like walking on a mine field :) There is no problem with current loadable-stats config in terms of data sufficiency. But we could optimize a lot of things on the way. Welp, guess not this iteration. |
Unfortunately, stats generated by loadable as over and misused already. Releasing a major version of plugin and introducing the breaking change in the API might break some integrations. So this can be done only with recommended replacement, a clear migration guide, and the minimal API surface from day 1, as switching that flag defaulting to minimal stats will introduce another breaking change. Thus I want to ask you - what exactly you need to change? What if in any circumstances the best way forward is creating a special MF-friendly version of ChunkExtractor and Loadable.bootstrap code? |
|
Let's move that discussion to the moment when initial implementation is done in pull request. Need to focus on the task first, then we'll see if some suggestions are worth doing. As for your questions:
|
|
Did not abandon the project :) Just got my hands busy elsewhere. First for the bad parts:
To sumarize, the goal yet again got pushed away by a few weeks of development :) |
Unfortunately, I just spent half of my day realizing that this happens when you use webpack 5 or above. Once you downgrade back to webpack 4 it functions normally. |
|
Razzle is quite unstable with wp5, needs help with rewrite btw ;) |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
How I got this to work for next.js was to embed the loadable stats into the remote entry’s runtime. So I have get,init,chunks - then at runtime I’ll collect the chunks from each remote that it’s exporting, then merge them with the known loadable manifest. For module ids. I didn’t have to worry about that, they were registered as the ids from the other container and merging the manifests at runtime provided the lookups I needed. Ontop of that, I’ve also changed how remote module is loaded, so when a remote modules get or init apis are called, I’ve got hooks into that and can see “who loaded what” so as it’s executing webpack is pushing requests into a queue that I can flush after render by getting the remotes chunk maps with global.remote.chunkMap |
|
The hook "who loaded what" seems like some new webpack event? Gave this whole thing a little thought and what's really holding the implementaion back - is WMF awesome async boundary, that just loads whatever it needs (hardcoded as Promise.all in resulting bundle). If that thing could be exposed, it would make things much easier. Also I don't see how that could be done during SSR (executing webpack), as the eager chunks are not the same for csr and ssr. |
🐛 Bug Report
loadable-components: failed to synchronously load component, which expected to be available { fileName: './src/shared/dedicated/index.js',
chunkName: 'Dedicated',
error: 'Cannot read property 'call' of undefined' }
(node:7562) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'call' of undefined
at webpack_require (/Users/ajay/Desktop/WebStrom/MicroFrontend/Edge/dist/server/main.js:394:42)
at Module../src/shared/components/Footer/index.js (/Users/ajay/Desktop/WebStrom/MicroFrontend/Edge/dist/server/Dedicated.server.js:21:71)
at webpack_require (/Users/ajay/Desktop/WebStrom/MicroFrontend/Edge/dist/server/main.js:394:42)
at Module../src/shared/dedicated/index.js (/Users/ajay/Desktop/WebStrom/MicroFrontend/Edge/dist/server/Dedicated.server.js:60:76)
at webpack_require (/Users/ajay/Desktop/WebStrom/MicroFrontend/Edge/dist/server/main.js:394:42)
at Object.requireSync (/Users/ajay/Desktop/WebStrom/MicroFrontend/Edge/dist/server/src_server_render_js.server.js:225:14)
at InnerLoadable.loadSync (/Users/ajay/Desktop/WebStrom/MicroFrontend/Edge/dist/server/vendors-node_modules_loadable_server_lib_index_js-node_modules_express_index_js-node_modules_-ee7ccd.js:420:35)
at new InnerLoadable (/Users/ajay/Desktop/WebStrom/MicroFrontend/Edge/dist/server/vendors-node_modules_loadable_server_lib_index_js-node_modules_express_index_js-node_modules_-ee7ccd.js:315:17)
at processChild (/Users/ajay/Desktop/WebStrom/MicroFrontend/Edge/dist/server/vendors-node_modules_loadable_server_lib_index_js-node_modules_express_index_js-node_modules_-ee7ccd.js:56603:14)
at resolve (/Users/ajay/Desktop/WebStrom/MicroFrontend/Edge/dist/server/vendors-node_modules_loadable_server_lib_index_js-node_modules_express_index_js-node_modules_-ee7ccd.js:56568:5)
(node:7562) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:7562) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
To Reproduce
routes.js -
import React from 'react'
import loadable from '@loadable/component'
// import Home from './home'
// import Dedicated from './dedicated'
const Home = loadable(() => import(/* webpackChunkName: "Home" / './home'))
const Dedicated = loadable( () => import(/ webpackChunkName: "Dedicated" */ './dedicated'))
const homeRoute = (path) => ({
path,
exact: true,
component: Home
})
const dedicatedRoute = (path) => ({
path,
exact: true,
component: Dedicated
})
export default () => [
homeRoute('/'),
dedicatedRoute('/:player(messi)')
]
Expected behavior
A clear and concise description of what you expected to happen.
Link to repl or repo (highly encouraged)
https://github.com/ajayjaggi/MicroForntEnd-Basic-Structure
Issues without a reproduction link are likely to stall.
The text was updated successfully, but these errors were encountered: