How to improve home page performance (merge chunks) after using module federation on library sharing? (Maybe HTTP2 is not so fine)

[stoneChen]

At first, this is not a bug. The webpack version I use: 5.88.1.

I work in a company who's business logic of products are very complex. Micro-Frontend architecture is quite common in my company, making module federation (MF) a suitable choice for us. Thanks MF.

We've been using MF for over a year now, and it has worked seamlessly for sharing React components across multiple projects, albeit without any shared libraries.

This year, our focus is on enhancing the performance of various products spanning multiple projects through Micro-Frontend. We have started sharing UI components in Micro-Frontend across these projects, with each component being packaged as an isolated npm package. To enable the sharing of any component, webpack(MF plugin) have split the code of each component into separate chunks. Initially, we didn't see any issues with having numerous chunks, as we believed in the power of HTTP2 (for HTTP2 multiplexing).

We conducted multiple tests in both local and online user environments, collecting and analyzing performance data. Our findings indicated that a smaller number of chunks significantly improves performance.

To address this, we made efforts to merge chunks using the optimization.splitChunks.cacheGroups option. However, it seems that when sharing components with Module Federation, the code splitting for these components takes precedence. We tried various options, but the results were not as expected.

One day, we found that eager option (of MF shared option) may help us. After setting eager: true for all the components the home page of a product(project), the number of chunks reduced noticeably (e.g. index.xxxx.js gets larger), resulting in improved performance.

However, we noticed that the MF remoteEntry.js file also increased in size. After a deep dive into the eager option documentation, we realized this was expected but not desirable for us. We consider remoteEntry.js as a resource list, similar to an HTML role, and expect it to have a small code size. Consumer projects load remoteEntry.js from another provider project using a timestamp (e.g., remoteEntry.js?t=160xxxxx) or with a no-cache HTTP header to access the latest code. While other resources (like index.[contenthash].js) use long-term caching from CDN. If remoteEntry.js becomes too large, it could negatively impact performance.

Just like this demo: https://github.com/module-federation/module-federation-examples/tree/master/shared-routes2/app2, after building in the dist directory, remoteEntry.js and main.js both have react and react-dom source code.

It appears that eager: true may not be the ideal solution for us to enhance performance, and using a no-cache strategy for remoteEntry.js may not be the best practice.

Then, We want to build two versions of source business code, one version with eager: true and without exposes, another one version without eager: true and with exposes, maybe it will work, but it is too heavy.

Could someone provide some advice on this matter? Thank you!

This image helps illustrate the usage of MF in our company:

[stoneChen]

Related webpack discussion: webpack/webpack#17697

[RussellCanfield]

I believe there are some reliability issues with using query strings to cache bust, ideally you should deploy the remotes versioned and reference those URLs, once that URL is unique you don't have to use query strings to cache bust the remoteEntry file.

Agreed on eager is not the answer here, how are you loading the remotes? Are these build time URLs in webpack or are you using dynamic remotes (or delegate modules)? The main reason I ask is because of module sharing and how that works, with dynamic remotes you are "stuck" with whatever the host's versions are of those modules. For example in the case of dynamic remotes:

Host -> lodash 1.0.0
Remote -> lodash 1.0.1

These won't share, and your remotes need to be "lodash": "1.0.0". This generally impacts the amount of files you get since you'd be sharing less

[stoneChen]

@RussellCanfield Thanks for response. If the code size of remoteEntry.js is small enough, I think using query strings to cache bust is not a big problem. Just like accessing a SPA html, the html file should be small.

The core problem is that using shared in MF splits out the components chunks(many chunks because of many components) for potentially sharing , but the application itself(e.g. home page) gets performance down.

The version not matching is not my point.

I drew a new graph and update the issue, please read it if you can. Thanks.

[ScriptedAlchemy]

you can use cacheGroups on split chunks with enforce: true on the rules, which will force shared modules into fewer chunks.

[ScriptedAlchemy]

In the future i do plan to provide some api or options to reduce chunk split granularity, i also plan to support "preferred origin" so during share module negotiations you can set preference that remotes will "pick" if it meets the needs, offering consistant origin loads.

For chunk split, you can use splitChunks or custom plugin to connectchunkandModule etc to merge it down. Id like to support common clusters like "react stuff" and so on, but likely the best will be to add onto "shared" some groupName, similar to share scope

[stoneChen]

@ScriptedAlchemy Thanks for response.
I tried again for setting splitChunks.cacheGroups, I found that eager is "positive" option while splitChunks.cacheGroups is "negative" option, and setting enforce: true or not the build result are the same.

eager is "positive" option means that if I setting package A with eager: true, then the direct and indirect dependencies of package A will ALL be included into entry chunk and remoteEntry.js chunk.

splitChunks.cacheGroups is "negative" option means that webpack give me a chance to check every module(original file) should be included in a cacheGroup.

Is that right?

I am mainly sure I can write a function to extract a chunk including the modules insteading of eager, though it is a little complex(I have written a recursive function using module.issuer which can "move" most modules into new chunk).

I realized that even though I can extract all the modules I need into a new chunk insteading of setting all the upstream package eager, but the remoteEntry.js will also load that chunk which is not necessary. I can understand why this happened which is just like non-module-federation scene.

So, I think if I want to deeply optimize chunks, maybe I should build two times(versions) severally for main application and MF exposes, though it is heavy, at present.

I am glad to wait for the new feature of chunk split!

[ScriptedAlchemy]

Eager make remote bigger, splitChunks lets us force chunks into fewer slices.

I dont recommend eager, expect for special cases. Eager moves code to the entrypoint (remoteEntry) and works similar to how a non code split app would where everything moves into the entrypoint.

There is ongoing work in here module-federation/core#1268 - where ive forked module federation to get around limitations of webpack and move development progress faster.

SplitChunks cache groups should work, ive done it in the past with success.

Now that ive forked federation i can likely control chunk grouping or add option to cluster shared modules.

timeline wise, this has some priority for me and bytedance but there are larger tasks higher in the queue, mainly unifying my ecosystem with bytedance internal infra.

From there we should be able to move as one, solving problems there will solve problems for community.

[stoneChen]

@ScriptedAlchemy Thanks, I will follow this PR.

[ScriptedAlchemy]

in the next version of module federation we will support splitChunks api so it doesnt collide anymore.

To reduce number of modules, we do plan to offer something like "chunkGroup" on shared so you can force shared modules into specific chunk groups

[levrik]

@ScriptedAlchemy Any updates on this? I'm using module federation to merge a legacy and new app into a single app. It's a bit annoying to have a lot of individual JS files downloaded for a few dependencies that are shared (React, Single-SPA, Material UI, etc) instead of having a single chunk with all these common dependencies combined.

[ScriptedAlchemy]

use splitChunks, module-federation/enhanced now works with custom chunk split rules.

[source-xm]

use splitChunks, module-federation/enhanced now works with custom chunk split rules.

How to use it, need to adjust the relevant configuration rules ?, and are there any relevant examples? :)

[ScriptedAlchemy]

think you can just use split chunks like you usually would as long as you're using federation/enhanced