Allow thread-loader configuration
#795
Conversation
Currently `thread-loader` is not configurable, meaning that in some CI environments where the OS reports a very large number of CPUs, `thread-loader` will create a very large worker pool. The only configuration that is currently available is setting `JOBS=1`, but the pool warmup call doesn't check `JOBS`, so always spawns the full number of workers. So this commit: * Skips warming the worker pool if `JOBS=1` and we know we won't be using `thread-loader` * Adds a `thread-loader` configuration object to the options interface that allows applications to pass their own `thread-loader` configuration, or `false` to disable the use of `thread-loader` entirely.
|
The context around this (for me) is that my embroider builds are failing in CI because the containers have 4GB of RAM, and that gets used up quickly spawning 35 or however many workers (which is unavoidable currently because of the worker pool warming). Just the change for But if there are any concerns about the configuration/options change, I'd be happy to split the change where |
| @@ -76,6 +66,7 @@ const Webpack: Packager<Options> = class Webpack implements PackagerInstance { | |||
| private extraConfig: Configuration | undefined; | |||
| private passthroughCache: Map<string, Stats> = new Map(); | |||
| private publicAssetURL: string | undefined; | |||
| private extraThreadLoaderOptions: object | false | undefined; | |||
There was a problem hiding this comment.
maybe replace object with ThreadLoaderOptions Type
There was a problem hiding this comment.
I couldn't find types for thread-loader anywhere, and wasn't sure we'd want to sign up for maintaining our own type. We certainly could though...
There was a problem hiding this comment.
Oh, I couldn't find either, nvm..
|
Thanks for digging into this, this looks great. |
rwjblue
changed the title
thread-loader configuration
Currently
thread-loaderis not configurable, meaning that in some CI environments where the OS reports a very large number of CPUs,thread-loaderwill create a very large worker pool.The only configuration that is currently available is setting
JOBS=1, but the pool warmup call doesn't checkJOBS, so always spawns the full number of workers.So this commit:
JOBS=1and we know we won't be usingthread-loaderthread-loaderconfiguration object to the options interface that allows applications to pass their ownthread-loaderconfiguration, orfalseto disable the use ofthread-loaderentirely.