Only serialize and send shared references to workers that need them #8589
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This reverts commit cf5b3ff.
3 tasks
devongovett
commented
Oct 31, 2022
| @@ -145,6 +145,7 @@ export type RunAPI = {| | |||
| getRequestResult<T>(contentKey: ContentKey): Async<?T>, | |||
| getPreviousResult<T>(ifMatch?: string): Async<?T>, | |||
| getSubRequests(): Array<StoredRequest>, | |||
| getInvalidSubRequests(): Array<StoredRequest>, | |||
There was a problem hiding this comment.
This new API is subtly different than api.getSubRequest().filter(req => !api.canSkipSubrequest(req.id)) in that it doesn't have the side effect of preserving all previous sub requests even if not used.
devongovett
commented
Oct 31, 2022
| @@ -110,7 +110,7 @@ export default class Parcel { | |||
| await resolvedOptions.cache.ensure(); | |||
|
|
|||
| let {dispose: disposeOptions, ref: optionsRef} = | |||
| await this.#farm.createSharedReference(resolvedOptions); | |||
| await this.#farm.createSharedReference(resolvedOptions, false); | |||
There was a problem hiding this comment.
Serializing the options aren't cacheable because of the MemoryFS which has a side effect in the serialize() method... 😬
Benchmark ResultsKitchen Sink ✅
Timings
Cold Bundles
Cached BundlesNo bundle changes detected. React HackerNews ✅
Timings
Cold BundlesNo bundle changes detected. Cached BundlesNo bundle changes detected. AtlasKit Editor ✅
Timings
Cold Bundles
Cached Bundles
Three.js ✅
Timings
Cold BundlesNo bundle changes detected. Cached BundlesNo bundle changes detected. |
joeyslater
approved these changes
Nov 1, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #8491
This is based on #8491, and in addition to packaging on the main thread, it also transforms on the main thread if there is only one file change which improves performance even more.
It also refactors the code a little to follow a slightly different approach: rather than avoiding creating the shared reference and needing a different way of passing the bundle graph around, it changes the way shared references are implemented in the worker farm. Instead of serializing the bundle graph upfront and always sending it to every worker, it lazily serializes and sends shared references to workers only when needed. This way, if using the main thread, no serialization occurs but the shared reference can still be accessed using the same API. In addition, if you only have 2 bundles but 8 workers, we only send the bundle graph to the 2 workers that need it rather than all 8.