Application profiling #8695

marcins · 2022-12-16T03:01:33Z

↪️ Pull Request

This is an implementation of the Tracing RFC - more detail on the background and motivations is in #8588. The main differences from the RFC is that the feature is called "application profiling" (and the existing CPU profiling is referred to as "Sampling Profiling").

Instrumentation has been added for:

high-level phases - building, bundling, packaging
transform (per-plugin)
Babel transform (per-babel plugin visitor)
bundling (bundle / optimize)
packaging (package / optimize)
other plugin types (resolving, naming, etc..)

🚨 Test instructions

An app profile can be generated by starting Parcel with --profile-application.

✔️ PR Todo

Inject the profiler for plugins
Create a low-overhead instrumentation method
Added/updated unit tests for this change
Filled out test instructions (In case there aren't any unit tests)
Included links to related issues/PRs

See also documentation PR: parcel-bundler/website#1075

packages/transformers/babel/src/babel7.js

packages/core/core/src/PackagerRunner.js

packages/core/core/src/Transformation.js

packages/core/core/src/requests/PathRequest.js

packages/core/profiler/src/ApplicationProfiler.js

packages/core/core/src/requests/ParcelBuildRequest.js

marcins · 2023-03-02T22:03:49Z

I've finally gotten to addressing the PR feedback from late last year. The main changes have been:

rebasing on latest v2 branch
“injecting” a PluginApplicationProfiler similar to PluginLogger for plugins to ensure the same instance is used
instrumenting the remaining plugin types in core

What’s still missing:

adding some test coverage to the new functionality

lettertwo

😍

…ion-profiling

devongovett · 2023-04-01T16:41:17Z

This will be a really useful feature! My only question is around naming. I'm not sure what "application" refers to here. It's not really profiling the user's application so much as it is profiling Parcel. It seems more like a "trace" to me. Maybe we could have --profile and --trace as flags, and trace or tracer as the object passed to plugins? What do you think?

Also rename: - @parcel/reporter-application-profiler -> @parcel/reporter-tracer - PluginApplicationProfiler -> PluginTracer - cli option `--profile-application` -> `--trace` - PluginOptions.applicationProfiler -> PluginOptions.tracer

lettertwo · 2023-04-04T18:59:41Z

Renamed ApplicationProfiler -> Tracer

Also renamed:

@parcel/reporter-application-profiler -> @parcel/reporter-tracer
PluginApplicationProfiler -> PluginTracer
cli option --profile-application -> --trace
PluginOptions.applicationProfiler -> PluginOptions.tracer

lettertwo · 2023-04-04T19:04:47Z

@devongovett I think the rename is good, but with one small point of confusion: The SamplingProfiler has an accompanying Trace that is now very similar in name to Tracer (formerly called ApplicationProfiler. Should we rename Trace to something that ties it to SamplingProfiler?

devongovett · 2023-04-05T00:33:49Z

That's not exposed externally to parcel though right? If not then doesn't seem like too big a deal.

devongovett · 2023-04-17T15:46:56Z

packages/core/workers/index.d.ts

@@ -11,6 +11,7 @@ export type FarmOptions = {
  workerPath?: FilePath,
  backend: BackendType,
  shouldPatchConsole?: boolean,
+  shouldProfileApplication?: boolean,


Suggested change

shouldProfileApplication?: boolean,

shouldTrace?: boolean,

devongovett · 2023-04-17T15:50:48Z

packages/transformers/babel/src/babel7.js

@@ -74,6 +76,28 @@ export default async function babel7(
    },
  };

+  if (tracer.enabled) {
+    config.wrapPluginVisitorMethod = (


Does this wrap visiting each individual AST node, or just once for each plugin? How expensive is that?

Does this wrap visiting each individual AST node, or just once for each plugin? How expensive is that?

I just ran two full production Jira builds back-to-back, one with tracing & one without (both with --no-cache). The timings were 18m 29s without tracing, and 19m 1s with tracing (on an M1 Mac). So minimal overhead for tracing in general.

The trace file produced is approximately 1.4GB - can't load it directly in the Perfetto UI (the WASM version), but works fine via the trace_processor CLI tool.

There are approximately 2.9m Babel transform entries - so yes, my understanding is that it wraps every visitor call for every AST type, but as you can see the overhead of this is relatively minimal.

…ion-profiling

mischnic reviewed Dec 16, 2022

View reviewed changes

packages/transformers/babel/src/babel7.js Outdated Show resolved Hide resolved

lettertwo requested changes Dec 16, 2022

View reviewed changes

lettertwo self-requested a review December 16, 2022 18:36

mattcompiles reviewed Dec 18, 2022

View reviewed changes

packages/core/core/src/requests/ParcelBuildRequest.js Outdated Show resolved Hide resolved

packages/core/core/src/requests/ParcelBuildRequest.js Outdated Show resolved Hide resolved

marcins added 24 commits March 2, 2023 09:20

Move profiler to new @parcel/profiler pkg

256072e

Refactor Profiler -> SamplingProfiler

4809e23

Initial implementation of ApplicationProfiler

523330d

Plumb in applicationProfiler to child/worker/reporter system

56a5336

Add initial application profile reporter

64b5887

Implement CLI integration for application profiler

cc8036f

Implement profiling for packaging

643bf9b

Implement profiling for bundling

ddce2f3

Implement enabling app profiling in workers

be15f38

Implement profiling for Babel transforms

bbd7f61

Fix race condition with closing trace writeStream

3b0a3a6

Implement profiling for top level phases

097cd15

Implement profiling for resolution

53425d6

Implement profiling for optimization

6984ff3

Implement profiling for bundle naming

a69f537

Bump profiler to 2.8.3 to match rest

5520673

Refactor to use PluginApplicationProfiler

5e75260

Refactor createMeasurement to avoid redundant obj allocations

4acce7c

Document PluginApplicationProfiler API

bda0790

Implement injection of PluginApplicationProfiler

be724e1

Implement reporting for reporters..

7b63807

Implement application profiling for applyRuntime

7113a75

Add application profiling to compressor

62b5530

Fix an errant relative path

66fffa1

marcins force-pushed the mszczepanski/application-profiling branch from 9809022 to 66fffa1 Compare March 2, 2023 22:00

marcins added 3 commits March 6, 2023 13:50

Move transform measurement to better location

d111d17

Minor improvements to some measurements

5e1e4b6

Prevent double-end of measurement

afeab07

marcins mentioned this pull request Mar 7, 2023

Initial docs for app profiling parcel-bundler/website#1075

Merged

marcins added 2 commits March 8, 2023 15:47

Improve profile reporter

89695ef

Add basic integration test for app profiling

b1c5af1

marcins marked this pull request as ready for review March 8, 2023 23:25

lettertwo approved these changes Mar 16, 2023

View reviewed changes

marcins changed the title ~~[WIP/Draft]: Application profiling~~ Application profiling Mar 16, 2023

marcins added 3 commits March 23, 2023 08:18

Merge remote-tracking branch 'upstream/v2' into mszczepanski/applicat…

96566ac

…ion-profiling

Merge remote-tracking branch 'upstream/v2' into mszczepanski/applicat…

d70dd06

…ion-profiling

Fix linting issues

ae25e62

lettertwo self-requested a review March 30, 2023 16:10

lettertwo approved these changes Mar 30, 2023

View reviewed changes

thebriando approved these changes Mar 30, 2023

View reviewed changes

Merge branch 'v2' into mszczepanski/application-profiling

0b91a04

lettertwo added 3 commits April 4, 2023 11:41

Merge branch 'v2' into mszczepanski/application-profiling

b8d4ec4

Rename ApplicationProfiler -> Tracer

3c45509

Also rename: - @parcel/reporter-application-profiler -> @parcel/reporter-tracer - PluginApplicationProfiler -> PluginTracer - cli option `--profile-application` -> `--trace` - PluginOptions.applicationProfiler -> PluginOptions.tracer

Merge branch 'v2' into mszczepanski/application-profiling

322e740

devongovett approved these changes Apr 17, 2023

View reviewed changes

marcins and others added 3 commits May 10, 2023 15:01

Merge remote-tracking branch 'upstream/v2' into mszczepanski/applicat…

7621579

…ion-profiling

Fix some missing shouldProfileApplication -> shouldTrace

5869002

Merge branch 'v2' into mszczepanski/application-profiling

414d499

devongovett approved these changes May 12, 2023

View reviewed changes

devongovett merged commit 482d23a into parcel-bundler:v2 May 12, 2023

mischnic mentioned this pull request Oct 27, 2023

High level profiling / tracing #8588

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Application profiling #8695

Application profiling #8695

marcins commented Dec 16, 2022 •

edited

Loading

marcins commented Mar 2, 2023

lettertwo left a comment

devongovett commented Apr 1, 2023

lettertwo commented Apr 4, 2023

lettertwo commented Apr 4, 2023

devongovett commented Apr 5, 2023

devongovett Apr 17, 2023

devongovett Apr 17, 2023

marcins May 11, 2023

Application profiling #8695

Application profiling #8695

Conversation

marcins commented Dec 16, 2022 • edited Loading

↪️ Pull Request

🚨 Test instructions

✔️ PR Todo

marcins commented Mar 2, 2023

lettertwo left a comment

Choose a reason for hiding this comment

devongovett commented Apr 1, 2023

lettertwo commented Apr 4, 2023

lettertwo commented Apr 4, 2023

devongovett commented Apr 5, 2023

devongovett Apr 17, 2023

Choose a reason for hiding this comment

devongovett Apr 17, 2023

Choose a reason for hiding this comment

marcins May 11, 2023

Choose a reason for hiding this comment

marcins commented Dec 16, 2022 •

edited

Loading