Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(dashmate)!: speedup dashmate command #1916

Open
wants to merge 324 commits into
base: master
Choose a base branch
from

Conversation

shumkov
Copy link
Member

@shumkov shumkov commented Jun 29, 2024

Issue being fixed or feature implemented

Dashmate commands are very slow which is very annoying in day-to-day use. The most time is taken by yarn and importing modules (not actual work).

What was done?

Benchmark 1: ./scripts/dashmate setup (new dev)
  Time (mean ± σ):     887.8 ms ±  19.1 ms    [User: 949.7 ms, System: 111.3 ms]
  Range (min … max):   875.3 ms … 940.0 ms    10 runs
  
Benchmark 2: yarn dashmate setup (old dev)
  Time (mean ± σ):      2.715 s ±  0.075 s    [User: 2.981 s, System: 0.318 s]
  Range (min … max):    2.600 s …  2.836 s    10 runs
 
Benchmark 3: dashmate (new production package)
  Time (mean ± σ):     647.6 ms ±   4.0 ms    [User: 690.5 ms, System: 49.9 ms]
  Range (min … max):   640.0 ms … 652.6 ms    10 runs
 
Summary
  dashmate (new production package) ran
    1.37 ± 0.03 times faster than ./scripts/dashmate setup (new dev)
    4.19 ± 0.12 times faster than yarn dashmate setup (old dev)
  • Incorporated esbuild to bundle dashmate in order to reduce imports
  • Used explicit definition of the Oclif commands (dashmate/src/index.js) to allow bundling
  • Changed the Awilix injection mode from classic to allow bundling
  • Updated yarn to latest version in order to fix an issue with esbuild
  • Replaced unmaintained ultra runner to yarn workspaces plugin
  • Removed the @dashevo/utils dependency from dash-spv due to tons of deprecation warnings after bundling
  • Introduced ./scripts/dashmate to avoid of using yarn .. to run scripts, which was 20% slower
  • Removed @oclif/plugin-help which needs to generate extended help for commands (development purposes)
  • Moved dashmate volume dump scripts to dashmate package
  • Enabled oliff development mode (with verbose logs and stack traces) when DEBUG=\* env is passed
  • Updated dashmate packaging script to leverage bundling

How Has This Been Tested?

  • Building dashmate locally and running commands
  • Tests are updated
  • CI is using bundled dashmate too

Breaking Changes

It shouldn't be any difference for end user, but developers needs to know:

  • Instead of using yarn dashmate .. now you need to use ./scripts/dashmate from the platform repo root.
  • To use dashmate it needs to be built (yarn build). During dashmate development, it is better to call it from packages/dashmate so you won't constantly rebuild everything.

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
  • I have made corresponding changes to the documentation if needed

For repository code-owners and collaborators only

  • I have assigned this pull request to a milestone

shumkov and others added 30 commits March 8, 2024 19:42
shumkov and others added 28 commits July 9, 2024 15:12
# Conflicts:
#	packages/dashmate/src/listr/tasks/setup/setupRegularPresetTaskFactory.js
Temporary disabled due to flakiness. These tests aren't important for now, since we are
//  going to release v1.0.0 with withdrawals disabled.
Temporary disabled due to flakiness. These tests aren't important for now, since we are
//  going to release v1.0.0 with withdrawals disabled.
# Conflicts:
#	packages/dashmate/src/listr/tasks/setup/setupLocalPresetTaskFactory.js
#	packages/dashmate/src/listr/tasks/setup/setupRegularPresetTaskFactory.js
# Conflicts:
#	packages/platform-test-suite/test/e2e/withdrawals.spec.js
#	scripts/configure_dotenv.sh
#	yarn.lock
@shumkov shumkov changed the base branch from v1.0-dev to master August 25, 2024 06:05
@shumkov shumkov requested a review from antouhou as a code owner August 25, 2024 06:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet