Skip to content
This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Conversation

@georgesdib
Copy link
Contributor

Companion for paritytech/substrate#11149

Added benchmarking to frame_election_provider_support.

@paritytech-ci paritytech-ci requested review from a team April 11, 2022 22:09
@kianenigma
Copy link
Contributor

/benchmark runtime polkadot frame_election_provider_support

@parity-benchapp
Copy link

parity-benchapp bot commented Apr 13, 2022

Benchmark Runtime Polkadot Pallet for branch "npos-election-benchmarking" with command cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=frame_election_provider_support --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

Toolchain: stable-x86_64-unknown-linux-gnu (default)
rustc 1.57.0 (f1edd0429 2021-11-29)

Results
error: no matching package named `pallet-election-provider-support-benchmarking` found
location searched: https://github.com/paritytech/substrate?branch=master
required by package `kusama-runtime v0.9.19 (/home/benchbot/bench-bot/git/polkadot/runtime/kusama)`

ERROR: Unable to commit file ./runtime/polkadot/src/weights/

@kianenigma
Copy link
Contributor

/benchmark runtime kusama frame_election_provider_support

@kianenigma
Copy link
Contributor

/benchmark runtime westend frame_election_provider_support

@parity-benchapp
Copy link

parity-benchapp bot commented Apr 13, 2022

Benchmark Runtime Kusama Pallet for branch "npos-election-benchmarking" with command cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=frame_election_provider_support --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

Toolchain: stable-x86_64-unknown-linux-gnu (default)
rustc 1.57.0 (f1edd0429 2021-11-29)

Results
error: no matching package named `pallet-election-provider-support-benchmarking` found
location searched: https://github.com/paritytech/substrate?branch=master
required by package `kusama-runtime v0.9.19 (/home/benchbot/bench-bot/git/polkadot/runtime/kusama)`

ERROR: Unable to commit file ./runtime/kusama/src/weights/

@parity-benchapp
Copy link

parity-benchapp bot commented Apr 13, 2022

Benchmark Runtime Westend Pallet for branch "npos-election-benchmarking" with command cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=frame_election_provider_support --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

Toolchain: stable-x86_64-unknown-linux-gnu (default)
rustc 1.57.0 (f1edd0429 2021-11-29)

Results
error: no matching package named `pallet-election-provider-support-benchmarking` found
location searched: https://github.com/paritytech/substrate?branch=master
required by package `kusama-runtime v0.9.19 (/home/benchbot/bench-bot/git/polkadot/runtime/kusama)`

ERROR: Unable to commit file ./runtime/westend/src/weights/

Copy link
Member

@bkchr bkchr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Runtime changes look reasonable. Trusting @kianenigma here ;)

@paritytech-ci paritytech-ci requested review from a team April 15, 2022 06:05
@paritytech-processbot
Copy link

Waiting for commit status.

@kianenigma
Copy link
Contributor

/benchmark runtime westend frame_election_provider_support

@parity-benchapp
Copy link

parity-benchapp bot commented Apr 15, 2022

Benchmark Runtime Westend Pallet for branch "npos-election-benchmarking" with command cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=frame_election_provider_support --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

Toolchain: stable-x86_64-unknown-linux-gnu (default)
rustc 1.57.0 (f1edd0429 2021-11-29)

Results
error: Found argument '--chain' which wasn't expected, or isn't valid in this context

	If you tried to supply `--chain` as a value rather than a flag, use `-- --chain`

USAGE:
    polkadot benchmark <SUBCOMMAND>

For more information try --help

ERROR: Unable to commit file ./runtime/westend/src/weights/

@kianenigma
Copy link
Contributor

/benchmark runtime kusama frame_election_provider_support

@kianenigma
Copy link
Contributor

/benchmark runtime polkadot frame_election_provider_support

@parity-benchapp
Copy link

parity-benchapp bot commented Apr 15, 2022

Benchmark Runtime Kusama Pallet for branch "npos-election-benchmarking" with command cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=frame_election_provider_support --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

Toolchain: stable-x86_64-unknown-linux-gnu (default)
rustc 1.57.0 (f1edd0429 2021-11-29)

Results
error: Found argument '--chain' which wasn't expected, or isn't valid in this context

	If you tried to supply `--chain` as a value rather than a flag, use `-- --chain`

USAGE:
    polkadot benchmark <SUBCOMMAND>

For more information try --help

ERROR: Unable to commit file ./runtime/kusama/src/weights/

@parity-benchapp
Copy link

parity-benchapp bot commented Apr 15, 2022

Benchmark Runtime Polkadot Pallet for branch "npos-election-benchmarking" with command cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=frame_election_provider_support --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

Toolchain: stable-x86_64-unknown-linux-gnu (default)
rustc 1.57.0 (f1edd0429 2021-11-29)

Results
error: Found argument '--chain' which wasn't expected, or isn't valid in this context

	If you tried to supply `--chain` as a value rather than a flag, use `-- --chain`

USAGE:
    polkadot benchmark <SUBCOMMAND>

For more information try --help

ERROR: Unable to commit file ./runtime/polkadot/src/weights/

@paritytech-processbot
Copy link

Merge cancelled due to error. Error: Checks failed for 93125f0

@joao-paulo-parity
Copy link
Contributor

@kianenigma please try the commands again, they should work after paritytech/bench-bot#91

@georgesdib
Copy link
Contributor Author

/benchmark runtime polkadot frame_election_provider_support

@parity-benchapp
Copy link

parity-benchapp bot commented Apr 15, 2022

Benchmark Runtime Polkadot Pallet for branch "npos-election-benchmarking" with command cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark pallet --chain=polkadot-dev --steps=50 --repeat=20 --pallet=frame_election_provider_support --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

Toolchain: stable-x86_64-unknown-linux-gnu (default)
rustc 1.57.0 (f1edd0429 2021-11-29)

Results
Pallet: "frame_election_provider_support", Extrinsic: "phragmen", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + v    18.39
    + t    0.086
    + d     2866
              µs

Reads = 0 + (0 * v) + (0 * t) + (0 * d)
Writes = 0 + (0 * v) + (0 * t) + (0 * d)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    v     t     d   mean µs  sigma µs       %
 1000  1000    16     18230     30.53    0.1%
 1020  1000    16     18800     110.8    0.5%
 1040  1000    16     18960     49.17    0.2%
 1060  1000    16     19350     25.88    0.1%
 1080  1000    16     19700     34.05    0.1%
 1100  1000    16     20050     26.32    0.1%
 1120  1000    16     20420      27.3    0.1%
 1140  1000    16     20900       121    0.5%
 1160  1000    16     21180     57.94    0.2%
 1180  1000    16     21620     63.94    0.2%
 1200  1000    16     21910     60.88    0.2%
 1220  1000    16     22270     50.36    0.2%
 1240  1000    16     22630     51.61    0.2%
 1260  1000    16     22970     42.14    0.1%
 1280  1000    16     23370     67.23    0.2%
 1300  1000    16     23700     43.33    0.1%
 1320  1000    16     24130     27.38    0.1%
 1340  1000    16     24500     48.74    0.1%
 1360  1000    16     24840     39.46    0.1%
 1380  1000    16     25190      64.7    0.2%
 1400  1000    16     25570     66.59    0.2%
 1420  1000    16     25960        64    0.2%
 1440  1000    16     26300     56.06    0.2%
 1460  1000    16     26660     47.03    0.1%
 1480  1000    16     27020     70.42    0.2%
 1500  1000    16     27370     50.87    0.1%
 1520  1000    16     27740     47.45    0.1%
 1540  1000    16     28130     45.28    0.1%
 1560  1000    16     28950     159.5    0.5%
 1580  1000    16     28820     30.55    0.1%
 1600  1000    16     29320     82.96    0.2%
 1620  1000    16     29650     32.65    0.1%
 1640  1000    16     29970     37.42    0.1%
 1660  1000    16     30330     40.76    0.1%
 1680  1000    16     30690     50.27    0.1%
 1700  1000    16     31060     51.18    0.1%
 1720  1000    16     31450     68.11    0.2%
 1740  1000    16     31900     24.66    0.0%
 1760  1000    16     32190     23.28    0.0%
 1780  1000    16     32570     70.83    0.2%
 1800  1000    16     32940     53.53    0.1%
 1820  1000    16     33280     65.52    0.1%
 1840  1000    16     33680      40.3    0.1%
 1860  1000    16     34060     35.44    0.1%
 1880  1000    16     34480     56.22    0.1%
 1900  1000    16     34820     53.34    0.1%
 1920  1000    16     35130     56.13    0.1%
 1940  1000    16     35580     68.82    0.1%
 1960  1000    16     35920     47.98    0.1%
 1980  1000    16     36270     86.04    0.2%
 2000   500    16     36570     55.59    0.1%
 2000   510    16     36650     96.58    0.2%
 2000   520    16     36680     43.46    0.1%
 2000   530    16     36630      80.6    0.2%
 2000   540    16     36690     48.76    0.1%
 2000   550    16     36690     74.63    0.2%
 2000   560    16     36640     75.78    0.2%
 2000   570    16     36690      68.9    0.1%
 2000   580    16     36640     84.72    0.2%
 2000   590    16     36580     67.82    0.1%
 2000   600    16     36640     86.42    0.2%
 2000   610    16     36580     80.33    0.2%
 2000   620    16     36590     78.22    0.2%
 2000   630    16     36670     74.84    0.2%
 2000   640    16     36610     62.95    0.1%
 2000   650    16     36600     83.96    0.2%
 2000   660    16     36680     77.75    0.2%
 2000   670    16     36680     67.75    0.1%
 2000   680    16     36640     82.37    0.2%
 2000   690    16     36670     73.21    0.1%
 2000   700    16     36790     185.9    0.5%
 2000   710    16     36640     63.08    0.1%
 2000   720    16     36640     38.59    0.1%
 2000   730    16     36780     48.45    0.1%
 2000   740    16     36620     41.91    0.1%
 2000   750    16     36670     70.88    0.1%
 2000   760    16     36650     56.96    0.1%
 2000   770    16     36620     33.45    0.0%
 2000   780    16     36740     85.94    0.2%
 2000   790    16     36620     50.22    0.1%
 2000   800    16     36670     67.93    0.1%
 2000   810    16     36620     57.28    0.1%
 2000   820    16     36710     92.82    0.2%
 2000   830    16     36640     32.34    0.0%
 2000   840    16     36700      65.1    0.1%
 2000   850    16     36650     60.88    0.1%
 2000   860    16     36660     41.18    0.1%
 2000   870    16     36660     29.73    0.0%
 2000   880    16     36590     87.81    0.2%
 2000   890    16     36890     259.6    0.7%
 2000   900    16     36660     82.34    0.2%
 2000   910    16     36650     46.49    0.1%
 2000   920    16     36660     25.41    0.0%
 2000   930    16     36670     57.73    0.1%
 2000   940    16     36670     75.49    0.2%
 2000   950    16     36660     66.95    0.1%
 2000   960    16     36670     48.76    0.1%
 2000   970    16     36640     84.86    0.2%
 2000   980    16     36640     80.67    0.2%
 2000   990    16     36750     49.85    0.1%
 2000  1000     5      6532     20.45    0.3%
 2000  1000     6      8204     19.74    0.2%
 2000  1000     7      9946     27.25    0.2%
 2000  1000     8     12020     28.46    0.2%
 2000  1000     9     14900     45.72    0.3%
 2000  1000    10     17530     55.67    0.3%
 2000  1000    11     20250     48.39    0.2%
 2000  1000    12     22960     50.31    0.2%
 2000  1000    13     26020     47.65    0.1%
 2000  1000    14     29380     52.64    0.1%
 2000  1000    15     32910      55.4    0.1%
 2000  1000    16     36700     85.51    0.2%

Quality and confidence:
param     error
v         0.051
t         0.102
d         7.074

Model:
Time ~=        0
    + v    18.13
    + t        0
    + d     2953
              µs

Reads = 0 + (0 * v) + (0 * t) + (0 * d)
Writes = 0 + (0 * v) + (0 * t) + (0 * d)

Pallet: "frame_election_provider_support", Extrinsic: "phragmms", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + v    16.15
    + t        0
    + d     2621
              µs

Reads = 0 + (0 * v) + (0 * t) + (0 * d)
Writes = 0 + (0 * v) + (0 * t) + (0 * d)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    v     t     d   mean µs  sigma µs       %
 1000  1000    16     16160     42.62    0.2%
 1020  1000    16     16460     40.33    0.2%
 1040  1000    16     16800      33.8    0.2%
 1060  1000    16     17190     51.05    0.2%
 1080  1000    16     17430     37.32    0.2%
 1100  1000    16     17750     52.51    0.2%
 1120  1000    16     18070     42.76    0.2%
 1140  1000    16     18390     35.02    0.1%
 1160  1000    16     18760     43.16    0.2%
 1180  1000    16     19050     21.71    0.1%
 1200  1000    16     19370     23.46    0.1%
 1220  1000    16     19710     34.65    0.1%
 1240  1000    16     20090     29.29    0.1%
 1260  1000    16     20360     39.78    0.1%
 1280  1000    16     20670     34.16    0.1%
 1300  1000    16     21010     37.12    0.1%
 1320  1000    16     21330     40.84    0.1%
 1340  1000    16     21630     30.64    0.1%
 1360  1000    16     21980     36.47    0.1%
 1380  1000    16     22260     49.52    0.2%
 1400  1000    16     22590     63.91    0.2%
 1420  1000    16     22910     37.38    0.1%
 1440  1000    16     23250     43.18    0.1%
 1460  1000    16     23620     47.14    0.1%
 1480  1000    16     23930      45.8    0.1%
 1500  1000    16     24250     69.63    0.2%
 1520  1000    16     24540     97.89    0.3%
 1540  1000    16     24890     41.89    0.1%
 1560  1000    16     25190     46.18    0.1%
 1580  1000    16     25530     47.59    0.1%
 1600  1000    16     25820     38.99    0.1%
 1620  1000    16     26120      41.9    0.1%
 1640  1000    16     26470     57.99    0.2%
 1660  1000    16     26810     64.42    0.2%
 1680  1000    16     27180     65.57    0.2%
 1700  1000    16     27400     66.19    0.2%
 1720  1000    16     27790     47.54    0.1%
 1740  1000    16     28080        36    0.1%
 1760  1000    16     28400     51.35    0.1%
 1780  1000    16     28730     68.63    0.2%
 1800  1000    16     29140      86.3    0.2%
 1820  1000    16     29370     32.67    0.1%
 1840  1000    16     29760     66.33    0.2%
 1860  1000    16     30110     50.66    0.1%
 1880  1000    16     30350     50.84    0.1%
 1900  1000    16     30690     65.69    0.2%
 1920  1000    16     30960     62.43    0.2%
 1940  1000    16     31400     76.29    0.2%
 1960  1000    16     31630     51.85    0.1%
 1980  1000    16     31950     41.12    0.1%
 2000   500    16     32370     54.34    0.1%
 2000   510    16     32390     78.49    0.2%
 2000   520    16     32410     83.96    0.2%
 2000   530    16     32380     42.97    0.1%
 2000   540    16     32350     73.17    0.2%
 2000   550    16     32310     42.74    0.1%
 2000   560    16     32830     101.8    0.3%
 2000   570    16     32350        59    0.1%
 2000   580    16     32340     99.05    0.3%
 2000   590    16     32430     71.98    0.2%
 2000   600    16     32650     214.1    0.6%
 2000   610    16     32830     56.59    0.1%
 2000   620    16     32350     91.48    0.2%
 2000   630    16     32380     89.58    0.2%
 2000   640    16     32440     50.24    0.1%
 2000   650    16     32460     80.49    0.2%
 2000   660    16     32340     69.52    0.2%
 2000   670    16     32390     73.43    0.2%
 2000   680    16     32350     49.84    0.1%
 2000   690    16     32420     95.72    0.2%
 2000   700    16     32340      63.5    0.1%
 2000   710    16     32720     39.57    0.1%
 2000   720    16     32390     85.07    0.2%
 2000   730    16     32800     56.05    0.1%
 2000   740    16     32340     63.87    0.1%
 2000   750    16     32830      51.7    0.1%
 2000   760    16     32400     43.01    0.1%
 2000   770    16     32420     62.18    0.1%
 2000   780    16     32510     142.1    0.4%
 2000   790    16     32390     56.85    0.1%
 2000   800    16     32370     72.03    0.2%
 2000   810    16     32370     56.39    0.1%
 2000   820    16     32460     42.74    0.1%
 2000   830    16     32340     79.24    0.2%
 2000   840    16     32420     57.97    0.1%
 2000   850    16     32370     57.76    0.1%
 2000   860    16     32850     49.45    0.1%
 2000   870    16     32380     41.87    0.1%
 2000   880    16     32470     46.64    0.1%
 2000   890    16     32440     81.32    0.2%
 2000   900    16     32350     55.87    0.1%
 2000   910    16     32380     51.61    0.1%
 2000   920    16     32350     51.43    0.1%
 2000   930    16     32370     51.13    0.1%
 2000   940    16     32460     62.72    0.1%
 2000   950    16     32350     43.79    0.1%
 2000   960    16     32380     71.86    0.2%
 2000   970    16     32370     34.43    0.1%
 2000   980    16     32360     57.76    0.1%
 2000   990    16     32330     32.71    0.1%
 2000  1000     5      4950     25.96    0.5%
 2000  1000     6      6385     14.56    0.2%
 2000  1000     7      8004     14.05    0.1%
 2000  1000     8      9826     25.22    0.2%
 2000  1000     9     12280     27.45    0.2%
 2000  1000    10     14580     27.35    0.1%
 2000  1000    11     17050     28.85    0.1%
 2000  1000    12     19660     34.12    0.1%
 2000  1000    13     22770     124.9    0.5%
 2000  1000    14     25590     51.63    0.2%
 2000  1000    15     28900     51.04    0.1%
 2000  1000    16     32380     63.68    0.1%

Quality and confidence:
param     error
v         0.051
t         0.102
d         7.126

Model:
Time ~=        0
    + v    15.93
    + t        0
    + d     2704
              µs

Reads = 0 + (0 * v) + (0 * t) + (0 * d)
Writes = 0 + (0 * v) + (0 * t) + (0 * d)


@georgesdib
Copy link
Contributor Author

/benchmark runtime kusama frame_election_provider_support

@georgesdib
Copy link
Contributor Author

/benchmark runtime westend frame_election_provider_support

…-- benchmark pallet --chain=polkadot-dev --steps=50 --repeat=20 --pallet=frame_election_provider_support --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/
@parity-benchapp
Copy link

parity-benchapp bot commented Apr 15, 2022

Benchmark Runtime Kusama Pallet for branch "npos-election-benchmarking" with command cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark pallet --chain=kusama-dev --steps=50 --repeat=20 --pallet=frame_election_provider_support --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

Toolchain: stable-x86_64-unknown-linux-gnu (default)
rustc 1.57.0 (f1edd0429 2021-11-29)

Results
Pallet: "frame_election_provider_support", Extrinsic: "phragmen", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + v    18.52
    + t    0.025
    + d     2896
              µs

Reads = 0 + (0 * v) + (0 * t) + (0 * d)
Writes = 0 + (0 * v) + (0 * t) + (0 * d)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    v     t     d   mean µs  sigma µs       %
 1000  1000    16     18390      22.7    0.1%
 1020  1000    16     18780      17.1    0.0%
 1040  1000    16     19160     55.26    0.2%
 1060  1000    16     19560     28.24    0.1%
 1080  1000    16     20330     147.5    0.7%
 1100  1000    16     20360     51.61    0.2%
 1120  1000    16     20680     49.04    0.2%
 1140  1000    16     21070     67.83    0.3%
 1160  1000    16     21420     66.27    0.3%
 1180  1000    16     21730     19.91    0.0%
 1200  1000    16     22110     65.96    0.2%
 1220  1000    16     22490     59.36    0.2%
 1240  1000    16     22890     40.23    0.1%
 1260  1000    16     23220     67.62    0.2%
 1280  1000    16     23620     52.39    0.2%
 1300  1000    16     24440     418.4    1.7%
 1320  1000    16     24710     458.7    1.8%
 1340  1000    16     24710     58.59    0.2%
 1360  1000    16     25060      25.1    0.1%
 1380  1000    16     25450     45.11    0.1%
 1400  1000    16     25820     38.34    0.1%
 1420  1000    16     26180     50.63    0.1%
 1440  1000    16     26650     48.32    0.1%
 1460  1000    16     26910     22.54    0.0%
 1480  1000    16     27290     21.19    0.0%
 1500  1000    16     27670     31.39    0.1%
 1520  1000    16     28050     41.24    0.1%
 1540  1000    16     28430     52.59    0.1%
 1560  1000    16     28830     62.22    0.2%
 1580  1000    16     29140     30.87    0.1%
 1600  1000    16     29540      44.7    0.1%
 1620  1000    16     30270     212.2    0.7%
 1640  1000    16     30310     44.31    0.1%
 1660  1000    16     30610     33.99    0.1%
 1680  1000    16     30970     42.07    0.1%
 1700  1000    16     31330     37.75    0.1%
 1720  1000    16     31670     44.41    0.1%
 1740  1000    16     32080     48.58    0.1%
 1760  1000    16     32490     45.04    0.1%
 1780  1000    16     32950     33.94    0.1%
 1800  1000    16     33260     86.58    0.2%
 1820  1000    16     33590     37.69    0.1%
 1840  1000    16     33990     50.92    0.1%
 1860  1000    16     34330     51.74    0.1%
 1880  1000    16     34740     53.15    0.1%
 1900  1000    16     35130     48.38    0.1%
 1920  1000    16     35510     52.55    0.1%
 1940  1000    16     35930        36    0.1%
 1960  1000    16     36280     58.64    0.1%
 1980  1000    16     36620     63.55    0.1%
 2000   500    16     37010     55.34    0.1%
 2000   510    16     37000     69.45    0.1%
 2000   520    16     37230     154.2    0.4%
 2000   530    16     36990     128.4    0.3%
 2000   540    16     36970     74.48    0.2%
 2000   550    16     37020     93.03    0.2%
 2000   560    16     37010     60.62    0.1%
 2000   570    16     37010     47.12    0.1%
 2000   580    16     37060     44.79    0.1%
 2000   590    16     37020     60.09    0.1%
 2000   600    16     36990     52.34    0.1%
 2000   610    16     36980     72.38    0.1%
 2000   620    16     37000     75.86    0.2%
 2000   630    16     37030     64.21    0.1%
 2000   640    16     37040     62.17    0.1%
 2000   650    16     37010     53.29    0.1%
 2000   660    16     37030     38.33    0.1%
 2000   670    16     37060     87.51    0.2%
 2000   680    16     37090     117.5    0.3%
 2000   690    16     37120     108.6    0.2%
 2000   700    16     37040     71.77    0.1%
 2000   710    16     37020     51.06    0.1%
 2000   720    16     37010     48.85    0.1%
 2000   730    16     37040     70.05    0.1%
 2000   740    16     37050     48.09    0.1%
 2000   750    16     37030     91.71    0.2%
 2000   760    16     37010     62.56    0.1%
 2000   770    16     36970     63.75    0.1%
 2000   780    16     36970     67.42    0.1%
 2000   790    16     37120     85.23    0.2%
 2000   800    16     37000     45.22    0.1%
 2000   810    16     37020     52.92    0.1%
 2000   820    16     36980      59.9    0.1%
 2000   830    16     37020     70.71    0.1%
 2000   840    16     37010     76.58    0.2%
 2000   850    16     36980     47.98    0.1%
 2000   860    16     36980     61.44    0.1%
 2000   870    16     37160     116.2    0.3%
 2000   880    16     37020     73.92    0.1%
 2000   890    16     37100     67.51    0.1%
 2000   900    16     37030     50.13    0.1%
 2000   910    16     37050     95.21    0.2%
 2000   920    16     36960     44.62    0.1%
 2000   930    16     37020     53.23    0.1%
 2000   940    16     37090     59.79    0.1%
 2000   950    16     37030     68.71    0.1%
 2000   960    16     37030     48.08    0.1%
 2000   970    16     37020     34.55    0.0%
 2000   980    16     37080     57.29    0.1%
 2000   990    16     37000     52.85    0.1%
 2000  1000     5      6607     22.14    0.3%
 2000  1000     6      8185     18.84    0.2%
 2000  1000     7     10070     27.28    0.2%
 2000  1000     8     12190     33.78    0.2%
 2000  1000     9     15060     47.88    0.3%
 2000  1000    10     17760     46.41    0.2%
 2000  1000    11     20410     22.99    0.1%
 2000  1000    12     23200     47.05    0.2%
 2000  1000    13     26340     46.02    0.1%
 2000  1000    14     29780     51.38    0.1%
 2000  1000    15     33270     48.92    0.1%
 2000  1000    16     37040     72.18    0.1%

Quality and confidence:
param     error
v         0.051
t         0.102
d         7.075

Model:
Time ~=        0
    + v    18.24
    + t        0
    + d     2979
              µs

Reads = 0 + (0 * v) + (0 * t) + (0 * d)
Writes = 0 + (0 * v) + (0 * t) + (0 * d)

Pallet: "frame_election_provider_support", Extrinsic: "phragmms", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + v    16.26
    + t     0.06
    + d     2633
              µs

Reads = 0 + (0 * v) + (0 * t) + (0 * d)
Writes = 0 + (0 * v) + (0 * t) + (0 * d)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    v     t     d   mean µs  sigma µs       %
 1000  1000    16     16250     21.46    0.1%
 1020  1000    16     16570     31.74    0.1%
 1040  1000    16     16920     40.89    0.2%
 1060  1000    16     17240     68.99    0.4%
 1080  1000    16     17610     35.17    0.1%
 1100  1000    16     17990     74.81    0.4%
 1120  1000    16     18240     33.83    0.1%
 1140  1000    16     18610     86.19    0.4%
 1160  1000    16     18890     46.49    0.2%
 1180  1000    16     19170     31.89    0.1%
 1200  1000    16     19560     28.04    0.1%
 1220  1000    16     19840     37.36    0.1%
 1240  1000    16     20160     25.37    0.1%
 1260  1000    16     20480     35.41    0.1%
 1280  1000    16     20790     48.19    0.2%
 1300  1000    16     21140     64.85    0.3%
 1320  1000    16     21460     38.27    0.1%
 1340  1000    16     21770     56.68    0.2%
 1360  1000    16     22090     58.48    0.2%
 1380  1000    16     22430     41.31    0.1%
 1400  1000    16     22720     42.13    0.1%
 1420  1000    16     23080     60.87    0.2%
 1440  1000    16     23390     36.22    0.1%
 1460  1000    16     23700     42.67    0.1%
 1480  1000    16     24030     41.95    0.1%
 1500  1000    16     24350     93.57    0.3%
 1520  1000    16     24700     110.3    0.4%
 1540  1000    16     25070     60.25    0.2%
 1560  1000    16     25350     47.22    0.1%
 1580  1000    16     25670     51.23    0.1%
 1600  1000    16     26020     69.14    0.2%
 1620  1000    16     26630     124.7    0.4%
 1640  1000    16     26690     42.56    0.1%
 1660  1000    16     26980     35.61    0.1%
 1680  1000    16     27310     77.25    0.2%
 1700  1000    16     27720      53.6    0.1%
 1720  1000    16     27970     47.03    0.1%
 1740  1000    16     28360     41.89    0.1%
 1760  1000    16     28550     43.85    0.1%
 1780  1000    16     28930     26.32    0.0%
 1800  1000    16     29270     34.31    0.1%
 1820  1000    16     29580     40.79    0.1%
 1840  1000    16     29930     62.03    0.2%
 1860  1000    16     30210     53.62    0.1%
 1880  1000    16     30630     81.04    0.2%
 1900  1000    16     30920      40.8    0.1%
 1920  1000    16     31260     55.94    0.1%
 1940  1000    16     31530     89.18    0.2%
 1960  1000    16     32030     47.74    0.1%
 1980  1000    16     32270        71    0.2%
 2000   500    16     32600     52.18    0.1%
 2000   510    16     32590     102.1    0.3%
 2000   520    16     32570     65.52    0.2%
 2000   530    16     32560     57.41    0.1%
 2000   540    16     32580      60.3    0.1%
 2000   550    16     32530      55.2    0.1%
 2000   560    16     32540     53.67    0.1%
 2000   570    16     32660     123.3    0.3%
 2000   580    16     32640     63.73    0.1%
 2000   590    16     32530     50.39    0.1%
 2000   600    16     32540     91.07    0.2%
 2000   610    16     32580     50.22    0.1%
 2000   620    16     32540     93.03    0.2%
 2000   630    16     32530     82.81    0.2%
 2000   640    16     32580     105.6    0.3%
 2000   650    16     32600     66.26    0.2%
 2000   660    16     32470     55.22    0.1%
 2000   670    16     32540     93.03    0.2%
 2000   680    16     32490     58.92    0.1%
 2000   690    16     32570     81.21    0.2%
 2000   700    16     32610     79.46    0.2%
 2000   710    16     32570     114.2    0.3%
 2000   720    16     32660     80.37    0.2%
 2000   730    16     32610       102    0.3%
 2000   740    16     32530     67.81    0.2%
 2000   750    16     32590       128    0.3%
 2000   760    16     32600      53.8    0.1%
 2000   770    16     32590     58.12    0.1%
 2000   780    16     32580     55.22    0.1%
 2000   790    16     32640     33.96    0.1%
 2000   800    16     32590     91.66    0.2%
 2000   810    16     32550     53.33    0.1%
 2000   820    16     32580     66.16    0.2%
 2000   830    16     32530     110.5    0.3%
 2000   840    16     32590     33.39    0.1%
 2000   850    16     32620      56.8    0.1%
 2000   860    16     32660     62.59    0.1%
 2000   870    16     32530     57.88    0.1%
 2000   880    16     32610     87.35    0.2%
 2000   890    16     32480     70.92    0.2%
 2000   900    16     32580     98.51    0.3%
 2000   910    16     32630     45.69    0.1%
 2000   920    16     32690     107.8    0.3%
 2000   930    16     32710     122.6    0.3%
 2000   940    16     32610     49.31    0.1%
 2000   950    16     32620     87.37    0.2%
 2000   960    16     32620     54.94    0.1%
 2000   970    16     32690     62.52    0.1%
 2000   980    16     32560     57.18    0.1%
 2000   990    16     32530      49.7    0.1%
 2000  1000     5      5007     11.57    0.2%
 2000  1000     6      6516     19.53    0.2%
 2000  1000     7      8115     25.91    0.3%
 2000  1000     8      9961     27.46    0.2%
 2000  1000     9     12520     40.31    0.3%
 2000  1000    10     14810      36.5    0.2%
 2000  1000    11     17270     58.66    0.3%
 2000  1000    12     19880     56.71    0.2%
 2000  1000    13     22830     83.26    0.3%
 2000  1000    14     25970     91.19    0.3%
 2000  1000    15     29090     71.03    0.2%
 2000  1000    16     32560     77.52    0.2%

Quality and confidence:
param     error
v         0.049
t         0.098
d         6.844

Model:
Time ~=        0
    + v    16.04
    + t        0
    + d     2712
              µs

Reads = 0 + (0 * v) + (0 * t) + (0 * d)
Writes = 0 + (0 * v) + (0 * t) + (0 * d)


…-- benchmark pallet --chain=kusama-dev --steps=50 --repeat=20 --pallet=frame_election_provider_support --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/
@parity-benchapp
Copy link

parity-benchapp bot commented Apr 15, 2022

Benchmark Runtime Westend Pallet for branch "npos-election-benchmarking" with command cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark pallet --chain=westend-dev --steps=50 --repeat=20 --pallet=frame_election_provider_support --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

Toolchain: stable-x86_64-unknown-linux-gnu (default)
rustc 1.57.0 (f1edd0429 2021-11-29)

Results
Pallet: "frame_election_provider_support", Extrinsic: "phragmen", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + v    19.06
    + t    0.013
    + d     2956
              µs

Reads = 0 + (0 * v) + (0 * t) + (0 * d)
Writes = 0 + (0 * v) + (0 * t) + (0 * d)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    v     t     d   mean µs  sigma µs       %
 1000  1000    16     18900     43.71    0.2%
 1020  1000    16     19330     66.79    0.3%
 1040  1000    16     19650     27.69    0.1%
 1060  1000    16     20030      29.5    0.1%
 1080  1000    16     20410     44.36    0.2%
 1100  1000    16     20770     31.86    0.1%
 1120  1000    16     21160     48.77    0.2%
 1140  1000    16     21510     30.11    0.1%
 1160  1000    16     21950     45.56    0.2%
 1180  1000    16     22280     48.54    0.2%
 1200  1000    16     22700     54.71    0.2%
 1220  1000    16     23050     54.33    0.2%
 1240  1000    16     23450     42.05    0.1%
 1260  1000    16     23800     33.67    0.1%
 1280  1000    16     24150     51.47    0.2%
 1300  1000    16     24570     33.14    0.1%
 1320  1000    16     24920     42.03    0.1%
 1340  1000    16     25360     47.97    0.1%
 1360  1000    16     25760     18.44    0.0%
 1380  1000    16     26120     59.52    0.2%
 1400  1000    16     26450     49.93    0.1%
 1420  1000    16     26840      41.9    0.1%
 1440  1000    16     27200      60.7    0.2%
 1460  1000    16     27600     65.36    0.2%
 1480  1000    16     28230     221.3    0.7%
 1500  1000    16     28330     52.68    0.1%
 1520  1000    16     28780     71.06    0.2%
 1540  1000    16     29450       143    0.4%
 1560  1000    16     29730     126.7    0.4%
 1580  1000    16     29890     43.18    0.1%
 1600  1000    16     30260     28.36    0.0%
 1620  1000    16     30620     48.42    0.1%
 1640  1000    16     30990     54.56    0.1%
 1660  1000    16     31410     48.12    0.1%
 1680  1000    16     31830     122.6    0.3%
 1700  1000    16     32210      87.3    0.2%
 1720  1000    16     32580     27.07    0.0%
 1740  1000    16     32990     51.79    0.1%
 1760  1000    16     33300     56.59    0.1%
 1780  1000    16     33780     59.93    0.1%
 1800  1000    16     34090     34.67    0.1%
 1820  1000    16     34570     58.52    0.1%
 1840  1000    16     34890     65.54    0.1%
 1860  1000    16     35260     93.48    0.2%
 1880  1000    16     35760     101.4    0.2%
 1900  1000    16     36080     45.85    0.1%
 1920  1000    16     36530       144    0.3%
 1940  1000    16     36800     41.82    0.1%
 1960  1000    16     37200     50.99    0.1%
 1980  1000    16     37680     89.17    0.2%
 2000   500    16     37930      76.3    0.2%
 2000   510    16     37930     59.27    0.1%
 2000   520    16     37910     87.09    0.2%
 2000   530    16     37990     93.79    0.2%
 2000   540    16     37950     93.09    0.2%
 2000   550    16     37980     76.93    0.2%
 2000   560    16     37990     56.49    0.1%
 2000   570    16     37900     62.99    0.1%
 2000   580    16     37950     42.25    0.1%
 2000   590    16     37930     56.91    0.1%
 2000   600    16     37920     63.02    0.1%
 2000   610    16     37980      40.9    0.1%
 2000   620    16     38040     71.92    0.1%
 2000   630    16     37970     60.38    0.1%
 2000   640    16     37950     49.93    0.1%
 2000   650    16     37930     73.97    0.1%
 2000   660    16     37920     55.02    0.1%
 2000   670    16     38090       133    0.3%
 2000   680    16     37960     85.97    0.2%
 2000   690    16     38000     94.42    0.2%
 2000   700    16     37970     72.58    0.1%
 2000   710    16     37930     59.96    0.1%
 2000   720    16     37970     70.21    0.1%
 2000   730    16     37990      64.5    0.1%
 2000   740    16     37940     76.65    0.2%
 2000   750    16     37950     92.94    0.2%
 2000   760    16     37870     41.61    0.1%
 2000   770    16     37890     47.72    0.1%
 2000   780    16     37960     63.45    0.1%
 2000   790    16     37990     73.79    0.1%
 2000   800    16     38000     53.62    0.1%
 2000   810    16     37990     67.59    0.1%
 2000   820    16     37910     29.71    0.0%
 2000   830    16     37920     56.26    0.1%
 2000   840    16     37910     73.51    0.1%
 2000   850    16     38000     63.44    0.1%
 2000   860    16     38040     65.75    0.1%
 2000   870    16     38000     64.44    0.1%
 2000   880    16     37960     43.89    0.1%
 2000   890    16     37940      54.5    0.1%
 2000   900    16     37920     63.76    0.1%
 2000   910    16     37900     68.66    0.1%
 2000   920    16     37970      58.9    0.1%
 2000   930    16     38100     44.22    0.1%
 2000   940    16     37900     29.54    0.0%
 2000   950    16     37900     50.71    0.1%
 2000   960    16     37910     71.89    0.1%
 2000   970    16     37900     26.98    0.0%
 2000   980    16     37980     65.43    0.1%
 2000   990    16     37940     71.07    0.1%
 2000  1000     5      6884     14.17    0.2%
 2000  1000     6      8553     17.89    0.2%
 2000  1000     7     10530     67.51    0.6%
 2000  1000     8     12640     41.84    0.3%
 2000  1000     9     15540     40.97    0.2%
 2000  1000    10     18320     58.79    0.3%
 2000  1000    11     21050      50.8    0.2%
 2000  1000    12     24110     108.5    0.4%
 2000  1000    13     27160     61.86    0.2%
 2000  1000    14     30500     49.07    0.1%
 2000  1000    15     34160     46.01    0.1%
 2000  1000    16     37980     61.14    0.1%

Quality and confidence:
param     error
v         0.049
t         0.099
d         6.922

Model:
Time ~=        0
    + v     18.8
    + t        0
    + d     3040
              µs

Reads = 0 + (0 * v) + (0 * t) + (0 * d)
Writes = 0 + (0 * v) + (0 * t) + (0 * d)

Pallet: "frame_election_provider_support", Extrinsic: "phragmms", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + v    16.24
    + t    0.132
    + d     2622
              µs

Reads = 0 + (0 * v) + (0 * t) + (0 * d)
Writes = 0 + (0 * v) + (0 * t) + (0 * d)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    v     t     d   mean µs  sigma µs       %
 1000  1000    16     16220     53.22    0.3%
 1020  1000    16     16550      39.3    0.2%
 1040  1000    16     16900     37.03    0.2%
 1060  1000    16     17190     24.11    0.1%
 1080  1000    16     17530     34.34    0.1%
 1100  1000    16     17820     33.23    0.1%
 1120  1000    16     18120     34.25    0.1%
 1140  1000    16     18450     42.75    0.2%
 1160  1000    16     18780     55.17    0.2%
 1180  1000    16     19120     30.17    0.1%
 1200  1000    16     19460     56.61    0.2%
 1220  1000    16     19770     46.25    0.2%
 1240  1000    16     20090     30.11    0.1%
 1260  1000    16     21100     209.8    0.9%
 1280  1000    16     20740      32.2    0.1%
 1300  1000    16     21110      37.2    0.1%
 1320  1000    16     21420     27.87    0.1%
 1340  1000    16     21740     42.85    0.1%
 1360  1000    16     22010     35.02    0.1%
 1380  1000    16     22350     50.86    0.2%
 1400  1000    16     22620     23.33    0.1%
 1420  1000    16     23040     45.77    0.1%
 1440  1000    16     23340     41.23    0.1%
 1460  1000    16     23700     61.01    0.2%
 1480  1000    16     23930     54.42    0.2%
 1500  1000    16     24310     33.57    0.1%
 1520  1000    16     24640     56.99    0.2%
 1540  1000    16     24920     60.11    0.2%
 1560  1000    16     25270     42.95    0.1%
 1580  1000    16     25570     68.73    0.2%
 1600  1000    16     25980     72.64    0.2%
 1620  1000    16     26240      55.8    0.2%
 1640  1000    16     26650      67.2    0.2%
 1660  1000    16     26990     78.87    0.2%
 1680  1000    16     27270     48.17    0.1%
 1700  1000    16     27580     57.43    0.2%
 1720  1000    16     27890     28.49    0.1%
 1740  1000    16     28250     38.87    0.1%
 1760  1000    16     28500     57.88    0.2%
 1780  1000    16     28900     51.57    0.1%
 1800  1000    16     29260     87.65    0.2%
 1820  1000    16     29560     63.81    0.2%
 1840  1000    16     29800     43.09    0.1%
 1860  1000    16     30170     45.45    0.1%
 1880  1000    16     30540        53    0.1%
 1900  1000    16     30800     59.73    0.1%
 1920  1000    16     31230     97.54    0.3%
 1940  1000    16     31540     82.27    0.2%
 1960  1000    16     31810     84.45    0.2%
 1980  1000    16     32120     50.06    0.1%
 2000   500    16     32460     70.31    0.2%
 2000   510    16     32520     104.6    0.3%
 2000   520    16     32480     79.38    0.2%
 2000   530    16     32450     69.93    0.2%
 2000   540    16     32460     82.16    0.2%
 2000   550    16     32470     76.47    0.2%
 2000   560    16     32410     70.35    0.2%
 2000   570    16     32450     42.67    0.1%
 2000   580    16     32480     63.09    0.1%
 2000   590    16     32470     72.53    0.2%
 2000   600    16     32470     86.94    0.2%
 2000   610    16     32440     37.36    0.1%
 2000   620    16     32530     66.08    0.2%
 2000   630    16     32470     86.53    0.2%
 2000   640    16     32510     52.45    0.1%
 2000   650    16     32570     75.21    0.2%
 2000   660    16     32480     71.98    0.2%
 2000   670    16     32420     77.21    0.2%
 2000   680    16     32430     81.24    0.2%
 2000   690    16     36390     948.5    2.6%
 2000   700    16     32450     75.49    0.2%
 2000   710    16     32430     60.13    0.1%
 2000   720    16     32520     69.69    0.2%
 2000   730    16     32710     124.5    0.3%
 2000   740    16     32620     106.4    0.3%
 2000   750    16     32560      61.3    0.1%
 2000   760    16     32430     62.29    0.1%
 2000   770    16     32420     60.67    0.1%
 2000   780    16     32470     69.97    0.2%
 2000   790    16     32470     71.92    0.2%
 2000   800    16     32490     73.38    0.2%
 2000   810    16     32480      81.6    0.2%
 2000   820    16     32470     103.6    0.3%
 2000   830    16     32510     64.67    0.1%
 2000   840    16     32520     68.65    0.2%
 2000   850    16     32540     97.03    0.2%
 2000   860    16     32460     78.34    0.2%
 2000   870    16     32470     75.98    0.2%
 2000   880    16     32580     66.93    0.2%
 2000   890    16     32630     96.42    0.2%
 2000   900    16     32520     97.91    0.3%
 2000   910    16     32530     155.8    0.4%
 2000   920    16     32460     82.36    0.2%
 2000   930    16     32610     69.98    0.2%
 2000   940    16     32500     27.11    0.0%
 2000   950    16     32470     77.11    0.2%
 2000   960    16     32650     108.9    0.3%
 2000   970    16     32500     81.17    0.2%
 2000   980    16     32650     106.1    0.3%
 2000   990    16     32540     107.6    0.3%
 2000  1000     5      5025     16.15    0.3%
 2000  1000     6      6502     13.64    0.2%
 2000  1000     7      8120     26.01    0.3%
 2000  1000     8      9955     28.25    0.2%
 2000  1000     9     12420     12.65    0.1%
 2000  1000    10     14770     37.68    0.2%
 2000  1000    11     17210     23.67    0.1%
 2000  1000    12     19910     62.16    0.3%
 2000  1000    13     22680     51.36    0.2%
 2000  1000    14     25820     81.12    0.3%
 2000  1000    15     29130     84.18    0.2%
 2000  1000    16     32510     79.33    0.2%

Quality and confidence:
param     error
v         0.066
t         0.132
d         9.193

Model:
Time ~=        0
    + v    16.02
    + t        0
    + d     2707
              µs

Reads = 0 + (0 * v) + (0 * t) + (0 * d)
Writes = 0 + (0 * v) + (0 * t) + (0 * d)


…-- benchmark pallet --chain=westend-dev --steps=50 --repeat=20 --pallet=frame_election_provider_support --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
@bkchr bkchr added B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit. A0-please_review Pull request needs code review. labels Apr 15, 2022
@bkchr bkchr merged commit 74c7544 into paritytech:master Apr 15, 2022
@georgesdib georgesdib deleted the npos-election-benchmarking branch April 15, 2022 15:18
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants