Skip to content
This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Paras inherent benchmark with keystore#4110

Merged
emostov merged 4 commits intozeke-benchmark-paras-inherentfrom
zeke-bench-get-sigs-to-work
Oct 20, 2021
Merged

Paras inherent benchmark with keystore#4110
emostov merged 4 commits intozeke-benchmark-paras-inherentfrom
zeke-bench-get-sigs-to-work

Conversation

@emostov
Copy link
Contributor

@emostov emostov commented Oct 19, 2021

target branch: zeke-benchmark-paras-inherent

testing benchmarks

@github-actions github-actions bot added the A0-please_review Pull request needs code review. label Oct 19, 2021
@emostov
Copy link
Contributor Author

emostov commented Oct 19, 2021

/benchmark runtime kusama runtime_parachains::paras_inherent

@parity-benchapp
Copy link

parity-benchapp bot commented Oct 19, 2021 •
edited
Loading

Benchmark Runtime Kusama Pallet for branch "zeke-bench-get-sigs-to-work" with command cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras_inherent.rs

Results 
Pallet: "runtime_parachains::paras_inherent", Extrinsic: "enter_dispute_dominant", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParaInherent Included (r:1 w:1)
Storage: System ParentHash (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: ParaScheduler AvailabilityCores (r:1 w:1)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: Paras Parachains (r:1 w:0)
Storage: ParaInclusion PendingAvailability (r:41 w:40)
Storage: ParaInclusion PendingAvailabilityCommitments (r:40 w:40)
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: Session Validators (r:1 w:0)
Storage: ParasShared ActiveValidatorIndices (r:1 w:0)
Storage: Staking ActiveEra (r:1 w:0)
Storage: Staking ErasRewardPoints (r:1 w:1)
Storage: Dmp DownwardMessageQueues (r:40 w:1)
Storage: Hrmp HrmpChannelDigests (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:40 w:0)
Storage: ParaScheduler SessionStartBlock (r:1 w:0)
Storage: ParaScheduler ParathreadQueue (r:1 w:1)
Storage: ParaScheduler Scheduled (r:1 w:1)
Storage: ParaScheduler ValidatorGroups (r:1 w:0)
Storage: System BlockWeight (r:1 w:0)
Storage: Paras PastCodeMeta (r:40 w:0)
Storage: Paras CurrentCodeHash (r:40 w:0)
Storage: Ump RelayDispatchQueueSize (r:40 w:0)
Storage: Hrmp HrmpWatermarks (r:39 w:1)
Storage: Paras Heads (r:39 w:1)
Storage: Ump NeedsDispatch (r:1 w:1)
Storage: Ump NextDispatchRoundStartWith (r:1 w:1)
Storage: ParaInclusion AvailabilityBitfields (r:0 w:200)
Storage: ParaInherent OnChainVotes (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    27870
    + d        0
              µs

Reads = 378 + (0 * d)
Writes = 292 + (0 * d)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    d   mean µs  sigma µs       %
    0     27660     45.98    0.1%
    1     27550     23.35    0.0%
    2     27200     35.32    0.1%
    3     26880     38.62    0.1%
    4     26580     28.23    0.1%
    5     26260     30.22    0.1%
    6     25910     32.27    0.1%
    7     25600     35.73    0.1%
    8     25290     47.37    0.1%
    9     25010     37.76    0.1%
   10     24650     32.32    0.1%
   11     24300     26.01    0.1%
   12     24030     32.56    0.1%
   13     23680     31.67    0.1%
   14     23370     30.57    0.1%
   15     23060     50.15    0.2%
   16     22740     39.23    0.1%
   17     22370     42.12    0.1%
   18     22060     30.02    0.1%
   19     21720     43.35    0.1%
   20     21330     31.62    0.1%
   21     21050     22.61    0.1%
   22     20750     33.57    0.1%
   23     20380     25.12    0.1%
   24     20030     25.04    0.1%
   25     19890     43.29    0.2%
   26     19510     40.83    0.2%
   27     19190     40.76    0.2%
   28     18830      18.6    0.0%
   29     18530     30.18    0.1%
   30     18220     24.37    0.1%
   31     17880     39.05    0.2%
   32     17550     21.35    0.1%
   33     17210     33.13    0.1%
   34     16860     57.26    0.3%
   35     16540     23.82    0.1%
   36     16180     36.93    0.2%
   37     15910     33.59    0.2%
   38     15560     36.01    0.2%
   39     15220     32.49    0.2%
   40     14790     43.42    0.2%

Quality and confidence:
param     error
d         0.268

Model:
Time ~=    27860
    + d        0
              µs

Reads = 378 + (0 * d)
Writes = 292 + (0 * d)

Pallet: "runtime_parachains::paras_inherent", Extrinsic: "enter_disputes_only", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParaInherent Included (r:1 w:1)
Storage: System ParentHash (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: ParaScheduler AvailabilityCores (r:1 w:1)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: Paras Parachains (r:1 w:0)
Storage: ParaInclusion PendingAvailability (r:41 w:0)
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParaScheduler SessionStartBlock (r:1 w:0)
Storage: ParaScheduler ParathreadQueue (r:1 w:1)
Storage: ParaScheduler Scheduled (r:1 w:1)
Storage: ParaScheduler ValidatorGroups (r:1 w:0)
Storage: System BlockWeight (r:1 w:0)
Storage: Ump NeedsDispatch (r:1 w:1)
Storage: Ump NextDispatchRoundStartWith (r:1 w:1)
Storage: ParaInclusion AvailabilityBitfields (r:0 w:200)
Storage: ParaInherent OnChainVotes (r:0 w:1)
Storage: ParaInclusion PendingAvailabilityCommitments (r:1 w:1)
Storage: Session Validators (r:1 w:0)
Storage: ParasShared ActiveValidatorIndices (r:1 w:0)
Storage: Staking ActiveEra (r:1 w:0)
Storage: Staking ErasRewardPoints (r:1 w:1)
Storage: Dmp DownwardMessageQueues (r:1 w:1)
Storage: Hrmp HrmpChannelDigests (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:1 w:0)
Storage: Hrmp HrmpWatermarks (r:0 w:1)
Storage: Paras Heads (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    10770
    + d    102.5
              µs

Reads = 62 + (1 * d)
Writes = 212 + (2 * d)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    d   mean µs  sigma µs       %
    0     10770     18.64    0.1%
    1     10900     18.09    0.1%
    2     10950     19.39    0.1%
    3     11110     26.32    0.2%
    4     11180     28.07    0.2%
    5     11280        12    0.1%
    6     11380     26.45    0.2%
    7     11490     23.05    0.2%
    8     11600     19.25    0.1%
    9     11720     46.38    0.3%
   10     11800     23.58    0.1%
   11     11890      18.5    0.1%
   12     11970     15.78    0.1%
   13     12130     33.17    0.2%
   14     12210     13.84    0.1%
   15     12330     22.92    0.1%
   16     12420     22.55    0.1%
   17     12540     25.24    0.2%
   18     12620     31.22    0.2%
   19     12720     26.85    0.2%
   20     12780     10.02    0.0%
   21     12900     32.96    0.2%
   22     12990     16.28    0.1%
   23     13110     20.26    0.1%
   24     13290     37.67    0.2%
   25     13390     19.44    0.1%
   26     13500     25.52    0.1%
   27     13580     14.58    0.1%
   28     13660     27.02    0.1%
   29     13790     33.84    0.2%
   30     13900     36.71    0.2%
   31     14010      28.7    0.2%
   32     14040     21.11    0.1%
   33     14190     40.84    0.2%
   34     14250     28.75    0.2%
   35     14360     29.39    0.2%
   36     14480     26.54    0.1%
   37     14550      18.9    0.1%
   38     14650     35.53    0.2%
   39     14730     24.19    0.1%
   40     14860     47.22    0.3%

Quality and confidence:
param     error
d         0.163

Model:
Time ~=    10780
    + d    102.6
              µs

Reads = 61 + (1 * d)
Writes = 212 + (2 * d)

Pallet: "runtime_parachains::paras_inherent", Extrinsic: "enter_backed_dominant", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParaInherent Included (r:1 w:1)
Storage: System ParentHash (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: ParaInclusion PendingAvailability (r:41 w:40)
Storage: ParaScheduler AvailabilityCores (r:1 w:1)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: Paras Parachains (r:1 w:0)
Storage: ParaInclusion PendingAvailabilityCommitments (r:40 w:40)
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: Session Validators (r:1 w:0)
Storage: ParasShared ActiveValidatorIndices (r:1 w:0)
Storage: Staking ActiveEra (r:1 w:0)
Storage: Staking ErasRewardPoints (r:1 w:1)
Storage: Dmp DownwardMessageQueues (r:1 w:1)
Storage: Hrmp HrmpChannelDigests (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:1 w:0)
Storage: ParaScheduler SessionStartBlock (r:1 w:0)
Storage: ParaScheduler ParathreadQueue (r:1 w:1)
Storage: ParaScheduler Scheduled (r:1 w:1)
Storage: ParaScheduler ValidatorGroups (r:1 w:0)
Storage: System BlockWeight (r:1 w:0)
Storage: Ump NeedsDispatch (r:1 w:1)
Storage: Ump NextDispatchRoundStartWith (r:1 w:1)
Storage: ParaInclusion AvailabilityBitfields (r:0 w:200)
Storage: ParaInherent OnChainVotes (r:0 w:1)
Storage: Hrmp HrmpWatermarks (r:0 w:1)
Storage: Paras Heads (r:0 w:1)
Storage: Paras PastCodeMeta (r:1 w:0)
Storage: Paras CurrentCodeHash (r:1 w:0)
Storage: Ump RelayDispatchQueueSize (r:1 w:0)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    14920
    + b    323.6
              µs

Reads = 98 + (7 * b)
Writes = 292 + (0 * b)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    b   mean µs  sigma µs       %
    0     14840     37.95    0.2%
    1     15230     42.65    0.2%
    2     15580     32.22    0.2%
    3     15870     27.03    0.1%
    4     16190     28.44    0.1%
    5     16520     30.45    0.1%
    6     16860     26.32    0.1%
    7     17170     22.06    0.1%
    8     17500     16.75    0.0%
    9     17870     30.14    0.1%
   10     18210     18.56    0.1%
   11     18510     31.89    0.1%
   12     18890     43.92    0.2%
   13     19170     28.07    0.1%
   14     19500     54.36    0.2%
   15     19800     30.46    0.1%
   16     20010     42.46    0.2%
   17     20340     26.29    0.1%
   18     20740     56.35    0.2%
   19     21060     33.59    0.1%
   20     21390     13.49    0.0%
   21     21710      44.9    0.2%
   22     22030     28.48    0.1%
   23     22320     27.15    0.1%
   24     22690     35.91    0.1%
   25     23030     21.32    0.0%
   26     23360     27.93    0.1%
   27     23810     105.8    0.4%
   28     24040     42.11    0.1%
   29     24320     36.14    0.1%
   30     24620     16.33    0.0%
   31     24970     44.25    0.1%
   32     25330     29.03    0.1%
   33     25610     15.84    0.0%
   34     25980     36.72    0.1%
   35     26220     30.31    0.1%
   36     26620     33.42    0.1%
   37     26870     36.83    0.1%
   38     27230     40.57    0.1%
   39     27540     26.24    0.0%
   40     27670     30.46    0.1%

Quality and confidence:
param     error
b         0.269

Model:
Time ~=    14920
    + b    323.6
              µs

Reads = 98 + (7 * b)
Writes = 292 + (0 * b)

Pallet: "runtime_parachains::paras_inherent", Extrinsic: "enter_backed_only", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParaInherent Included (r:1 w:1)
Storage: System ParentHash (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: ParaScheduler AvailabilityCores (r:1 w:1)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: Paras Parachains (r:1 w:0)
Storage: ParaInclusion PendingAvailability (r:41 w:0)
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParaScheduler SessionStartBlock (r:1 w:0)
Storage: ParaScheduler ParathreadQueue (r:1 w:1)
Storage: ParaScheduler Scheduled (r:1 w:1)
Storage: ParaScheduler ValidatorGroups (r:1 w:0)
Storage: System BlockWeight (r:1 w:0)
Storage: Ump NeedsDispatch (r:1 w:1)
Storage: Ump NextDispatchRoundStartWith (r:1 w:1)
Storage: ParaInclusion AvailabilityBitfields (r:0 w:200)
Storage: ParaInherent OnChainVotes (r:0 w:1)
Storage: ParaInclusion PendingAvailabilityCommitments (r:1 w:1)
Storage: Session Validators (r:1 w:0)
Storage: ParasShared ActiveValidatorIndices (r:1 w:0)
Storage: Staking ActiveEra (r:1 w:0)
Storage: Staking ErasRewardPoints (r:1 w:1)
Storage: Dmp DownwardMessageQueues (r:1 w:1)
Storage: Hrmp HrmpChannelDigests (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:1 w:0)
Storage: Paras PastCodeMeta (r:1 w:0)
Storage: Paras CurrentCodeHash (r:1 w:0)
Storage: Ump RelayDispatchQueueSize (r:1 w:0)
Storage: Hrmp HrmpWatermarks (r:0 w:1)
Storage: Paras Heads (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    10810
    + b    421.2
              µs

Reads = 58 + (8 * b)
Writes = 212 + (2 * b)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    b   mean µs  sigma µs       %
    0     10730     15.38    0.1%
    1     11240     17.99    0.1%
    2     11680     27.64    0.2%
    3     12090      25.7    0.2%
    4     12510     19.75    0.1%
    5     12930     24.94    0.1%
    6     13350     33.91    0.2%
    7     13770     28.67    0.2%
    8     14210     41.51    0.2%
    9     14590      13.6    0.0%
   10     15030     24.73    0.1%
   11     15450      17.2    0.1%
   12     15870     22.01    0.1%
   13     16270     19.06    0.1%
   14     16690     22.34    0.1%
   15     17130     32.71    0.1%
   16     17550     27.21    0.1%
   17     17970     35.22    0.1%
   18     18390     43.25    0.2%
   19     18780     14.83    0.0%
   20     19220     17.18    0.0%
   21     19650     37.71    0.1%
   22     20100     30.63    0.1%
   23     20530     32.43    0.1%
   24     20930     25.83    0.1%
   25     21350     33.68    0.1%
   26     21790     21.73    0.0%
   27     22220     25.86    0.1%
   28     22630     24.31    0.1%
   29     23000     25.89    0.1%
   30     23430     32.05    0.1%
   31     23880     25.73    0.1%
   32     24280     26.97    0.1%
   33     24690      27.5    0.1%
   34     25140     48.79    0.1%
   35     25530     46.76    0.1%
   36     25950     36.76    0.1%
   37     26430     36.86    0.1%
   38     26850     38.58    0.1%
   39     27260     38.29    0.1%
   40     27680     37.25    0.1%

Quality and confidence:
param     error
b         0.159

Model:
Time ~=    10810
    + b    421.5
              µs

Reads = 58 + (8 * b)
Writes = 212 + (2 * b)

ERROR: Unable to push ./runtime/kusama/src/weights/runtime_parachains_paras_inherent.rs

emostov
emostov commented Oct 19, 2021
View reviewed changes
runtime/parachains/src/mock.rs
Comment on lines +313 to +314
let mut ext: TestExternalities = t.into();
ext.register_extension(KeystoreExt(Arc::new(KeyStore::new()) as SyncCryptoStorePtr));
Copy link
Contributor Author

@emostov emostov Oct 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems to fix the issue with using the keystore when running tests

@emostov emostov removed the A0-please_review Pull request needs code review. label Oct 20, 2021
@emostov emostov merged commit 6fde324 into zeke-benchmark-paras-inherent Oct 20, 2021
@emostov emostov deleted the zeke-bench-get-sigs-to-work branch October 20, 2021 17:54
emostov added a commit that referenced this pull request Nov 1, 2021
@emostov
Paras inherent benchmark with keystore (#4110)
1ac35a3 
* WIP transition to signing with Public

* Add test ext to testing mock

* Maybe working? But so slow its hard to tell

* add some primitive debug logs
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@emostov