Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[storage] use different metric name for each shard #14666

Merged
merged 2 commits into from
Sep 26, 2024
Merged

Conversation

areshand
Copy link
Contributor

Description

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Other (specify)

How Has This Been Tested?

Key Areas to Review

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

Copy link

trunk-io bot commented Sep 18, 2024

⏱️ 5h 4m total CI duration on this PR
Job Cumulative Duration Recent Runs
execution-performance / single-node-performance 1h 15m 🟩🟩🟥🟩
forge-compat-test / forge 1h 🟩🟩🟩
forge-e2e-test / forge 44m 🟩🟩🟩
execution-performance / test-target-determinator 19m 🟩🟩🟩🟩
forge-framework-upgrade-test / forge 15m 🟩
check-dynamic-deps 14m 🟩🟩🟩🟩🟩 (+2 more)
test-target-determinator 13m 🟩🟩🟩
check 11m 🟩🟩🟩
general-lints 11m 🟩🟩🟩🟩🟩 (+1 more)
rust-cargo-deny 11m 🟩🟩🟩🟩🟩 (+1 more)
rust-doc-tests 5m 🟩
rust-doc-tests 5m 🟩
indexer-grpc-e2e-tests / test-indexer-grpc-docker-compose 4m 🟥🟩🟩
semgrep/ci 3m 🟩🟩🟩🟩🟩 (+2 more)
rust-move-tests 2m 🟩
rust-move-tests 2m 🟩
rust-move-tests 2m 🟩
rust-move-tests 2m 🟩
rust-move-tests 1m 🟩
file_change_determinator 1m 🟩🟩🟩🟩🟩 (+1 more)
file_change_determinator 1m 🟩🟩🟩🟩 (+2 more)
rust-move-tests 1m
rust-doc-tests 1m 🟥
file_change_determinator 36s 🟩🟩🟩
permission-check 30s 🟩🟩🟩🟩🟩 (+2 more)
permission-check 21s 🟩🟩🟩🟩 (+1 more)
permission-check 17s 🟩🟩🟩🟩🟩 (+1 more)
permission-check 15s 🟩🟩🟩🟩 (+1 more)
permission-check 14s 🟩🟩🟩
determine-docker-build-metadata 11s 🟩🟩🟩
permission-check 7s 🟩
Backport PR 3s 🟥
rust-move-tests 1s
rust-doc-tests 1s

🚨 2 jobs on the last run were significantly faster/slower than expected

Job Duration vs 7d avg Delta
execution-performance / single-node-performance 23m 18m +28%
forge-compat-test / forge 23m 19m +23%

settingsfeedbackdocs ⋅ learn more about trunk.io

@areshand areshand force-pushed the one_shard_per_metric branch 2 times, most recently from a83abd2 to 8e6fcc3 Compare September 18, 2024 17:45
@areshand areshand added the CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR label Sep 19, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

@areshand areshand force-pushed the one_shard_per_metric branch from 8e6fcc3 to 64a1f51 Compare September 19, 2024 20:40

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

@msmouse msmouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks boss!

@areshand areshand force-pushed the one_shard_per_metric branch from 64a1f51 to 29f4f33 Compare September 25, 2024 00:44
@areshand areshand enabled auto-merge (rebase) September 25, 2024 00:44

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on 66a8bceb9b4e2483c50734db8406b7cc76163682

two traffics test: inner traffic : committed: 14486.40 txn/s, latency: 2741.97 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 3200 ms), latency samples: 5508040
two traffics test : committed: 99.94 txn/s, latency: 1601.66 ms, (p50: 1500 ms, p70: 1500, p90: 1600 ms, p99: 7900 ms), latency samples: 1780
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.251, avg: 0.226", "QsPosToProposal: max: 1.074, avg: 1.052", "ConsensusProposalToOrdered: max: 0.328, avg: 0.294", "ConsensusOrderedToCommit: max: 0.411, avg: 0.400", "ConsensusProposalToCommit: max: 0.701, avg: 0.693"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 1.60s no progress at version 23974 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 8.61s no progress at version 3038464 (avg 8.61s) [limit 15].
Test Ok

Copy link
Contributor

✅ Forge suite framework_upgrade success on b6187e1794550f4a6b25cab0e5d40188574241da ==> 66a8bceb9b4e2483c50734db8406b7cc76163682

Compatibility test results for b6187e1794550f4a6b25cab0e5d40188574241da ==> 66a8bceb9b4e2483c50734db8406b7cc76163682 (PR)
Upgrade the nodes to version: 66a8bceb9b4e2483c50734db8406b7cc76163682
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1318.65 txn/s, submitted: 1320.71 txn/s, failed submission: 2.05 txn/s, expired: 2.05 txn/s, latency: 2515.16 ms, (p50: 2400 ms, p70: 2700, p90: 4200 ms, p99: 5400 ms), latency samples: 115520
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1248.01 txn/s, submitted: 1250.49 txn/s, failed submission: 2.48 txn/s, expired: 2.48 txn/s, latency: 2595.37 ms, (p50: 2400 ms, p70: 2700, p90: 4500 ms, p99: 5700 ms), latency samples: 110780
5. check swarm health
Compatibility test for b6187e1794550f4a6b25cab0e5d40188574241da ==> 66a8bceb9b4e2483c50734db8406b7cc76163682 passed
Upgrade the remaining nodes to version: 66a8bceb9b4e2483c50734db8406b7cc76163682
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1213.25 txn/s, submitted: 1215.31 txn/s, failed submission: 2.06 txn/s, expired: 2.06 txn/s, latency: 2771.66 ms, (p50: 2400 ms, p70: 2700, p90: 4500 ms, p99: 6600 ms), latency samples: 105900
Test Ok

Copy link
Contributor

✅ Forge suite compat success on b6187e1794550f4a6b25cab0e5d40188574241da ==> 66a8bceb9b4e2483c50734db8406b7cc76163682

Compatibility test results for b6187e1794550f4a6b25cab0e5d40188574241da ==> 66a8bceb9b4e2483c50734db8406b7cc76163682 (PR)
1. Check liveness of validators at old version: b6187e1794550f4a6b25cab0e5d40188574241da
compatibility::simple-validator-upgrade::liveness-check : committed: 14625.11 txn/s, latency: 2299.34 ms, (p50: 2000 ms, p70: 2100, p90: 2700 ms, p99: 7000 ms), latency samples: 477840
2. Upgrading first Validator to new version: 66a8bceb9b4e2483c50734db8406b7cc76163682
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 6399.77 txn/s, latency: 4321.66 ms, (p50: 4800 ms, p70: 5100, p90: 5700 ms, p99: 6000 ms), latency samples: 119520
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 5895.27 txn/s, latency: 5445.39 ms, (p50: 5800 ms, p70: 6400, p90: 7000 ms, p99: 8000 ms), latency samples: 200220
3. Upgrading rest of first batch to new version: 66a8bceb9b4e2483c50734db8406b7cc76163682
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 7044.09 txn/s, latency: 3944.77 ms, (p50: 4300 ms, p70: 4500, p90: 5100 ms, p99: 5400 ms), latency samples: 136440
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 7558.06 txn/s, latency: 4233.90 ms, (p50: 4300 ms, p70: 4400, p90: 6500 ms, p99: 6700 ms), latency samples: 253100
4. upgrading second batch to new version: 66a8bceb9b4e2483c50734db8406b7cc76163682
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 10431.65 txn/s, latency: 2672.58 ms, (p50: 2500 ms, p70: 2700, p90: 4500 ms, p99: 4700 ms), latency samples: 206000
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 10149.42 txn/s, latency: 3018.12 ms, (p50: 2500 ms, p70: 2700, p90: 7100 ms, p99: 8800 ms), latency samples: 331000
5. check swarm health
Compatibility test for b6187e1794550f4a6b25cab0e5d40188574241da ==> 66a8bceb9b4e2483c50734db8406b7cc76163682 passed
Test Ok

@areshand areshand merged commit 55fdf3f into main Sep 26, 2024
45 of 47 checks passed
@areshand areshand deleted the one_shard_per_metric branch September 26, 2024 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants