Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write block info when restoring sharded DB #15567

Merged
merged 2 commits into from
Dec 12, 2024
Merged

Conversation

areshand
Copy link
Contributor

@areshand areshand commented Dec 11, 2024

Description

How Has This Been Tested?

added new tests

Key Areas to Review

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Move Compiler
  • Other (specify)

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

Copy link

trunk-io bot commented Dec 11, 2024

⏱️ 2h 15m total CI duration on this PR
Slowest 15 Jobs Cumulative Duration Recent Runs
execution-performance / single-node-performance 30m 🟥🟩
rust-move-tests 13m 🟩
rust-move-tests 13m 🟩
rust-move-tests 12m 🟩
rust-move-tests 12m 🟩
rust-cargo-deny 11m 🟩🟩🟩🟩🟩 (+1 more)
check-dynamic-deps 7m 🟩🟩🟩🟩🟩 (+2 more)
rust-move-tests 6m
rust-doc-tests 5m 🟩
execution-performance / test-target-determinator 5m 🟩
test-target-determinator 5m 🟩
check 4m 🟩
semgrep/ci 3m 🟩🟩🟩🟩🟩 (+2 more)
general-lints 3m 🟩🟩🟩🟩🟩 (+1 more)
rust-move-tests 2m

🚨 1 job on the last run was significantly faster/slower than expected

Job Duration vs 7d avg Delta
execution-performance / single-node-performance 24m 20m +20%

settingsfeedbackdocs ⋅ learn more about trunk.io

storage/aptosdb/src/ledger_db/ledger_metadata_db.rs Outdated Show resolved Hide resolved
storage/db-tool/src/tests.rs Outdated Show resolved Hide resolved
@areshand areshand force-pushed the rm_dep_on_internal_indexer branch 2 times, most recently from 0e4c1b7 to 5e62932 Compare December 12, 2024 17:24
@areshand areshand force-pushed the rm_dep_on_internal_indexer branch from 5e62932 to ff1ed14 Compare December 12, 2024 17:30
@areshand areshand changed the title debug Write block info when restoring sharded DB Dec 12, 2024
@areshand areshand force-pushed the rm_dep_on_internal_indexer branch from ff1ed14 to 26eb559 Compare December 12, 2024 17:33
@areshand areshand marked this pull request as ready for review December 12, 2024 17:34
@areshand areshand enabled auto-merge (rebase) December 12, 2024 19:07

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite compat success on 3c6e693a27339e73520f41030dce8fc9cd504967 ==> d30ea4945b54965fec09b3342e2e684f54e6229b

Compatibility test results for 3c6e693a27339e73520f41030dce8fc9cd504967 ==> d30ea4945b54965fec09b3342e2e684f54e6229b (PR)
1. Check liveness of validators at old version: 3c6e693a27339e73520f41030dce8fc9cd504967
compatibility::simple-validator-upgrade::liveness-check : committed: 16870.88 txn/s, latency: 1996.76 ms, (p50: 1900 ms, p70: 2100, p90: 2400 ms, p99: 3300 ms), latency samples: 547960
2. Upgrading first Validator to new version: d30ea4945b54965fec09b3342e2e684f54e6229b
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 6314.64 txn/s, latency: 4550.57 ms, (p50: 5200 ms, p70: 5500, p90: 5700 ms, p99: 5900 ms), latency samples: 113360
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 6647.25 txn/s, latency: 4884.83 ms, (p50: 5200 ms, p70: 5300, p90: 5300 ms, p99: 5400 ms), latency samples: 232880
3. Upgrading rest of first batch to new version: d30ea4945b54965fec09b3342e2e684f54e6229b
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 6421.22 txn/s, latency: 4492.17 ms, (p50: 5000 ms, p70: 5300, p90: 5700 ms, p99: 5800 ms), latency samples: 121640
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 6717.50 txn/s, latency: 4930.97 ms, (p50: 5300 ms, p70: 5400, p90: 5600 ms, p99: 5800 ms), latency samples: 225960
4. upgrading second batch to new version: d30ea4945b54965fec09b3342e2e684f54e6229b
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 12403.47 txn/s, latency: 2244.84 ms, (p50: 2300 ms, p70: 2700, p90: 2800 ms, p99: 3100 ms), latency samples: 216280
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 12920.98 txn/s, latency: 2444.48 ms, (p50: 2400 ms, p70: 2800, p90: 2900 ms, p99: 3300 ms), latency samples: 421200
5. check swarm health
Compatibility test for 3c6e693a27339e73520f41030dce8fc9cd504967 ==> d30ea4945b54965fec09b3342e2e684f54e6229b passed
Test Ok

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on d30ea4945b54965fec09b3342e2e684f54e6229b

two traffics test: inner traffic : committed: 14778.36 txn/s, latency: 2685.76 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 3300 ms), latency samples: 5619060
two traffics test : committed: 100.06 txn/s, latency: 1410.99 ms, (p50: 1400 ms, p70: 1400, p90: 1500 ms, p99: 1600 ms), latency samples: 1700
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 1.554, avg: 1.489", "ConsensusProposalToOrdered: max: 0.313, avg: 0.292", "ConsensusOrderedToCommit: max: 0.378, avg: 0.366", "ConsensusProposalToCommit: max: 0.669, avg: 0.657"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.56s no progress at version 29626 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.69s no progress at version 2261723 (avg 0.69s) [limit 16].
Test Ok

Copy link
Contributor

✅ Forge suite framework_upgrade success on 3c6e693a27339e73520f41030dce8fc9cd504967 ==> d30ea4945b54965fec09b3342e2e684f54e6229b

Compatibility test results for 3c6e693a27339e73520f41030dce8fc9cd504967 ==> d30ea4945b54965fec09b3342e2e684f54e6229b (PR)
Upgrade the nodes to version: d30ea4945b54965fec09b3342e2e684f54e6229b
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1487.68 txn/s, submitted: 1491.99 txn/s, failed submission: 4.30 txn/s, expired: 4.30 txn/s, latency: 2020.72 ms, (p50: 2000 ms, p70: 2100, p90: 2700 ms, p99: 3900 ms), latency samples: 131380
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1313.49 txn/s, submitted: 1315.90 txn/s, failed submission: 2.41 txn/s, expired: 2.41 txn/s, latency: 2245.12 ms, (p50: 2100 ms, p70: 2400, p90: 3000 ms, p99: 4500 ms), latency samples: 119840
5. check swarm health
Compatibility test for 3c6e693a27339e73520f41030dce8fc9cd504967 ==> d30ea4945b54965fec09b3342e2e684f54e6229b passed
Upgrade the remaining nodes to version: d30ea4945b54965fec09b3342e2e684f54e6229b
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1502.47 txn/s, submitted: 1503.10 txn/s, failed submission: 0.63 txn/s, expired: 0.63 txn/s, latency: 2818.11 ms, (p50: 2100 ms, p70: 3000, p90: 6000 ms, p99: 7400 ms), latency samples: 95620
Test Ok

@areshand areshand merged commit 8616cb6 into main Dec 12, 2024
79 of 88 checks passed
@areshand areshand deleted the rm_dep_on_internal_indexer branch December 12, 2024 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants