Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[move] Use workspace dependencies for third-party/move #15461

Merged
merged 2 commits into from
Dec 5, 2024

Conversation

georgemitenkov
Copy link
Contributor

@georgemitenkov georgemitenkov commented Dec 3, 2024

Description

We have been using workspace vs path dependencies for third_party/move pretty randomly, ususally copy pasting them. This PR scans through all cargo dependencies and ensures that

  • all move crates use workspace dependency keys if possible
  • move crates reuse external crates from workspace for the same version (e.g., I had to keep rand at two versions. Can we upgrade on Aptos side?)
  • move dependencies are properly ordered

The goal is to make sure syncing with move-on-aptos repo is easy: when adding new crates and dependencies we would only need to modify the root Cargo.toml there to add a new create/dependency. Right now, this is not possible.

How Has This Been Tested?

Key Areas to Review

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Move Compiler
  • Other (specify)

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

Copy link

trunk-io bot commented Dec 3, 2024

⏱️ 1h 35m total CI duration on this PR
Slowest 15 Jobs Cumulative Duration Recent Runs
execution-performance / single-node-performance 24m 🟩
rust-move-tests 13m 🟩
rust-move-tests 12m 🟩
rust-move-tests 8m
rust-cargo-deny 7m 🟩🟩🟩🟩
rust-doc-tests 6m 🟩
check-dynamic-deps 4m 🟩🟩🟩🟩
execution-performance / test-target-determinator 4m 🟩
test-target-determinator 4m 🟩
check 4m 🟩
rust-move-tests 2m
general-lints 2m 🟩🟩🟩🟩
semgrep/ci 2m 🟩🟩🟩🟩
fetch-last-released-docker-image-tag 1m 🟩
file_change_determinator 46s 🟩🟩🟩🟩

🚨 1 job on the last run was significantly faster/slower than expected

Job Duration vs 7d avg Delta
execution-performance / single-node-performance 24m 20m +22%

settingsfeedbackdocs ⋅ learn more about trunk.io

Copy link
Contributor Author

georgemitenkov commented Dec 3, 2024

Copy link
Contributor

@vineethk vineethk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

I added a comment in places where I thought we could use "workspace = true", but there could be good reasons why you left them as is (but it was not obvious to me). I realized the comments applied to the same small set of packages in various places, so if you justify them once per unique package, that should suffice.

testsuite/fuzzer/Cargo.toml Outdated Show resolved Hide resolved
third_party/move/evm/move-to-yul/Cargo.toml Show resolved Hide resolved
move-cli = { path = "../../tools/move-cli" }
move-package = { path = "../../tools/move-package" }
move-cli = { workspace = true }
move-package = { workspace = true }
move-stdlib = { path = "../../move-stdlib", features = ["testing"] }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be made { workspace = true }?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For stdlib, I am not sure mostly because we have 2 of those😄 one we use for Aptos (as a workspace dependency aptos-move-stdlib), and one defined in third-party move. I think the reason for this is gas, and we want to move some of gas infra (cc @vgao1996 to confirm my thinking) to third party to be able to use it directly. I did not add it as a workspace dependency to avoid confusion between the two.

third_party/move/move-compiler/Cargo.toml Show resolved Hide resolved
third_party/move/tools/move-cli/Cargo.toml Show resolved Hide resolved
move-stdlib = { path = "../../move-stdlib", features = ["testing"] }
move-symbol-pool = { path = "../../move-symbol-pool" }
move-symbol-pool = { workspace = true }
move-table-extension = { path = "../../extensions/move-table-extension" }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be made { workspace = true }?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here it is the same as for move-stdlib. We have a copy of table extension in our aptos tree due to gas charging (aptos-move-table-extension or aptos-table-extension iirc), and so this one is actually not used, so we should not add it to the workspace to avoid having two extensions :)

@georgemitenkov georgemitenkov changed the base branch from george/copybara-move-on-aptos to main December 5, 2024 18:35
@georgemitenkov georgemitenkov force-pushed the george/move-on-aptos-fixes branch from b020c0c to e88cfa2 Compare December 5, 2024 18:35
@georgemitenkov georgemitenkov enabled auto-merge (squash) December 5, 2024 18:38

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

github-actions bot commented Dec 5, 2024

✅ Forge suite compat success on c2969f0b0ecd826aa1516a386aed59496a247958 ==> e88cfa24deb0e4e798115559920ec568a1945bad

Compatibility test results for c2969f0b0ecd826aa1516a386aed59496a247958 ==> e88cfa24deb0e4e798115559920ec568a1945bad (PR)
1. Check liveness of validators at old version: c2969f0b0ecd826aa1516a386aed59496a247958
compatibility::simple-validator-upgrade::liveness-check : committed: 15260.57 txn/s, latency: 2193.70 ms, (p50: 2100 ms, p70: 2200, p90: 2800 ms, p99: 4500 ms), latency samples: 502100
2. Upgrading first Validator to new version: e88cfa24deb0e4e798115559920ec568a1945bad
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 5787.47 txn/s, latency: 5140.89 ms, (p50: 5800 ms, p70: 6100, p90: 6300 ms, p99: 6400 ms), latency samples: 113460
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 6020.72 txn/s, latency: 5456.32 ms, (p50: 6000 ms, p70: 6100, p90: 6400 ms, p99: 6600 ms), latency samples: 205200
3. Upgrading rest of first batch to new version: e88cfa24deb0e4e798115559920ec568a1945bad
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 7345.13 txn/s, latency: 3883.58 ms, (p50: 4300 ms, p70: 4600, p90: 4700 ms, p99: 4800 ms), latency samples: 136700
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 7322.71 txn/s, latency: 4411.00 ms, (p50: 4700 ms, p70: 4800, p90: 4800 ms, p99: 4900 ms), latency samples: 252300
4. upgrading second batch to new version: e88cfa24deb0e4e798115559920ec568a1945bad
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 9470.96 txn/s, latency: 2844.15 ms, (p50: 2500 ms, p70: 3400, p90: 4300 ms, p99: 5200 ms), latency samples: 173740
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 10083.22 txn/s, latency: 3158.10 ms, (p50: 2500 ms, p70: 3900, p90: 5100 ms, p99: 5200 ms), latency samples: 335460
5. check swarm health
Compatibility test for c2969f0b0ecd826aa1516a386aed59496a247958 ==> e88cfa24deb0e4e798115559920ec568a1945bad passed
Test Ok

Copy link
Contributor

github-actions bot commented Dec 5, 2024

✅ Forge suite realistic_env_max_load success on e88cfa24deb0e4e798115559920ec568a1945bad

two traffics test: inner traffic : committed: 14523.30 txn/s, latency: 2729.83 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 3300 ms), latency samples: 5522100
two traffics test : committed: 100.07 txn/s, latency: 1502.19 ms, (p50: 1400 ms, p70: 1500, p90: 1600 ms, p99: 2500 ms), latency samples: 1780
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 1.553, avg: 1.492", "ConsensusProposalToOrdered: max: 0.328, avg: 0.297", "ConsensusOrderedToCommit: max: 0.411, avg: 0.390", "ConsensusProposalToCommit: max: 0.706, avg: 0.686"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.57s no progress at version 17515 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.66s no progress at version 2846113 (avg 0.66s) [limit 16].
Test Ok

Copy link
Contributor

github-actions bot commented Dec 5, 2024

✅ Forge suite framework_upgrade success on c2969f0b0ecd826aa1516a386aed59496a247958 ==> e88cfa24deb0e4e798115559920ec568a1945bad

Compatibility test results for c2969f0b0ecd826aa1516a386aed59496a247958 ==> e88cfa24deb0e4e798115559920ec568a1945bad (PR)
Upgrade the nodes to version: e88cfa24deb0e4e798115559920ec568a1945bad
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1551.09 txn/s, submitted: 1555.78 txn/s, failed submission: 4.69 txn/s, expired: 4.69 txn/s, latency: 1931.54 ms, (p50: 1800 ms, p70: 2100, p90: 2400 ms, p99: 3600 ms), latency samples: 138980
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1376.44 txn/s, submitted: 1379.99 txn/s, failed submission: 3.55 txn/s, expired: 3.55 txn/s, latency: 2186.46 ms, (p50: 2100 ms, p70: 2400, p90: 3300 ms, p99: 4500 ms), latency samples: 124020
5. check swarm health
Compatibility test for c2969f0b0ecd826aa1516a386aed59496a247958 ==> e88cfa24deb0e4e798115559920ec568a1945bad passed
Upgrade the remaining nodes to version: e88cfa24deb0e4e798115559920ec568a1945bad
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1321.55 txn/s, submitted: 1326.68 txn/s, failed submission: 4.91 txn/s, expired: 5.12 txn/s, latency: 2316.74 ms, (p50: 2100 ms, p70: 2400, p90: 3600 ms, p99: 4800 ms), latency samples: 118381
Test Ok

@georgemitenkov georgemitenkov merged commit 95e0d68 into main Dec 5, 2024
85 of 88 checks passed
@georgemitenkov georgemitenkov deleted the george/move-on-aptos-fixes branch December 5, 2024 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants