Fix node db cache and e2e workflow improvements #3031
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
piotr-iohk
force-pushed
the
piotr/adp-1196/node-db-cache
branch
from
6a7b133 to
0d752a0
Compare
rvl
approved these changes
Nov 23, 2021
test/e2e/Rakefile
Outdated
| # $ rake datetime | ||
| # 20211122-94332 | ||
| task :datetime do | ||
| puts Time.now.strftime('%Y%m%d-%-k%M%S') |
There was a problem hiding this comment.
It should be HHMMSS, right?
Suggested change
| puts Time.now.strftime('%Y%m%d-%-k%M%S') | |
| puts Time.now.strftime('%Y%m%d-%H%M%S') |
| key: ${{ runner.os }}-node-cache-v2-${{ env.NETWORK }} | ||
| key: node-db-${{ runner.os }}-${{ env.NETWORK }}-${{ steps.date-time.outputs.value }} | ||
| restore-keys: | | ||
| node-db-${{ runner.os }}-${{ env.NETWORK }}- |
|
bors r+ |
iohk-bors bot
added a commit
that referenced
this pull request
Nov 23, 2021
3031: Fix node db cache and e2e workflow improvements r=piotr-iohk a=piotr-iohk - 5cccf82 Attempt to fix node db caching - 5356643 More granular steps in e2e workflows - 087f7a5 Use Linux workflow cache if other not available - 0d752a0 Docker workflow adjustments - e2b2ef5 update README ### Comments Improvements made here and in #3027 bring significant improvements in speed and reliability of e2e tests. Tests on MacOS and Linux used to take 2-3h overall and on Windows even up to 6h (which often led to exceeding GH actions timeout). This was mainly due to incorrect use of node db caching which caused a lot of time spend of syncing node db against testnet. Improved execution times are as follows: | Linux|MacOS|Windows|Docker| |--|--|--|--| |[1h](https://github.com/input-output-hk/cardano-wallet/actions/runs/1490250065)|[1h 6m](https://github.com/input-output-hk/cardano-wallet/actions/runs/1490370180)|[1h 31m](https://github.com/input-output-hk/cardano-wallet/actions/runs/1490442118)|[1h](https://github.com/input-output-hk/cardano-wallet/actions/runs/1493877513)| Due to node db caching, which is happening properly now on every run, node db syncing time is cut from few hours to few minutes 🎉. Majority of the remaining test suite time is still spend on syncing fixture wallets, but it is good test on it's own so there seems to be no need for further improvements. Execution times seem to be satisfactory for nightly e2e tests and also in the need for an ad-hoc e2e run. ### Issue Number ADP-1196 3032: Remove redundant imports and constraints. r=Anviking a=jonathanknowles ## Issue Number None ## Summary This PR removes some redundant imports and constraints. I collected these patches while working on other things. It seemed sensible to put them in their own PR. Co-authored-by: Piotr Stachyra <[email protected]> Co-authored-by: Jonathan Knowles <[email protected]>
|
This PR was included in a batch that timed out, it will be automatically retried |
piotr-iohk
force-pushed
the
piotr/adp-1196/node-db-cache
branch
from
be4bed8 to
7f3cdce
Compare
|
Canceled. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
5cccf82
Attempt to fix node db caching
5356643
More granular steps in e2e workflows
087f7a5
Use Linux workflow cache if other not available
0d752a0
Docker workflow adjustments
e2b2ef5
update README
Comments
Improvements made here and in #3027 bring significant improvements in speed and reliability of e2e tests. Tests on MacOS and Linux used to take 2-3h overall and on Windows even up to 6h (which often led to exceeding GH actions timeout). This was mainly due to incorrect use of node db caching which caused a lot of time spend of syncing node db against testnet. Improved execution times are as follows:
Due to node db caching, which is happening properly now on every run, node db syncing time is cut from few hours to few minutes 🎉. Majority of the remaining test suite time is still spend on syncing fixture wallets, but it is good test on it's own so there seems to be no need for further improvements. Execution times seem to be satisfactory for nightly e2e tests and also in the need for an ad-hoc e2e run.
Issue Number
ADP-1196