Compress KM dumps and perf ETLs before upload. #3029
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rectified95
changed the title
rectified95
force-pushed
the
compress_artifact
branch
from
ac86abe
to
9e994ff
Compare
added 2 commits
November 7, 2023 17:19
rectified95
force-pushed
the
compress_artifact
branch
from
9e994ff
to
f35f000
Compare
rectified95
force-pushed
the
compress_artifact
branch
from
9f39512
to
417a074
Compare
rectified95
requested review from
dthaler,
poornagmsft,
Alan-Jowett,
saxena-anurag,
shankarseal,
dv-msft,
gtrevi,
shpalani,
matthewige and
mtfriesen
as code owners
Alan-Jowett
approved these changes
Nov 8, 2023
gtrevi
approved these changes
Nov 8, 2023
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Related #2948
The bad runner which failed to upload crash dumps and perf traces seems better now - connected the vSwitch to a faster adapter.
To further facilitate uploads, first compress the large files.
KM dumps 600MB -> 60MB
Perf ETLs 1.1GB -> 110MB
Testing
Modified the km_stress and perf_with_profile jobs to run on PR as a one-off, and observed the files were successfully uploaded:
km_stress crash dumps upload logs:
https://github.com/microsoft/ebpf-for-windows/actions/runs/6792240647/job/18465751588
perf profile upload logs:
https://github.com/microsoft/ebpf-for-windows/actions/runs/6794724707/job/18471866156