Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pubsys: limit threads during validate-repo #1564

Merged
merged 1 commit into from
May 7, 2021
Merged

pubsys: limit threads during validate-repo #1564

merged 1 commit into from
May 7, 2021

Conversation

webern
Copy link
Contributor

@webern webern commented May 6, 2021

Issue number:

Closes #1522

Description of changes:

In a previous quick fix we spawned a thread for every target during
pubsys validate-repo. Now we limit the number of threads with a rayon
thread pool.

Testing done:

  • I ran cargo make validate-repo for a production repo with no issues.
  • I did the same for a personal dev repo, no issues.
  • I manipulated a target in the dev repo and tried again and saw an appropriate failure:

Hash mismatch for [snip]/targets/[snip].migrate_v1.1.0_schnauzer-paws.lz4: calculated 2bad2467bcdf174dbb915978bdbcd18c0e79b457b8f52b92750ac5089b6c082e, expected ddc5fa22e8359c66789ce47f437b42652aab80d9a837be61e55cc42a46baca94

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

tools/pubsys/Cargo.toml Outdated Show resolved Hide resolved
tools/pubsys/src/repo/validate_repo/mod.rs Show resolved Hide resolved
tools/pubsys/src/repo/validate_repo/mod.rs Outdated Show resolved Hide resolved
tools/pubsys/src/repo/validate_repo/mod.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@zmrow zmrow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🐕

@webern
Copy link
Contributor Author

webern commented May 7, 2021

webern force-pushed the webern:no-thread-bombing branch from 16cf3fc to c9f6fb7

Uh-oh that's a git mistake horror show. Reverting

@webern
Copy link
Contributor Author

webern commented May 7, 2021

webern force-pushed the webern:no-thread-bombing branch from c9f6fb7 to 16cf3fc now

Puts us back to the initial sha the PR was opened with.

In a previous quick fix we spawned a thread for every target during
pubsys validate-repo. Now we limit the number of threads with a rayon
thread pool.
@webern
Copy link
Contributor Author

webern commented May 7, 2021

webern force-pushed the webern:no-thread-bombing branch from 16cf3fc to 6dee545

implement @tjkirch's suggestions, this time without a git bomb.

@webern webern merged commit 867a7ca into bottlerocket-os:develop May 7, 2021
@webern webern deleted the no-thread-bombing branch May 7, 2021 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pubsys: don't thread bomb in validate-repo
4 participants