Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG] fixing cross-platform cibuildwheel action for release #2384

Merged
merged 11 commits into from
Dec 2, 2022

Conversation

ctb
Copy link
Contributor

@ctb ctb commented Dec 2, 2022

The QEMU/cibuildwheel based wheel-builds were exiting on cargo update:

2022-12-01T17:46:44.0138993Z   copying src/sourmash/cli/sketch/translate.py -> build/lib/sourmash/cli/sketch
2022-12-01T17:46:47.2377148Z       Updating crates.io index
2022-12-01T17:48:47.6753837Z   error: subprocess-exited-with-error
2022-12-01T17:48:47.6779277Z   
2022-12-01T17:48:47.6796452Z   × Building wheel for sourmash (pyproject.toml) did not run successfully.
2022-12-01T17:48:47.6834234Z   │ exit code: 247
2022-12-01T17:48:47.6839824Z   ╰─> See above for output.
2022-12-01T17:48:47.6851498Z   

which seems to have been caused by an out-of-memory error; see https://users.rust-lang.org/t/cargo-uses-too-much-memory-being-run-in-qemu/76531 for what looks like the same problem.

This PR fixes the problem using the same fix as in https://users.rust-lang.org/t/cargo-uses-too-much-memory-being-run-in-qemu/76531.

@codecov
Copy link

codecov bot commented Dec 2, 2022

Codecov Report

Merging #2384 (738305d) into latest (6f25fbf) will increase coverage by 8.11%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           latest    #2384      +/-   ##
==========================================
+ Coverage   84.08%   92.19%   +8.11%     
==========================================
  Files         130      101      -29     
  Lines       15070    11522    -3548     
  Branches     2212     2212              
==========================================
- Hits        12671    10623    -2048     
+ Misses       2103      603    -1500     
  Partials      296      296              
Flag Coverage Δ
python 92.19% <ø> (ø)
rust ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/core/src/from.rs
src/core/src/index/bigsi.rs
src/core/src/ffi/hyperloglog.rs
src/core/src/ffi/index/revindex.rs
src/core/src/index/search.rs
src/core/src/signature.rs
src/core/src/sketch/minhash.rs
src/core/tests/minhash.rs
src/core/src/storage.rs
src/core/src/encodings.rs
... and 19 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@ctb
Copy link
Contributor Author

ctb commented Dec 2, 2022

AH-HAH! 9367075 induces the failure,

  info: downloading component 'rust-std' for 'aarch64-apple-darwin'
  info: installing component 'rust-std' for 'aarch64-apple-darwin'
      Updating crates.io index
  .ci/install_cargo.sh: line 8:   219 Killed                  cargo update
                                                            ✕ 167.23s
Error: Command ['sh', '-c', 'source .ci/install_cargo.sh'] failed with code 137. 

and this eventually led me to THIS:

https://users.rust-lang.org/t/cargo-uses-too-much-memory-being-run-in-qemu/76531

which seems to be the same problem we're facing!!!

Now to see if their solution works...

@ctb
Copy link
Contributor Author

ctb commented Dec 2, 2022

aaaaand this issue too - rust-lang/cargo#10583

@ctb
Copy link
Contributor Author

ctb commented Dec 2, 2022

it worked! firing for effect...

@ctb ctb marked this pull request as ready for review December 2, 2022 06:01
@ctb ctb changed the title [WIP] Work on fixing cibuildwheel [MRG] fixing cross-platform cibuildwheel action for release Dec 2, 2022
@ctb
Copy link
Contributor Author

ctb commented Dec 2, 2022

Merging into latest to check that it works there, too.

@ctb ctb merged commit fa46307 into latest Dec 2, 2022
@ctb ctb deleted the fix/ubuntu_wheel_build branch December 2, 2022 06:02
ctb added a commit that referenced this pull request Dec 2, 2022
ctb added a commit that referenced this pull request Dec 2, 2022
This PR continues the cibuildwheel exploration that started with a fix
for building the wheels
(#2384 and #2385).

The saga continues like so:

When building the release candidates, we don't get "clean" rcN wheels. I
think this is because when I run `cargo update` in the cibuildwheel
action it's actually updating the Rust packages and we're getting a
version bump from that. Yay? (See screenshot of releases page with 'dev'
yada in wheel names.)

<img width="1025" alt="Screen Shot 2022-12-02 at 7 15 52 AM"
src="https://user-images.githubusercontent.com/51016/205325234-9ec0c6c2-7cbc-42c7-9727-fd700ef59320.png">

This PR changes `cargo update` to `cargo update --dry-run` which should
update the crates.io index without actually changing `Cargo.lock` and
pushing a version bump.

We Shall See!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant