memory allocation failure during test on 32-bit debian CI

[dkg]

debian continuous integration for xxhash-rust shows failures for the three 32-bit architectures covered: i386 armel and armhf, all for xxhash-rust version 0.8.5. (64-bit platforms don't seem to have a problem).

All the errors on 32-bit platforms appear to be the same, if i'm reading the logs correctly: when the xxh3 test is enabled (or in the --all-targets --all-features test, named rust-xxhash-rust:@, which also has xxh3 enabled), there's a memory allocation failure. From the i386 test logs:

    Running `CARGO=/usr/bin/cargo CARGO_CRATE_NAME=xxhash_rust CARGO_MANIFEST_DIR=/usr/share/cargo/registry/xxhash-rust-0.8.5 CARGO_PKG_AUTHORS='Douman <[email protected]>' CARGO_PKG_DESCRIPTION='Implementation of xxhash' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE=BSL-1.0 CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=xxhash-rust CARGO_PKG_REPOSITORY='https://github.com/DoumanAsh/xxhash-rust' CARGO_PKG_VERSION=0.8.5 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=8 CARGO_PKG_VERSION_PATCH=5 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH='/tmp/tmp.CxqByNHB3a/target/debug/deps:/usr/lib' rustc --crate-name xxhash_rust --edition=2018 src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 --test --cfg 'feature="xxh3"' -C metadata=ee812a8640e9d0d0 -C extra-filename=-ee812a8640e9d0d0 --out-dir /tmp/tmp.CxqByNHB3a/target/i686-unknown-linux-gnu/debug/deps --target i686-unknown-linux-gnu -C incremental=/tmp/tmp.CxqByNHB3a/target/i686-unknown-linux-gnu/debug/incremental -L dependency=/tmp/tmp.CxqByNHB3a/target/i686-unknown-linux-gnu/debug/deps -L dependency=/tmp/tmp.CxqByNHB3a/target/debug/deps --extern getrandom=/tmp/tmp.CxqByNHB3a/target/i686-unknown-linux-gnu/debug/deps/libgetrandom-322ba22e80140a7b.rlib --extern xxhash_c_sys=/tmp/tmp.CxqByNHB3a/target/i686-unknown-linux-gnu/debug/deps/libxxhash_c_sys-9cb2dbdfa5d3f6f6.rlib -C debuginfo=2 --cap-lints warn -C linker=i686-linux-gnu-gcc -C link-arg=-Wl,-z,relro --remap-path-prefix /usr/share/cargo/registry/xxhash-rust-0.8.5=/usr/share/cargo/registry/xxhash-rust-0.8.5 -L native=/usr/lib/i386-linux-gnu`
memory allocation of 39344 bytes failed
error: could not compile `xxhash-rust`

Caused by:
  process didn't exit successfully: `CARGO=/usr/bin/cargo CARGO_CRATE_NAME=test_vectors CARGO_MANIFEST_DIR=/usr/share/cargo/registry/xxhash-rust-0.8.5 CARGO_PKG_AUTHORS='Douman <[email protected]>' CARGO_PKG_DESCRIPTION='Implementation of xxhash' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE=BSL-1.0 CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=xxhash-rust CARGO_PKG_REPOSITORY='https://github.com/DoumanAsh/xxhash-rust' CARGO_PKG_VERSION=0.8.5 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=8 CARGO_PKG_VERSION_PATCH=5 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 CARGO_TARGET_TMPDIR=/tmp/tmp.CxqByNHB3a/target/i686-unknown-linux-gnu/tmp LD_LIBRARY_PATH='/tmp/tmp.CxqByNHB3a/target/debug/deps:/usr/lib' rustc --crate-name test_vectors --edition=2018 tests/test-vectors.rs --error-format=json --json=diagnostic-rendered-ansi --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 --test --cfg 'feature="xxh3"' -C metadata=e84fb12084e50348 -C extra-filename=-e84fb12084e50348 --out-dir /tmp/tmp.CxqByNHB3a/target/i686-unknown-linux-gnu/debug/deps --target i686-unknown-linux-gnu -C incremental=/tmp/tmp.CxqByNHB3a/target/i686-unknown-linux-gnu/debug/incremental -L dependency=/tmp/tmp.CxqByNHB3a/target/i686-unknown-linux-gnu/debug/deps -L dependency=/tmp/tmp.CxqByNHB3a/target/debug/deps --extern getrandom=/tmp/tmp.CxqByNHB3a/target/i686-unknown-linux-gnu/debug/deps/libgetrandom-322ba22e80140a7b.rlib --extern xxhash_c_sys=/tmp/tmp.CxqByNHB3a/target/i686-unknown-linux-gnu/debug/deps/libxxhash_c_sys-9cb2dbdfa5d3f6f6.rlib --extern xxhash_rust=/tmp/tmp.CxqByNHB3a/target/i686-unknown-linux-gnu/debug/deps/libxxhash_rust-35df15a22e0c4ab4.rlib -C debuginfo=2 --cap-lints warn -C linker=i686-linux-gnu-gcc -C link-arg=-Wl,-z,relro --remap-path-prefix /usr/share/cargo/registry/xxhash-rust-0.8.5=/usr/share/cargo/registry/xxhash-rust-0.8.5 -L native=/usr/lib/i386-linux-gnu` (signal: 6, SIGABRT: process abort signal)
autopkgtest [07:29:05]: test librust-xxhash-rust-dev:xxh3: -----------------------]
autopkgtest [07:29:05]: test librust-xxhash-rust-dev:xxh3:  - - - - - - - - - - results - - - - - - - - - -
librust-xxhash-rust-dev:xxh3 FAIL non-zero exit status 101

I'm happy to try to patch the package to gain more debugging information, if that would be useful, or to execute reasonable-looking commands in a test environment. Do you have suggestions on what I should be looking for? If you're running tests upstream on 32-bit platforms that include this feature, can you point me to recent updates, so i can review the differences?

[DoumanAsh]

It is weird that you cannot compile for 32bit target due to insufficient memory.
Is runner for debian CI somehow constrained?

I'm running my tests like that
https://github.com/DoumanAsh/xxhash-rust/runs/5983601494?check_suite_focus=true

cross test --features xxh32,const_xxh32,xxh64,const_xxh64,xxh3,const_xxh3 --release --target i586-unknown-linux-musl

Where cross is tool for cross-compilation, but otherwise it is the same cargo

P.s. It is not really xxh3 fault I think, it is literally compiler not having memory to compile

P.s.s try to remove test-vectors since this is quite heavy test

[dkg]

I agree that test-vectors.rs is where rustc starts to get into runaway RAM usage (several gigs on the amd64 hardware i'm testing on). But i'd also rather not disable any test vectors that you're able to run upstream. (my goal here is to be able to replicate the upstream results!)

Looking at that file, i don't think i understand why rustc would need multiple gigs of RAM to build it. what am i misunderstanding?

Addtionally, i'm a little surprised that the const_xxh3 feature is able to be tested on its own but not the xxh3 feature on its own.

Can you try running a test with only enabling the xxh3 feature on an i386 platform?

something like:

cargo test --no-default-features --features xxh3

[dkg]

I don't know enough about the cross tool, but if it's running on a 64-bit platform, cross-compiling to target a 32-bit platform, that might explain why it's not failing for you, but it is for me.

The debian CI daemons are on machines that have lots of RAM. the smallest ones are 8GiB, and they run all the way into the several hundred GiB range. the i386 example above runs on a machine with 8GiB, and the armhf example ran on a machine with 256GiB.

But rustc on a 32-bit platform is still a 32-bit process, which means it can only address at most 4GiB of RAM (including the kernel). So if test-vectors.rs when run with only the xxh3 feature enabled causes rustc to chew up more than ~3GiB of RAM, it is guaranteed to fail on a 32-bit platform, even if the CI server has a hundred gigs available.

[DoumanAsh]

Can you try running a test with only enabling the xxh3 feature on an i386 platform?

I don't really have i386 so I can only run it in emulation
But it works fine via cross
It doesn't really matter how you enable features, whether it would be all or only xxh3 result is likely to be the same

Addtionally, i'm a little surprised that the const_xxh3 feature is able to be tested on its own but not the xxh3 feature on its own.

Well, test-vectors are only targeting xxh3 and const_xxh3 doesn't really participate in it.

I don't know enough about the cross tool, but if it's running on a 64-bit platform, cross-compiling to target a 32-bit platform, that might explain why it's not failing for you, but it is for me.

It runs build in docker image, likely alpine

I don't really know how much memory rustc would need
You can try to modify codegen-units to 1 to avoid spawning shit ton of threads

[DoumanAsh]

I merged #20 into master
Can you please check it and confirm CI works fine now

[dkg]

Thanks @DoumanAsh -- #20 looks reasonable to me; i've tested it locally (on amd64, not i386) and wow rustc is a lot faster building test-vectors.rs ☺

I had already pushed #19 to debian infrastructure as 0.8.5-2, and it passed the 32-bit CI: https://ci.debian.net/packages/r/rust-xxhash-rust/ -- so we're on the right track.

Now in debian i've rolled back #19, and pushed #20 as 0.8.5-3 . it hasn't hit the CI infrastructure yet, but i'll let you know when it does.

Thanks for persisting on sorting this out!

[dkg]

all the 32-bit architectures tested in the debian CI have now passed 0.8.5-3, so i think that confirms the fix. Whenever you release the next version, i'll pull it into debian and we can drop that patch.:+1: