asm: Since nightly-2024-08-01 (LLVM 19), in and out is sometime allocated to the same register

[taiki-e]

Repro:

# Prerequisite: Environment that can compile or cross-compile aach64-linux-gnu
git clone https://github.com/taiki-e/atomic-maybe-uninit.git
cd atomic-maybe-uninit
git checkout v0.3.2
RUSTFLAGS='-C target-feature=+lse' \
  ./tools/test.sh --tests --target aarch64-unknown-linux-gnu --release --no-run

I expected to see this happen: compile pass

Instead, this happened:

error: unpredictable STXP instruction, status is also a source
   --> src/arch/aarch64.rs:571:37
    |
571 | ...                   concat!("st", $release, "xp {tmp:w}, {val_lo}, {val_hi}, [{dst}]"),
    |                       ^
    |
note: instantiated into assembly here
   --> <inline asm>:3:6
    |
3   | stxp w8, x8, x23, [x9]
    |      ^

error: unpredictable STXP instruction, status is also a source
   --> src/arch/aarch64.rs:571:37
    |
571 | ...                   concat!("st", $release, "xp {tmp:w}, {val_lo}, {val_hi}, [{dst}]"),
    |                       ^
    |
note: instantiated into assembly here
   --> <inline asm>:3:7
    |
3   | stlxp w8, x8, x23, [x9]
    |       ^

This is the inline assembly that received the error, and tmp and val_lo which the compiler assigned the same registers are out and in, respectively.

                                val_lo = in(reg) val.pair.lo,
                                val_hi = in(reg) val.pair.hi,
                                tmp = out(reg) _,

Those two must not be allocated the same register, but for some reason the compiler seems to be treating out as if it were lateout.

lateout(<reg>) <expr>
Identical to out except that the register allocator can reuse a register allocated to an in.

I have confirmed that at least the following three targets are affected:

• aarch64-unknown-linux-gnu (ci log)
• aarch64_be-unknown-linux-gnu (ci log)
• riscv64gc-unknown-linux-gnu (ci log)

As for riscv64gc-unknown-linux-gnu, I consistently get the unexpected result at runtime (not compile error) in exactly the same version range.

In the aarch64 case, the problem appeared in a form that the assembler could detect as an error, resulting in a compilation error, but in some situations, as in the riscv64 case, the code may be silently miscompiled.

Meta

rustc --version --verbose:

rustc 1.82.0-nightly (f167efad2 2024-08-24)
binary: rustc
commit-hash: f167efad2f51088d86180ee89177b3d7c9e7c2f5
commit-date: 2024-08-24
host: aarch64-unknown-linux-gnu
release: 1.82.0-nightly
LLVM version: 19.1.0

It can be reproduced with nightly-2024-08-01 or later.

rustc 1.82.0-nightly (28a58f2fa 2024-07-31)
binary: rustc
commit-hash: 28a58f2fa7f0c46b8fab8237c02471a915924fe5
commit-date: 2024-07-31
host: aarch64-unknown-linux-gnu
release: 1.82.0-nightly
LLVM version: 19.1.0

The last version not affected by the problem is nightly-2024-07-31, which uses LLVM 18, so it seems likely that the update to LLVM 19 (#127513) caused this regression.
cc @nikic

rustc 1.82.0-nightly (f8060d282 2024-07-30)
binary: rustc
commit-hash: f8060d282d42770fadd73905e3eefb85660d3278
commit-date: 2024-07-30
host: aarch64-unknown-linux-gnu
release: 1.82.0-nightly
LLVM version: 18.1.7

---

@rustbot label +I-unsound, +A-LLVM, +A-inline-assembly

[apiraino]

WG-prioritization assigning priority (Zulip discussion).

@rustbot label -I-prioritize +P-high

[nikic]

Upstream issue: llvm/llvm-project#106380

[nikic]

Worth noting that the LLVM IR reduction also fails in the same way in previous LLVM versions, so this is not an LLVM 19 regression in register allocations -- likely things just get perturbed enough that we now hit a pre-existing issue.

[nikic]

This is fixed upstream now, but the required changes were fairly extensive, so this won't get backported to LLVM 19. I'd suggest working around this by laundering the possibly-uninitialized inputs through a black_box. That should prevent LLVM from incorrectly allocating the registers.