Compiling drop(Box<large array>) causes runaway compiler memory usage

[adlerd]

Attempting to compile this code causes extremely large memory usage by rustc.
By experimentation, the important features of the code are: large LIMIT, the element type needs to have drop glue, and the drop can't be skipped with forget or some other mechanism. It also must be a program, not a library function.

src/main.rs

fn main() {
    const LIMIT: usize = 64_000_000;
    // NB. if it compiled successfully, this would just panic, it wouldn't even allocate...
    let _arr: Box<[String; LIMIT]> = Vec::new().into_boxed_slice().try_into().ok().unwrap();
}

Cargo.toml

[package]
name = "repro"
version = "0.1.0"
edition = "2021"

The problematic compiler invocation: cargo build.

I expected to see this happen: The compiler memory usage shouldn't run away due to large LIMIT; in fact the compiler memory usage should be almost entirely independent of LIMIT for this program.

Instead, this happened: When LIMIT is large, rustc uses a lot of memory and takes a long time. I have not successfully compiled the above program with the given LIMIT of 64M.

Meta

I repro'd this on 1.65.0, 1.68.0, and finally:

rustc 1.70.0-nightly (39f2657d1 2023-03-09)
binary: rustc
commit-hash: 39f2657d1101b50f9b71ae460b762d330cc8426b
commit-date: 2023-03-09
host: x86_64-unknown-linux-gnu
release: 1.70.0-nightly
LLVM version: 15.0.7

(I did not capture a compiler backtrace.)

Thoughts

I surmise that there's a correlation between LIMIT and compiler behavior but I haven't explored it beyond verifying a small LIMIT doesn't repro the bad behavior.

I thought perhaps this program tries to use the stack for the slice->array cast, and somehow the compiler is getting stuck on such a large stack frame. Thankfully, this is ruled out by checking that src/alloc/boxed.rs:boxed_slice_as_array_unchecked does indeed do the slice->array cast entirely using pointers. So the compiler shouldn't "see" how big the array is; it's behind a pointer.

Appending std::mem::forget(_arr); to main bypasses the issue, so it appears to be drop glue for the box which causes the problem. I don't see a particular reason why the drop glue for this type should be problematic; it should be no different than dropping the initialized part of a large Vec<String>.

[clubby789]

DEBUG rustc_mir_transform::shim make_shim(DropGlue(DefId(2:2152 ~ core[1238]::ptr::drop_in_place), Some([std::string::String; 64000000])))
DEBUG rustc_mir_transform::shim build_drop_shim(def_id=DefId(2:2152 ~ core[1238]::ptr::drop_in_place), ty=Some([std::string::String; 64000000]))

is what shows up in logging right before hanging + memory explodes

---

rust/compiler/rustc_mir_dataflow/src/elaborate_drops.rs

Line 739 in ff4b772

fn open_drop_for_array(&mut self, ety: Ty<'tcx>, opt_size: Option<u64>) -> BasicBlock {

seems to be the culprit. (0..64000000).map(..).collect()...