Unable to install rust on Raspberry Pi 3A+ #2564
Comments
|
Frankly this looks like it's running out of RAM installing the component which suggests that you're likely to hit similar problems compiling any non-trivial programs. Either add swap, or cross-build from another more powerful system. I am not sure what else to suggest. |
|
Happens to me too, it worked fine before (1 year ago for sure), for what exactly does rustup needs so much memory? |
|
Try exporting RUSTUP_IO_THREADS=1 before running rustup. The underlying problem is that rust has some very large files (single files > 200MB), and we need to untar the file and write it to disk, but because of interactions with other system constraints like Windows Defender and NFS servers we've settled on concurrent file writing across the board (it took several evolutions to get here, even though it doesn't look like an obvious place to aim); our next iteration is to introduce chunking for large files to deal with this issue, but none of our contributors have had time to write the fairly modest changes needed to do this. |
|
It helped a bit, it failed first time but after restart it probably fit into memory and got through. Oh, I had no idea, I thought the install process just downloads binaries and untars them somewhere (with system tar if possible). Ok, thanks a lot. BTW: could you outline the work needed for handling big files on devices with limited memory? |
|
I am not certain of the details since I've not looked at that part of the codebase in a while, however in brief the work would involve detecting when a particularly large file is encountered in the incoming stream of files, and rather than reading it all in at once and passing it to an IO thread to be written out, operating on the main thread to do that particular file in a streaming fashion. I'm sure @rbtcollins could do a better job of explaining how that might look, if he has some time. |
Fixes rust-lang#2632, rust-lang#2145, rust-lang#2564 Files over 16M are now written incrementally chunks rather than buffered in memory in one full linear buffer. This chunk size is not configurable. For threaded unpacking, the entire memory buffer will be used to buffer chunks and a single worker thread will dispatch IO operations from the buffer, so minimal performance impact should be anticipated (file size/16M round trips at worst, and most network file systems will latency hide linear writes). For immediate unpacking, each chunk is dispatched directly to disk, which may impact performance as less latency hiding is possible - but for immediate unpacking clarity of behaviour is the priority.
Fixes rust-lang#2632, rust-lang#2145, rust-lang#2564 Files over 16M are now written incrementally chunks rather than buffered in memory in one full linear buffer. This chunk size is not configurable. For threaded unpacking, the entire memory buffer will be used to buffer chunks and a single worker thread will dispatch IO operations from the buffer, so minimal performance impact should be anticipated (file size/16M round trips at worst, and most network file systems will latency hide linear writes). For immediate unpacking, each chunk is dispatched directly to disk, which may impact performance as less latency hiding is possible - but for immediate unpacking clarity of behaviour is the priority.
|
Fixed by #2707 |
Fixes rust-lang#2632, rust-lang#2145, rust-lang#2564 Files over 16M are now written incrementally chunks rather than buffered in memory in one full linear buffer. This chunk size is not configurable. For threaded unpacking, the entire memory buffer will be used to buffer chunks and a single worker thread will dispatch IO operations from the buffer, so minimal performance impact should be anticipated (file size/16M round trips at worst, and most network file systems will latency hide linear writes). For immediate unpacking, each chunk is dispatched directly to disk, which may impact performance as less latency hiding is possible - but for immediate unpacking clarity of behaviour is the priority.
Fixes rust-lang#2632, rust-lang#2145, rust-lang#2564 Files over 16M are now written incrementally chunks rather than buffered in memory in one full linear buffer. This chunk size is not configurable. For threaded unpacking, the entire memory buffer will be used to buffer chunks and a single worker thread will dispatch IO operations from the buffer, so minimal performance impact should be anticipated (file size/16M round trips at worst, and most network file systems will latency hide linear writes). For immediate unpacking, each chunk is dispatched directly to disk, which may impact performance as less latency hiding is possible - but for immediate unpacking clarity of behaviour is the priority.
|
For whomever finds this thread while trying to install on raspberry pi zero 2w, I had to |
Sorry for ditching standard template, but It's not really applicable here for me because I hit issue earlier on the process.
Regardless what I do I'm not able to install
ruston myRaspberry Pi 3A+with 512MB of memory.And even worse, installer leaves me with partially broken installation which I'm not able to cleanup afterwards.
Here is the log of
rustupexecution:This issue is related to #2365, but since it was closed, creating a new one.
The text was updated successfully, but these errors were encountered: