std::fs::File::read tries to read more than INT_MAX on OSX and causes EINVAL

[cillian64]

std::fs::File::read calls the libc read function passing the buffer length to read's nbyte (or count) argument [0]. From POSIX the maximum read is SSIZE_MAX [1], although 64-bit MacOS libc is buggy and limits the read to INT_MAX [2].

I tried this code on OSX 10.11.6 (El Capitan):

use std::io::prelude::*;
fn main() {
let mut buf = vec![0u8; 2147483648]; // 2^31
let mut file = std::fs::OpenOptions::new().read(true).open("/etc/group")
.unwrap();
file.read(&mut buf[..]).unwrap();
}

I would expect this to run happily and read the small number of bytes in /etc/group. Instead, libc read returns -1 with the perror message "Invalid argument" causing the last unwrap to panic. I can confirm that this code produces the expected result on 64-bit glibc (ubuntu 16.04).

In theory this could cause problems even on systems without a buggy libc. A slice's maximum size is limited by usize (size_t). On a 32-bit system it should be easy enough (about 2GiB) to create a buffer smaller than SIZE_MAX but bigger than SSIZE_MAX which would cause this problem with glibc. I have not confirmed this.

I feel that File::read should shield the user from these odd implementation issues. Given the uncertain number of bytes read by File::read, I see no harm in capping the read size at INT_MAX or SSIZE_MAX.

[0]: rust/src/libstd/sys/unix/fd.rs :40
[1]: https://linux.die.net/man/2/read
[2]: https://opensource.apple.com/source/Git/Git-61/src/git/wrapper.c?txt :172

Meta

rustc 1.13.0
binary: rustc
commit-hash: unknown
commit-date: unknown
host: x86_64-apple-darwin
release: 1.13.0

[retep998]

For comparison, on Windows Rust already caps the amount that you can read/write in a single call to 0xFFFFFFFF (the maximum value of a DWORD). I see no issue with enforcing similar limits on other platforms.