-
Notifications
You must be signed in to change notification settings - Fork 17.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
internal/poll: CopyFileRange returns ENOTSUP on Linux 3.10.0 kernel on NFS mount #40731
Comments
@gopherbot Please open a backport to 1.15. This bug causes failures doing ordinary file operations on files opened on NFS. The code involved is new in 1.15 FYI @acln0 |
Backport issue(s) opened: #40739 (for 1.15). Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases. |
Change https://golang.org/cl/248258 mentions this issue: |
@nwidger I sent https://golang.org/cl/248258. I'm pretty sure it will fix the problem, as it's approximately what you tried already. Would you mind giving it a try to see if it fixes the problem? Thanks. We'll have to backport this to the 1.15 release branch for the 1.15.1 release. |
@ianlancetaylor I gave your fix a try and it does indeed resolve the problem. Thank you for the quick turnaround! |
@ianlancetaylor Sorry for the trouble. I wasn't aware of this one. It's not documented in the copy_file_range man page, but, after all, I suppose individual file systems can return any error they please. While we are at it, apart from EOPNOTSUPP (which, in hindsight, seems to be one the code should have been aware of), are any other errors that we may have to treat similarly? I can't think of any at the moment, but now would be the best time to deal with them, if they exist. |
@acln0 Linux kernel 3.10 used by the issue reporter doesn't support copy_file_range at all, since the feature has been introduced with 4.5 released in March 2017. So it isn't an individual file system issue. The copy_file_range man file on man7.org contains following in the VERSION section
|
But I thought that was the ENOSYS case, which we covered in the initial implementation. |
@ulikunitz I saw that same comment and wondered the same thing. I've checked and I am running the 3.10.0 kernel with glibc 2.17, therefore the system has neither kernel or user-space emulation support for copy_file_range. After adding some println's to $GOROOT/src/internal/poll/copy_file_range_linux.go, I see that copy_file_range returns EOPNOTSUPP when running my test program within an NFS mount, and ENOSYS within a non-NFS mount such as /tmp. So I think that all makes sense. Let me know if there are any other experiments I can do. |
CentOS provides a vendor kernel that includes backports, which may result in the observed behavior. |
Change https://golang.org/cl/249197 mentions this issue: |
Hi, before I open an issue, I'd like to understand what folks think about this scenario I just encountered: I build my binaries on CircleCI in a Docker container. CircleCI allows the user to pin a specific Docker engine, but they default to a version (17.11.0-ce) that runs on a Docker Host powered by Linux Kernel Despite the above kernel does support the directive
This seems to correspond to the I've filed an issue report with the vendor asking why this is happening and I'm waiting for a confirmation: is this a scenario that we would like to take into account, and fallback as well to the other approach? I spawn up a VirtualBox environment and tried to build the same code that breaks on CircleCI on three different kernels and all the builds were successful, proving that the issue I've encountered is probably related to some limitation/sandboxing artificially imposed by the vendor:
Unfortunately I can't really reproduce the exact setup as CircleCI has its own internal AMIs/images that are not accessible to me, but I asked for the Kernel details and they told me the Docker Host that breaks my linker/builds runs on I'll wait for your advise before opening an issue in case you reckon this is one: I don't think it technically is, but I thought it would be worth it submitting my experience as it might be more common than we think. |
@thoeni your issue appears to be related to a different problem, the original error was |
Thanks Dave, just wanted to make sure I wasn't going to create unnecessary "noise", but I'll open a separate issue as you suggested and if not appropriate we can close it as a "won't fix". Thanks for your feedback on this 🙏 |
Duplicates are a much easier burden to manage vs issues which approach Tolstoy. Thank you for taking the time to raise a new issue. |
…PP as not-handled For #40731 Fixes #40739 Change-Id: I3e29878d597318acf5edcc38497aa2624f72be35 Reviewed-on: https://go-review.googlesource.com/c/go/+/248258 Run-TryBot: Ian Lance Taylor <[email protected]> TryBot-Result: Gobot Gobot <[email protected]> Reviewed-by: Emmanuel Odeke <[email protected]> Reviewed-by: Tobias Klauser <[email protected]> (cherry picked from commit d3a411b) Reviewed-on: https://go-review.googlesource.com/c/go/+/249197 Run-TryBot: Tobias Klauser <[email protected]>
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (
go env
)?Running on CentOS 7.8 with a 3.10.0 kernel.
go env
OutputWhat did you do?
I ran the following program within a directory on an NFS mount.
https://play.golang.org/p/M_6eEgranwf
What did you expect to see?
I expect the program to exit with a zero status code and for the "dst" file to contain the string "test".
What did you see instead?
Exit with status 1 with the following output:
Running the program with strace shows the following output:
here's the full output in case it helps:
strace
OutputI was able to resolve the issue by editing $GOROOT/src/internal/poll/copy_file_range_linux.go such that syscall.ENOTSUP is handled identically to syscall.ENOSYS in poll.CopyFileRange. However, I don't know if this is the correct solution or if there's something else at play here.
The text was updated successfully, but these errors were encountered: