-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
File.Copy: Clone file when possible on MacOS #77835
Comments
Tagging subscribers to this area: @dotnet/area-system-io Issue DetailsFor runtime/src/native/libs/System.Native/pal_io.c Line 1257 in d320fd9
If we instead use copyfile() we can use the COPYFILE_CLONE option (with the existing COPYFILE_ALL obviously), which allows the file to be instantly copied on supported filesystems, and falls back to exactly the same implementation when it's not available (note - this behaves identically (except with symlinks (ie. this copies the symlink itself I think) and when the destination exists, which we can special case), it just allows APFS to immediately set the data location to the existing file, and when it's updated it will be create a copy since APFS is a copy-on-write filesystem). This is obviously much faster (I tested it on a hard drive with a 30GB file and it copied instantly).
We can implement that in this file:
by splitting the OSX-like platforms code, like we have with the code that sets the creation date. We should be able to implement it with exactly the same behaviour as whatever the current behaviour is with relative ease. I'd like to implement this in a PR if approved (ideally by .NET 8).
|
Can someone also tag this with the performance improvement tag (tenet I think). |
It was discussed in the past: #26713 This comment is most interesting: #26713 (comment)
cc @filipnavara who contributed dotnet/corefx#37583 |
There's more than one closed PR that attempted to do this. The work can be resurrected. The main problem was duplicating logic between the managed side of the code and the native side of the code that dealt with the file sharing emulation. |
The behaviour discovered in mono/mono#10020 is now documented in the man pages:
|
I think the last attempt was #33159. |
If you could find links to all of the previous attempts that'd be great (if they all link to each other in the description that is fine as well) so I can see what was there and what went wrong with it. If you're happy for me to have a go, can you please 'assign' this issue to me, thanks. |
Can you please explain to me what the issue with file locking is specifically related to this feature. I read through the issue you commented with and some other linked ones (including the ones about file locking itself), and I don't understand what it has to do with The solution that I plan to implement is (with appropriate platform checks and error handling):
|
.NET uses the POSIX advisory locks to implement the FileShare option of If |
• Use copyfile (with COPYFILE_CLONE_FORCE) when possible on macOS to clone the file while still keeping file locking logic intact • Split common Unix logic into multiple functions that the macOS implementation uses parts of at different times • Add string version of ResolveLinkTarget to save the allocation since part of the code needs it • Need to add tests to check the file is actually cloned so we know if it works or not
• I missed setting StringMarshalling on the LibraryImport • I missed partial on the CopyFile method implementations
• Apparently 'Both partial method declarations must be unsafe or neither may be unsafe'
• I love partial methods • Move unsafe keyword into the OSX method rather than in method declaration • Fix accessibility modifier of CopyFile in FileSystem.Unix.cs
• Fix indentation (should have used spaces) • Import Microsoft.Win32.SafeHandles in FileSystem.CopyFile.OSX.cs file • Fix for SA1205 'Partial elements should declare an access modifier' • Fix typo in FileSystem.CopyFile.OtherUnix.cs of 'startedCopyFile' • Fix missing openDst parameter code in StartCopyFile • Fix not checking the nullability in StandardCopyFile, which led to 'Possible null reference argument for parameter'
• Add missing parameter value (openNewFile) to OpenCopyFileDstHandle • Fix misnamed variable usages throughout some code • Properly qualify Interop.ErrorInfo • This should be the last fix for build issues for this set
• Add missing nullable ? for dstHandle (which obtains the file lock)
• Test failures were since copying a file onto itself should cause an error, this fixes that
• Fix nullability issues
…ain) • The variable in the if statements weren't updated • Handling for if readlink fails
…t#77835' • The source path should have used sourceFullPath rather than destPath
Can someone re-open my PR #79243 please. |
For
File.Copy
on macOS we're currently doing the following:runtime/src/native/libs/System.Native/pal_io.c
Line 1257 in d320fd9
If we instead use
copyfile()
we can use theCOPYFILE_CLONE
option (with the existingCOPYFILE_ALL
obviously, which I think the clone option may include all of the flags of anyway), which allows the file to be instantly copied on supported filesystems, and falls back to exactly the same implementation when it's not available (note - this behaves identically (except with symlinks (ie. this copies the symlink itself I think) and when the destination exists, which we can special case), it just allows APFS to immediately set the data location to the existing file, and when it's updated it will be create a copy since APFS is a copy-on-write filesystem). This is obviously much faster (I tested it on a hard drive with a 30GB file and it copied instantly).We can implement that in this file:
runtime/src/libraries/System.Private.CoreLib/src/System/IO/FileSystem.Unix.cs
Line 31 in 21e9913
by splitting the OSX-like platforms code, like we have with the code that sets the creation date.
We should be able to implement it with effectively the same behaviour as whatever the current behaviour is with relative ease. Note: apps like finder clone files in this way.
I'd like to implement this in a PR if approved (ideally by .NET 8).
The text was updated successfully, but these errors were encountered: