Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clone files on OSX-like platforms when possible, instead of copying the whole file #79243

Merged
merged 85 commits into from
Jun 13, 2023

Commits on Dec 5, 2022

  1. Initial implementation of fix for dotnet#77835 - cloning files on macOS

    • Use copyfile (with COPYFILE_CLONE_FORCE) when possible on macOS to clone the file while still keeping file locking logic intact
    • Split common Unix logic into multiple functions that the macOS implementation uses parts of at different times
    • Add string version of ResolveLinkTarget to save the allocation since part of the code needs it
    • Need to add tests to check the file is actually cloned so we know if it works or not
    hamarb123 committed Dec 5, 2022
    Configuration menu
    Copy the full SHA
    a373c59 View commit details
    Browse the repository at this point in the history
  2. Fix some build issues from previous commit for dotnet#77835

    • I missed setting StringMarshalling on the LibraryImport
    • I missed partial on the CopyFile method implementations
    hamarb123 committed Dec 5, 2022
    Configuration menu
    Copy the full SHA
    f21a5ae View commit details
    Browse the repository at this point in the history
  3. Fix some build issues from previous commit for dotnet#77835 again

    • Apparently 'Both partial method declarations must be unsafe or neither may be unsafe'
    hamarb123 committed Dec 5, 2022
    Configuration menu
    Copy the full SHA
    1890115 View commit details
    Browse the repository at this point in the history
  4. Fix some build issues from previous commit for dotnet#77835 again (2)

    • I love partial methods
    • Move unsafe keyword into the OSX method rather than in method declaration
    • Fix accessibility modifier of CopyFile in FileSystem.Unix.cs
    hamarb123 committed Dec 5, 2022
    Configuration menu
    Copy the full SHA
    e490d5e View commit details
    Browse the repository at this point in the history
  5. Fix some build issues from previous commit for dotnet#77835 again (3)

    • Fix indentation (should have used spaces)
    • Import Microsoft.Win32.SafeHandles in FileSystem.CopyFile.OSX.cs file
    • Fix for SA1205 'Partial elements should declare an access modifier'
    • Fix typo in FileSystem.CopyFile.OtherUnix.cs of 'startedCopyFile'
    • Fix missing openDst parameter code in StartCopyFile
    • Fix not checking the nullability in StandardCopyFile, which led to 'Possible null reference argument for parameter'
    hamarb123 committed Dec 5, 2022
    Configuration menu
    Copy the full SHA
    1e43a91 View commit details
    Browse the repository at this point in the history
  6. Fix some build issues from previous commit for dotnet#77835 again (4)

    • Add missing parameter value (openNewFile) to OpenCopyFileDstHandle
    • Fix misnamed variable usages throughout some code
    • Properly qualify Interop.ErrorInfo
    • This should be the last fix for build issues for this set
    hamarb123 committed Dec 5, 2022
    Configuration menu
    Copy the full SHA
    4c5a995 View commit details
    Browse the repository at this point in the history
  7. Fix some build issues from previous commit for dotnet#77835 again (5)

    • Add missing nullable ? for dstHandle (which obtains the file lock)
    hamarb123 committed Dec 5, 2022
    Configuration menu
    Copy the full SHA
    c65e229 View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2022

  1. Fix copying a file onto itself logic for dotnet#77835

    • Test failures were since copying a file onto itself should cause an error, this fixes that
    hamarb123 committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    5377efb View commit details
    Browse the repository at this point in the history
  2. Fix some nullability issues from previous commit for dotnet#77835

    • Fix nullability issues
    hamarb123 committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    8d8328a View commit details
    Browse the repository at this point in the history
  3. Fix some nullability issues from previous commit for dotnet#77835 (ag…

    …ain)
    
    • The variable in the if statements weren't updated
    • Handling for if readlink fails
    hamarb123 committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    835fc2f View commit details
    Browse the repository at this point in the history
  4. Fix bug introduced in 'Fix copying a file onto itself logic for dotne…

    …t#77835'
    
    • The source path should have used sourceFullPath rather than destPath
    hamarb123 committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    05f9d26 View commit details
    Browse the repository at this point in the history

Commits on Dec 8, 2022

  1. Configuration menu
    Copy the full SHA
    5fe4cd2 View commit details
    Browse the repository at this point in the history

Commits on Jan 24, 2023

  1. Merge branch 'main' of https://github.com/dotnet/runtime

    Merge remote-tracking branch 'upstream'
    hamarb123 committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    28ef104 View commit details
    Browse the repository at this point in the history
  2. Add extra test

    • From dotnet#33159
    • As requested by feedback
    • I assumed this test already existed, so it's good we have it now
    hamarb123 committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    21144e0 View commit details
    Browse the repository at this point in the history
  3. Use correct comment formatting

    • Add a space after //
    hamarb123 committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    049627f View commit details
    Browse the repository at this point in the history
  4. Use clonefile API instead

    • Use clonefile API instead
    • Update OpenReadOnly API to return the whole FileStatus
    • Revert changes to ResolveLinkTarget
    • Incorporate some feedback about how CopyFile should be implemented
    - Use st_dev and st_ino to optimise checks
    - Read them from an already required fstat call
    - Avoid calling clonefile when it would be likely to fail
    - Avoid expensive destination file locking scheme when clonefile won't be called
    - Fail fast when copying file onto itself
    hamarb123 committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    ab18c05 View commit details
    Browse the repository at this point in the history
  5. Remove redundant equality

    hamarb123 committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    776f500 View commit details
    Browse the repository at this point in the history

Commits on Jan 30, 2023

  1. Clone ACLs and make CopyOntoLockedFile test conditional

    • When using clonefile, specify CLONE_ACL to ensure ACLs are cloned, matching previous behaviour
    • Make CopyOntoLockedFile test conditional to platforms that support file locking
    hamarb123 committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    dcddd8f View commit details
    Browse the repository at this point in the history
  2. Remove retry logic

    • Remove retry logic after feedback that it is not necessary
    hamarb123 committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    ac27788 View commit details
    Browse the repository at this point in the history

Commits on May 14, 2023

  1. Address feedback

    Simplify error throwing, rename readStatus to isReadOnly, prepare for removal of StartedCopyFileState type.
    
    Co-authored-by: Stephen Toub <[email protected]>
    
    Squashed 19 commits, which addressed most of Stephen Toub's feedback. Thanks GH for all of those separate commits :)
    hamarb123 committed May 14, 2023
    Configuration menu
    Copy the full SHA
    f320a70 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    3d4a962 View commit details
    Browse the repository at this point in the history
  3. Fix compilation issues by me thinking I was smart

    • Add missing filePermissions field to return of StartCopyFile (which I removed because I didn't see it being used anywhere, but of course I didn't search for it)
    hamarb123 committed May 14, 2023
    Configuration menu
    Copy the full SHA
    a908a79 View commit details
    Browse the repository at this point in the history
  4. Fix nullability

    hamarb123 committed May 14, 2023
    Configuration menu
    Copy the full SHA
    16cedc6 View commit details
    Browse the repository at this point in the history

Commits on May 16, 2023

  1. Implement feedback

    • Implement feedback to remove StandardCopyFile, and to remove StartCopyFile
    • Add new line before CLONE_ACL line
    hamarb123 committed May 16, 2023
    Configuration menu
    Copy the full SHA
    d15fed3 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    617afbf View commit details
    Browse the repository at this point in the history
  3. Fix compilation issues

    hamarb123 committed May 16, 2023
    Configuration menu
    Copy the full SHA
    eb97585 View commit details
    Browse the repository at this point in the history
  4. Fix missed line in FileSystem.CopyFile.OSX.cs

    • Missed line in FileSystem.CopyFile.OSX.cs referencing the now-removed StandardCopyFile
    hamarb123 committed May 16, 2023
    Configuration menu
    Copy the full SHA
    70c6e8b View commit details
    Browse the repository at this point in the history
  5. Update src/libraries/System.Private.CoreLib/src/System/IO/FileSystem.…

    …CopyFile.OSX.cs
    
    Co-authored-by: Stephen Toub <[email protected]>
    hamarb123 and stephentoub authored May 16, 2023
    Configuration menu
    Copy the full SHA
    f866a3d View commit details
    Browse the repository at this point in the history
  6. Update src/libraries/System.Private.CoreLib/src/System/IO/FileSystem.…

    …CopyFile.OSX.cs
    
    Co-authored-by: Stephen Toub <[email protected]>
    hamarb123 and stephentoub authored May 16, 2023
    Configuration menu
    Copy the full SHA
    bf67649 View commit details
    Browse the repository at this point in the history
  7. Update src/libraries/System.Private.CoreLib/src/System/IO/FileSystem.…

    …CopyFile.OSX.cs
    
    Co-authored-by: Stephen Toub <[email protected]>
    hamarb123 and stephentoub authored May 16, 2023
    Configuration menu
    Copy the full SHA
    ff108db View commit details
    Browse the repository at this point in the history
  8. Update src/libraries/System.Private.CoreLib/src/System/IO/FileSystem.…

    …CopyFile.OSX.cs
    
    Co-authored-by: Stephen Toub <[email protected]>
    hamarb123 and stephentoub authored May 16, 2023
    Configuration menu
    Copy the full SHA
    69071eb View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    20c42f9 View commit details
    Browse the repository at this point in the history
  10. Update comment for clarity

    • Explain more clearly why we open a lock to the destination file before deleting it
    hamarb123 committed May 16, 2023
    Configuration menu
    Copy the full SHA
    1280a8a View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    f33ae9c View commit details
    Browse the repository at this point in the history
  12. Fix compilation errors

    hamarb123 committed May 16, 2023
    Configuration menu
    Copy the full SHA
    068a8aa View commit details
    Browse the repository at this point in the history

Commits on May 17, 2023

  1. Remove some comments as per feedback

    • Remove comments saying to keep code in sync as per feedback
    hamarb123 committed May 17, 2023
    Configuration menu
    Copy the full SHA
    2c48aad View commit details
    Browse the repository at this point in the history
  2. Changes from feedback

    • Update some comments to make things clear
    • Shorten comment and code that interprets the error as per feedback
    hamarb123 committed May 17, 2023
    Configuration menu
    Copy the full SHA
    ef84010 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    8924afe View commit details
    Browse the repository at this point in the history
  4. Remove old comment

    hamarb123 committed May 17, 2023
    Configuration menu
    Copy the full SHA
    1649d43 View commit details
    Browse the repository at this point in the history

Commits on May 18, 2023

  1. Implement some feedback

    hamarb123 committed May 18, 2023
    Configuration menu
    Copy the full SHA
    a82767b View commit details
    Browse the repository at this point in the history
  2. Implement suggestion that cuts down on duplication

    • Implement suggestion from @tmds that greatly cuts down on code duplication :)
    hamarb123 committed May 18, 2023
    Configuration menu
    Copy the full SHA
    c0c7e33 View commit details
    Browse the repository at this point in the history
  3. Remove unnecessary using

    hamarb123 committed May 18, 2023
    Configuration menu
    Copy the full SHA
    18b8c2e View commit details
    Browse the repository at this point in the history
  4. Remove some other unnecessary usings

    • This time in FileSystem.TryCloneFile.OSX.cs
    hamarb123 committed May 18, 2023
    Configuration menu
    Copy the full SHA
    00b68ed View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    afa73ca View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    a684785 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    61c0576 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    7c6c80b View commit details
    Browse the repository at this point in the history

Commits on May 23, 2023

  1. Configuration menu
    Copy the full SHA
    9cdb4b3 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    2dc08e4 View commit details
    Browse the repository at this point in the history
  3. Fix whitespace

    hamarb123 committed May 23, 2023
    Configuration menu
    Copy the full SHA
    02cfaf6 View commit details
    Browse the repository at this point in the history
  4. Fix compilation errors

    hamarb123 committed May 23, 2023
    Configuration menu
    Copy the full SHA
    1fc02f7 View commit details
    Browse the repository at this point in the history
  5. Fix more compilation issues

    hamarb123 committed May 23, 2023
    Configuration menu
    Copy the full SHA
    71bc96e View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    aa76f24 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    3945a9f View commit details
    Browse the repository at this point in the history
  8. Add missing cast

    hamarb123 committed May 23, 2023
    Configuration menu
    Copy the full SHA
    3457dcc View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    788216e View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    233217a View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    20caacd View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    6d40467 View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    705aa4f View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    a5de060 View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    7221560 View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    1db28fd View commit details
    Browse the repository at this point in the history
  17. Configuration menu
    Copy the full SHA
    d360ae4 View commit details
    Browse the repository at this point in the history
  18. Add some code to deal with EINVAL

    • We can get EINVAL resulting from trying to copy ACLs, so we retry with with the flag to copy ACLs disabled (up to 1 time)
    hamarb123 committed May 23, 2023
    Configuration menu
    Copy the full SHA
    323cf67 View commit details
    Browse the repository at this point in the history
  19. Configuration menu
    Copy the full SHA
    44ef8f3 View commit details
    Browse the repository at this point in the history
  20. Configuration menu
    Copy the full SHA
    50195af View commit details
    Browse the repository at this point in the history
  21. Configuration menu
    Copy the full SHA
    53f52bc View commit details
    Browse the repository at this point in the history
  22. Remove unnecessary diffs

    • Let me know if I should add these comments back
    hamarb123 committed May 23, 2023
    Configuration menu
    Copy the full SHA
    aee3ea8 View commit details
    Browse the repository at this point in the history

Commits on May 24, 2023

  1. Fix compilation errors

    hamarb123 committed May 24, 2023
    Configuration menu
    Copy the full SHA
    c6174e6 View commit details
    Browse the repository at this point in the history
  2. Update src/libraries/System.Private.CoreLib/src/System/IO/FileSystem.…

    …TryCloneFile.OSX.cs
    
    Co-authored-by: Dan Moseley <[email protected]>
    hamarb123 and danmoseley authored May 24, 2023
    Configuration menu
    Copy the full SHA
    0838d9f View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    013c775 View commit details
    Browse the repository at this point in the history
  4. Fix test failures

    hamarb123 committed May 24, 2023
    Configuration menu
    Copy the full SHA
    487aee9 View commit details
    Browse the repository at this point in the history
  5. Extract attempting clonefile to a separate function

    • As per feedback. It allows us to avoid the gotos.
    • Also reorder the check on the second retry.
    hamarb123 committed May 24, 2023
    Configuration menu
    Copy the full SHA
    37b5183 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    bc99ec9 View commit details
    Browse the repository at this point in the history
  7. Use a debug statement to check EINVAL instead

    • Since it should only happen for an invalid filesystem anyway
    hamarb123 committed May 24, 2023
    Configuration menu
    Copy the full SHA
    e1c7739 View commit details
    Browse the repository at this point in the history
  8. Improve & fix comment

    hamarb123 committed May 24, 2023
    Configuration menu
    Copy the full SHA
    aec1eac View commit details
    Browse the repository at this point in the history
  9. Fix compilation errors

    hamarb123 committed May 24, 2023
    Configuration menu
    Copy the full SHA
    483858d View commit details
    Browse the repository at this point in the history
  10. Implement feedback

    hamarb123 committed May 24, 2023
    Configuration menu
    Copy the full SHA
    f67bd18 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    5cf74af View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2023

  1. Implement feedback to use partial functions

    • Implement feedback to use partial functions so that TryCloneFile doesn't have to exist other than when it's actually needed
    hamarb123 committed Jun 6, 2023
    Configuration menu
    Copy the full SHA
    10011a5 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    a793bb7 View commit details
    Browse the repository at this point in the history
  3. Implement other feedback

    hamarb123 committed Jun 6, 2023
    Configuration menu
    Copy the full SHA
    550a76a View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    32c5463 View commit details
    Browse the repository at this point in the history

Commits on Jun 13, 2023

  1. Configuration menu
    Copy the full SHA
    de964d0 View commit details
    Browse the repository at this point in the history