btrfs-progs: receive: workaround full file clone quirk #899
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[BUG]
The following script can lead to receive failure with latest kernel:
dev="/dev/test/scratch1"
mnt="/mnt/btrfs"
mkfs.btrfs -f $dev
mount $dev $mnt
btrfs subv create $mnt/subv1
xfs_io -f -c "pwrite 0 4000" $mnt/subv1/source
xfs_io -f -c "reflink $mnt/subv1/source" $mnt/subv1/dest
btrfs subv snap -r $mnt/subv1 $mnt/ro_subv1
btrfs subv snap $mnt/subv1 $mnt/snap1
xfs_io -f -c "pwrite -S 0xff 0 3900" -c "truncate 3900" $mnt/snap1/source
truncate -s 0 $mnt/snap1/dest
xfs_io -f -c "reflink $mnt/snap1/source" $mnt/snap1/dest
btrfs subv snap -r $mnt/snap1 $mnt/ro_snap1
btrfs send $mnt/ro_subv1 -f /tmp/ro_subv1.stream
btrfs send -p $mnt/ro_subv1 $mnt/ro_snap1 -f /tmp/ro_snap1.stream
umount $mnt
mkfs.btrfs -f $dev
mount $dev $mnt
btrfs receive -f /tmp/ro_subv1.stream $mnt
btrfs receive -f /tmp/ro_snap1.stream $mnt
At snapshot ro_snap1
ERROR: failed to clone extents to dest: Invalid argument
[CAUSE]
Since kernel commit 46a6e10a1ab1 ("btrfs: send: allow cloning
non-aligned extent if it ends at i_size"), kernel can send out clone
stream if we're cloning a full file, even if the size of the file is not
sector aligned, like this one:
snapshot ./ro_snap1 uuid=2a3e2b70-c606-d446-b60b-baab458be6da transid=9 parent_uuid=d8ff9b9e-3ffc-6343-b53e-e22f8bbb7c25 parent_transid=7
write ./ro_snap1/source offset=0 len=4700
truncate ./ro_snap1/source size=4700
utimes ./ro_snap1/source atime=2024-09-27T13:08:54+0800 mtime=2024-09-27T13:08:54+0800 ctime=2024-09-27T13:08:54+0800
clone ./ro_snap1/dest offset=0 len=4700 from=./ro_snap1/source clone_offset=0
truncate ./ro_snap1/dest size=4700
utimes ./ro_snap1/dest atime=2024-09-27T13:08:54+0800 mtime=2024-09-27T13:08:54+0800 ctime=2024-09-27T13:08:54+0800
However for the clone command, if the file inside the source subvolume
is larger than the new size, kernel will reject the clone operation, as
the resulted layout may read beyond the EOF of the clone source.
This should be addressed by the kernel, by doing the truncation before
the clone to ensure the destination file is no larger than the source.
[FIX]
It won't hurt for "btrfs receive" command to workaround the
problem, by truncating the destination file first.
Here we choose to truncate the file size to 0, other than the
source/destination file size.
As truncating to an unaligned size can cause the fs to do extra page
dirty and zero the tailing part.
Since we know it's a full file clone, truncating the file to size 0 will
avoid the extra page dirty, and allow the later clone to be done.
Reported-by: Ben Millwood <[email protected]>
Link: https://lore.kernel.org/linux-btrfs/CAJhrHS2z+WViO2h=ojYvBPDLsATwLbg+7JaNCyYomv0fUxEpQQ@mail.gmail.com/
Signed-off-by: Qu Wenruo <[email protected]>
…e clone
The new test case will utilize the following send stream:
Stream 1:
subvol ./ro_subv1 uuid=769f87d1-98b2-824f-bf18-9d98178ad2e2 transid=7
chown ./ro_subv1/ gid=0 uid=0
chmod ./ro_subv1/ mode=755
mkfile ./ro_subv1/o257-7-0
rename ./ro_subv1/o257-7-0 dest=./ro_subv1/source
write ./ro_subv1/source offset=0 len=4000
chown ./ro_subv1/source gid=0 uid=0
chmod ./ro_subv1/source mode=600
utimes ./ro_subv1/source atime=2024-09-27T13:26:25+0800 mtime=2024-09-27T13:26:25+0800 ctime=2024-09-27T13:26:25+0800
mkfile ./ro_subv1/o258-7-0
rename ./ro_subv1/o258-7-0 dest=./ro_subv1/dest
clone ./ro_subv1/dest offset=0 len=4000 from=./ro_subv1/source clone_offset=0
chown ./ro_subv1/dest gid=0 uid=0
chmod ./ro_subv1/dest mode=600
utimes ./ro_subv1/dest atime=2024-09-27T13:26:25+0800 mtime=2024-09-27T13:26:25+0800 ctime=2024-09-27T13:26:25+0800
utimes ./ro_subv1/ atime=2024-09-27T13:26:25+0800 mtime=2024-09-27T13:26:25+0800 ctime=2024-09-27T13:26:25+0800
Stream 2:
snapshot ./ro_snap1 uuid=e78c9b7c-1bea-fc48-aaf3-6a4d98eba473 transid=9 parent_uuid=769f87d1-98b2-824f-bf18-9d98178ad2e2 parent_transid=7
write ./ro_snap1/source offset=0 len=3900
truncate ./ro_snap1/source size=3900
utimes ./ro_snap1/source atime=2024-09-27T13:26:25+0800 mtime=2024-09-27T13:26:25+0800 ctime=2024-09-27T13:26:25+0800
clone ./ro_snap1/dest offset=0 len=3900 from=./ro_snap1/source clone_offset=0
truncate ./ro_snap1/dest size=3900
utimes ./ro_snap1/dest atime=2024-09-27T13:26:25+0800 mtime=2024-09-27T13:26:25+0800 ctime=2024-09-27T13:26:25+0800
The stream is generated using v6.11 kernel, as upstream commit
46a6e10a1ab1 ("btrfs: send: allow cloning non-aligned extent if it ends
at i_size") allows full file clone to further reduce the stream size.
Verify that "btrfs receive" command has the proper workaround to handle
such full file clone correctly.
Signed-off-by: Qu Wenruo <[email protected]>
|
Discarded as Filipe will fix the kernel part. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Since kernel commit 46a6e10a1ab1 ("btrfs: send: allow cloning
non-aligned extent if it ends at i_size"), we can have clone commands to
clone a full file.
However such full file clone can have an unaligned length, and such
clone is only allowed if the destination file is no larger than the
clone range.
But we can create a case where such clone called on a destination file
which has a larger size.
This can lead to kernel reject such clone, and fail the receive.
This patchset will handle such quirk by truncate the file size to 0 if
we detect this is a full file clone.
Also add a new test case to verify the behaivor.