btrfs: allow reflinks into NODATASUM files

[PATCH 0/2] btrfs: allow reflinks into NODATASUM files

Posted by Daan De Meyer via B4 Relay 1 week, 6 days ago

The primary use case for this series is building a NOCOW VM image from
individual partition images that are COW and checksummed. Today those
partitions cannot be cloned into the NODATACOW and NODATASUM destination,
so tools fall back to a full copy. This makes provisioning slower and
duplicates all of the image data up front.

Allow cloning and deduplication from a checksummed file into a NODATASUM
file. The VM image can then share extents with the partition images.
Existing checksummed extents are COWed once when modified, protecting the
source checksums, while newly allocated extents use the destination's
normal NOCOW behavior.

The reverse direction remains rejected because the destination would
expect checksums that do not exist.

Patch 1 prevents swap activation from bypassing the COW protection when a
NODATASUM file references checksummed extents. Patch 2 relaxes the reflink
restriction in the safe direction.

The xfstests branch is available at:
https://github.com/kdave/xfstests/pull/6

Signed-off-by: Daan De Meyer <daan@amutable.com>
---
Daan De Meyer (2):
      btrfs: reject swapfile activation if any extent has checksums
      btrfs: allow reflinking from checksummed files into nodatasum files

 fs/btrfs/inode.c   | 29 +++++++++++++++++++++++++++++
 fs/btrfs/reflink.c | 18 ++++++++++++++----
 2 files changed, 43 insertions(+), 4 deletions(-)
---
base-commit: cab9e339cfbc1a4e075e53e281dfb00391e1a6bb
change-id: 20260712-reflink-into-nodatasum-aff962f3794e

Best regards,
--  
Daan De Meyer <daan@amutable.com>

Re: [PATCH 0/2] btrfs: allow reflinks into NODATASUM files

Posted by Qu Wenruo 1 week, 6 days ago


在 2026/7/12 23:55, Daan De Meyer via B4 Relay 写道:
> The primary use case for this series is building a NOCOW VM image from
> individual partition images that are COW and checksummed. Today those
> partitions cannot be cloned into the NODATACOW and NODATASUM destination,
> so tools fall back to a full copy. This makes provisioning slower and
> duplicates all of the image data up front.
> 
> Allow cloning and deduplication from a checksummed file into a NODATASUM
> file. The VM image can then share extents with the partition images.
> Existing checksummed extents are COWed once when modified, protecting the
> source checksums, while newly allocated extents use the destination's
> normal NOCOW behavior.

No, the change will easily cause btrfs check errors, in both lowmem and 
original modes.

And before you do anything weird, I'd prefer you to address the 
regression caused by your new 32bit compatible ioctl first:

https://lore.kernel.org/linux-btrfs/66470c2e-56b9-49a9-9cd2-aa49fac75bbf@suse.com/
> 
> The reverse direction remains rejected because the destination would
> expect checksums that do not exist.
> 
> Patch 1 prevents swap activation from bypassing the COW protection when a
> NODATASUM file references checksummed extents. Patch 2 relaxes the reflink
> restriction in the safe direction.
> 
> The xfstests branch is available at:
> https://github.com/kdave/xfstests/pull/6
> 
> Signed-off-by: Daan De Meyer <daan@amutable.com>
> ---
> Daan De Meyer (2):
>        btrfs: reject swapfile activation if any extent has checksums
>        btrfs: allow reflinking from checksummed files into nodatasum files
> 
>   fs/btrfs/inode.c   | 29 +++++++++++++++++++++++++++++
>   fs/btrfs/reflink.c | 18 ++++++++++++++----
>   2 files changed, 43 insertions(+), 4 deletions(-)
> ---
> base-commit: cab9e339cfbc1a4e075e53e281dfb00391e1a6bb
> change-id: 20260712-reflink-into-nodatasum-aff962f3794e
> 
> Best regards,
> --
> Daan De Meyer <daan@amutable.com>
> 
> 
>

Re: [PATCH 0/2] btrfs: allow reflinks into NODATASUM files

Posted by Neal Gompa 1 week, 6 days ago

On Sun, Jul 12, 2026 at 10:25 AM Daan De Meyer via B4 Relay
<devnull+daan.amutable.com@kernel.org> wrote:
>
> The primary use case for this series is building a NOCOW VM image from
> individual partition images that are COW and checksummed. Today those
> partitions cannot be cloned into the NODATACOW and NODATASUM destination,
> so tools fall back to a full copy. This makes provisioning slower and
> duplicates all of the image data up front.
>
> Allow cloning and deduplication from a checksummed file into a NODATASUM
> file. The VM image can then share extents with the partition images.
> Existing checksummed extents are COWed once when modified, protecting the
> source checksums, while newly allocated extents use the destination's
> normal NOCOW behavior.
>
> The reverse direction remains rejected because the destination would
> expect checksums that do not exist.
>
> Patch 1 prevents swap activation from bypassing the COW protection when a
> NODATASUM file references checksummed extents. Patch 2 relaxes the reflink
> restriction in the safe direction.
>
> The xfstests branch is available at:
> https://github.com/kdave/xfstests/pull/6
>
> Signed-off-by: Daan De Meyer <daan@amutable.com>
> ---
> Daan De Meyer (2):
>       btrfs: reject swapfile activation if any extent has checksums
>       btrfs: allow reflinking from checksummed files into nodatasum files
>
>  fs/btrfs/inode.c   | 29 +++++++++++++++++++++++++++++
>  fs/btrfs/reflink.c | 18 ++++++++++++++----
>  2 files changed, 43 insertions(+), 4 deletions(-)
> ---
> base-commit: cab9e339cfbc1a4e075e53e281dfb00391e1a6bb
> change-id: 20260712-reflink-into-nodatasum-aff962f3794e
>

The patch series looks good to me.

Reviewed-by: Neal Gompa <neal@gompa.dev>

That said, I have a question: Is there no reason we couldn't reflink
into a checksummed area and generate *new* checksums for that purpose?
That would be useful for staging for regular backup.

I can foresee that being useful for archiving OS trees or VM images as
snapshots by creating checksums for the newly created snapshots...



--
真実はいつも一つ！/ Always, there's only one truth!

Re: [PATCH 0/2] btrfs: allow reflinks into NODATASUM files

Posted by Qu Wenruo 1 week, 5 days ago


在 2026/7/13 00:36, Neal Gompa 写道:
> On Sun, Jul 12, 2026 at 10:25 AM Daan De Meyer via B4 Relay
> <devnull+daan.amutable.com@kernel.org> wrote:
>>
>> The primary use case for this series is building a NOCOW VM image from
>> individual partition images that are COW and checksummed. Today those
>> partitions cannot be cloned into the NODATACOW and NODATASUM destination,
>> so tools fall back to a full copy. This makes provisioning slower and
>> duplicates all of the image data up front.
>>
>> Allow cloning and deduplication from a checksummed file into a NODATASUM
>> file. The VM image can then share extents with the partition images.
>> Existing checksummed extents are COWed once when modified, protecting the
>> source checksums, while newly allocated extents use the destination's
>> normal NOCOW behavior.
>>
>> The reverse direction remains rejected because the destination would
>> expect checksums that do not exist.
>>
>> Patch 1 prevents swap activation from bypassing the COW protection when a
>> NODATASUM file references checksummed extents. Patch 2 relaxes the reflink
>> restriction in the safe direction.
>>
>> The xfstests branch is available at:
>> https://github.com/kdave/xfstests/pull/6
>>
>> Signed-off-by: Daan De Meyer <daan@amutable.com>
>> ---
>> Daan De Meyer (2):
>>        btrfs: reject swapfile activation if any extent has checksums
>>        btrfs: allow reflinking from checksummed files into nodatasum files
>>
>>   fs/btrfs/inode.c   | 29 +++++++++++++++++++++++++++++
>>   fs/btrfs/reflink.c | 18 ++++++++++++++----
>>   2 files changed, 43 insertions(+), 4 deletions(-)
>> ---
>> base-commit: cab9e339cfbc1a4e075e53e281dfb00391e1a6bb
>> change-id: 20260712-reflink-into-nodatasum-aff962f3794e
>>
> 
> The patch series looks good to me.
> 
> Reviewed-by: Neal Gompa <neal@gompa.dev>

I appreciate your feedback as an end user/sys admin, but I doubt if you 
really understand the requirement or tried the patch by yourself.

I'd appreciate if you put more weight behind your reviewed-by tags.

> 
> That said, I have a question: Is there no reason we couldn't reflink
> into a checksummed area and generate *new* checksums for that purpose?
> That would be useful for staging for regular backup.

Because btrfs' NODATASUM/NODATACOW flag is per-inode, as that's the only 
way we can store that flag.

If you really want proper per-extent NODATASUM/NODATACOW, there will 
need a huge change in the on-disk format, e.g. introducing new flags 
into btrfs_file_extent_item, and do the extra work handling corner cases 
like preallocation.

It's definitely not as simple two small patches.

> 
> I can foresee that being useful for archiving OS trees or VM images as
> snapshots by creating checksums for the newly created snapshots...

Nope, one should not combine VM images with snapshots on btrfs.

Even if we have fixed all the direct IO problems, the extent bookend 
will waste a lot of space.

> 
> 
> 
> --
> 真実はいつも一つ！/ Always, there's only one truth!
>

Re: [PATCH 0/2] btrfs: allow reflinks into NODATASUM files

Posted by Neal Gompa 1 week, 5 days ago

On Sun, Jul 12, 2026 at 6:19 PM Qu Wenruo <wqu@suse.com> wrote:
>
>
>
> 在 2026/7/13 00:36, Neal Gompa 写道:
> > On Sun, Jul 12, 2026 at 10:25 AM Daan De Meyer via B4 Relay
> > <devnull+daan.amutable.com@kernel.org> wrote:
> >>
> >> The primary use case for this series is building a NOCOW VM image from
> >> individual partition images that are COW and checksummed. Today those
> >> partitions cannot be cloned into the NODATACOW and NODATASUM destination,
> >> so tools fall back to a full copy. This makes provisioning slower and
> >> duplicates all of the image data up front.
> >>
> >> Allow cloning and deduplication from a checksummed file into a NODATASUM
> >> file. The VM image can then share extents with the partition images.
> >> Existing checksummed extents are COWed once when modified, protecting the
> >> source checksums, while newly allocated extents use the destination's
> >> normal NOCOW behavior.
> >>
> >> The reverse direction remains rejected because the destination would
> >> expect checksums that do not exist.
> >>
> >> Patch 1 prevents swap activation from bypassing the COW protection when a
> >> NODATASUM file references checksummed extents. Patch 2 relaxes the reflink
> >> restriction in the safe direction.
> >>
> >> The xfstests branch is available at:
> >> https://github.com/kdave/xfstests/pull/6
> >>
> >> Signed-off-by: Daan De Meyer <daan@amutable.com>
> >> ---
> >> Daan De Meyer (2):
> >>        btrfs: reject swapfile activation if any extent has checksums
> >>        btrfs: allow reflinking from checksummed files into nodatasum files
> >>
> >>   fs/btrfs/inode.c   | 29 +++++++++++++++++++++++++++++
> >>   fs/btrfs/reflink.c | 18 ++++++++++++++----
> >>   2 files changed, 43 insertions(+), 4 deletions(-)
> >> ---
> >> base-commit: cab9e339cfbc1a4e075e53e281dfb00391e1a6bb
> >> change-id: 20260712-reflink-into-nodatasum-aff962f3794e
> >>
> >
> > The patch series looks good to me.
> >
> > Reviewed-by: Neal Gompa <neal@gompa.dev>
>
> I appreciate your feedback as an end user/sys admin, but I doubt if you
> really understand the requirement or tried the patch by yourself.
>
> I'd appreciate if you put more weight behind your reviewed-by tags.
>

I already don't review patches very often because of responses like
this. Please don't make me feel like I should pull out entirely.

> >
> > That said, I have a question: Is there no reason we couldn't reflink
> > into a checksummed area and generate *new* checksums for that purpose?
> > That would be useful for staging for regular backup.
>
> Because btrfs' NODATASUM/NODATACOW flag is per-inode, as that's the only
> way we can store that flag.
>
> If you really want proper per-extent NODATASUM/NODATACOW, there will
> need a huge change in the on-disk format, e.g. introducing new flags
> into btrfs_file_extent_item, and do the extra work handling corner cases
> like preallocation.
>
> It's definitely not as simple two small patches.
>

I wasn't implying that it was simple. I asked the question because the
patches themselves don't explain why it's not done, only that it's
"not safe". That is not a sufficient enough answer in my book. And
while I can intuit the reason from my own experience working on CoW
stuff before, I wanted to know why it was dismissed as "unsafe" when I
am aware of ways for it to be done.

> >
> > I can foresee that being useful for archiving OS trees or VM images as
> > snapshots by creating checksums for the newly created snapshots...
>
> Nope, one should not combine VM images with snapshots on btrfs.
>
> Even if we have fixed all the direct IO problems, the extent bookend
> will waste a lot of space.
>

In the real world, this combination happens *a lot*. In fact, multiple
businesses are built on this idea (though not using Btrfs
specifically). It is not out of the realm of possibility to me that
Btrfs would support it and that the weaknesses could be minimized.
Throwing the idea completely out is beyond silly because of how common
containers and virtual machines are. Both are used more and more, even
without people knowing it, so the idea that we shouldn't at least look
at making that pattern workable seems foolhardy.


-- 
真実はいつも一つ！/ Always, there's only one truth!