Hi everyone,

When using overlayfs with the mount option index=on, the first time a directory is
used as upper dir, overlayfs stores in a xattr "overlay.origin" the UUID of the
filesystem being used in the layers. If the upper dir is reused, overlayfs
refuses to mount for a different filesystem, by comparing the UUID with what's
stored at overlay.origin, and it fails with "failed to verify upper root origin"
on dmesg. Remounting with the very same fs is supported and works fine.

However, btrfs mounts may have volatiles UUIDs. When mounting the exact same
disk image with btrfs, a random UUID is assigned for the following disks each
time they are mounted, stored at temp_fsid and used across the kernel as the
disk UUID. `btrfs filesystem show` presents that. Calling statfs() however shows
the original (and duplicated) UUID for all disks.

This feature doesn't work well with overlayfs with index=on, as when the image
is mounted a second time, will get a different UUID and ovl will refuse to
mount, breaking the user expectation that using the same image should work. A
small script can be find in the end of this cover letter that illustrates this.

From this, I can think of some options:

- Use statfs() internally to always get the fsid, that is persistent. The patch
here illustrates that approach, but doesn't fully implement it.
- Create a new sb op, called get_uuid() so the filesystem returns what's
appropriated.
- Have a workaround in ovl for btrfs.
- Document this as unsupported, and userland needs to erase overlay.origin each
time it wants to remount.
- If ovl detects that temp_fsid and index are being used at the same time,
refuses to mount.

I'm not sure which one would be better here, so I would like to hear some ideas
on this.

Thanks!
	André

---

To reproduce:

mkdir -p dir1 dir2

fallocate -l 300m ./disk1.img
mkfs.btrfs -q -f ./disk1.img

# cloning the disks
cp disk1.img disk2.img
sudo mount -o loop ./disk1.img dir1
sudo mount -o loop ./disk2.img dir2

mkdir -p dir2/lower aux/upper aux/work

# this works
sudo mount -t overlay -o lowerdir=dir2/lower,upperdir=aux/upper,workdir=aux/work,userxattr none dir2/lower

sudo umount dir2/lower
sudo umount dir2

sudo mount -o loop ./disk2.img dir2

# this doesn't works
sudo mount -t overlay -o lowerdir=dir2/lower,upperdir=aux/upper,workdir=aux/work,userxattr none dir2/lower

André Almeida (1):
  ovl: Use fsid as unique identifier for trusted origin

 fs/overlayfs/copy_up.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

-- 
2.51.0

On Tue, Oct 14, 2025 at 3:57 AM André Almeida <andrealmeid@igalia.com> wrote:
>
> Hi everyone,
>
> When using overlayfs with the mount option index=on, the first time a directory is
> used as upper dir, overlayfs stores in a xattr "overlay.origin" the UUID of the
> filesystem being used in the layers. If the upper dir is reused, overlayfs
> refuses to mount for a different filesystem, by comparing the UUID with what's
> stored at overlay.origin, and it fails with "failed to verify upper root origin"
> on dmesg. Remounting with the very same fs is supported and works fine.
>
> However, btrfs mounts may have volatiles UUIDs. When mounting the exact same
> disk image with btrfs, a random UUID is assigned for the following disks each
> time they are mounted, stored at temp_fsid and used across the kernel as the
> disk UUID. `btrfs filesystem show` presents that. Calling statfs() however shows
> the original (and duplicated) UUID for all disks.
>
> This feature doesn't work well with overlayfs with index=on, as when the image
> is mounted a second time, will get a different UUID and ovl will refuse to
> mount, breaking the user expectation that using the same image should work. A
> small script can be find in the end of this cover letter that illustrates this.
>
> From this, I can think of some options:
>
> - Use statfs() internally to always get the fsid, that is persistent. The patch
> here illustrates that approach, but doesn't fully implement it.
> - Create a new sb op, called get_uuid() so the filesystem returns what's
> appropriated.

FWIW this operation already exists in export_operations.
It is currently only used by pnfs and only implemented by xfs.
I would nor object for overlayfs to use this method if implemented
and fall back to copying uuid directly from s_uuid
(better yet make it a vfs helper)
Note that commit
8f720d9f892e0 xfs: publish UUID in struct super_block
was done for a similar reason.
The xfs mount option nouuid is the poor man's solution for
mounting cloned disk images.

Thanks,
Amir.

On Wed, Oct 15, 2025 at 01:09:26PM +0200, Amir Goldstein wrote:
> FWIW this operation already exists in export_operations.
> It is currently only used by pnfs and only implemented by xfs.
> I would nor object for overlayfs to use this method if implemented
> and fall back to copying uuid directly from s_uuid

The get_uuid export operation is specifically about an on-disk superblock
that the pnfs client can match.  Overloading it for in-core information
would be very confusing and also problematic if it doesn't exist on-disk
in this format.  In retrospective it should be called get_disk_uuid or
something similar.

在 2025/10/14 12:27, André Almeida 写道:
> Hi everyone,
> 
> When using overlayfs with the mount option index=on, the first time a directory is
> used as upper dir, overlayfs stores in a xattr "overlay.origin" the UUID of the
> filesystem being used in the layers. If the upper dir is reused, overlayfs
> refuses to mount for a different filesystem, by comparing the UUID with what's
> stored at overlay.origin, and it fails with "failed to verify upper root origin"
> on dmesg. Remounting with the very same fs is supported and works fine.
> 
> However, btrfs mounts may have volatiles UUIDs. When mounting the exact same
> disk image with btrfs, a random UUID is assigned for the following disks each
> time they are mounted, stored at temp_fsid and used across the kernel as the
> disk UUID. `btrfs filesystem show` presents that. Calling statfs() however shows
> the original (and duplicated) UUID for all disks.

Yep, that's the btrfs' hack to allowing mounting cloned devices (as long 
as they are all single-device only btrfs)

Although I'm not a huge fan for that, without that you can not even 
mount any cloned btrfs in the first place.

> 
> This feature doesn't work well with overlayfs with index=on, as when the image
> is mounted a second time, will get a different UUID and ovl will refuse to
> mount, breaking the user expectation that using the same image should work. A
> small script can be find in the end of this cover letter that illustrates this.
> 
>  From this, I can think of some options:
> 
> - Use statfs() internally to always get the fsid, that is persistent. The patch
> here illustrates that approach, but doesn't fully implement it.
> - Create a new sb op, called get_uuid() so the filesystem returns what's
> appropriated.
> - Have a workaround in ovl for btrfs.
> - Document this as unsupported, and userland needs to erase overlay.origin each
> time it wants to remount.
> - If ovl detects that temp_fsid and index are being used at the same time,
> refuses to mount.

Or, let btrfs to reject the cloned device in the first place.

> 
> I'm not sure which one would be better here, so I would like to hear some ideas
> on this.
> 
> Thanks!
> 	André
> 
> ---
> 
> To reproduce:
> 
> mkdir -p dir1 dir2
> 
> fallocate -l 300m ./disk1.img
> mkfs.btrfs -q -f ./disk1.img
> 
> # cloning the disks
> cp disk1.img disk2.img

If you really want to use the same copied fs, at least you can use
`btrfstune -m disk2.img` to change it to a new metadata uuid (without 
re-writing all metadata).

Then everything should work.

Thanks,
Qu
> sudo mount -o loop ./disk1.img dir1
> sudo mount -o loop ./disk2.img dir2
> 
> mkdir -p dir2/lower aux/upper aux/work
> 
> # this works
> sudo mount -t overlay -o lowerdir=dir2/lower,upperdir=aux/upper,workdir=aux/work,userxattr none dir2/lower
> 
> sudo umount dir2/lower
> sudo umount dir2
> 
> sudo mount -o loop ./disk2.img dir2
> 
> # this doesn't works
> sudo mount -t overlay -o lowerdir=dir2/lower,upperdir=aux/upper,workdir=aux/work,userxattr none dir2/lower
> 
> André Almeida (1):
>    ovl: Use fsid as unique identifier for trusted origin
> 
>   fs/overlayfs/copy_up.c | 8 +++++++-
>   1 file changed, 7 insertions(+), 1 deletion(-)
>