Hi all,

We would like to support the usage of casefold filesystems with
overlayfs. This patchset do some of the work needed for that, but I'm
sure there are more places that need to be tweaked so please share your
feedback for this work.

* Implementation

The most obvious place that required change was the strncmp() inside of
ovl_cache_entry_find(), that I managed to convert to use d_same_name(),
that will then call the generic_ci_d_compare function if it's set for
the dentry. There are more strncmp() around ovl, but I would rather hear
feedback about this approach first than already implementing this around
the code.

* Testing

I used tmpfs to create a small ovl like this:

sudo mount -t tmpfs -o casefold tmpfs mnt/
cd mnt/
mkdir dir
chattr +F dir
cd dir/
mkdir upper lower
mkdir lower/A lower/b lower/c
mkdir upper/a upper/b upper/d
mkdir merged work
sudo mount -t overlay overlay -olowerdir=lower,upperdir=upper,workdir=work, merged
ls /tmp/mnt/dir/merged/
a  b  c  d

And ovl is respecting the equivalent names. `a` points to a merged dir
between `A` and `a`, but giving that upperdir has a lowercase `a`, this
is the name displayed here.

Thanks,
	André

---
André Almeida (3):
      ovl: Make ovl_cache_entry_find support casefold
      ovl: Make ovl_dentry_weird() accept casefold dentries
      ovl: Enable support for casefold filesystems

 fs/overlayfs/namei.c     | 11 ++++++-----
 fs/overlayfs/overlayfs.h |  2 +-
 fs/overlayfs/ovl_entry.h |  1 +
 fs/overlayfs/params.c    |  5 +++--
 fs/overlayfs/readdir.c   | 32 +++++++++++++++++++++-----------
 fs/overlayfs/util.c      | 12 +++++++-----
 6 files changed, 39 insertions(+), 24 deletions(-)
---
base-commit: a24588245776dafc227243a01bfbeb8a59bafba9
change-id: 20250409-tonyk-overlayfs-591f5e4d407a

Best regards,
-- 
André Almeida <andrealmeid@igalia.com>

On Wed, Apr 9, 2025 at 5:01 PM André Almeida <andrealmeid@igalia.com> wrote:
>
> Hi all,
>
> We would like to support the usage of casefold filesystems with
> overlayfs. This patchset do some of the work needed for that, but I'm
> sure there are more places that need to be tweaked so please share your
> feedback for this work.
>
> * Implementation
>
> The most obvious place that required change was the strncmp() inside of
> ovl_cache_entry_find(), that I managed to convert to use d_same_name(),

That's a very niche part of overlayfs where comparison of names matter.

Please look very closely at ovl_lookup() and how an overlay entry stack is
composed from several layers including the option to redirect to different names
via redirect xattr, so there is really very much to deal with other
than readdir.

I suggest that you start with a design proposal of how you intend to tackle this
task and what are your requirements?
Any combination of casefold supported layers?

Thanks,
Amir.

> that will then call the generic_ci_d_compare function if it's set for
> the dentry. There are more strncmp() around ovl, but I would rather hear
> feedback about this approach first than already implementing this around
> the code.
>
> * Testing
>
> I used tmpfs to create a small ovl like this:
>
> sudo mount -t tmpfs -o casefold tmpfs mnt/
> cd mnt/
> mkdir dir
> chattr +F dir
> cd dir/
> mkdir upper lower
> mkdir lower/A lower/b lower/c
> mkdir upper/a upper/b upper/d
> mkdir merged work
> sudo mount -t overlay overlay -olowerdir=lower,upperdir=upper,workdir=work, merged
> ls /tmp/mnt/dir/merged/
> a  b  c  d
>
> And ovl is respecting the equivalent names. `a` points to a merged dir
> between `A` and `a`, but giving that upperdir has a lowercase `a`, this
> is the name displayed here.
>
> Thanks,
>         André
>
> ---
> André Almeida (3):
>       ovl: Make ovl_cache_entry_find support casefold
>       ovl: Make ovl_dentry_weird() accept casefold dentries
>       ovl: Enable support for casefold filesystems
>
>  fs/overlayfs/namei.c     | 11 ++++++-----
>  fs/overlayfs/overlayfs.h |  2 +-
>  fs/overlayfs/ovl_entry.h |  1 +
>  fs/overlayfs/params.c    |  5 +++--
>  fs/overlayfs/readdir.c   | 32 +++++++++++++++++++++-----------
>  fs/overlayfs/util.c      | 12 +++++++-----
>  6 files changed, 39 insertions(+), 24 deletions(-)
> ---
> base-commit: a24588245776dafc227243a01bfbeb8a59bafba9
> change-id: 20250409-tonyk-overlayfs-591f5e4d407a
>
> Best regards,
> --
> André Almeida <andrealmeid@igalia.com>
>

Hi Amir,

Sorry for my delay.

Em 09/04/2025 14:17, Amir Goldstein escreveu:
> On Wed, Apr 9, 2025 at 5:01 PM André Almeida <andrealmeid@igalia.com> wrote:
>>
>> Hi all,
>>
>> We would like to support the usage of casefold filesystems with
>> overlayfs. This patchset do some of the work needed for that, but I'm
>> sure there are more places that need to be tweaked so please share your
>> feedback for this work.
>>
>> * Implementation
>>
>> The most obvious place that required change was the strncmp() inside of
>> ovl_cache_entry_find(), that I managed to convert to use d_same_name(),
> 
> That's a very niche part of overlayfs where comparison of names matter.
> 
> Please look very closely at ovl_lookup() and how an overlay entry stack is
> composed from several layers including the option to redirect to different names
> via redirect xattr, so there is really very much to deal with other
> than readdir.
> 
> I suggest that you start with a design proposal of how you intend to tackle this
> task and what are your requirements?
> Any combination of casefold supported layers?
> 

The intended use case here is to use overlayfs as a container layer for 
games. The lower layer will have the common libraries required for 
games, and the upper layer will be a container for the running game, so 
the game will be able to have write permission and even change the 
common libraries if needed without impacting the original libraries. For 
that, we would use case-folded enable ext4 mounting points.

This use case doesn't need layers redirection, or to combine different 
layers of enabled/disable case-fold. We would have just two layers, 
upper and lower, both with case-fold enabled prior to mounting. If the 
layers doesn't agree on the casefold flags/version/status, we can refuse 
mounting it.

To avoid complexity and corner cases, I propose to have this feature 
enabled only for the layout described above: one upper and one lower 
layer, with both layers with the same casefold status and to refuse 
otherwise.

The implementation would be, on top of this patchset, to create 
restrictions on the mounting options if casefold is enabled in a 
mounting point.

Thoughts?

> Thanks,
> Amir.
> 

On Thu, Jul 10, 2025 at 10:54 PM André Almeida <andrealmeid@igalia.com> wrote:
>
> Hi Amir,
>
> Sorry for my delay.
>
> Em 09/04/2025 14:17, Amir Goldstein escreveu:
> > On Wed, Apr 9, 2025 at 5:01 PM André Almeida <andrealmeid@igalia.com> wrote:
> >>
> >> Hi all,
> >>
> >> We would like to support the usage of casefold filesystems with
> >> overlayfs. This patchset do some of the work needed for that, but I'm
> >> sure there are more places that need to be tweaked so please share your
> >> feedback for this work.
> >>
> >> * Implementation
> >>
> >> The most obvious place that required change was the strncmp() inside of
> >> ovl_cache_entry_find(), that I managed to convert to use d_same_name(),
> >
> > That's a very niche part of overlayfs where comparison of names matter.
> >
> > Please look very closely at ovl_lookup() and how an overlay entry stack is
> > composed from several layers including the option to redirect to different names
> > via redirect xattr, so there is really very much to deal with other
> > than readdir.
> >
> > I suggest that you start with a design proposal of how you intend to tackle this
> > task and what are your requirements?
> > Any combination of casefold supported layers?
> >
>
> The intended use case here is to use overlayfs as a container layer for
> games. The lower layer will have the common libraries required for
> games, and the upper layer will be a container for the running game, so
> the game will be able to have write permission and even change the
> common libraries if needed without impacting the original libraries. For
> that, we would use case-folded enable ext4 mounting points.
>
> This use case doesn't need layers redirection, or to combine different
> layers of enabled/disable case-fold. We would have just two layers,
> upper and lower, both with case-fold enabled prior to mounting. If the
> layers doesn't agree on the casefold flags/version/status, we can refuse
> mounting it.
>
> To avoid complexity and corner cases, I propose to have this feature
> enabled only for the layout described above: one upper and one lower
> layer, with both layers with the same casefold status and to refuse
> otherwise.
>
> The implementation would be, on top of this patchset, to create
> restrictions on the mounting options if casefold is enabled in a
> mounting point.
>
> Thoughts?
>

Good plan, but I don't think it is enough.

First of all take a look at this patch already queued for next:
https://lore.kernel.org/linux-unionfs/20250602171702.1941891-1-amir73il@gmail.com/

We will now check for ovl_dentry_casefolded() per dir on every lookup,
not only at mount time, so you need to rebase your patches and adjust the logic
to per-ovl-dir logic - all entries in ovl_stack need to have same casefold.

The second thing is that from vfs POV, your patches do not mark the overlayfs
dirs as casefolded and do not define d_compare()/d_hash() methods
for overlayfs. I think this is wrong and will result in odd behavior
w.r.t overlayfs
dcache.

I think that ovl_fill_super(), in the case where all layers are on same fs and
that fs sb_has_encoding(), assign ovl sb->s_encoding = upper_sb->s_encoding
and call generic_set_sb_d_ops().

As a matter of fact, I don't think we even need to restrict to all
layers on same
sb, just to all layers use the same sb->s_encoding and I think this
will be pretty
easy to implement on-the-fly.

Then in ovl_lookup() when composing the overlayfs stack for ovl_inode,
IFF ovl has_encoding make sure that all or none dentry on stack are
ovl_dentry_casefolded() and if they are casefolded, mark also the overlay
inode S_CASEFOLDED as well.

Please make sure to write fstests for the new functionality.
You can base you test on my WIP test for the patch that is in linux-next:
https://github.com/amir73il/xfstests/commits/ovl-casefold/

Your changes are going to change the results of my test
to allow lookup in a lower casefolded subdir
and so will need to change test expectation and add file name
case insensitive lookups to test this case.

Thanks,
Amir.

André Almeida <andrealmeid@igalia.com> writes:

> Hi all,
>
> We would like to support the usage of casefold filesystems with
> overlayfs. This patchset do some of the work needed for that, but I'm
> sure there are more places that need to be tweaked so please share your
> feedback for this work.

I didn't look the patches yet, but this is going to be quite tricky.
For a start, consider the semantics when mixing volumes with different
case settings for lower/upper/work directories.  And that could be any
setting, such as whether the directory has +F, the encoding version and
the encoding flags (strict mode).  Any mismatch will look bonkers and
you want to forbid the mount.

Consider upperdir is case-sensitive but lowerdir is not.  In this case,
I suspect the case-exact name would be hidden by the upper, but the
inexact-case would still resolve from the lower when it shouldn't, and
can't be raised again.  If we have the other way around, the upper
will hide more than one file from the lower and it is a matter of luck
which file we are getting.

In addition, if we have a case-insensitive on top of a case-sensitive,
there is no way we can do the case-insensitive lookup on the lower
without doing a sequential search across the entire directory.  Then it
is again a matter of luck which file we are getting.

The issue can appear even on the same volume, since case-insensitiveness
is actually per-directory and can be flipped when a directory is empty.
If something like the below is possible, we are in the same situation
again:

mkdir lower/ci
chattr +F lower/ci
touch lower/ci/BLA
mount -o overlay none upperdir=upper,lowerdir=lower,workdir=work merged
rm -r merged/ci/BLA    // creates a whiteout in upper
                       // merged looks empty and should be allowed to drop +F
chattr -F merged/ci

So we'd also need to always forbid clearing the +F attribute and perhaps
forbid it from ever being set on the merged directory.  We also want to
require the encoding version and flags to match.

> * Implementation
>
> The most obvious place that required change was the strncmp() inside of
> ovl_cache_entry_find(), that I managed to convert to use d_same_name(),
> that will then call the generic_ci_d_compare function if it's set for
> the dentry. There are more strncmp() around ovl, but I would rather hear
> feedback about this approach first than already implementing this around
> the code.

I'd suggest marking it as an RFC since it is not a functional
implementation yet, IIUC.

>> * Testing
> sudo mount -t tmpfs -o casefold tmpfs mnt/
> cd mnt/
> mkdir dir
> chattr +F dir
> cd dir/
> mkdir upper lower
> mkdir lower/A lower/b lower/c
> mkdir upper/a upper/b upper/d
> mkdir merged work
> sudo mount -t overlay overlay -olowerdir=lower,upperdir=upper,workdir=work, merged
> ls /tmp/mnt/dir/merged/
> a  b  c  d
>
> And ovl is respecting the equivalent names. `a` points to a merged dir
> between `A` and `a`, but giving that upperdir has a lowercase `a`, this
> is the name displayed here.

Did you try fstests generic/556?  It might require some work to make it
run over ovl, but it will exercise several cases that are quite
hard to spot.


-- 
Gabriel Krisman Bertazi

Hi Gabriel!

Em 09/04/2025 13:52, Gabriel Krisman Bertazi escreveu:
> André Almeida <andrealmeid@igalia.com> writes:
> 
>> Hi all,
>>
>> We would like to support the usage of casefold filesystems with
>> overlayfs. This patchset do some of the work needed for that, but I'm
>> sure there are more places that need to be tweaked so please share your
>> feedback for this work.
> 
> I didn't look the patches yet, but this is going to be quite tricky.
> For a start, consider the semantics when mixing volumes with different
> case settings for lower/upper/work directories.  And that could be any
> setting, such as whether the directory has +F, the encoding version and
> the encoding flags (strict mode).  Any mismatch will look bonkers and
> you want to forbid the mount.
> 
> Consider upperdir is case-sensitive but lowerdir is not.  In this case,
> I suspect the case-exact name would be hidden by the upper, but the
> inexact-case would still resolve from the lower when it shouldn't, and
> can't be raised again.  If we have the other way around, the upper
> will hide more than one file from the lower and it is a matter of luck
> which file we are getting.
> 
> In addition, if we have a case-insensitive on top of a case-sensitive,
> there is no way we can do the case-insensitive lookup on the lower
> without doing a sequential search across the entire directory.  Then it
> is again a matter of luck which file we are getting.
> 
> The issue can appear even on the same volume, since case-insensitiveness
> is actually per-directory and can be flipped when a directory is empty.
> If something like the below is possible, we are in the same situation
> again:
> 
> mkdir lower/ci
> chattr +F lower/ci
> touch lower/ci/BLA
> mount -o overlay none upperdir=upper,lowerdir=lower,workdir=work merged
> rm -r merged/ci/BLA    // creates a whiteout in upper
>                         // merged looks empty and should be allowed to drop +F
> chattr -F merged/ci
> 
> So we'd also need to always forbid clearing the +F attribute and perhaps
> forbid it from ever being set on the merged directory.  We also want to
> require the encoding version and flags to match.
> 

Thank you for the prompt response. I agree with you, for the next 
version I will implement such restrictions. I think it will be better to 
start with very restrict possibilities and then slowly dropping then, if 
they prove to be not problematic.

>> * Implementation
>>
>> The most obvious place that required change was the strncmp() inside of
>> ovl_cache_entry_find(), that I managed to convert to use d_same_name(),
>> that will then call the generic_ci_d_compare function if it's set for
>> the dentry. There are more strncmp() around ovl, but I would rather hear
>> feedback about this approach first than already implementing this around
>> the code.
> 
> I'd suggest marking it as an RFC since it is not a functional
> implementation yet, IIUC.
> 

Ops, you are right, I forgot to tag it as such.

>>> * Testing
>> sudo mount -t tmpfs -o casefold tmpfs mnt/
>> cd mnt/
>> mkdir dir
>> chattr +F dir
>> cd dir/
>> mkdir upper lower
>> mkdir lower/A lower/b lower/c
>> mkdir upper/a upper/b upper/d
>> mkdir merged work
>> sudo mount -t overlay overlay -olowerdir=lower,upperdir=upper,workdir=work, merged
>> ls /tmp/mnt/dir/merged/
>> a  b  c  d
>>
>> And ovl is respecting the equivalent names. `a` points to a merged dir
>> between `A` and `a`, but giving that upperdir has a lowercase `a`, this
>> is the name displayed here.
> 
> Did you try fstests generic/556?  It might require some work to make it
> run over ovl, but it will exercise several cases that are quite
> hard to spot.
> 
> 

I haven't tried, I'm not sure which directory should I point to fstests 
to use for the test, the merged one?