> > Even simpler: allow O_PATH descriptors for f*xattr().
> 
> Attached patch.  Will post shortly.
> 
> However, I think it would make sense to fix virtiofsd as well, as this will take time to percolate down, even if Al doesn't find
> anything wrong with it.

Thanks for you comments.

Though I'm still learning virtiofsd code, if nobody will try I'm willing to work on this.

> Doing unshare(CLONE_FS) after thread startup seems safe, though must be careful to change the working directory to the root of
> the mount
> *before* starting any threads.

I think working directry is changed in setup_sandbox() -> setup_mount_namespace() -> setup_pivot_root().
So, can we just add unshare(CLONE_FS) in fv_queue_worker()?

Sorry if I'm totally misunderstood the situation.

Thanks,
Misono

On Fri, Oct 18, 2019 at 08:51:20AM +0000, misono.tomohiro@fujitsu.com wrote:
> > Doing unshare(CLONE_FS) after thread startup seems safe, though must be careful to change the working directory to the root of
> > the mount
> > *before* starting any threads.
> 
> I think working directry is changed in setup_sandbox() -> setup_mount_namespace() -> setup_pivot_root().
> So, can we just add unshare(CLONE_FS) in fv_queue_worker()?

fv_queue_worker() is the thread pool worker function that is called for
each request.  Calling unshare(CLONE_FS) for each request is not
necessary and will reduce performance.

A thread-local variable can be used to avoid repeated calls to
unshare(CLONE_FS) from the same worker thread:

  static __thread bool clone_fs_called;

  static void fv_queue_worker(gpointer data, gpointer user_data)
  {
      ...
      if (!clone_fs_called) {
          int ret;

	  ret = unshare(CLONE_FS);
	  assert(ret == 0); /* errors not expected according to man page */

	  clone_fs_called = true;
      }

Another issue is the seccomp policy.  Since worker threads are spawned
at runtime it is necessary to add the unshare(2) syscall to the seccomp
whitelist in contrib/virtiofsd/seccomp.c.

Stefan

> -----Original Message-----
> From: Stefan Hajnoczi [mailto:stefanha@redhat.com]
> Sent: Monday, October 21, 2019 6:41 PM
> To: Misono, Tomohiro/味曽野 智礼 <misono.tomohiro@fujitsu.com>
> Cc: 'Miklos Szeredi' <mszeredi@redhat.com>; virtio-fs@redhat.com; qemu-devel@nongnu.org
> Subject: Re: [Virtio-fs] [PATCH 0/2] virtiofsd: Two fix for xattr operation
> 
> On Fri, Oct 18, 2019 at 08:51:20AM +0000, misono.tomohiro@fujitsu.com wrote:
> > > Doing unshare(CLONE_FS) after thread startup seems safe, though must
> > > be careful to change the working directory to the root of the mount
> > > *before* starting any threads.
> >
> > I think working directry is changed in setup_sandbox() -> setup_mount_namespace() -> setup_pivot_root().
> > So, can we just add unshare(CLONE_FS) in fv_queue_worker()?
> 
> fv_queue_worker() is the thread pool worker function that is called for each request.  Calling unshare(CLONE_FS) for each request
> is not necessary and will reduce performance.
> 
> A thread-local variable can be used to avoid repeated calls to
> unshare(CLONE_FS) from the same worker thread:
> 
>   static __thread bool clone_fs_called;
> 
>   static void fv_queue_worker(gpointer data, gpointer user_data)
>   {
>       ...
>       if (!clone_fs_called) {
>           int ret;
> 
> 	  ret = unshare(CLONE_FS);
> 	  assert(ret == 0); /* errors not expected according to man page */
> 
> 	  clone_fs_called = true;
>       }
> 
> Another issue is the seccomp policy.  Since worker threads are spawned at runtime it is necessary to add the unshare(2) syscall
> to the seccomp whitelist in contrib/virtiofsd/seccomp.c.
> 

Thanks for suggesting.

I tried above code with fchdir() + ...xattr() + fchdir() in lo_...xattr
and it solves all the problem about xattr I see.

However, it seems the fix causes some performance issue in stress test
as ACL check issues getxattr() and a lot of fchdir() happens. So, I may
try to combine the old method for regular file and this method for special
files.

Thanks,
Misono

On Wed, Oct 23, 2019 at 11:42:50AM +0000, misono.tomohiro@fujitsu.com wrote:
> > -----Original Message-----
> > From: Stefan Hajnoczi [mailto:stefanha@redhat.com]
> > Sent: Monday, October 21, 2019 6:41 PM
> > To: Misono, Tomohiro/味曽野 智礼 <misono.tomohiro@fujitsu.com>
> > Cc: 'Miklos Szeredi' <mszeredi@redhat.com>; virtio-fs@redhat.com; qemu-devel@nongnu.org
> > Subject: Re: [Virtio-fs] [PATCH 0/2] virtiofsd: Two fix for xattr operation
> > 
> > On Fri, Oct 18, 2019 at 08:51:20AM +0000, misono.tomohiro@fujitsu.com wrote:
> > > > Doing unshare(CLONE_FS) after thread startup seems safe, though must
> > > > be careful to change the working directory to the root of the mount
> > > > *before* starting any threads.
> > >
> > > I think working directry is changed in setup_sandbox() -> setup_mount_namespace() -> setup_pivot_root().
> > > So, can we just add unshare(CLONE_FS) in fv_queue_worker()?
> > 
> > fv_queue_worker() is the thread pool worker function that is called for each request.  Calling unshare(CLONE_FS) for each request
> > is not necessary and will reduce performance.
> > 
> > A thread-local variable can be used to avoid repeated calls to
> > unshare(CLONE_FS) from the same worker thread:
> > 
> >   static __thread bool clone_fs_called;
> > 
> >   static void fv_queue_worker(gpointer data, gpointer user_data)
> >   {
> >       ...
> >       if (!clone_fs_called) {
> >           int ret;
> > 
> > 	  ret = unshare(CLONE_FS);
> > 	  assert(ret == 0); /* errors not expected according to man page */
> > 
> > 	  clone_fs_called = true;
> >       }
> > 
> > Another issue is the seccomp policy.  Since worker threads are spawned at runtime it is necessary to add the unshare(2) syscall
> > to the seccomp whitelist in contrib/virtiofsd/seccomp.c.
> > 
> 
> Thanks for suggesting.
> 
> I tried above code with fchdir() + ...xattr() + fchdir() in lo_...xattr
> and it solves all the problem about xattr I see.
> 
> However, it seems the fix causes some performance issue in stress test
> as ACL check issues getxattr() and a lot of fchdir() happens. So, I may
> try to combine the old method for regular file and this method for special
> files.

Sounds good.

Stefan