[v1] vhost-user: fix watcher need be removed when vhost-user hotplug

[Qemu-devel] [PATCH] vhost-user: fix watcher need be removed when vhost-user hotplug

Posted by Yunjian Wang 6 years, 11 months ago

From: w00273186 <wangyunjian@huawei.com>

"nc" is freed after hotplug vhost-user, but the watcher don't be removed.
The QEMU crash when the watcher access the "nc" on socket disconnect.

Call Trace:
    #0  object_get_class (obj=obj@entry=0x2) at qom/object.c:751
    #1  0x00007fc031c79f41 in qemu_chr_fe_disconnect (be=<optimized out>) at chardev/char.c:1048
    #2  0x00007fc031bd62e0 in net_vhost_user_watch (chan=<optimized out>, cond=<optimized out>, opaque=<optimized out>) at net/vhost-user.c:191
    #3  0x00007fc02c23e99a in g_main_context_dispatch () from /lib64/libglib-2.0.so.0
    #4  0x00007fc031ccfc0c in glib_pollfds_poll () at util/main-loop.c:213
    #5  os_host_main_loop_wait (timeout=<optimized out>) at util/main-loop.c:261
    #6  main_loop_wait (nonblocking=nonblocking@entry=0) at util/main-loop.c:517
    #7  0x00007fc03193bc87 in main_loop () at vl.c:1899
    #8  main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4719

Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
---
 net/vhost-user.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/net/vhost-user.c b/net/vhost-user.c
index 00a0c1c..5cc2178 100644
--- a/net/vhost-user.c
+++ b/net/vhost-user.c
@@ -155,6 +155,10 @@ static void vhost_user_cleanup(NetClientState *nc)
 
         qemu_chr_fe_deinit(&s->chr);
         object_unparent(OBJECT(chr));
+        if (s->watch) {
+            g_source_remove(s->watch);
+            s->watch = 0;
+        }
     }
 
     qemu_purge_queued_packets(nc);
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH] vhost-user: fix watcher need be removed when vhost-user hotplug

Posted by Marc-André Lureau 6 years, 11 months ago

On Tue, May 9, 2017 at 5:04 PM Yunjian Wang <wangyunjian@huawei.com> wrote:

> From: w00273186 <wangyunjian@huawei.com>
>
> "nc" is freed after hotplug vhost-user, but the watcher don't be removed.
>


> The QEMU crash when the watcher access the "nc" on socket disconnect.
>
>
Do you have a reproducer? thanks


> Call Trace:
>     #0  object_get_class (obj=obj@entry=0x2) at qom/object.c:751
>     #1  0x00007fc031c79f41 in qemu_chr_fe_disconnect (be=<optimized out>)
> at chardev/char.c:1048
>     #2  0x00007fc031bd62e0 in net_vhost_user_watch (chan=<optimized out>,
> cond=<optimized out>, opaque=<optimized out>) at net/vhost-user.c:191
>     #3  0x00007fc02c23e99a in g_main_context_dispatch () from
> /lib64/libglib-2.0.so.0
>     #4  0x00007fc031ccfc0c in glib_pollfds_poll () at util/main-loop.c:213
>     #5  os_host_main_loop_wait (timeout=<optimized out>) at
> util/main-loop.c:261
>     #6  main_loop_wait (nonblocking=nonblocking@entry=0) at
> util/main-loop.c:517
>     #7  0x00007fc03193bc87 in main_loop () at vl.c:1899
>     #8  main (argc=<optimized out>, argv=<optimized out>, envp=<optimized
> out>) at vl.c:4719
>
> Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
> ---
>  net/vhost-user.c | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/net/vhost-user.c b/net/vhost-user.c
> index 00a0c1c..5cc2178 100644
> --- a/net/vhost-user.c
> +++ b/net/vhost-user.c
> @@ -155,6 +155,10 @@ static void vhost_user_cleanup(NetClientState *nc)
>
>          qemu_chr_fe_deinit(&s->chr);
>          object_unparent(OBJECT(chr));
> +        if (s->watch) {
> +            g_source_remove(s->watch);
> +            s->watch = 0;
> +        }
>      }
>
>      qemu_purge_queued_packets(nc);
> --
> 1.8.3.1
>
>
>
> --
Marc-André Lureau

Re: [Qemu-devel] [PATCH] vhost-user: fix watcher need be removed when vhost-user hotplug

Posted by wangyunjian 6 years, 11 months ago

On Tue, May 9, 2017 at 5:04 PM Yunjian Wang <wangyunjian@huawei.com> wrote:

> > From: w00273186 <wangyunjian@huawei.com>
> >
> > "nc" is freed after hotplug vhost-user, but the watcher don't be removed.
> >
> 
> 
> > The QEMU crash when the watcher access the "nc" on socket disconnect.
> >
> >
> Do you have a reproducer? thanks
>

reproduce steps:

1. virsh attach-device vm0 vhost-user.xml
2. virsh detach-device vm0 vhost-user.xml
3. virsh attach-device vm0 vhost-user.xml
4. service openvswitch restart
5. repeat step 2~4

The vhost-user xml:

    <interface type='vhostuser'>
      <mac address='00:01:01:00:10:41'/>
      <source type='unix' path='/var/run/vhost-user/tap1' mode='client'/>
      <model type='virtio'/>
      <driver queues='8'/>
　</interface>

Thanks

> 
> > Call Trace:
> >     #0  object_get_class (obj=obj@entry=0x2) at qom/object.c:751
> >     #1  0x00007fc031c79f41 in qemu_chr_fe_disconnect (be=<optimized out>)
> > at chardev/char.c:1048
> >     #2  0x00007fc031bd62e0 in net_vhost_user_watch (chan=<optimized out>,
> > cond=<optimized out>, opaque=<optimized out>) at net/vhost-user.c:191
> >     #3  0x00007fc02c23e99a in g_main_context_dispatch () from
> > /lib64/libglib-2.0.so.0
> >     #4  0x00007fc031ccfc0c in glib_pollfds_poll () at util/main-loop.c:213
> >     #5  os_host_main_loop_wait (timeout=<optimized out>) at
> > util/main-loop.c:261
> >     #6  main_loop_wait (nonblocking=nonblocking@entry=0) at
> > util/main-loop.c:517
> >     #7  0x00007fc03193bc87 in main_loop () at vl.c:1899
> >     #8  main (argc=<optimized out>, argv=<optimized out>, envp=<optimized
> > out>) at vl.c:4719
> >
> > Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
> > ---
> >  net/vhost-user.c | 4 ++++
> >  1 file changed, 4 insertions(+)
> >
> > diff --git a/net/vhost-user.c b/net/vhost-user.c
> > index 00a0c1c..5cc2178 100644
> > --- a/net/vhost-user.c
> > +++ b/net/vhost-user.c
> > @@ -155,6 +155,10 @@ static void vhost_user_cleanup(NetClientState *nc)
> >
> >          qemu_chr_fe_deinit(&s->chr);
> >          object_unparent(OBJECT(chr));
> > +        if (s->watch) {
> > +            g_source_remove(s->watch);
> > +            s->watch = 0;
> > +        }
> >      }
> >
> >      qemu_purge_queued_packets(nc);
> > --
> > 1.8.3.1
> >
> >
> >
> > --
> Marc-André Lureau

Re: [Qemu-devel] [PATCH] vhost-user: fix watcher need be removed when vhost-user hotplug

Posted by Marc-André Lureau 6 years, 11 months ago

On Wed, May 10, 2017 at 5:55 AM wangyunjian <wangyunjian@huawei.com> wrote:

> On Tue, May 9, 2017 at 5:04 PM Yunjian Wang <wangyunjian@huawei.com>
> wrote:
>
> > > From: w00273186 <wangyunjian@huawei.com>
> > >
> > > "nc" is freed after hotplug vhost-user, but the watcher don't be
> removed.
> > >
> >
> >
> > > The QEMU crash when the watcher access the "nc" on socket disconnect.
> > >
> > >
> > Do you have a reproducer? thanks
> >
>
> reproduce steps:
>
> 1. virsh attach-device vm0 vhost-user.xml
> 2. virsh detach-device vm0 vhost-user.xml
> 3. virsh attach-device vm0 vhost-user.xml
> 4. service openvswitch restart
> 5. repeat step 2~4
>
> The vhost-user xml:
>
>     <interface type='vhostuser'>
>       <mac address='00:01:01:00:10:41'/>
>       <source type='unix' path='/var/run/vhost-user/tap1' mode='client'/>
>       <model type='virtio'/>
>       <driver queues='8'/>
> </interface>
>
> Thanks
>
> >
> > > Call Trace:
> > >     #0  object_get_class (obj=obj@entry=0x2) at qom/object.c:751
> > >     #1  0x00007fc031c79f41 in qemu_chr_fe_disconnect (be=<optimized
> out>)
> > > at chardev/char.c:1048
> > >     #2  0x00007fc031bd62e0 in net_vhost_user_watch (chan=<optimized
> out>,
> > > cond=<optimized out>, opaque=<optimized out>) at net/vhost-user.c:191
> > >     #3  0x00007fc02c23e99a in g_main_context_dispatch () from
> > > /lib64/libglib-2.0.so.0
> > >     #4  0x00007fc031ccfc0c in glib_pollfds_poll () at
> util/main-loop.c:213
> > >     #5  os_host_main_loop_wait (timeout=<optimized out>) at
> > > util/main-loop.c:261
> > >     #6  main_loop_wait (nonblocking=nonblocking@entry=0) at
> > > util/main-loop.c:517
> > >     #7  0x00007fc03193bc87 in main_loop () at vl.c:1899
> > >     #8  main (argc=<optimized out>, argv=<optimized out>,
> envp=<optimized
> > > out>) at vl.c:4719
> > >
> > > Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
> > > ---
> > >  net/vhost-user.c | 4 ++++
> > >  1 file changed, 4 insertions(+)
> > >
> > > diff --git a/net/vhost-user.c b/net/vhost-user.c
> > > index 00a0c1c..5cc2178 100644
> > > --- a/net/vhost-user.c
> > > +++ b/net/vhost-user.c
> > > @@ -155,6 +155,10 @@ static void vhost_user_cleanup(NetClientState *nc)
> > >
> > >          qemu_chr_fe_deinit(&s->chr);
> > >          object_unparent(OBJECT(chr));
> > > +        if (s->watch) {
> > > +            g_source_remove(s->watch);
> > > +            s->watch = 0;
> > > +        }
> > >      }
>

This change looks ok, and happens because libvirt calls netdev_del before
the chardev is closed or removed.

I would move the chunk before deinit/unparent though, since that's when the
close happens.

> >
> > >      qemu_purge_queued_packets(nc);
> > > --
> > > 1.8.3.1
> > >
> > >
> > >
> > > --
> > Marc-André Lureau
>
-- 
Marc-André Lureau