[PATCH v2 0/4] nbd/server: Quiesce coroutines on context switch

Sergio Lopez posted 4 patches 3 years, 4 months ago
Failed in applying to current master (apply log)
There is a newer version of this series
block.c                         |  27 ++++++-
block/export/export.c           |  10 +--
blockdev-nbd.c                  |   2 +-
hw/block/dataplane/virtio-blk.c |   4 ++
hw/block/dataplane/xen-block.c  |   7 +-
hw/scsi/virtio-scsi.c           |   6 +-
include/block/export.h          |   4 +-
nbd/server.c                    | 120 ++++++++++++++++++++++++++++----
qemu-nbd.c                      |   2 +-
stubs/blk-exp-close-all.c       |   2 +-
10 files changed, 156 insertions(+), 28 deletions(-)
[PATCH v2 0/4] nbd/server: Quiesce coroutines on context switch
Posted by Sergio Lopez 3 years, 4 months ago
This series allows the NBD server to properly switch between AIO contexts,
having quiesced recv_coroutine and send_coroutine before doing the transition.

We need this because we send back devices running in IO Thread owned contexts
to the main context when stopping the data plane, something that can happen
multiple times during the lifetime of a VM (usually during the boot sequence or
on a reboot), and we drag the NBD server of the correspoing export with it.

While there, fix also a problem caused by a cross-dependency between
closing the export's client connections and draining the block
layer. The visible effect of this problem was QEMU getting hung when
the guest request a power off while there's an active NBD client.

v2:
 - Replace "virtio-blk: Acquire context while switching them on
 dataplane start" with "block: Honor blk_set_aio_context() context
 requirements" (Kevin Wolf)
 - Add "block: Avoid processing BDS twice in
 bdrv_set_aio_context_ignore()"
 - Add "block: Close block exports in two steps"
 - Rename nbd_read_eof() to nbd_server_read_eof() (Eric Blake)
 - Fix double space and typo in comment. (Eric Blake)

Sergio Lopez (4):
  block: Honor blk_set_aio_context() context requirements
  block: Avoid processing BDS twice in bdrv_set_aio_context_ignore()
  nbd/server: Quiesce coroutines on context switch
  block: Close block exports in two steps

 block.c                         |  27 ++++++-
 block/export/export.c           |  10 +--
 blockdev-nbd.c                  |   2 +-
 hw/block/dataplane/virtio-blk.c |   4 ++
 hw/block/dataplane/xen-block.c  |   7 +-
 hw/scsi/virtio-scsi.c           |   6 +-
 include/block/export.h          |   4 +-
 nbd/server.c                    | 120 ++++++++++++++++++++++++++++----
 qemu-nbd.c                      |   2 +-
 stubs/blk-exp-close-all.c       |   2 +-
 10 files changed, 156 insertions(+), 28 deletions(-)

-- 
2.26.2



Re: [PATCH v2 0/4] nbd/server: Quiesce coroutines on context switch
Posted by Eric Blake 3 years, 2 months ago
On 12/14/20 11:05 AM, Sergio Lopez wrote:
> This series allows the NBD server to properly switch between AIO contexts,
> having quiesced recv_coroutine and send_coroutine before doing the transition.
> 
> We need this because we send back devices running in IO Thread owned contexts
> to the main context when stopping the data plane, something that can happen
> multiple times during the lifetime of a VM (usually during the boot sequence or
> on a reboot), and we drag the NBD server of the correspoing export with it.
> 
> While there, fix also a problem caused by a cross-dependency between
> closing the export's client connections and draining the block
> layer. The visible effect of this problem was QEMU getting hung when
> the guest request a power off while there's an active NBD client.
> 
> v2:
>  - Replace "virtio-blk: Acquire context while switching them on
>  dataplane start" with "block: Honor blk_set_aio_context() context
>  requirements" (Kevin Wolf)
>  - Add "block: Avoid processing BDS twice in
>  bdrv_set_aio_context_ignore()"
>  - Add "block: Close block exports in two steps"
>  - Rename nbd_read_eof() to nbd_server_read_eof() (Eric Blake)
>  - Fix double space and typo in comment. (Eric Blake)

ping - where do we stand on this series?  Patches 1 and 3 have positive
reviews, I'll queue them now; patches 2 and 4 had enough comments that
I'm guessing I should wait for a v3?


> 
> Sergio Lopez (4):
>   block: Honor blk_set_aio_context() context requirements
>   block: Avoid processing BDS twice in bdrv_set_aio_context_ignore()
>   nbd/server: Quiesce coroutines on context switch
>   block: Close block exports in two steps
> 
>  block.c                         |  27 ++++++-
>  block/export/export.c           |  10 +--
>  blockdev-nbd.c                  |   2 +-
>  hw/block/dataplane/virtio-blk.c |   4 ++
>  hw/block/dataplane/xen-block.c  |   7 +-
>  hw/scsi/virtio-scsi.c           |   6 +-
>  include/block/export.h          |   4 +-
>  nbd/server.c                    | 120 ++++++++++++++++++++++++++++----
>  qemu-nbd.c                      |   2 +-
>  stubs/blk-exp-close-all.c       |   2 +-
>  10 files changed, 156 insertions(+), 28 deletions(-)
> 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org


Re: [PATCH v2 0/4] nbd/server: Quiesce coroutines on context switch
Posted by Sergio Lopez 3 years, 2 months ago
On Wed, Jan 20, 2021 at 02:49:14PM -0600, Eric Blake wrote:
> On 12/14/20 11:05 AM, Sergio Lopez wrote:
> > This series allows the NBD server to properly switch between AIO contexts,
> > having quiesced recv_coroutine and send_coroutine before doing the transition.
> > 
> > We need this because we send back devices running in IO Thread owned contexts
> > to the main context when stopping the data plane, something that can happen
> > multiple times during the lifetime of a VM (usually during the boot sequence or
> > on a reboot), and we drag the NBD server of the correspoing export with it.
> > 
> > While there, fix also a problem caused by a cross-dependency between
> > closing the export's client connections and draining the block
> > layer. The visible effect of this problem was QEMU getting hung when
> > the guest request a power off while there's an active NBD client.
> > 
> > v2:
> >  - Replace "virtio-blk: Acquire context while switching them on
> >  dataplane start" with "block: Honor blk_set_aio_context() context
> >  requirements" (Kevin Wolf)
> >  - Add "block: Avoid processing BDS twice in
> >  bdrv_set_aio_context_ignore()"
> >  - Add "block: Close block exports in two steps"
> >  - Rename nbd_read_eof() to nbd_server_read_eof() (Eric Blake)
> >  - Fix double space and typo in comment. (Eric Blake)
> 
> ping - where do we stand on this series?  Patches 1 and 3 have positive
> reviews, I'll queue them now; patches 2 and 4 had enough comments that
> I'm guessing I should wait for a v3?

Yes, I have almost ready a v3. Will send it out today. I think it'd be
better to pull all four patches at the same time, as "block: Honor
blk_set_aio_context() context requirements" may cause trouble without
the patch to avoid double processing in
"bdrv_set_aio_context_ignore()".

Thanks,
Sergio.
 
> > 
> > Sergio Lopez (4):
> >   block: Honor blk_set_aio_context() context requirements
> >   block: Avoid processing BDS twice in bdrv_set_aio_context_ignore()
> >   nbd/server: Quiesce coroutines on context switch
> >   block: Close block exports in two steps
> > 
> >  block.c                         |  27 ++++++-
> >  block/export/export.c           |  10 +--
> >  blockdev-nbd.c                  |   2 +-
> >  hw/block/dataplane/virtio-blk.c |   4 ++
> >  hw/block/dataplane/xen-block.c  |   7 +-
> >  hw/scsi/virtio-scsi.c           |   6 +-
> >  include/block/export.h          |   4 +-
> >  nbd/server.c                    | 120 ++++++++++++++++++++++++++++----
> >  qemu-nbd.c                      |   2 +-
> >  stubs/blk-exp-close-all.c       |   2 +-
> >  10 files changed, 156 insertions(+), 28 deletions(-)
> > 
> 
> -- 
> Eric Blake, Principal Software Engineer
> Red Hat, Inc.           +1-919-301-3226
> Virtualization:  qemu.org | libvirt.org
>