[PATCH v3 0/2] hw/nvme: add irqfd support

Klaus Jensen posted 2 patches 1 year, 8 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20220825201454.259190-1-its@irrelevant.dk
Maintainers: Keith Busch <kbusch@kernel.org>, Klaus Jensen <its@irrelevant.dk>
hw/nvme/ctrl.c       | 261 ++++++++++++++++++++++++++++++++++++++++---
hw/nvme/nvme.h       |   6 +
hw/nvme/trace-events |   3 +
3 files changed, 253 insertions(+), 17 deletions(-)
[PATCH v3 0/2] hw/nvme: add irqfd support
Posted by Klaus Jensen 1 year, 8 months ago
From: Klaus Jensen <k.jensen@samsung.com>

This is a re-spin of Jinhao's irqfd support series that fixes msix
vector masking/unmasking to work correctly.

I kept being bugged out about that msi route not getting updated, so I
hit the code into submission with a hammer.

I finally noticed the core issue:

  1. The vector notifiers was never set because msix is not enabled at
     the point where nvme_init_pci() is called.

     Move this call to nvme_start_ctrl().

Since the unmask callback was suddenly getting called now, another fix
was needed:

  2. Call kvm_irqchip_add_irqfd_notifier_gsi() in the unmask handler -
     not in nvme_init_irq_notifier(). The vectors may potentially be
     masked/unmasked and shall cause a pair of add_irqfd and
     remove_irqfd calls. Removing it from nvme_init_irq_notifier() makes
     sure we do not try to double add.

Now it does what it is supposed to; no hacks required :)

Jinhao Fan (2):
  hw/nvme: support irq(de)assertion with eventfd
  hw/nvme: use KVM irqfd when available

 hw/nvme/ctrl.c       | 261 ++++++++++++++++++++++++++++++++++++++++---
 hw/nvme/nvme.h       |   6 +
 hw/nvme/trace-events |   3 +
 3 files changed, 253 insertions(+), 17 deletions(-)

-- 
2.37.2

Re: [PATCH v3 0/2] hw/nvme: add irqfd support
Posted by Jinhao Fan 1 year, 8 months ago
at 4:14 AM, Klaus Jensen <its@irrelevant.dk> wrote:

> From: Klaus Jensen <k.jensen@samsung.com>
> 
> This is a re-spin of Jinhao's irqfd support series that fixes msix
> vector masking/unmasking to work correctly.
> 
> I kept being bugged out about that msi route not getting updated, so I
> hit the code into submission with a hammer.
> 
> I finally noticed the core issue:
> 
>  1. The vector notifiers was never set because msix is not enabled at
>     the point where nvme_init_pci() is called.
> 
>     Move this call to nvme_start_ctrl().
> 
> Since the unmask callback was suddenly getting called now, another fix
> was needed:
> 
>  2. Call kvm_irqchip_add_irqfd_notifier_gsi() in the unmask handler -
>     not in nvme_init_irq_notifier(). The vectors may potentially be
>     masked/unmasked and shall cause a pair of add_irqfd and
>     remove_irqfd calls. Removing it from nvme_init_irq_notifier() makes
>     sure we do not try to double add.
> 
> Now it does what it is supposed to; no hacks required :)
> 
> Jinhao Fan (2):
>  hw/nvme: support irq(de)assertion with eventfd
>  hw/nvme: use KVM irqfd when available
> 
> hw/nvme/ctrl.c       | 261 ++++++++++++++++++++++++++++++++++++++++---
> hw/nvme/nvme.h       |   6 +
> hw/nvme/trace-events |   3 +
> 3 files changed, 253 insertions(+), 17 deletions(-)
> 
> -- 
> 2.37.2

This patch series works well on my machine without a vIOMMU. 

Just to confirm my understanding, the workflow of updating msi route is:
Host driver update MSI msg and addr -> unmask callback get called ->
kvm_irqchip_update_msi_route() does the work. Right?
Re: [PATCH v3 0/2] hw/nvme: add irqfd support
Posted by Klaus Jensen 1 year, 8 months ago
On Aug 26 10:03, Jinhao Fan wrote:
> at 4:14 AM, Klaus Jensen <its@irrelevant.dk> wrote:
> 
> > From: Klaus Jensen <k.jensen@samsung.com>
> > 
> > This is a re-spin of Jinhao's irqfd support series that fixes msix
> > vector masking/unmasking to work correctly.
> > 
> > I kept being bugged out about that msi route not getting updated, so I
> > hit the code into submission with a hammer.
> > 
> > I finally noticed the core issue:
> > 
> >  1. The vector notifiers was never set because msix is not enabled at
> >     the point where nvme_init_pci() is called.
> > 
> >     Move this call to nvme_start_ctrl().
> > 
> > Since the unmask callback was suddenly getting called now, another fix
> > was needed:
> > 
> >  2. Call kvm_irqchip_add_irqfd_notifier_gsi() in the unmask handler -
> >     not in nvme_init_irq_notifier(). The vectors may potentially be
> >     masked/unmasked and shall cause a pair of add_irqfd and
> >     remove_irqfd calls. Removing it from nvme_init_irq_notifier() makes
> >     sure we do not try to double add.
> > 
> > Now it does what it is supposed to; no hacks required :)
> > 
> > Jinhao Fan (2):
> >  hw/nvme: support irq(de)assertion with eventfd
> >  hw/nvme: use KVM irqfd when available
> > 
> > hw/nvme/ctrl.c       | 261 ++++++++++++++++++++++++++++++++++++++++---
> > hw/nvme/nvme.h       |   6 +
> > hw/nvme/trace-events |   3 +
> > 3 files changed, 253 insertions(+), 17 deletions(-)
> > 
> > -- 
> > 2.37.2
> 
> This patch series works well on my machine without a vIOMMU. 
> 
> Just to confirm my understanding, the workflow of updating msi route is:
> Host driver update MSI msg and addr -> unmask callback get called ->
> kvm_irqchip_update_msi_route() does the work. Right?
> 

Yes :)