1. Patchew
  2. QEMU
  3. View series

RE: [PATCH v1 0/4] Support dynamic MSI-X allocation

Liu, Jing2 posted 4 patches 8 months ago
Only 0 patches received!
There is a newer version of this series
RE: [PATCH v1 0/4] Support dynamic MSI-X allocation
Posted by Liu, Jing2 8 months ago 
Friendly ping to have your valuable inputs and comments. 
Thanks very much.

BRs,
Jing

> On 8/22/2023 3:29 PM, Jing Liu wrote:
> Changes since RFC v1:
> - RFC v1: https://www.mail-archive.com/qemu-
> devel@nongnu.org/msg978637.html
> - Revise the comments. (Alex)
> - Report error of getting irq info and remove the trace of failure
>   case. (Alex, Cédric)
> - Only store dynamic allocation flag as a bool type and test
>   accordingly. (Alex)
> - Move dynamic allocation detection to vfio_msix_early_setup(). (Alex)
> - Change the condition logic in vfio_msix_vector_do_use() that moving
>   the defer_kvm_irq_routing test out and create a common place to update
>   nr_vectors. (Alex)
> - Consolidate the way of MSI-X enabling during device initialization and
>   interrupt restoring that uses fd = -1 trick. Create a function doing
>   that. (Alex)
> 
> Before kernel v6.5, dynamic allocation of MSI-X interrupts was not supported.
> Qemu therefore when allocating a new interrupt, should first release all
> previously allocated interrupts (including disable of MSI-X) and re-allocate all
> interrupts that includes the new one.
> 
> The kernel series [1] adds the support of dynamic MSI-X allocation to vfio-pci
> and uses the existing flag VFIO_IRQ_INFO_NORESIZE to guide user space, that
> when dynamic MSI-X is supported the flag is cleared.
> 
> This series makes the behavior for VFIO PCI devices when dynamic MSI-X
> allocation is supported. When guest unmasks an interrupt, Qemu can directly
> allocate an interrupt on host for this and has nothing to do with the previously
> allocated ones. Therefore, host only allocates interrupts for those unmasked
> (enabled) interrupts inside guest when dynamic MSI-X allocation is supported by
> device.
> 
> When guests enable MSI-X with all of the vectors masked, Qemu need match the
> state to enable MSI-X with no vector enabled. During migration restore, Qemu
> also need enable MSI-X first in dynamic allocation mode, to avoid the guest
> unused vectors being allocated on host. To consolidate them, we use vector 0
> with an invalid fd to get MSI-X enabled and create a common function for this.
> This is cleaner than setting userspace triggering and immediately release.
> 
> Any feedback is appreciated.
> 
> Jing
> 
> [1] https://lwn.net/Articles/931679/
> 
> Jing Liu (4):
>   vfio/pci: detect the support of dynamic MSI-X allocation
>   vfio/pci: enable vector on dynamic MSI-X allocation
>   vfio/pci: use an invalid fd to enable MSI-X
>   vfio/pci: enable MSI-X in interrupt restoring on dynamic allocation
> 
>  hw/vfio/pci.c        | 126 +++++++++++++++++++++++++++++++++----------
>  hw/vfio/pci.h        |   1 +
>  hw/vfio/trace-events |   2 +-
>  3 files changed, 101 insertions(+), 28 deletions(-)
> 
> --
> 2.27.0
Re: [PATCH v1 0/4] Support dynamic MSI-X allocation
Posted by Cédric Le Goater 8 months ago 
On 9/15/23 09:40, Liu, Jing2 wrote:
> Friendly ping to have your valuable inputs and comments.
> Thanks very much.

I think that was done. We are waiting for the v2.

Thanks,

C.


> 
> BRs,
> Jing
> 
>> On 8/22/2023 3:29 PM, Jing Liu wrote:
>> Changes since RFC v1:
>> - RFC v1: https://www.mail-archive.com/qemu-
>> devel@nongnu.org/msg978637.html
>> - Revise the comments. (Alex)
>> - Report error of getting irq info and remove the trace of failure
>>    case. (Alex, Cédric)
>> - Only store dynamic allocation flag as a bool type and test
>>    accordingly. (Alex)
>> - Move dynamic allocation detection to vfio_msix_early_setup(). (Alex)
>> - Change the condition logic in vfio_msix_vector_do_use() that moving
>>    the defer_kvm_irq_routing test out and create a common place to update
>>    nr_vectors. (Alex)
>> - Consolidate the way of MSI-X enabling during device initialization and
>>    interrupt restoring that uses fd = -1 trick. Create a function doing
>>    that. (Alex)
>>
>> Before kernel v6.5, dynamic allocation of MSI-X interrupts was not supported.
>> Qemu therefore when allocating a new interrupt, should first release all
>> previously allocated interrupts (including disable of MSI-X) and re-allocate all
>> interrupts that includes the new one.
>>
>> The kernel series [1] adds the support of dynamic MSI-X allocation to vfio-pci
>> and uses the existing flag VFIO_IRQ_INFO_NORESIZE to guide user space, that
>> when dynamic MSI-X is supported the flag is cleared.
>>
>> This series makes the behavior for VFIO PCI devices when dynamic MSI-X
>> allocation is supported. When guest unmasks an interrupt, Qemu can directly
>> allocate an interrupt on host for this and has nothing to do with the previously
>> allocated ones. Therefore, host only allocates interrupts for those unmasked
>> (enabled) interrupts inside guest when dynamic MSI-X allocation is supported by
>> device.
>>
>> When guests enable MSI-X with all of the vectors masked, Qemu need match the
>> state to enable MSI-X with no vector enabled. During migration restore, Qemu
>> also need enable MSI-X first in dynamic allocation mode, to avoid the guest
>> unused vectors being allocated on host. To consolidate them, we use vector 0
>> with an invalid fd to get MSI-X enabled and create a common function for this.
>> This is cleaner than setting userspace triggering and immediately release.
>>
>> Any feedback is appreciated.
>>
>> Jing
>>
>> [1] https://lwn.net/Articles/931679/
>>
>> Jing Liu (4):
>>    vfio/pci: detect the support of dynamic MSI-X allocation
>>    vfio/pci: enable vector on dynamic MSI-X allocation
>>    vfio/pci: use an invalid fd to enable MSI-X
>>    vfio/pci: enable MSI-X in interrupt restoring on dynamic allocation
>>
>>   hw/vfio/pci.c        | 126 +++++++++++++++++++++++++++++++++----------
>>   hw/vfio/pci.h        |   1 +
>>   hw/vfio/trace-events |   2 +-
>>   3 files changed, 101 insertions(+), 28 deletions(-)
>>
>> --
>> 2.27.0
>
RE: [PATCH v1 0/4] Support dynamic MSI-X allocation
Posted by Liu, Jing2 8 months ago 
Hi Cédric,
Thanks for this information. I'll send v2 later.

Jing

> On 9/15/2023 3:42 PM, Cédric Le Goater <clg@redhat.com> wrote:
> 
> On 9/15/23 09:40, Liu, Jing2 wrote:
> > Friendly ping to have your valuable inputs and comments.
> > Thanks very much.
> 
> I think that was done. We are waiting for the v2.
> 
> Thanks,
> 
> C.
> 
> 
> >
> > BRs,
> > Jing
> >
> >> On 8/22/2023 3:29 PM, Jing Liu wrote:
> >> Changes since RFC v1:
> >> - RFC v1: https://www.mail-archive.com/qemu-
> >> devel@nongnu.org/msg978637.html
> >> - Revise the comments. (Alex)
> >> - Report error of getting irq info and remove the trace of failure
> >>    case. (Alex, Cédric)
> >> - Only store dynamic allocation flag as a bool type and test
> >>    accordingly. (Alex)
> >> - Move dynamic allocation detection to vfio_msix_early_setup().
> >> (Alex)
> >> - Change the condition logic in vfio_msix_vector_do_use() that moving
> >>    the defer_kvm_irq_routing test out and create a common place to update
> >>    nr_vectors. (Alex)
> >> - Consolidate the way of MSI-X enabling during device initialization and
> >>    interrupt restoring that uses fd = -1 trick. Create a function doing
> >>    that. (Alex)
> >>
> >> Before kernel v6.5, dynamic allocation of MSI-X interrupts was not supported.
> >> Qemu therefore when allocating a new interrupt, should first release
> >> all previously allocated interrupts (including disable of MSI-X) and
> >> re-allocate all interrupts that includes the new one.
> >>
> >> The kernel series [1] adds the support of dynamic MSI-X allocation to
> >> vfio-pci and uses the existing flag VFIO_IRQ_INFO_NORESIZE to guide
> >> user space, that when dynamic MSI-X is supported the flag is cleared.
> >>
> >> This series makes the behavior for VFIO PCI devices when dynamic
> >> MSI-X allocation is supported. When guest unmasks an interrupt, Qemu
> >> can directly allocate an interrupt on host for this and has nothing
> >> to do with the previously allocated ones. Therefore, host only
> >> allocates interrupts for those unmasked
> >> (enabled) interrupts inside guest when dynamic MSI-X allocation is
> >> supported by device.
> >>
> >> When guests enable MSI-X with all of the vectors masked, Qemu need
> >> match the state to enable MSI-X with no vector enabled. During
> >> migration restore, Qemu also need enable MSI-X first in dynamic
> >> allocation mode, to avoid the guest unused vectors being allocated on
> >> host. To consolidate them, we use vector 0 with an invalid fd to get MSI-X
> enabled and create a common function for this.
> >> This is cleaner than setting userspace triggering and immediately release.
> >>
> >> Any feedback is appreciated.
> >>
> >> Jing
> >>
> >> [1] https://lwn.net/Articles/931679/
> >>
> >> Jing Liu (4):
> >>    vfio/pci: detect the support of dynamic MSI-X allocation
> >>    vfio/pci: enable vector on dynamic MSI-X allocation
> >>    vfio/pci: use an invalid fd to enable MSI-X
> >>    vfio/pci: enable MSI-X in interrupt restoring on dynamic
> >> allocation
> >>
> >>   hw/vfio/pci.c        | 126 +++++++++++++++++++++++++++++++++----------
> >>   hw/vfio/pci.h        |   1 +
> >>   hw/vfio/trace-events |   2 +-
> >>   3 files changed, 101 insertions(+), 28 deletions(-)
> >>
> >> --
> >> 2.27.0
> >

Patchew 2.1-devel-0870f84

© 2016 - 2024 Red Hat, Inc.