[PATCH v4 0/5] Allow dyn MSI-X vector allocation of MANA

Shradha Gupta posted 5 patches 6 months, 3 weeks ago
There is a newer version of this series
.../net/ethernet/microsoft/mana/gdma_main.c   | 356 ++++++++++++++----
drivers/pci/controller/pci-hyperv.c           |   5 +-
drivers/pci/msi/irqdomain.c                   |   5 +-
include/linux/msi.h                           |   2 +
include/net/mana/gdma.h                       |   8 +-
5 files changed, 293 insertions(+), 83 deletions(-)
[PATCH v4 0/5] Allow dyn MSI-X vector allocation of MANA
Posted by Shradha Gupta 6 months, 3 weeks ago
In this patchset we want to enable the MANA driver to be able to
allocate MSI-X vectors in PCI dynamically.

The first patch exports pci_msix_prepare_desc() in PCI to be able to
correctly prepare descriptors for dynamically added MSI-X vectors.

The second patch adds the support of dynamic vector allocation in
pci-hyperv PCI controller by enabling the MSI_FLAG_PCI_MSIX_ALLOC_DYN
flag and using the pci_msix_prepare_desc() exported in first patch.

The third patch adds a detailed description of the irq_setup(), to
help understand the function design better.

The fourth patch is a preparation patch for mana changes to support
dynamic IRQ allocation. It contains changes in irq_setup() to allow
skipping first sibling CPU sets, in case certain IRQs are already
affinitized to them.

The fifth patch has the changes in MANA driver to be able to allocate
MSI-X vectors dynamically. If the support does not exist it defaults to
older behavior.
---
 Change in v4
 * add a patch describing the functionality of irq_setup() through a 
   comment
 * In irq_setup(), avoid using a label next_cpumask:
 * modify the changes in MANA patch about restructuring the error
   handling path in mana_gd_setup_dyn_irqs()
 * modify the mana_gd_setup_irqs() to simplify handling around
   start_irq_index
 * add warning if an invalid gic is returned
 * place the xa_destroy() cleanup in mana_gd_remove
---
 Changes in v3
 * split the 3rd patch into preparation patch around irq_setup() and
   changes in mana driver to allow dynamic IRQ allocation
 * Add arm64 support for dynamic MSI-X allocation in pci_hyperv
   controller
---
 Changes in v2
 * split the first patch into two(exporting the preapre_desc
   func and using the function and flag in pci-hyperv)
 * replace 'pci vectors' by 'MSI-X vectors'
 * Change the cover letter description to align with changes made
---

Shradha Gupta (5):
  PCI/MSI: Export pci_msix_prepare_desc() for dynamic MSI-X allocations
  PCI: hv: Allow dynamic MSI-X vector allocation
  net: mana: explain irq_setup() algorithm
  net: mana: Allow irq_setup() to skip cpus for affinity
  net: mana: Allocate MSI-X vectors dynamically

 .../net/ethernet/microsoft/mana/gdma_main.c   | 356 ++++++++++++++----
 drivers/pci/controller/pci-hyperv.c           |   5 +-
 drivers/pci/msi/irqdomain.c                   |   5 +-
 include/linux/msi.h                           |   2 +
 include/net/mana/gdma.h                       |   8 +-
 5 files changed, 293 insertions(+), 83 deletions(-)

-- 
2.34.1
Re: [PATCH v4 0/5] Allow dyn MSI-X vector allocation of MANA
Posted by Simon Horman 6 months, 3 weeks ago
On Tue, May 27, 2025 at 08:57:33AM -0700, Shradha Gupta wrote:
> In this patchset we want to enable the MANA driver to be able to
> allocate MSI-X vectors in PCI dynamically.
> 
> The first patch exports pci_msix_prepare_desc() in PCI to be able to
> correctly prepare descriptors for dynamically added MSI-X vectors.
> 
> The second patch adds the support of dynamic vector allocation in
> pci-hyperv PCI controller by enabling the MSI_FLAG_PCI_MSIX_ALLOC_DYN
> flag and using the pci_msix_prepare_desc() exported in first patch.
> 
> The third patch adds a detailed description of the irq_setup(), to
> help understand the function design better.
> 
> The fourth patch is a preparation patch for mana changes to support
> dynamic IRQ allocation. It contains changes in irq_setup() to allow
> skipping first sibling CPU sets, in case certain IRQs are already
> affinitized to them.
> 
> The fifth patch has the changes in MANA driver to be able to allocate
> MSI-X vectors dynamically. If the support does not exist it defaults to
> older behavior.

Hi Shradha,

It's unclear what the target tree for this patch-set is.
But if it is net-next, which seems likely given the code under
drivers/net/, then:

Please include that target in the subject of each patch in the patch-set.

	Subject: [PATCH v5 net-next 0/5] ...

And, moreover, ...

## Form letter - net-next-closed

The merge window for v6.16 has begun and therefore net-next is closed
for new drivers, features, code refactoring and optimizations. We are
currently accepting bug fixes only.

Please repost when net-next reopens after June 8th.

RFC patches sent for review only are obviously welcome at any time.
Re: [PATCH v4 0/5] Allow dyn MSI-X vector allocation of MANA
Posted by Shradha Gupta 6 months, 3 weeks ago
On Wed, May 28, 2025 at 07:55:08PM +0100, Simon Horman wrote:
> On Tue, May 27, 2025 at 08:57:33AM -0700, Shradha Gupta wrote:
> > In this patchset we want to enable the MANA driver to be able to
> > allocate MSI-X vectors in PCI dynamically.
> > 
> > The first patch exports pci_msix_prepare_desc() in PCI to be able to
> > correctly prepare descriptors for dynamically added MSI-X vectors.
> > 
> > The second patch adds the support of dynamic vector allocation in
> > pci-hyperv PCI controller by enabling the MSI_FLAG_PCI_MSIX_ALLOC_DYN
> > flag and using the pci_msix_prepare_desc() exported in first patch.
> > 
> > The third patch adds a detailed description of the irq_setup(), to
> > help understand the function design better.
> > 
> > The fourth patch is a preparation patch for mana changes to support
> > dynamic IRQ allocation. It contains changes in irq_setup() to allow
> > skipping first sibling CPU sets, in case certain IRQs are already
> > affinitized to them.
> > 
> > The fifth patch has the changes in MANA driver to be able to allocate
> > MSI-X vectors dynamically. If the support does not exist it defaults to
> > older behavior.
> 
> Hi Shradha,
> 
> It's unclear what the target tree for this patch-set is.
> But if it is net-next, which seems likely given the code under
> drivers/net/, then:
> 
> Please include that target in the subject of each patch in the patch-set.
> 
> 	Subject: [PATCH v5 net-next 0/5] ...
> 
> And, moreover, ...
> 
> ## Form letter - net-next-closed
> 
> The merge window for v6.16 has begun and therefore net-next is closed
> for new drivers, features, code refactoring and optimizations. We are
> currently accepting bug fixes only.
> 
> Please repost when net-next reopens after June 8th.
> 
> RFC patches sent for review only are obviously welcome at any time.

Thank you Simon.

While posting this patchset I was a bit confused about what should be
the target tree. That's why in the cover letter of the V1 for this
series, I had requested more clarity on the same (since there are patches
from PCI and net-next both).

In such cases how do we decide which tree to target?

Also, noted about the next merge window for net-next :-)

Regards,
Shradha.
Re: [PATCH v4 0/5] Allow dyn MSI-X vector allocation of MANA
Posted by Simon Horman 6 months, 3 weeks ago
On Thu, May 29, 2025 at 06:28:45AM -0700, Shradha Gupta wrote:
> On Wed, May 28, 2025 at 07:55:08PM +0100, Simon Horman wrote:
> > On Tue, May 27, 2025 at 08:57:33AM -0700, Shradha Gupta wrote:
> > > In this patchset we want to enable the MANA driver to be able to
> > > allocate MSI-X vectors in PCI dynamically.
> > > 
> > > The first patch exports pci_msix_prepare_desc() in PCI to be able to
> > > correctly prepare descriptors for dynamically added MSI-X vectors.
> > > 
> > > The second patch adds the support of dynamic vector allocation in
> > > pci-hyperv PCI controller by enabling the MSI_FLAG_PCI_MSIX_ALLOC_DYN
> > > flag and using the pci_msix_prepare_desc() exported in first patch.
> > > 
> > > The third patch adds a detailed description of the irq_setup(), to
> > > help understand the function design better.
> > > 
> > > The fourth patch is a preparation patch for mana changes to support
> > > dynamic IRQ allocation. It contains changes in irq_setup() to allow
> > > skipping first sibling CPU sets, in case certain IRQs are already
> > > affinitized to them.
> > > 
> > > The fifth patch has the changes in MANA driver to be able to allocate
> > > MSI-X vectors dynamically. If the support does not exist it defaults to
> > > older behavior.
> > 
> > Hi Shradha,
> > 
> > It's unclear what the target tree for this patch-set is.
> > But if it is net-next, which seems likely given the code under
> > drivers/net/, then:
> > 
> > Please include that target in the subject of each patch in the patch-set.
> > 
> > 	Subject: [PATCH v5 net-next 0/5] ...
> > 
> > And, moreover, ...
> > 
> > ## Form letter - net-next-closed
> > 
> > The merge window for v6.16 has begun and therefore net-next is closed
> > for new drivers, features, code refactoring and optimizations. We are
> > currently accepting bug fixes only.
> > 
> > Please repost when net-next reopens after June 8th.
> > 
> > RFC patches sent for review only are obviously welcome at any time.
> 
> Thank you Simon.
> 
> While posting this patchset I was a bit confused about what should be
> the target tree. That's why in the cover letter of the V1 for this
> series, I had requested more clarity on the same (since there are patches
> from PCI and net-next both).
> 
> In such cases how do we decide which tree to target?

Yes, that isn't entirely clear to me either.
Hopefully the maintainers can negotiate this.

> 
> Also, noted about the next merge window for net-next :-)
> 
> Regards,
> Shradha.
>
Re: [PATCH v4 0/5] Allow dyn MSI-X vector allocation of MANA
Posted by Shradha Gupta 6 months, 2 weeks ago
On Fri, May 30, 2025 at 07:07:32PM +0100, Simon Horman wrote:
> On Thu, May 29, 2025 at 06:28:45AM -0700, Shradha Gupta wrote:
> > On Wed, May 28, 2025 at 07:55:08PM +0100, Simon Horman wrote:
> > > On Tue, May 27, 2025 at 08:57:33AM -0700, Shradha Gupta wrote:
> > > > In this patchset we want to enable the MANA driver to be able to
> > > > allocate MSI-X vectors in PCI dynamically.
> > > > 
> > > > The first patch exports pci_msix_prepare_desc() in PCI to be able to
> > > > correctly prepare descriptors for dynamically added MSI-X vectors.
> > > > 
> > > > The second patch adds the support of dynamic vector allocation in
> > > > pci-hyperv PCI controller by enabling the MSI_FLAG_PCI_MSIX_ALLOC_DYN
> > > > flag and using the pci_msix_prepare_desc() exported in first patch.
> > > > 
> > > > The third patch adds a detailed description of the irq_setup(), to
> > > > help understand the function design better.
> > > > 
> > > > The fourth patch is a preparation patch for mana changes to support
> > > > dynamic IRQ allocation. It contains changes in irq_setup() to allow
> > > > skipping first sibling CPU sets, in case certain IRQs are already
> > > > affinitized to them.
> > > > 
> > > > The fifth patch has the changes in MANA driver to be able to allocate
> > > > MSI-X vectors dynamically. If the support does not exist it defaults to
> > > > older behavior.
> > > 
> > > Hi Shradha,
> > > 
> > > It's unclear what the target tree for this patch-set is.
> > > But if it is net-next, which seems likely given the code under
> > > drivers/net/, then:
> > > 
> > > Please include that target in the subject of each patch in the patch-set.
> > > 
> > > 	Subject: [PATCH v5 net-next 0/5] ...
> > > 
> > > And, moreover, ...
> > > 
> > > ## Form letter - net-next-closed
> > > 
> > > The merge window for v6.16 has begun and therefore net-next is closed
> > > for new drivers, features, code refactoring and optimizations. We are
> > > currently accepting bug fixes only.
> > > 
> > > Please repost when net-next reopens after June 8th.
> > > 
> > > RFC patches sent for review only are obviously welcome at any time.
> > 
> > Thank you Simon.
> > 
> > While posting this patchset I was a bit confused about what should be
> > the target tree. That's why in the cover letter of the V1 for this
> > series, I had requested more clarity on the same (since there are patches
> > from PCI and net-next both).
> > 
> > In such cases how do we decide which tree to target?
> 
> Yes, that isn't entirely clear to me either.
> Hopefully the maintainers can negotiate this.
>

Thanks Simon. also since teh target tree is not entirely clear, can I
still send out an updated version with suggested changes?
 
> > 
> > Also, noted about the next merge window for net-next :-)
> > 
> > Regards,
> > Shradha.
> >
Re: [PATCH v4 0/5] Allow dyn MSI-X vector allocation of MANA
Posted by Zhu Yanjun 6 months, 3 weeks ago
在 2025/5/27 17:57, Shradha Gupta 写道:
> In this patchset we want to enable the MANA driver to be able to
> allocate MSI-X vectors in PCI dynamically.
> 
> The first patch exports pci_msix_prepare_desc() in PCI to be able to
> correctly prepare descriptors for dynamically added MSI-X vectors.
> 
> The second patch adds the support of dynamic vector allocation in
> pci-hyperv PCI controller by enabling the MSI_FLAG_PCI_MSIX_ALLOC_DYN
> flag and using the pci_msix_prepare_desc() exported in first patch.
> 
> The third patch adds a detailed description of the irq_setup(), to
> help understand the function design better.
> 
> The fourth patch is a preparation patch for mana changes to support
> dynamic IRQ allocation. It contains changes in irq_setup() to allow
> skipping first sibling CPU sets, in case certain IRQs are already
> affinitized to them.
> 
> The fifth patch has the changes in MANA driver to be able to allocate
> MSI-X vectors dynamically. If the support does not exist it defaults to
> older behavior.
> ---
>   Change in v4
>   * add a patch describing the functionality of irq_setup() through a
>     comment
>   * In irq_setup(), avoid using a label next_cpumask:
>   * modify the changes in MANA patch about restructuring the error
>     handling path in mana_gd_setup_dyn_irqs()
>   * modify the mana_gd_setup_irqs() to simplify handling around
>     start_irq_index
>   * add warning if an invalid gic is returned
>   * place the xa_destroy() cleanup in mana_gd_remove
> ---
>   Changes in v3
>   * split the 3rd patch into preparation patch around irq_setup() and
>     changes in mana driver to allow dynamic IRQ allocation
>   * Add arm64 support for dynamic MSI-X allocation in pci_hyperv
>     controller
> ---
>   Changes in v2
>   * split the first patch into two(exporting the preapre_desc
>     func and using the function and flag in pci-hyperv)
>   * replace 'pci vectors' by 'MSI-X vectors'
>   * Change the cover letter description to align with changes made
> ---
> 
> Shradha Gupta (5):
>    PCI/MSI: Export pci_msix_prepare_desc() for dynamic MSI-X allocations
>    PCI: hv: Allow dynamic MSI-X vector allocation
>    net: mana: explain irq_setup() algorithm
>    net: mana: Allow irq_setup() to skip cpus for affinity
>    net: mana: Allocate MSI-X vectors dynamically

In this patchset, base-commit seems missing.

Please see this link:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/submitting-patches.rst?h=v6.15#n868

"
When you open ``outgoing/0000-cover-letter.patch`` for editing, you will
notice that it will have the ``base-commit:`` trailer at the very
bottom, which provides the reviewer and the CI tools enough information
to properly perform ``git am`` without worrying about conflicts::
"

When creating patches:
"
git format-patch --base=main origin/main
"

This will include a base-commit: line in each patch file:

"
base-commit: abcdef1234567890...
"

This is useful when submitting patches to mailing lists or other tooling.

Please follow the submitting-patches.rst to add base-commit.

Best Regards,
Zhu Yanjun

> 
>   .../net/ethernet/microsoft/mana/gdma_main.c   | 356 ++++++++++++++----
>   drivers/pci/controller/pci-hyperv.c           |   5 +-
>   drivers/pci/msi/irqdomain.c                   |   5 +-
>   include/linux/msi.h                           |   2 +
>   include/net/mana/gdma.h                       |   8 +-
>   5 files changed, 293 insertions(+), 83 deletions(-)
> 

Re: [PATCH v4 0/5] Allow dyn MSI-X vector allocation of MANA
Posted by Shradha Gupta 6 months, 2 weeks ago
On Sun, Jun 01, 2025 at 04:53:31PM +0200, Zhu Yanjun wrote:
> ??? 2025/5/27 17:57, Shradha Gupta ??????:
> >In this patchset we want to enable the MANA driver to be able to
> >allocate MSI-X vectors in PCI dynamically.
> >
> >The first patch exports pci_msix_prepare_desc() in PCI to be able to
> >correctly prepare descriptors for dynamically added MSI-X vectors.
> >
> >The second patch adds the support of dynamic vector allocation in
> >pci-hyperv PCI controller by enabling the MSI_FLAG_PCI_MSIX_ALLOC_DYN
> >flag and using the pci_msix_prepare_desc() exported in first patch.
> >
> >The third patch adds a detailed description of the irq_setup(), to
> >help understand the function design better.
> >
> >The fourth patch is a preparation patch for mana changes to support
> >dynamic IRQ allocation. It contains changes in irq_setup() to allow
> >skipping first sibling CPU sets, in case certain IRQs are already
> >affinitized to them.
> >
> >The fifth patch has the changes in MANA driver to be able to allocate
> >MSI-X vectors dynamically. If the support does not exist it defaults to
> >older behavior.
> >---
> >  Change in v4
> >  * add a patch describing the functionality of irq_setup() through a
> >    comment
> >  * In irq_setup(), avoid using a label next_cpumask:
> >  * modify the changes in MANA patch about restructuring the error
> >    handling path in mana_gd_setup_dyn_irqs()
> >  * modify the mana_gd_setup_irqs() to simplify handling around
> >    start_irq_index
> >  * add warning if an invalid gic is returned
> >  * place the xa_destroy() cleanup in mana_gd_remove
> >---
> >  Changes in v3
> >  * split the 3rd patch into preparation patch around irq_setup() and
> >    changes in mana driver to allow dynamic IRQ allocation
> >  * Add arm64 support for dynamic MSI-X allocation in pci_hyperv
> >    controller
> >---
> >  Changes in v2
> >  * split the first patch into two(exporting the preapre_desc
> >    func and using the function and flag in pci-hyperv)
> >  * replace 'pci vectors' by 'MSI-X vectors'
> >  * Change the cover letter description to align with changes made
> >---
> >
> >Shradha Gupta (5):
> >   PCI/MSI: Export pci_msix_prepare_desc() for dynamic MSI-X allocations
> >   PCI: hv: Allow dynamic MSI-X vector allocation
> >   net: mana: explain irq_setup() algorithm
> >   net: mana: Allow irq_setup() to skip cpus for affinity
> >   net: mana: Allocate MSI-X vectors dynamically
> 
> In this patchset, base-commit seems missing.
> 
> Please see this link:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/submitting-patches.rst?h=v6.15#n868
> 
> "
> When you open ``outgoing/0000-cover-letter.patch`` for editing, you will
> notice that it will have the ``base-commit:`` trailer at the very
> bottom, which provides the reviewer and the CI tools enough information
> to properly perform ``git am`` without worrying about conflicts::
> "
> 
> When creating patches:
> "
> git format-patch --base=main origin/main
> "
> 
> This will include a base-commit: line in each patch file:
> 
> "
> base-commit: abcdef1234567890...
> "
> 
> This is useful when submitting patches to mailing lists or other tooling.
> 
> Please follow the submitting-patches.rst to add base-commit.
> 
> Best Regards,
> Zhu Yanjun
>

Thank you, I will make the necessary changes in the next version.

Regards,
Shradha. 
> >
> >  .../net/ethernet/microsoft/mana/gdma_main.c   | 356 ++++++++++++++----
> >  drivers/pci/controller/pci-hyperv.c           |   5 +-
> >  drivers/pci/msi/irqdomain.c                   |   5 +-
> >  include/linux/msi.h                           |   2 +
> >  include/net/mana/gdma.h                       |   8 +-
> >  5 files changed, 293 insertions(+), 83 deletions(-)
> >