[v2] PCI/P2PDMA: Avoid returning a provider for non_mappable_bars

[PATCH v2] PCI/P2PDMA: Avoid returning a provider for non_mappable_bars

Posted by Matt Evans 1 month, 3 weeks ago

Extend the checks in pcim_p2pdma_init() and pcim_p2pdma_provider() to
exclude functions that have pdev->non_mappable_bars set.

Consumers such as VFIO were previously able to map these for access by
the CPU or P2P.  Update the comment on non_mappable_bars to show it
refers to any access, not just userspace CPU access.

Fixes: 372d6d1b8ae3c ("PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation")
Signed-off-by: Matt Evans <mattev@meta.com>
---

This arises from Alex Williamson's suggestion to test
non_mappable_bars when getting the provider, with discussion here:

 https://lore.kernel.org/kvm/20260415181623.1021090-1-mattev@meta.com/

The goal was to prevent a hole where VFIO could export DMABUFs for
BARs marked non-mappable, and to fix for all users of the provider
rather than just VFIO.  Alex observed that non_mappable_bars should be
taken to mean BARs weren't usable by the CPU _or_ peers and,
considering that, its comment about userspace access wasn't quite
right.

== Changes ==

v2:
 - Also test non_mappable_bars in pcim_p2pdma_init(), as
   otherwise pci_p2pdma_add_resource() will WARN_ON
   pcim_p2pdma_provider() failing.

Niklas and Logan, I didn't re-add your R-B from v1 as (splitting
hairs...) the code's changed.

v1:
 https://lore.kernel.org/linux-pci/20260421174351.3897842-1-mattev@meta.com/


 drivers/pci/p2pdma.c | 6 +++++-
 include/linux/pci.h  | 2 +-
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c
index 7c898542af8d..adb17a4f6939 100644
--- a/drivers/pci/p2pdma.c
+++ b/drivers/pci/p2pdma.c
@@ -262,6 +262,9 @@ int pcim_p2pdma_init(struct pci_dev *pdev)
 	struct pci_p2pdma *p2p;
 	int i, ret;
 
+	if (pdev->non_mappable_bars)
+		return -EOPNOTSUPP;
+
 	p2p = rcu_dereference_protected(pdev->p2pdma, 1);
 	if (p2p)
 		return 0;
@@ -318,7 +321,8 @@ struct p2pdma_provider *pcim_p2pdma_provider(struct pci_dev *pdev, int bar)
 {
 	struct pci_p2pdma *p2p;
 
-	if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM))
+	if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM) ||
+	    pdev->non_mappable_bars)
 		return NULL;
 
 	p2p = rcu_dereference_protected(pdev->p2pdma, 1);
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 2c4454583c11..1e6802017d6b 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -508,7 +508,7 @@ struct pci_dev {
 	unsigned int	no_command_memory:1;	/* No PCI_COMMAND_MEMORY */
 	unsigned int	rom_bar_overlap:1;	/* ROM BAR disable broken */
 	unsigned int	rom_attr_enabled:1;	/* Display of ROM attribute enabled? */
-	unsigned int	non_mappable_bars:1;	/* BARs can't be mapped to user-space  */
+	unsigned int	non_mappable_bars:1;	/* BARs can't be mapped by CPU or peers */
 	pci_dev_flags_t dev_flags;
 	atomic_t	enable_cnt;	/* pci_enable_device has been called */
 
-- 
2.47.3

Re: [PATCH v2] PCI/P2PDMA: Avoid returning a provider for non_mappable_bars

Posted by Bjorn Helgaas 1 month, 3 weeks ago

On Thu, Apr 23, 2026 at 10:30:51AM -0700, Matt Evans wrote:
> Extend the checks in pcim_p2pdma_init() and pcim_p2pdma_provider() to
> exclude functions that have pdev->non_mappable_bars set.
> 
> Consumers such as VFIO were previously able to map these for access by
> the CPU or P2P.  Update the comment on non_mappable_bars to show it
> refers to any access, not just userspace CPU access.
> 
> Fixes: 372d6d1b8ae3c ("PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation")
> Signed-off-by: Matt Evans <mattev@meta.com>

Applied to pci/p2pdma for v7.2, thanks!  Will be rebased after
v7.1-rc1.

> ---
> 
> This arises from Alex Williamson's suggestion to test
> non_mappable_bars when getting the provider, with discussion here:
> 
>  https://lore.kernel.org/kvm/20260415181623.1021090-1-mattev@meta.com/
> 
> The goal was to prevent a hole where VFIO could export DMABUFs for
> BARs marked non-mappable, and to fix for all users of the provider
> rather than just VFIO.  Alex observed that non_mappable_bars should be
> taken to mean BARs weren't usable by the CPU _or_ peers and,
> considering that, its comment about userspace access wasn't quite
> right.
> 
> == Changes ==
> 
> v2:
>  - Also test non_mappable_bars in pcim_p2pdma_init(), as
>    otherwise pci_p2pdma_add_resource() will WARN_ON
>    pcim_p2pdma_provider() failing.
> 
> Niklas and Logan, I didn't re-add your R-B from v1 as (splitting
> hairs...) the code's changed.
> 
> v1:
>  https://lore.kernel.org/linux-pci/20260421174351.3897842-1-mattev@meta.com/
> 
> 
>  drivers/pci/p2pdma.c | 6 +++++-
>  include/linux/pci.h  | 2 +-
>  2 files changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c
> index 7c898542af8d..adb17a4f6939 100644
> --- a/drivers/pci/p2pdma.c
> +++ b/drivers/pci/p2pdma.c
> @@ -262,6 +262,9 @@ int pcim_p2pdma_init(struct pci_dev *pdev)
>  	struct pci_p2pdma *p2p;
>  	int i, ret;
>  
> +	if (pdev->non_mappable_bars)
> +		return -EOPNOTSUPP;
> +
>  	p2p = rcu_dereference_protected(pdev->p2pdma, 1);
>  	if (p2p)
>  		return 0;
> @@ -318,7 +321,8 @@ struct p2pdma_provider *pcim_p2pdma_provider(struct pci_dev *pdev, int bar)
>  {
>  	struct pci_p2pdma *p2p;
>  
> -	if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM))
> +	if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM) ||
> +	    pdev->non_mappable_bars)
>  		return NULL;
>  
>  	p2p = rcu_dereference_protected(pdev->p2pdma, 1);
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index 2c4454583c11..1e6802017d6b 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -508,7 +508,7 @@ struct pci_dev {
>  	unsigned int	no_command_memory:1;	/* No PCI_COMMAND_MEMORY */
>  	unsigned int	rom_bar_overlap:1;	/* ROM BAR disable broken */
>  	unsigned int	rom_attr_enabled:1;	/* Display of ROM attribute enabled? */
> -	unsigned int	non_mappable_bars:1;	/* BARs can't be mapped to user-space  */
> +	unsigned int	non_mappable_bars:1;	/* BARs can't be mapped by CPU or peers */
>  	pci_dev_flags_t dev_flags;
>  	atomic_t	enable_cnt;	/* pci_enable_device has been called */
>  
> -- 
> 2.47.3
>

Re: [PATCH v2] PCI/P2PDMA: Avoid returning a provider for non_mappable_bars

Posted by Alex Williamson 1 month, 3 weeks ago

On Thu, 23 Apr 2026 10:30:51 -0700
Matt Evans <mattev@meta.com> wrote:

> Extend the checks in pcim_p2pdma_init() and pcim_p2pdma_provider() to
> exclude functions that have pdev->non_mappable_bars set.
> 
> Consumers such as VFIO were previously able to map these for access by
> the CPU or P2P.  Update the comment on non_mappable_bars to show it
> refers to any access, not just userspace CPU access.
> 
> Fixes: 372d6d1b8ae3c ("PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation")
> Signed-off-by: Matt Evans <mattev@meta.com>
> ---
> 
> This arises from Alex Williamson's suggestion to test
> non_mappable_bars when getting the provider, with discussion here:
> 
>  https://lore.kernel.org/kvm/20260415181623.1021090-1-mattev@meta.com/
> 
> The goal was to prevent a hole where VFIO could export DMABUFs for
> BARs marked non-mappable, and to fix for all users of the provider
> rather than just VFIO.  Alex observed that non_mappable_bars should be
> taken to mean BARs weren't usable by the CPU _or_ peers and,
> considering that, its comment about userspace access wasn't quite
> right.
> 
> == Changes ==
> 
> v2:
>  - Also test non_mappable_bars in pcim_p2pdma_init(), as
>    otherwise pci_p2pdma_add_resource() will WARN_ON
>    pcim_p2pdma_provider() failing.
> 
> Niklas and Logan, I didn't re-add your R-B from v1 as (splitting
> hairs...) the code's changed.
> 
> v1:
>  https://lore.kernel.org/linux-pci/20260421174351.3897842-1-mattev@meta.com/
> 
> 
>  drivers/pci/p2pdma.c | 6 +++++-
>  include/linux/pci.h  | 2 +-
>  2 files changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c
> index 7c898542af8d..adb17a4f6939 100644
> --- a/drivers/pci/p2pdma.c
> +++ b/drivers/pci/p2pdma.c
> @@ -262,6 +262,9 @@ int pcim_p2pdma_init(struct pci_dev *pdev)
>  	struct pci_p2pdma *p2p;
>  	int i, ret;
>  
> +	if (pdev->non_mappable_bars)
> +		return -EOPNOTSUPP;
> +
>  	p2p = rcu_dereference_protected(pdev->p2pdma, 1);
>  	if (p2p)
>  		return 0;
> @@ -318,7 +321,8 @@ struct p2pdma_provider *pcim_p2pdma_provider(struct pci_dev *pdev, int bar)
>  {
>  	struct pci_p2pdma *p2p;
>  
> -	if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM))
> +	if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM) ||
> +	    pdev->non_mappable_bars)
>  		return NULL;
>  
>  	p2p = rcu_dereference_protected(pdev->p2pdma, 1);
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index 2c4454583c11..1e6802017d6b 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -508,7 +508,7 @@ struct pci_dev {
>  	unsigned int	no_command_memory:1;	/* No PCI_COMMAND_MEMORY */
>  	unsigned int	rom_bar_overlap:1;	/* ROM BAR disable broken */
>  	unsigned int	rom_attr_enabled:1;	/* Display of ROM attribute enabled? */
> -	unsigned int	non_mappable_bars:1;	/* BARs can't be mapped to user-space  */
> +	unsigned int	non_mappable_bars:1;	/* BARs can't be mapped by CPU or peers */
>  	pci_dev_flags_t dev_flags;
>  	atomic_t	enable_cnt;	/* pci_enable_device has been called */
>  

Reviewed-by: Alex Williamson <alex@shazbot.org>
Suggested-by: Alex Williamson <alex@shazbot.org>

Thanks,
Alex

Re: [PATCH v2] PCI/P2PDMA: Avoid returning a provider for non_mappable_bars

Posted by Niklas Schnelle 1 month, 3 weeks ago

On Thu, 2026-04-23 at 10:30 -0700, Matt Evans wrote:
> Extend the checks in pcim_p2pdma_init() and pcim_p2pdma_provider() to
> exclude functions that have pdev->non_mappable_bars set.
> 
> Consumers such as VFIO were previously able to map these for access by
> the CPU or P2P.  Update the comment on non_mappable_bars to show it
> refers to any access, not just userspace CPU access.
> 
> Fixes: 372d6d1b8ae3c ("PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation")
> Signed-off-by: Matt Evans <mattev@meta.com>
> ---
> 
> This arises from Alex Williamson's suggestion to test
> non_mappable_bars when getting the provider, with discussion here:
> 
>  https://lore.kernel.org/kvm/20260415181623.1021090-1-mattev@meta.com/
> 
> The goal was to prevent a hole where VFIO could export DMABUFs for
> BARs marked non-mappable, and to fix for all users of the provider
> rather than just VFIO.  Alex observed that non_mappable_bars should be
> taken to mean BARs weren't usable by the CPU _or_ peers and,
> considering that, its comment about userspace access wasn't quite
> right.
> 
> == Changes ==
> 
> v2:
>  - Also test non_mappable_bars in pcim_p2pdma_init(), as
>    otherwise pci_p2pdma_add_resource() will WARN_ON
>    pcim_p2pdma_provider() failing.
> 
> Niklas and Logan, I didn't re-add your R-B from v1 as (splitting
> hairs...) the code's changed.
> 

Makes sense, now feel free to re-add ;)

Reviewed-by: Niklas Schnelle <schnelle@linux.ibm.com> 

Thanks,
Niklas