[PATCH v4 01/13] gpu: nova-core: Set correct DMA mask

Alistair Popple posted 13 patches 4 months ago
There is a newer version of this series
[PATCH v4 01/13] gpu: nova-core: Set correct DMA mask
Posted by Alistair Popple 4 months ago
Set the correct DMA mask. Without this DMA will fail on some setups.

Signed-off-by: Alistair Popple <apopple@nvidia.com>

---

Changes for v4:
 - Use a const (GPU_DMA_BITS) instead of a magic number

Changes for v2:
 - Update DMA mask to correct value for Ampere/Turing (47 bits)
---
 drivers/gpu/nova-core/driver.rs | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/gpu/nova-core/driver.rs b/drivers/gpu/nova-core/driver.rs
index edc72052e27a..84fe4a45eb6a 100644
--- a/drivers/gpu/nova-core/driver.rs
+++ b/drivers/gpu/nova-core/driver.rs
@@ -3,6 +3,8 @@
 use kernel::{
     auxiliary, c_str,
     device::Core,
+    dma::Device,
+    dma::DmaMask,
     pci,
     pci::{Class, ClassMask, Vendor},
     prelude::*,
@@ -20,6 +22,10 @@ pub(crate) struct NovaCore {
 }
 
 const BAR0_SIZE: usize = SZ_16M;
+
+// For now we only support Ampere which can use up to 47-bit DMA addresses.
+const GPU_DMA_BITS: u32 = 47;
+
 pub(crate) type Bar0 = pci::Bar<BAR0_SIZE>;
 
 kernel::pci_device_table!(
@@ -57,6 +63,9 @@ fn probe(pdev: &pci::Device<Core>, _info: &Self::IdInfo) -> Result<Pin<KBox<Self
         pdev.enable_device_mem()?;
         pdev.set_master();
 
+        // SAFETY: No DMA allocations have been made yet
+        unsafe { pdev.dma_set_mask_and_coherent(DmaMask::new::<GPU_DMA_BITS>())? };
+
         let devres_bar = Arc::pin_init(
             pdev.iomap_region_sized::<BAR0_SIZE>(0, c_str!("nova-core/bar0")),
             GFP_KERNEL,
-- 
2.50.1
Re: [PATCH v4 01/13] gpu: nova-core: Set correct DMA mask
Posted by Danilo Krummrich 4 months ago
On Wed Oct 8, 2025 at 2:12 AM CEST, Alistair Popple wrote:
> Set the correct DMA mask. Without this DMA will fail on some setups.
>
> Signed-off-by: Alistair Popple <apopple@nvidia.com>
>
> ---
>
> Changes for v4:
>  - Use a const (GPU_DMA_BITS) instead of a magic number
>
> Changes for v2:
>  - Update DMA mask to correct value for Ampere/Turing (47 bits)
> ---
>  drivers/gpu/nova-core/driver.rs | 9 +++++++++
>  1 file changed, 9 insertions(+)
>
> diff --git a/drivers/gpu/nova-core/driver.rs b/drivers/gpu/nova-core/driver.rs
> index edc72052e27a..84fe4a45eb6a 100644
> --- a/drivers/gpu/nova-core/driver.rs
> +++ b/drivers/gpu/nova-core/driver.rs
> @@ -3,6 +3,8 @@
>  use kernel::{
>      auxiliary, c_str,
>      device::Core,
> +    dma::Device,
> +    dma::DmaMask,
>      pci,
>      pci::{Class, ClassMask, Vendor},
>      prelude::*,
> @@ -20,6 +22,10 @@ pub(crate) struct NovaCore {
>  }
>  
>  const BAR0_SIZE: usize = SZ_16M;
> +
> +// For now we only support Ampere which can use up to 47-bit DMA addresses.
> +const GPU_DMA_BITS: u32 = 47;

IIRC, the idea was to abstract this properly with a subsequent patch worked on
by John. In that case, please add a TODO.

>  pub(crate) type Bar0 = pci::Bar<BAR0_SIZE>;
>  
>  kernel::pci_device_table!(
> @@ -57,6 +63,9 @@ fn probe(pdev: &pci::Device<Core>, _info: &Self::IdInfo) -> Result<Pin<KBox<Self
>          pdev.enable_device_mem()?;
>          pdev.set_master();
>  
> +        // SAFETY: No DMA allocations have been made yet

I think you forgot to address my comment from v2:

	It's not really about DMA allocations that have been made previously, there is
	no unsafe behavior in that.
	
	It's about the method must not be called concurrently with any DMA allocation or
	mapping primitives.
	
	Can you please adjust the comment correspondingly?

In general, I recommend having a look at the safety requirements of the
corresponding function.

NIT: Please end with a period.

> +        unsafe { pdev.dma_set_mask_and_coherent(DmaMask::new::<GPU_DMA_BITS>())? };
> +
>          let devres_bar = Arc::pin_init(
>              pdev.iomap_region_sized::<BAR0_SIZE>(0, c_str!("nova-core/bar0")),
>              GFP_KERNEL,
> -- 
> 2.50.1
Re: [PATCH v4 01/13] gpu: nova-core: Set correct DMA mask
Posted by Alistair Popple 4 months ago
On 2025-10-08 at 21:30 +1100, Danilo Krummrich <dakr@kernel.org> wrote...
> On Wed Oct 8, 2025 at 2:12 AM CEST, Alistair Popple wrote:
> > Set the correct DMA mask. Without this DMA will fail on some setups.
> >
> > Signed-off-by: Alistair Popple <apopple@nvidia.com>
> >
> > ---
> >
> > Changes for v4:
> >  - Use a const (GPU_DMA_BITS) instead of a magic number
> >
> > Changes for v2:
> >  - Update DMA mask to correct value for Ampere/Turing (47 bits)
> > ---
> >  drivers/gpu/nova-core/driver.rs | 9 +++++++++
> >  1 file changed, 9 insertions(+)
> >
> > diff --git a/drivers/gpu/nova-core/driver.rs b/drivers/gpu/nova-core/driver.rs
> > index edc72052e27a..84fe4a45eb6a 100644
> > --- a/drivers/gpu/nova-core/driver.rs
> > +++ b/drivers/gpu/nova-core/driver.rs
> > @@ -3,6 +3,8 @@
> >  use kernel::{
> >      auxiliary, c_str,
> >      device::Core,
> > +    dma::Device,
> > +    dma::DmaMask,
> >      pci,
> >      pci::{Class, ClassMask, Vendor},
> >      prelude::*,
> > @@ -20,6 +22,10 @@ pub(crate) struct NovaCore {
> >  }
> >  
> >  const BAR0_SIZE: usize = SZ_16M;
> > +
> > +// For now we only support Ampere which can use up to 47-bit DMA addresses.
> > +const GPU_DMA_BITS: u32 = 47;
> 
> IIRC, the idea was to abstract this properly with a subsequent patch worked on
> by John. In that case, please add a TODO.

Yep. I have added the following for v5:

// TODO: Add an abstraction for this to support newer GPUs which may support
// larger DMA addresses. Restricting these to smaller address widths wont have any
// adverse impacts for now.

> >  pub(crate) type Bar0 = pci::Bar<BAR0_SIZE>;
> >  
> >  kernel::pci_device_table!(
> > @@ -57,6 +63,9 @@ fn probe(pdev: &pci::Device<Core>, _info: &Self::IdInfo) -> Result<Pin<KBox<Self
> >          pdev.enable_device_mem()?;
> >          pdev.set_master();
> >  
> > +        // SAFETY: No DMA allocations have been made yet
> 
> I think you forgot to address my comment from v2:

Indeed I did sorry.
 
> 	It's not really about DMA allocations that have been made previously, there is
> 	no unsafe behavior in that.
> 	
> 	It's about the method must not be called concurrently with any DMA allocation or
> 	mapping primitives.
> 	
> 	Can you please adjust the comment correspondingly?

Have changed it to this for v5:

        // SAFETY: No concurrent DMA allocations or mappings can be made because
        // the device is still being probed and therefore isn't being used by
        // other threads of execution.

> In general, I recommend having a look at the safety requirements of the
> corresponding function.

Ack, that is my general approach. Whether I adaquately explain how they are met
is a different question :)

> NIT: Please end with a period.
> 
> > +        unsafe { pdev.dma_set_mask_and_coherent(DmaMask::new::<GPU_DMA_BITS>())? };
> > +
> >          let devres_bar = Arc::pin_init(
> >              pdev.iomap_region_sized::<BAR0_SIZE>(0, c_str!("nova-core/bar0")),
> >              GFP_KERNEL,
> > -- 
> > 2.50.1
>