Set the correct DMA mask. Without this DMA will fail on some setups.
Signed-off-by: Alistair Popple <apopple@nvidia.com>
---
Changes for v2:
- Update DMA mask to correct value for Ampere/Turing (47 bits)
---
drivers/gpu/nova-core/driver.rs | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/nova-core/driver.rs b/drivers/gpu/nova-core/driver.rs
index 1380b47617f7..ccc97340206e 100644
--- a/drivers/gpu/nova-core/driver.rs
+++ b/drivers/gpu/nova-core/driver.rs
@@ -1,6 +1,9 @@
// SPDX-License-Identifier: GPL-2.0
-use kernel::{auxiliary, bindings, c_str, device::Core, pci, prelude::*, sizes::SZ_16M, sync::Arc};
+use kernel::{
+ auxiliary, bindings, c_str, device::Core, dma::Device, dma::DmaMask, pci, prelude::*,
+ sizes::SZ_16M, sync::Arc,
+};
use crate::gpu::Gpu;
@@ -34,6 +37,9 @@ fn probe(pdev: &pci::Device<Core>, _info: &Self::IdInfo) -> Result<Pin<KBox<Self
pdev.enable_device_mem()?;
pdev.set_master();
+ // SAFETY: No DMA allocations have been made yet
+ unsafe { pdev.dma_set_mask_and_coherent(DmaMask::new::<47>())? };
+
let devres_bar = Arc::pin_init(
pdev.iomap_region_sized::<BAR0_SIZE>(0, c_str!("nova-core/bar0")),
GFP_KERNEL,
--
2.50.1On 9/22/25 1:30 PM, Alistair Popple wrote:
> + // SAFETY: No DMA allocations have been made yet
It's not really about DMA allocations that have been made previously, there is
no unsafe behavior in that.
It's about the method must not be called concurrently with any DMA allocation or
mapping primitives.
Can you please adjust the comment correspondingly?
> + unsafe { pdev.dma_set_mask_and_coherent(DmaMask::new::<47>())? };
As Boqun mentioned, we shouldn't have a magic number for this. I don't know if
it will change for future chips, but maybe we should move this to gpu::Spec to
be safe.
At least, create a constant for it (also in gpu::Spec?); in Nouveau I named this
NOUVEAU_VA_SPACE_BITS back then. Not a great name, if you have a better idea,
please go for it. :)
On 9/22/25 9:08 AM, Danilo Krummrich wrote:
> On 9/22/25 1:30 PM, Alistair Popple wrote:
>> + // SAFETY: No DMA allocations have been made yet
>
> It's not really about DMA allocations that have been made previously, there is
> no unsafe behavior in that.
>
> It's about the method must not be called concurrently with any DMA allocation or
> mapping primitives.
>
> Can you please adjust the comment correspondingly?
>
>> + unsafe { pdev.dma_set_mask_and_coherent(DmaMask::new::<47>())? };
>
> As Boqun mentioned, we shouldn't have a magic number for this. I don't know if
> it will change for future chips, but maybe we should move this to gpu::Spec to
It changes to 52 bits for GH100+ (Hopper/Blackwell+). When I post those
patches, I'll use a HAL to select the value.
> be safe.
>
> At least, create a constant for it (also in gpu::Spec?); in Nouveau I named this
> NOUVEAU_VA_SPACE_BITS back then. Not a great name, if you have a better idea,
> please go for it. :)
GPU_DMA_BIT_WIDTH, for now?
thanks,
--
John Hubbard
On 2025-09-23 at 12:16 +1000, John Hubbard <jhubbard@nvidia.com> wrote...
> On 9/22/25 9:08 AM, Danilo Krummrich wrote:
> > On 9/22/25 1:30 PM, Alistair Popple wrote:
> >> + // SAFETY: No DMA allocations have been made yet
> >
> > It's not really about DMA allocations that have been made previously, there is
> > no unsafe behavior in that.
> >
> > It's about the method must not be called concurrently with any DMA allocation or
> > mapping primitives.
> >
> > Can you please adjust the comment correspondingly?
Sure.
> >> + unsafe { pdev.dma_set_mask_and_coherent(DmaMask::new::<47>())? };
> >
> > As Boqun mentioned, we shouldn't have a magic number for this. I don't know if
> > it will change for future chips, but maybe we should move this to gpu::Spec to
>
> It changes to 52 bits for GH100+ (Hopper/Blackwell+). When I post those
> patches, I'll use a HAL to select the value.
>
> > be safe.
> >
> > At least, create a constant for it (also in gpu::Spec?); in Nouveau I named this
> > NOUVEAU_VA_SPACE_BITS back then. Not a great name, if you have a better idea,
> > please go for it. :)
Well it's certainly not the VA_SPACE width ... that's a different address space :-)
I thought from the context that the magic number was pretty obviously the
supported DMA address width in bits, especially given the extra decoration
of the DmaMask type. Certainly that's been the accepted practice for the rest
of the kernel where pretty much all drivers just use something of the form
dma_set_mask(drm_dev->dev, DMA_BIT_MASK(44)) or whatever DMA address widths
they support.
> GPU_DMA_BIT_WIDTH, for now?
Works for me.
> thanks,
> --
> John Hubbard
>
On Tue Sep 23, 2025 at 6:29 AM CEST, Alistair Popple wrote:
> On 2025-09-23 at 12:16 +1000, John Hubbard <jhubbard@nvidia.com> wrote...
>> On 9/22/25 9:08 AM, Danilo Krummrich wrote:
>> > On 9/22/25 1:30 PM, Alistair Popple wrote:
>> >> + // SAFETY: No DMA allocations have been made yet
>> >
>> > It's not really about DMA allocations that have been made previously, there is
>> > no unsafe behavior in that.
>> >
>> > It's about the method must not be called concurrently with any DMA allocation or
>> > mapping primitives.
>> >
>> > Can you please adjust the comment correspondingly?
>
> Sure.
>
>> >> + unsafe { pdev.dma_set_mask_and_coherent(DmaMask::new::<47>())? };
>> >
>> > As Boqun mentioned, we shouldn't have a magic number for this. I don't know if
>> > it will change for future chips, but maybe we should move this to gpu::Spec to
>>
>> It changes to 52 bits for GH100+ (Hopper/Blackwell+). When I post those
>> patches, I'll use a HAL to select the value.
>>
>> > be safe.
>> >
>> > At least, create a constant for it (also in gpu::Spec?); in Nouveau I named this
>> > NOUVEAU_VA_SPACE_BITS back then. Not a great name, if you have a better idea,
>> > please go for it. :)
>
> Well it's certainly not the VA_SPACE width ... that's a different address space :-)
I mean, sure. But isn't the limitation of 47 bits coming from the MMU and hence
defines the VA space width and the DMA bit width we can support?
> I thought from the context that the magic number was pretty obviously the
> supported DMA address width in bits, especially given the extra decoration
> of the DmaMask type. Certainly that's been the accepted practice for the rest
> of the kernel where pretty much all drivers just use something of the form
> dma_set_mask(drm_dev->dev, DMA_BIT_MASK(44)) or whatever DMA address widths
> they support.
>
>> GPU_DMA_BIT_WIDTH, for now?
>
> Works for me.
>
>> thanks,
>> --
>> John Hubbard
>>
On 2025-09-26 at 22:00 +1000, Danilo Krummrich <dakr@kernel.org> wrote...
> On Tue Sep 23, 2025 at 6:29 AM CEST, Alistair Popple wrote:
> > On 2025-09-23 at 12:16 +1000, John Hubbard <jhubbard@nvidia.com> wrote...
> >> On 9/22/25 9:08 AM, Danilo Krummrich wrote:
> >> > On 9/22/25 1:30 PM, Alistair Popple wrote:
> >> >> + // SAFETY: No DMA allocations have been made yet
> >> >
> >> > It's not really about DMA allocations that have been made previously, there is
> >> > no unsafe behavior in that.
> >> >
> >> > It's about the method must not be called concurrently with any DMA allocation or
> >> > mapping primitives.
> >> >
> >> > Can you please adjust the comment correspondingly?
> >
> > Sure.
> >
> >> >> + unsafe { pdev.dma_set_mask_and_coherent(DmaMask::new::<47>())? };
> >> >
> >> > As Boqun mentioned, we shouldn't have a magic number for this. I don't know if
> >> > it will change for future chips, but maybe we should move this to gpu::Spec to
> >>
> >> It changes to 52 bits for GH100+ (Hopper/Blackwell+). When I post those
> >> patches, I'll use a HAL to select the value.
> >>
> >> > be safe.
> >> >
> >> > At least, create a constant for it (also in gpu::Spec?); in Nouveau I named this
> >> > NOUVEAU_VA_SPACE_BITS back then. Not a great name, if you have a better idea,
> >> > please go for it. :)
> >
> > Well it's certainly not the VA_SPACE width ... that's a different address space :-)
>
> I mean, sure. But isn't the limitation of 47 bits coming from the MMU and hence
> defines the VA space width and the DMA bit width we can support?
Not at all. The 47 bit limitation comes from what the DMA engines can physically
address, whilst the MMU converts virtual addresses to physical DMA addresses.
So the two address spaces are different and can have different widths. Indeed
most of our current GPUs have a virtual address space of 49 bits whilst only
supporting 47 bits of DMA address space.
> > I thought from the context that the magic number was pretty obviously the
> > supported DMA address width in bits, especially given the extra decoration
> > of the DmaMask type. Certainly that's been the accepted practice for the rest
> > of the kernel where pretty much all drivers just use something of the form
> > dma_set_mask(drm_dev->dev, DMA_BIT_MASK(44)) or whatever DMA address widths
> > they support.
> >
> >> GPU_DMA_BIT_WIDTH, for now?
> >
> > Works for me.
> >
> >> thanks,
> >> --
> >> John Hubbard
> >>
>
On Mon Sep 29, 2025 at 2:19 AM CEST, Alistair Popple wrote:
> On 2025-09-26 at 22:00 +1000, Danilo Krummrich <dakr@kernel.org> wrote...
>> On Tue Sep 23, 2025 at 6:29 AM CEST, Alistair Popple wrote:
>> > On 2025-09-23 at 12:16 +1000, John Hubbard <jhubbard@nvidia.com> wrote...
>> >> On 9/22/25 9:08 AM, Danilo Krummrich wrote:
>> >> > On 9/22/25 1:30 PM, Alistair Popple wrote:
>> >> >> + // SAFETY: No DMA allocations have been made yet
>> >> >
>> >> > It's not really about DMA allocations that have been made previously, there is
>> >> > no unsafe behavior in that.
>> >> >
>> >> > It's about the method must not be called concurrently with any DMA allocation or
>> >> > mapping primitives.
>> >> >
>> >> > Can you please adjust the comment correspondingly?
>> >
>> > Sure.
>> >
>> >> >> + unsafe { pdev.dma_set_mask_and_coherent(DmaMask::new::<47>())? };
>> >> >
>> >> > As Boqun mentioned, we shouldn't have a magic number for this. I don't know if
>> >> > it will change for future chips, but maybe we should move this to gpu::Spec to
>> >>
>> >> It changes to 52 bits for GH100+ (Hopper/Blackwell+). When I post those
>> >> patches, I'll use a HAL to select the value.
>> >>
>> >> > be safe.
>> >> >
>> >> > At least, create a constant for it (also in gpu::Spec?); in Nouveau I named this
>> >> > NOUVEAU_VA_SPACE_BITS back then. Not a great name, if you have a better idea,
>> >> > please go for it. :)
>> >
>> > Well it's certainly not the VA_SPACE width ... that's a different address space :-)
>>
>> I mean, sure. But isn't the limitation of 47 bits coming from the MMU and hence
>> defines the VA space width and the DMA bit width we can support?
>
> Not at all. The 47 bit limitation comes from what the DMA engines can physically
> address, whilst the MMU converts virtual addresses to physical DMA addresses.
I'm well aware -- what I'm saying is that the number given to
dma_set_mask_and_coherent() does not necessarily only depend on the physical bus
and DMA controller capabilities.
It may also depend on the MMU, since we still need to be able to map DMA memory
in the GPU's virtual address space.
> So the two address spaces are different and can have different widths. Indeed
> most of our current GPUs have a virtual address space of 49 bits whilst only
> supporting 47 bits of DMA address space.
Now, it seems that in this case the DMA engine is the actual limiting factor,
but is this the case for all architectures or may we have cases where the MMU
(or something else) becomes the limiting factor, e.g. in future architectures?
On 2025-09-29 at 17:06 +1000, Danilo Krummrich <dakr@kernel.org> wrote...
> On Mon Sep 29, 2025 at 2:19 AM CEST, Alistair Popple wrote:
> > On 2025-09-26 at 22:00 +1000, Danilo Krummrich <dakr@kernel.org> wrote...
> >> On Tue Sep 23, 2025 at 6:29 AM CEST, Alistair Popple wrote:
> >> > On 2025-09-23 at 12:16 +1000, John Hubbard <jhubbard@nvidia.com> wrote...
> >> >> On 9/22/25 9:08 AM, Danilo Krummrich wrote:
> >> >> > On 9/22/25 1:30 PM, Alistair Popple wrote:
> >> >> >> + // SAFETY: No DMA allocations have been made yet
> >> >> >
> >> >> > It's not really about DMA allocations that have been made previously, there is
> >> >> > no unsafe behavior in that.
> >> >> >
> >> >> > It's about the method must not be called concurrently with any DMA allocation or
> >> >> > mapping primitives.
> >> >> >
> >> >> > Can you please adjust the comment correspondingly?
> >> >
> >> > Sure.
> >> >
> >> >> >> + unsafe { pdev.dma_set_mask_and_coherent(DmaMask::new::<47>())? };
> >> >> >
> >> >> > As Boqun mentioned, we shouldn't have a magic number for this. I don't know if
> >> >> > it will change for future chips, but maybe we should move this to gpu::Spec to
> >> >>
> >> >> It changes to 52 bits for GH100+ (Hopper/Blackwell+). When I post those
> >> >> patches, I'll use a HAL to select the value.
> >> >>
> >> >> > be safe.
> >> >> >
> >> >> > At least, create a constant for it (also in gpu::Spec?); in Nouveau I named this
> >> >> > NOUVEAU_VA_SPACE_BITS back then. Not a great name, if you have a better idea,
> >> >> > please go for it. :)
> >> >
> >> > Well it's certainly not the VA_SPACE width ... that's a different address space :-)
> >>
> >> I mean, sure. But isn't the limitation of 47 bits coming from the MMU and hence
> >> defines the VA space width and the DMA bit width we can support?
> >
> > Not at all. The 47 bit limitation comes from what the DMA engines can physically
> > address, whilst the MMU converts virtual addresses to physical DMA addresses.
>
> I'm well aware -- what I'm saying is that the number given to
> dma_set_mask_and_coherent() does not necessarily only depend on the physical bus
> and DMA controller capabilities.
>
> It may also depend on the MMU, since we still need to be able to map DMA memory
> in the GPU's virtual address space.
Sure, I'm probably being a bit loose with terminology - I'm not saying it
doesn't depend on the MMU capabilities just that the physical addressing limits
are independent of the virtual addressing limits so setting the DMA mask based
on VA_SPACE_BITS (ie. virtual addressing limits) seems incorrect.
> > So the two address spaces are different and can have different widths. Indeed
> > most of our current GPUs have a virtual address space of 49 bits whilst only
> > supporting 47 bits of DMA address space.
>
> Now, it seems that in this case the DMA engine is the actual limiting factor,
> but is this the case for all architectures or may we have cases where the MMU
> (or something else) becomes the limiting factor, e.g. in future architectures?
Hmm. I'm not sure I follow - the virtual addressing capabilities of the GPU MMU
are entirely indepedent of the DMA addressing capabilities of the GPU and bus.
For example you can still use 49-bit GPU virtual addresses with 47-bits of DMA
bits or less and vice-versa.
So the DMA address mask has nothing to do with the virtual address (ie.
VA_SPACE) width AFAICT? But maybe we've got slightly different terminology?
On Mon Sep 29, 2025 at 9:39 AM CEST, Alistair Popple wrote:
> On 2025-09-29 at 17:06 +1000, Danilo Krummrich <dakr@kernel.org> wrote...
>> On Mon Sep 29, 2025 at 2:19 AM CEST, Alistair Popple wrote:
>> > On 2025-09-26 at 22:00 +1000, Danilo Krummrich <dakr@kernel.org> wrote...
>> >> On Tue Sep 23, 2025 at 6:29 AM CEST, Alistair Popple wrote:
>> >> > On 2025-09-23 at 12:16 +1000, John Hubbard <jhubbard@nvidia.com> wrote...
>> >> >> On 9/22/25 9:08 AM, Danilo Krummrich wrote:
>> >> >> > On 9/22/25 1:30 PM, Alistair Popple wrote:
>> >> >> >> + // SAFETY: No DMA allocations have been made yet
>> >> >> >
>> >> >> > It's not really about DMA allocations that have been made previously, there is
>> >> >> > no unsafe behavior in that.
>> >> >> >
>> >> >> > It's about the method must not be called concurrently with any DMA allocation or
>> >> >> > mapping primitives.
>> >> >> >
>> >> >> > Can you please adjust the comment correspondingly?
>> >> >
>> >> > Sure.
>> >> >
>> >> >> >> + unsafe { pdev.dma_set_mask_and_coherent(DmaMask::new::<47>())? };
>> >> >> >
>> >> >> > As Boqun mentioned, we shouldn't have a magic number for this. I don't know if
>> >> >> > it will change for future chips, but maybe we should move this to gpu::Spec to
>> >> >>
>> >> >> It changes to 52 bits for GH100+ (Hopper/Blackwell+). When I post those
>> >> >> patches, I'll use a HAL to select the value.
>> >> >>
>> >> >> > be safe.
>> >> >> >
>> >> >> > At least, create a constant for it (also in gpu::Spec?); in Nouveau I named this
>> >> >> > NOUVEAU_VA_SPACE_BITS back then. Not a great name, if you have a better idea,
>> >> >> > please go for it. :)
>> >> >
>> >> > Well it's certainly not the VA_SPACE width ... that's a different address space :-)
>> >>
>> >> I mean, sure. But isn't the limitation of 47 bits coming from the MMU and hence
>> >> defines the VA space width and the DMA bit width we can support?
>> >
>> > Not at all. The 47 bit limitation comes from what the DMA engines can physically
>> > address, whilst the MMU converts virtual addresses to physical DMA addresses.
>>
>> I'm well aware -- what I'm saying is that the number given to
>> dma_set_mask_and_coherent() does not necessarily only depend on the physical bus
>> and DMA controller capabilities.
>>
>> It may also depend on the MMU, since we still need to be able to map DMA memory
>> in the GPU's virtual address space.
>
> Sure, I'm probably being a bit loose with terminology - I'm not saying it
> doesn't depend on the MMU capabilities just that the physical addressing limits
> are independent of the virtual addressing limits so setting the DMA mask based
> on VA_SPACE_BITS (ie. virtual addressing limits) seems incorrect.
I think no one said that physical addressing limits depend on virtual addressing
limits.
What I'm saying is that the DMA mask may depend on more than the physical
addressing limits or the DMA controller limits -- that's a different statement.
>> > So the two address spaces are different and can have different widths. Indeed
>> > most of our current GPUs have a virtual address space of 49 bits whilst only
>> > supporting 47 bits of DMA address space.
>>
>> Now, it seems that in this case the DMA engine is the actual limiting factor,
>> but is this the case for all architectures or may we have cases where the MMU
>> (or something else) becomes the limiting factor, e.g. in future architectures?
>
> Hmm. I'm not sure I follow - the virtual addressing capabilities of the GPU MMU
> are entirely indepedent of the DMA addressing capabilities of the GPU and bus.
> For example you can still use 49-bit GPU virtual addresses with 47-bits of DMA
> bits or less and vice-versa.
>
> So the DMA address mask has nothing to do with the virtual address (ie.
> VA_SPACE) width AFAICT? But maybe we've got slightly different terminology?
Again, no one said it has anything to do with virtual address space width, but
it has something to do with the physical addresses the MMU can handle.
Otherwise, let me answer with a question: What do we set the DMA mask to if the
DMA controller supports wider addresses than the MMU does? We still want to be
able to map DMA buffers in the GPU's virtual address space, no?
In other words, the value for the DMA mask may depend on multiple device
capabilities, i.e. physical bus, DMA controller, MMU, etc.
Hence, the DMA mask should be the minimum of all of those.
Whether we define all of them and compute the minimum, or just create a global
constant is a different question. But should at least document it cleanly.
On 2025-09-29 at 22:49 +1000, Danilo Krummrich <dakr@kernel.org> wrote...
> On Mon Sep 29, 2025 at 9:39 AM CEST, Alistair Popple wrote:
> > On 2025-09-29 at 17:06 +1000, Danilo Krummrich <dakr@kernel.org> wrote...
> >> On Mon Sep 29, 2025 at 2:19 AM CEST, Alistair Popple wrote:
> >> > On 2025-09-26 at 22:00 +1000, Danilo Krummrich <dakr@kernel.org> wrote...
> >> >> On Tue Sep 23, 2025 at 6:29 AM CEST, Alistair Popple wrote:
> >> >> > On 2025-09-23 at 12:16 +1000, John Hubbard <jhubbard@nvidia.com> wrote...
> >> >> >> On 9/22/25 9:08 AM, Danilo Krummrich wrote:
> >> >> >> > On 9/22/25 1:30 PM, Alistair Popple wrote:
> >> >> >> >> + // SAFETY: No DMA allocations have been made yet
> >> >> >> >
> >> >> >> > It's not really about DMA allocations that have been made previously, there is
> >> >> >> > no unsafe behavior in that.
> >> >> >> >
> >> >> >> > It's about the method must not be called concurrently with any DMA allocation or
> >> >> >> > mapping primitives.
> >> >> >> >
> >> >> >> > Can you please adjust the comment correspondingly?
> >> >> >
> >> >> > Sure.
> >> >> >
> >> >> >> >> + unsafe { pdev.dma_set_mask_and_coherent(DmaMask::new::<47>())? };
> >> >> >> >
> >> >> >> > As Boqun mentioned, we shouldn't have a magic number for this. I don't know if
> >> >> >> > it will change for future chips, but maybe we should move this to gpu::Spec to
> >> >> >>
> >> >> >> It changes to 52 bits for GH100+ (Hopper/Blackwell+). When I post those
> >> >> >> patches, I'll use a HAL to select the value.
> >> >> >>
> >> >> >> > be safe.
> >> >> >> >
> >> >> >> > At least, create a constant for it (also in gpu::Spec?); in Nouveau I named this
> >> >> >> > NOUVEAU_VA_SPACE_BITS back then. Not a great name, if you have a better idea,
> >> >> >> > please go for it. :)
> >> >> >
> >> >> > Well it's certainly not the VA_SPACE width ... that's a different address space :-)
> >> >>
> >> >> I mean, sure. But isn't the limitation of 47 bits coming from the MMU and hence
> >> >> defines the VA space width and the DMA bit width we can support?
> >> >
> >> > Not at all. The 47 bit limitation comes from what the DMA engines can physically
> >> > address, whilst the MMU converts virtual addresses to physical DMA addresses.
> >>
> >> I'm well aware -- what I'm saying is that the number given to
> >> dma_set_mask_and_coherent() does not necessarily only depend on the physical bus
> >> and DMA controller capabilities.
> >>
> >> It may also depend on the MMU, since we still need to be able to map DMA memory
> >> in the GPU's virtual address space.
> >
> > Sure, I'm probably being a bit loose with terminology - I'm not saying it
> > doesn't depend on the MMU capabilities just that the physical addressing limits
> > are independent of the virtual addressing limits so setting the DMA mask based
> > on VA_SPACE_BITS (ie. virtual addressing limits) seems incorrect.
>
> I think no one said that physical addressing limits depend on virtual addressing
> limits.
>
> What I'm saying is that the DMA mask may depend on more than the physical
> addressing limits or the DMA controller limits -- that's a different statement.
>
> >> > So the two address spaces are different and can have different widths. Indeed
> >> > most of our current GPUs have a virtual address space of 49 bits whilst only
> >> > supporting 47 bits of DMA address space.
> >>
> >> Now, it seems that in this case the DMA engine is the actual limiting factor,
> >> but is this the case for all architectures or may we have cases where the MMU
> >> (or something else) becomes the limiting factor, e.g. in future architectures?
> >
> > Hmm. I'm not sure I follow - the virtual addressing capabilities of the GPU MMU
> > are entirely indepedent of the DMA addressing capabilities of the GPU and bus.
> > For example you can still use 49-bit GPU virtual addresses with 47-bits of DMA
> > bits or less and vice-versa.
> >
> > So the DMA address mask has nothing to do with the virtual address (ie.
> > VA_SPACE) width AFAICT? But maybe we've got slightly different terminology?
>
> Again, no one said it has anything to do with virtual address space width, but
> it has something to do with the physical addresses the MMU can handle.
Huh? I'm confused - this started with the name for a constant and the suggestion
was this constant was called `NOUVEAU_VA_SPACE_BITS` in Nouveau. That very much
implies to me at least it has something to do with virtual address width? I was
just trying to point out (maybe poorly) that it doesn't.
> Otherwise, let me answer with a question: What do we set the DMA mask to if the
> DMA controller supports wider addresses than the MMU does? We still want to be
> able to map DMA buffers in the GPU's virtual address space, no?
Lets be explicit with terminology - which MMU are you referring to here? The GPU
MMU (GMMU) or the CPU MMU or the CPU IOMMU?
Not that it matters because the device driver needs to set the DMA mask to the
maximum width DMA address that the device HW is capable of producing, which in
this case is 47-bits. Theoretically I suppose it's possible for someone to build
a GPU which could generate DMA addresses greater than what it's own GMMU could
address after translation, but that seems pretty strange and not something I'm
aware of or expect to happen in any of our devices.
> In other words, the value for the DMA mask may depend on multiple device
> capabilities, i.e. physical bus, DMA controller, MMU, etc.
But that doesn't impact what the GPU device should set it's DMA mask
to be. If the GPU can generate 47-bits of DMA addresses it should call
dma_set_mask(DmaMask::new::<47>).
IOW it's not up to the GPU device driver to decide what other devices in the
chain are capable of, that's what the kernel DMA API is for. For example if
the physical bus the GPU is plugged into is limited to 32-bits for some reason
the DMA API will ensure dma_map_page(), etc. wont return addresses greater than
32 bits.
> Hence, the DMA mask should be the minimum of all of those.
Right, but I don't think that impacts the GPU device driver in anyi way. The
GPU supports 47-bits of DMA addresses, so we set it to that. Obviously different
models of GPUs may have different capabilities, so some kind of HAL will be
needed to look that up, but I don't see a need for any kind of computation in
the driver.
> Whether we define all of them and compute the minimum, or just create a global
> constant is a different question. But should at least document it cleanly.
On Mon, Sep 22, 2025 at 09:30:17PM +1000, Alistair Popple wrote:
> Set the correct DMA mask. Without this DMA will fail on some setups.
>
> Signed-off-by: Alistair Popple <apopple@nvidia.com>
>
> ---
>
> Changes for v2:
>
> - Update DMA mask to correct value for Ampere/Turing (47 bits)
> ---
> drivers/gpu/nova-core/driver.rs | 8 +++++++-
> 1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/nova-core/driver.rs b/drivers/gpu/nova-core/driver.rs
> index 1380b47617f7..ccc97340206e 100644
> --- a/drivers/gpu/nova-core/driver.rs
> +++ b/drivers/gpu/nova-core/driver.rs
> @@ -1,6 +1,9 @@
> // SPDX-License-Identifier: GPL-2.0
>
> -use kernel::{auxiliary, bindings, c_str, device::Core, pci, prelude::*, sizes::SZ_16M, sync::Arc};
> +use kernel::{
> + auxiliary, bindings, c_str, device::Core, dma::Device, dma::DmaMask, pci, prelude::*,
> + sizes::SZ_16M, sync::Arc,
> +};
>
> use crate::gpu::Gpu;
>
> @@ -34,6 +37,9 @@ fn probe(pdev: &pci::Device<Core>, _info: &Self::IdInfo) -> Result<Pin<KBox<Self
> pdev.enable_device_mem()?;
> pdev.set_master();
>
> + // SAFETY: No DMA allocations have been made yet
> + unsafe { pdev.dma_set_mask_and_coherent(DmaMask::new::<47>())? };
Any chance you give it this a const name? Otherwise 6 months later it
becomes a magic number ;-)
Otherwise than this,
Reviewed-by: Boqun Feng <boqun.feng@gmail.com>
Regards,
Boqun
> +
> let devres_bar = Arc::pin_init(
> pdev.iomap_region_sized::<BAR0_SIZE>(0, c_str!("nova-core/bar0")),
> GFP_KERNEL,
> --
> 2.50.1
>
© 2016 - 2025 Red Hat, Inc.