The RX buffer virtual address may reside in vmalloc space depending on
the allocation path, where virt_to_page() is invalid.

Use a helper that chooses vmalloc_to_page() or virt_to_page() as
appropriate. This is safe since the buffer is guaranteed to be
physically contiguous.

Signed-off-by: Koichiro Den <den@valinux.co.jp>
---
 drivers/ntb/ntb_transport.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/ntb/ntb_transport.c b/drivers/ntb/ntb_transport.c
index eb875e3db2e3..b9f9d2e0feb3 100644
--- a/drivers/ntb/ntb_transport.c
+++ b/drivers/ntb/ntb_transport.c
@@ -1546,6 +1546,11 @@ static void ntb_memcpy_rx(struct ntb_queue_entry *entry, void *offset)
 	ntb_rx_copy_callback(entry, NULL);
 }
 
+static inline struct page *ntb_vaddr_to_page(const void *addr)
+{
+	return is_vmalloc_addr(addr) ? vmalloc_to_page(addr) : virt_to_page(addr);
+}
+
 static int ntb_async_rx_submit(struct ntb_queue_entry *entry, void *offset)
 {
 	struct dma_async_tx_descriptor *txd;
@@ -1570,7 +1575,7 @@ static int ntb_async_rx_submit(struct ntb_queue_entry *entry, void *offset)
 		goto err;
 
 	unmap->len = len;
-	unmap->addr[0] = dma_map_page(device->dev, virt_to_page(offset),
+	unmap->addr[0] = dma_map_page(device->dev, ntb_vaddr_to_page(offset),
 				      pay_off, len, DMA_TO_DEVICE);
 	if (dma_mapping_error(device->dev, unmap->addr[0]))
 		goto err_get_unmap;
-- 
2.48.1

On 2025-10-26 18:43, Koichiro Den wrote:
> The RX buffer virtual address may reside in vmalloc space depending on
> the allocation path, where virt_to_page() is invalid.
> 
> Use a helper that chooses vmalloc_to_page() or virt_to_page() as
> appropriate. This is safe since the buffer is guaranteed to be
> physically contiguous.

I think this statement needs some explanation.

vmalloc memory is generally not contiguous and using vmalloc_to_page()
like this seems very questionable.

I did a very quick look and found that "offset" may come from
dma_alloc_attrs() which can also return coherent memory that would be in
vmalloc space and would be contiguous.

However, in my cursory look, it appears that the kernel address returned
by dma_alloc_attrs() is eventually passed to dma_map_page() in order to
obtain the dma address a second time. This is really ugly, and almost
certainly not expected by the dma layer.

This requires a bit of a change, but it seems to me that if
dma_alloc_attrs() is used, the dma address it returns should be used
directly and a second map should be avoided completely. Then we wouldn't
need the unusual use of vmalloc_to_page().

At the very least, I think these issues need to be mentioned in the
commit message.

Logan

On Mon, Oct 27, 2025 at 10:30:52AM -0600, Logan Gunthorpe wrote:
> 
> 
> On 2025-10-26 18:43, Koichiro Den wrote:
> > The RX buffer virtual address may reside in vmalloc space depending on
> > the allocation path, where virt_to_page() is invalid.
> > 
> > Use a helper that chooses vmalloc_to_page() or virt_to_page() as
> > appropriate. This is safe since the buffer is guaranteed to be
> > physically contiguous.
> 
> I think this statement needs some explanation.
> 
> vmalloc memory is generally not contiguous and using vmalloc_to_page()
> like this seems very questionable.

Yes generally it is, which is why I wrote the last sentence "... since the
buffer is guaranteed to be physically contiguous."

> 
> I did a very quick look and found that "offset" may come from
> dma_alloc_attrs() which can also return coherent memory that would be in
> vmalloc space and would be contiguous.
> 
> However, in my cursory look, it appears that the kernel address returned
> by dma_alloc_attrs() is eventually passed to dma_map_page() in order to
> obtain the dma address a second time. This is really ugly, and almost
> certainly not expected by the dma layer.
> 
> This requires a bit of a change, but it seems to me that if
> dma_alloc_attrs() is used, the dma address it returns should be used
> directly and a second map should be avoided completely. Then we wouldn't
> need the unusual use of vmalloc_to_page().

I agree there's room for improvement around this "double" mapping.
I'll think about a follow-up patch to clean this up.

> 
> At the very least, I think these issues need to be mentioned in the
> commit message.

As for the commit message, I think adding one more line like:
"See relevant commit 061a785a114f ("ntb: Force physically contiguous allocation of rx ring buffers")"
should be sufficient. What do you think?

Thanks for reviewing.

-Koichiro

> 
> Logan

On 2025-10-27 19:14, Koichiro Den wrote:
> 
> I agree there's room for improvement around this "double" mapping.
> I'll think about a follow-up patch to clean this up.
> 
>>
>> At the very least, I think these issues need to be mentioned in the
>> commit message.
> 
> As for the commit message, I think adding one more line like:
> "See relevant commit 061a785a114f ("ntb: Force physically contiguous allocation of rx ring buffers")"
> should be sufficient. What do you think?

Ah, I see why that is now. Thanks for the explanation.

Adding the link helps, but I still think the commit message needs to
mention why the memory is guaranteed to be physically contiguous.

Thanks,

Logan

On Tue, Oct 28, 2025 at 09:45:30AM -0600, Logan Gunthorpe wrote:
> 
> 
> On 2025-10-27 19:14, Koichiro Den wrote:
> > 
> > I agree there's room for improvement around this "double" mapping.
> > I'll think about a follow-up patch to clean this up.
> > 
> >>
> >> At the very least, I think these issues need to be mentioned in the
> >> commit message.
> > 
> > As for the commit message, I think adding one more line like:
> > "See relevant commit 061a785a114f ("ntb: Force physically contiguous allocation of rx ring buffers")"
> > should be sufficient. What do you think?
> 
> Ah, I see why that is now. Thanks for the explanation.
> 
> Adding the link helps, but I still think the commit message needs to
> mention why the memory is guaranteed to be physically contiguous.

That makes sense, I'll do so. Thanks for the review!

-Koichiro

> 
> Thanks,
> 
> Logan

On Tue, Oct 28, 2025 at 10:14:00AM +0900, Koichiro Den wrote:
> On Mon, Oct 27, 2025 at 10:30:52AM -0600, Logan Gunthorpe wrote:
> > 
> > 
> > On 2025-10-26 18:43, Koichiro Den wrote:
> > > The RX buffer virtual address may reside in vmalloc space depending on
> > > the allocation path, where virt_to_page() is invalid.
> > > 
> > > Use a helper that chooses vmalloc_to_page() or virt_to_page() as
> > > appropriate. This is safe since the buffer is guaranteed to be
> > > physically contiguous.
> > 
> > I think this statement needs some explanation.
> > 
> > vmalloc memory is generally not contiguous and using vmalloc_to_page()
> > like this seems very questionable.
> 
> Yes generally it is, which is why I wrote the last sentence "... since the
s/generally it is/generally it is non-contiguous/
Sorry for the confusion

-Koichiro

> buffer is guaranteed to be physically contiguous."
> 
> > 
> > I did a very quick look and found that "offset" may come from
> > dma_alloc_attrs() which can also return coherent memory that would be in
> > vmalloc space and would be contiguous.
> > 
> > However, in my cursory look, it appears that the kernel address returned
> > by dma_alloc_attrs() is eventually passed to dma_map_page() in order to
> > obtain the dma address a second time. This is really ugly, and almost
> > certainly not expected by the dma layer.
> > 
> > This requires a bit of a change, but it seems to me that if
> > dma_alloc_attrs() is used, the dma address it returns should be used
> > directly and a second map should be avoided completely. Then we wouldn't
> > need the unusual use of vmalloc_to_page().
> 
> I agree there's room for improvement around this "double" mapping.
> I'll think about a follow-up patch to clean this up.
> 
> > 
> > At the very least, I think these issues need to be mentioned in the
> > commit message.
> 
> As for the commit message, I think adding one more line like:
> "See relevant commit 061a785a114f ("ntb: Force physically contiguous allocation of rx ring buffers")"
> should be sufficient. What do you think?
> 
> Thanks for reviewing.
> 
> -Koichiro
> 
> > 
> > Logan