block/blk-mq-dma.c | 19 +++++---- drivers/nvme/host/pci.c | 90 +++++++++++++++++++++++++++++++++++-------- include/linux/bio-integrity.h | 1 - include/linux/blk-integrity.h | 14 ------- include/linux/blk-mq-dma.h | 28 +++++++------- include/linux/blk_types.h | 2 - 6 files changed, 99 insertions(+), 55 deletions(-)
Changelog:
v4:
* Changed double "if" to be "else if".
* Added missed PCI_P2PDMA_MAP_NONE case.
v3: https://patch.msgid.link/20251027-block-with-mmio-v3-0-ac3370e1f7b7@nvidia.com
* Encoded p2p map type in IOD flags instead of DMA attributes.
* Removed REQ_P2PDMA flag from block layer.
* Simplified map_phys conversion patch.
v2: https://lore.kernel.org/all/20251020-block-with-mmio-v2-0-147e9f93d8d4@nvidia.com/
* Added Chirstoph's Reviewed-by tag for first patch.
* Squashed patches
* Stored DMA MMIO attribute in NVMe IOD flags variable instead of block layer.
v1: https://patch.msgid.link/20251017-block-with-mmio-v1-0-3f486904db5e@nvidia.com
* Reordered patches.
* Dropped patch which tried to unify unmap flow.
* Set MMIO flag separately for data and integrity payloads.
v0: https://lore.kernel.org/all/cover.1760369219.git.leon@kernel.org/
----------------------------------------------------------------------
This patch series improves block layer and NVMe driver support for MMIO
memory regions, particularly for peer-to-peer (P2P) DMA transfers that
go through the host bridge.
The series addresses a critical gap where P2P transfers through the host
bridge (PCI_P2PDMA_MAP_THRU_HOST_BRIDGE) were not properly marked as
MMIO memory, leading to potential issues with:
- Inappropriate CPU cache synchronization operations on MMIO regions
- Incorrect DMA mapping/unmapping that doesn't respect MMIO semantics
- Missing IOMMU configuration for MMIO memory handling
This work is extracted from the larger DMA physical API improvement
series [1] and focuses specifically on block layer and NVMe requirements
for MMIO memory support.
Thanks
[1] https://lore.kernel.org/all/cover.1757423202.git.leonro@nvidia.com/
---
Leon Romanovsky (2):
nvme-pci: migrate to dma_map_phys instead of map_page
block-dma: properly take MMIO path
block/blk-mq-dma.c | 19 +++++----
drivers/nvme/host/pci.c | 90 +++++++++++++++++++++++++++++++++++--------
include/linux/bio-integrity.h | 1 -
include/linux/blk-integrity.h | 14 -------
include/linux/blk-mq-dma.h | 28 +++++++-------
include/linux/blk_types.h | 2 -
6 files changed, 99 insertions(+), 55 deletions(-)
---
base-commit: 3a8660878839faadb4f1a6dd72c3179c1df56787
change-id: 20251016-block-with-mmio-02acf4285427
Best regards,
--
Leon Romanovsky <leonro@nvidia.com>
On Wed, 12 Nov 2025 21:48:03 +0200, Leon Romanovsky wrote:
> Changelog:
> v4:
> * Changed double "if" to be "else if".
> * Added missed PCI_P2PDMA_MAP_NONE case.
> v3: https://patch.msgid.link/20251027-block-with-mmio-v3-0-ac3370e1f7b7@nvidia.com
> * Encoded p2p map type in IOD flags instead of DMA attributes.
> * Removed REQ_P2PDMA flag from block layer.
> * Simplified map_phys conversion patch.
> v2: https://lore.kernel.org/all/20251020-block-with-mmio-v2-0-147e9f93d8d4@nvidia.com/
> * Added Chirstoph's Reviewed-by tag for first patch.
> * Squashed patches
> * Stored DMA MMIO attribute in NVMe IOD flags variable instead of block layer.
> v1: https://patch.msgid.link/20251017-block-with-mmio-v1-0-3f486904db5e@nvidia.com
> * Reordered patches.
> * Dropped patch which tried to unify unmap flow.
> * Set MMIO flag separately for data and integrity payloads.
> v0: https://lore.kernel.org/all/cover.1760369219.git.leon@kernel.org/
>
> [...]
Applied, thanks!
[1/2] nvme-pci: migrate to dma_map_phys instead of map_page
commit: f10000db2f7cf29d8c2ade69266bed7b51c772cb
[2/2] block-dma: properly take MMIO path
commit: 8df2745e8b23fdbe34c5b0a24607f5aaf10ed7eb
Best regards,
--
Jens Axboe
On 11/13/25 9:39 AM, Jens Axboe wrote: > > On Wed, 12 Nov 2025 21:48:03 +0200, Leon Romanovsky wrote: >> Changelog: >> v4: >> * Changed double "if" to be "else if". >> * Added missed PCI_P2PDMA_MAP_NONE case. >> v3: https://patch.msgid.link/20251027-block-with-mmio-v3-0-ac3370e1f7b7@nvidia.com >> * Encoded p2p map type in IOD flags instead of DMA attributes. >> * Removed REQ_P2PDMA flag from block layer. >> * Simplified map_phys conversion patch. >> v2: https://lore.kernel.org/all/20251020-block-with-mmio-v2-0-147e9f93d8d4@nvidia.com/ >> * Added Chirstoph's Reviewed-by tag for first patch. >> * Squashed patches >> * Stored DMA MMIO attribute in NVMe IOD flags variable instead of block layer. >> v1: https://patch.msgid.link/20251017-block-with-mmio-v1-0-3f486904db5e@nvidia.com >> * Reordered patches. >> * Dropped patch which tried to unify unmap flow. >> * Set MMIO flag separately for data and integrity payloads. >> v0: https://lore.kernel.org/all/cover.1760369219.git.leon@kernel.org/ >> >> [...] > > Applied, thanks! > > [1/2] nvme-pci: migrate to dma_map_phys instead of map_page > commit: f10000db2f7cf29d8c2ade69266bed7b51c772cb > [2/2] block-dma: properly take MMIO path > commit: 8df2745e8b23fdbe34c5b0a24607f5aaf10ed7eb And now dropped again - this doesn't boot on neither my big test box with 33 nvme drives, nor even on my local test vm. Two different archs, and very different setups. Which begs the question, how on earth was this tested, if it doesn't boot on anything I have here?! -- Jens Axboe
On 11/13/25 10:12 AM, Jens Axboe wrote: > On 11/13/25 9:39 AM, Jens Axboe wrote: >> >> On Wed, 12 Nov 2025 21:48:03 +0200, Leon Romanovsky wrote: >>> Changelog: >>> v4: >>> * Changed double "if" to be "else if". >>> * Added missed PCI_P2PDMA_MAP_NONE case. >>> v3: https://patch.msgid.link/20251027-block-with-mmio-v3-0-ac3370e1f7b7@nvidia.com >>> * Encoded p2p map type in IOD flags instead of DMA attributes. >>> * Removed REQ_P2PDMA flag from block layer. >>> * Simplified map_phys conversion patch. >>> v2: https://lore.kernel.org/all/20251020-block-with-mmio-v2-0-147e9f93d8d4@nvidia.com/ >>> * Added Chirstoph's Reviewed-by tag for first patch. >>> * Squashed patches >>> * Stored DMA MMIO attribute in NVMe IOD flags variable instead of block layer. >>> v1: https://patch.msgid.link/20251017-block-with-mmio-v1-0-3f486904db5e@nvidia.com >>> * Reordered patches. >>> * Dropped patch which tried to unify unmap flow. >>> * Set MMIO flag separately for data and integrity payloads. >>> v0: https://lore.kernel.org/all/cover.1760369219.git.leon@kernel.org/ >>> >>> [...] >> >> Applied, thanks! >> >> [1/2] nvme-pci: migrate to dma_map_phys instead of map_page >> commit: f10000db2f7cf29d8c2ade69266bed7b51c772cb >> [2/2] block-dma: properly take MMIO path >> commit: 8df2745e8b23fdbe34c5b0a24607f5aaf10ed7eb > > And now dropped again - this doesn't boot on neither my big test box > with 33 nvme drives, nor even on my local test vm. Two different archs, > and very different setups. Which begs the question, how on earth was > this tested, if it doesn't boot on anything I have here?! I took a look, and what happens here is that iter.p2pdma.map is 0 as it never got set to anything. That is the same as PCI_P2PDMA_MAP_UNKNOWN, and hence we just end up in a BLK_STS_RESOURCE. First of all, returning BLK_STS_RESOURCE for that seems... highly suspicious. That should surely be a fatal error. And secondly, this just further backs up that there's ZERO testing done on this patchset at all. WTF? FWIW, the below makes it boot just fine, as expected, as a default zero filled iter then matches the UNKNOWN case. diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index e5ca8301bb8b..4cce69226773 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -1087,6 +1087,7 @@ static blk_status_t nvme_map_data(struct request *req) case PCI_P2PDMA_MAP_THRU_HOST_BRIDGE: iod->flags |= IOD_DATA_MMIO; break; + case PCI_P2PDMA_MAP_UNKNOWN: case PCI_P2PDMA_MAP_NONE: break; default: @@ -1122,6 +1123,7 @@ static blk_status_t nvme_pci_setup_meta_iter(struct request *req) case PCI_P2PDMA_MAP_THRU_HOST_BRIDGE: iod->flags |= IOD_META_MMIO; break; + case PCI_P2PDMA_MAP_UNKNOWN: case PCI_P2PDMA_MAP_NONE: break; default: -- Jens Axboe
On Thu, Nov 13, 2025 at 10:45:53AM -0700, Jens Axboe wrote: > I took a look, and what happens here is that iter.p2pdma.map is 0 as it > never got set to anything. That is the same as PCI_P2PDMA_MAP_UNKNOWN, > and hence we just end up in a BLK_STS_RESOURCE. First of all, returning > BLK_STS_RESOURCE for that seems... highly suspicious. That should surely > be a fatal error. And secondly, this just further backs up that there's > ZERO testing done on this patchset at all. WTF? > > FWIW, the below makes it boot just fine, as expected, as a default zero > filled iter then matches the UNKNOWN case. I think this must mean you don't have CONFIG_PCI_P2PDMA enabled. The state is never set in that case, but I think it should have been. --- diff --git a/include/linux/pci-p2pdma.h b/include/linux/pci-p2pdma.h index 951f81a38f3af..1dfcdafebf867 100644 --- a/include/linux/pci-p2pdma.h +++ b/include/linux/pci-p2pdma.h @@ -166,6 +166,8 @@ pci_p2pdma_state(struct pci_p2pdma_map_state *state, struct device *dev, __pci_p2pdma_update_state(state, dev, page); return state->map; } + + state->map = PCI_P2PDMA_MAP_NONE; return PCI_P2PDMA_MAP_NONE; } --
On 11/13/25 12:52 PM, Keith Busch wrote: > On Thu, Nov 13, 2025 at 10:45:53AM -0700, Jens Axboe wrote: >> I took a look, and what happens here is that iter.p2pdma.map is 0 as it >> never got set to anything. That is the same as PCI_P2PDMA_MAP_UNKNOWN, >> and hence we just end up in a BLK_STS_RESOURCE. First of all, returning >> BLK_STS_RESOURCE for that seems... highly suspicious. That should surely >> be a fatal error. And secondly, this just further backs up that there's >> ZERO testing done on this patchset at all. WTF? >> >> FWIW, the below makes it boot just fine, as expected, as a default zero >> filled iter then matches the UNKNOWN case. > > I think this must mean you don't have CONFIG_PCI_P2PDMA enabled. The Right, like most normal people :-) > state is never set in that case, but I think it should have been. If you want the patchset to boot, yes it should have been... -- Jens Axboe
On Thu, Nov 13, 2025 at 01:40:50PM -0700, Jens Axboe wrote: > On 11/13/25 12:52 PM, Keith Busch wrote: > > On Thu, Nov 13, 2025 at 10:45:53AM -0700, Jens Axboe wrote: > >> I took a look, and what happens here is that iter.p2pdma.map is 0 as it > >> never got set to anything. That is the same as PCI_P2PDMA_MAP_UNKNOWN, > >> and hence we just end up in a BLK_STS_RESOURCE. First of all, returning > >> BLK_STS_RESOURCE for that seems... highly suspicious. That should surely > >> be a fatal error. And secondly, this just further backs up that there's > >> ZERO testing done on this patchset at all. WTF? > >> > >> FWIW, the below makes it boot just fine, as expected, as a default zero > >> filled iter then matches the UNKNOWN case. > > > > I think this must mean you don't have CONFIG_PCI_P2PDMA enabled. The > > Right, like most normal people :-) It depends how you are declaring normal people :). In my Fedora OS, installed on my laptop, CONFIG_PCI_P2PDMA is enabled by default. https://src.fedoraproject.org/rpms/kernel/blob/rawhide/f/kernel-x86_64-fedora.config#_5567 and in RHEL too https://src.fedoraproject.org/rpms/kernel/blob/rawhide/f/kernel-x86_64-rhel.config#_4964 Thanks
On 11/14/25 1:16 AM, Leon Romanovsky wrote: > On Thu, Nov 13, 2025 at 01:40:50PM -0700, Jens Axboe wrote: >> On 11/13/25 12:52 PM, Keith Busch wrote: >>> On Thu, Nov 13, 2025 at 10:45:53AM -0700, Jens Axboe wrote: >>>> I took a look, and what happens here is that iter.p2pdma.map is 0 as it >>>> never got set to anything. That is the same as PCI_P2PDMA_MAP_UNKNOWN, >>>> and hence we just end up in a BLK_STS_RESOURCE. First of all, returning >>>> BLK_STS_RESOURCE for that seems... highly suspicious. That should surely >>>> be a fatal error. And secondly, this just further backs up that there's >>>> ZERO testing done on this patchset at all. WTF? >>>> >>>> FWIW, the below makes it boot just fine, as expected, as a default zero >>>> filled iter then matches the UNKNOWN case. >>> >>> I think this must mean you don't have CONFIG_PCI_P2PDMA enabled. The >> >> Right, like most normal people :-) > > It depends how you are declaring normal people :). > In my Fedora OS, installed on my laptop, CONFIG_PCI_P2PDMA is enabled by default. > https://src.fedoraproject.org/rpms/kernel/blob/rawhide/f/kernel-x86_64-fedora.config#_5567 > and in RHEL too > https://src.fedoraproject.org/rpms/kernel/blob/rawhide/f/kernel-x86_64-rhel.config#_4964 Distros tend to enable everything under the sun, that's hardly news or surprising to anyone. And also why the usual arguments of "oh but it only bloats foo or slows down bar when enabled" are utterly bogus. But I sure as hell don't run a distro config on my vm or my test boxes, or anything else I use for that matter, if it's not running a distro kernel. -- Jens Axboe
On Thu, Nov 13, 2025 at 10:45:53AM -0700, Jens Axboe wrote:
> On 11/13/25 10:12 AM, Jens Axboe wrote:
> > On 11/13/25 9:39 AM, Jens Axboe wrote:
> >>
> >> On Wed, 12 Nov 2025 21:48:03 +0200, Leon Romanovsky wrote:
> >>> Changelog:
> >>> v4:
> >>> * Changed double "if" to be "else if".
> >>> * Added missed PCI_P2PDMA_MAP_NONE case.
> >>> v3: https://patch.msgid.link/20251027-block-with-mmio-v3-0-ac3370e1f7b7@nvidia.com
> >>> * Encoded p2p map type in IOD flags instead of DMA attributes.
> >>> * Removed REQ_P2PDMA flag from block layer.
> >>> * Simplified map_phys conversion patch.
> >>> v2: https://lore.kernel.org/all/20251020-block-with-mmio-v2-0-147e9f93d8d4@nvidia.com/
> >>> * Added Chirstoph's Reviewed-by tag for first patch.
> >>> * Squashed patches
> >>> * Stored DMA MMIO attribute in NVMe IOD flags variable instead of block layer.
> >>> v1: https://patch.msgid.link/20251017-block-with-mmio-v1-0-3f486904db5e@nvidia.com
> >>> * Reordered patches.
> >>> * Dropped patch which tried to unify unmap flow.
> >>> * Set MMIO flag separately for data and integrity payloads.
> >>> v0: https://lore.kernel.org/all/cover.1760369219.git.leon@kernel.org/
> >>>
> >>> [...]
> >>
> >> Applied, thanks!
> >>
> >> [1/2] nvme-pci: migrate to dma_map_phys instead of map_page
> >> commit: f10000db2f7cf29d8c2ade69266bed7b51c772cb
> >> [2/2] block-dma: properly take MMIO path
> >> commit: 8df2745e8b23fdbe34c5b0a24607f5aaf10ed7eb
> >
> > And now dropped again - this doesn't boot on neither my big test box
> > with 33 nvme drives, nor even on my local test vm. Two different archs,
> > and very different setups. Which begs the question, how on earth was
> > this tested, if it doesn't boot on anything I have here?!
>
> I took a look, and what happens here is that iter.p2pdma.map is 0 as it
> never got set to anything. That is the same as PCI_P2PDMA_MAP_UNKNOWN,
> and hence we just end up in a BLK_STS_RESOURCE. First of all, returning
> BLK_STS_RESOURCE for that seems... highly suspicious. That should surely
> be a fatal error. And secondly, this just further backs up that there's
> ZERO testing done on this patchset at all. WTF?
>
> FWIW, the below makes it boot just fine, as expected, as a default zero
> filled iter then matches the UNKNOWN case.
>
>
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index e5ca8301bb8b..4cce69226773 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -1087,6 +1087,7 @@ static blk_status_t nvme_map_data(struct request *req)
> case PCI_P2PDMA_MAP_THRU_HOST_BRIDGE:
> iod->flags |= IOD_DATA_MMIO;
> break;
> + case PCI_P2PDMA_MAP_UNKNOWN:
> case PCI_P2PDMA_MAP_NONE:
> break;
> default:
> @@ -1122,6 +1123,7 @@ static blk_status_t nvme_pci_setup_meta_iter(struct request *req)
> case PCI_P2PDMA_MAP_THRU_HOST_BRIDGE:
> iod->flags |= IOD_META_MMIO;
> break;
> + case PCI_P2PDMA_MAP_UNKNOWN:
> case PCI_P2PDMA_MAP_NONE:
> break;
> default:
Sorry for troubles.
Can you please squash this fixup instead?
diff --git a/block/blk-mq-dma.c b/block/blk-mq-dma.c
index 98554929507a..807048644f2e 100644
--- a/block/blk-mq-dma.c
+++ b/block/blk-mq-dma.c
@@ -172,6 +172,7 @@ static bool blk_dma_map_iter_start(struct request *req, struct device *dma_dev,
memset(&iter->p2pdma, 0, sizeof(iter->p2pdma));
iter->status = BLK_STS_OK;
+ iter->p2pdma.map = PCI_P2PDMA_MAP_NONE;
/*
* Grab the first segment ASAP because we'll need it to check for P2P
>
> --
> Jens Axboe
On 11/13/25 12:50 PM, Leon Romanovsky wrote: > On Thu, Nov 13, 2025 at 10:45:53AM -0700, Jens Axboe wrote: >> On 11/13/25 10:12 AM, Jens Axboe wrote: >>> On 11/13/25 9:39 AM, Jens Axboe wrote: >>>> >>>> On Wed, 12 Nov 2025 21:48:03 +0200, Leon Romanovsky wrote: >>>>> Changelog: >>>>> v4: >>>>> * Changed double "if" to be "else if". >>>>> * Added missed PCI_P2PDMA_MAP_NONE case. >>>>> v3: https://patch.msgid.link/20251027-block-with-mmio-v3-0-ac3370e1f7b7@nvidia.com >>>>> * Encoded p2p map type in IOD flags instead of DMA attributes. >>>>> * Removed REQ_P2PDMA flag from block layer. >>>>> * Simplified map_phys conversion patch. >>>>> v2: https://lore.kernel.org/all/20251020-block-with-mmio-v2-0-147e9f93d8d4@nvidia.com/ >>>>> * Added Chirstoph's Reviewed-by tag for first patch. >>>>> * Squashed patches >>>>> * Stored DMA MMIO attribute in NVMe IOD flags variable instead of block layer. >>>>> v1: https://patch.msgid.link/20251017-block-with-mmio-v1-0-3f486904db5e@nvidia.com >>>>> * Reordered patches. >>>>> * Dropped patch which tried to unify unmap flow. >>>>> * Set MMIO flag separately for data and integrity payloads. >>>>> v0: https://lore.kernel.org/all/cover.1760369219.git.leon@kernel.org/ >>>>> >>>>> [...] >>>> >>>> Applied, thanks! >>>> >>>> [1/2] nvme-pci: migrate to dma_map_phys instead of map_page >>>> commit: f10000db2f7cf29d8c2ade69266bed7b51c772cb >>>> [2/2] block-dma: properly take MMIO path >>>> commit: 8df2745e8b23fdbe34c5b0a24607f5aaf10ed7eb >>> >>> And now dropped again - this doesn't boot on neither my big test box >>> with 33 nvme drives, nor even on my local test vm. Two different archs, >>> and very different setups. Which begs the question, how on earth was >>> this tested, if it doesn't boot on anything I have here?! >> >> I took a look, and what happens here is that iter.p2pdma.map is 0 as it >> never got set to anything. That is the same as PCI_P2PDMA_MAP_UNKNOWN, >> and hence we just end up in a BLK_STS_RESOURCE. First of all, returning >> BLK_STS_RESOURCE for that seems... highly suspicious. That should surely >> be a fatal error. And secondly, this just further backs up that there's >> ZERO testing done on this patchset at all. WTF? >> >> FWIW, the below makes it boot just fine, as expected, as a default zero >> filled iter then matches the UNKNOWN case. >> >> >> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c >> index e5ca8301bb8b..4cce69226773 100644 >> --- a/drivers/nvme/host/pci.c >> +++ b/drivers/nvme/host/pci.c >> @@ -1087,6 +1087,7 @@ static blk_status_t nvme_map_data(struct request *req) >> case PCI_P2PDMA_MAP_THRU_HOST_BRIDGE: >> iod->flags |= IOD_DATA_MMIO; >> break; >> + case PCI_P2PDMA_MAP_UNKNOWN: >> case PCI_P2PDMA_MAP_NONE: >> break; >> default: >> @@ -1122,6 +1123,7 @@ static blk_status_t nvme_pci_setup_meta_iter(struct request *req) >> case PCI_P2PDMA_MAP_THRU_HOST_BRIDGE: >> iod->flags |= IOD_META_MMIO; >> break; >> + case PCI_P2PDMA_MAP_UNKNOWN: >> case PCI_P2PDMA_MAP_NONE: >> break; >> default: > > Sorry for troubles. > > Can you please squash this fixup instead? > diff --git a/block/blk-mq-dma.c b/block/blk-mq-dma.c > index 98554929507a..807048644f2e 100644 > --- a/block/blk-mq-dma.c > +++ b/block/blk-mq-dma.c > @@ -172,6 +172,7 @@ static bool blk_dma_map_iter_start(struct request *req, struct device *dma_dev, > > memset(&iter->p2pdma, 0, sizeof(iter->p2pdma)); > iter->status = BLK_STS_OK; > + iter->p2pdma.map = PCI_P2PDMA_MAP_NONE; > > /* > * Grab the first segment ASAP because we'll need it to check for P2P Please send out a v5, and then also base it on the current tree. I had to hand apply one hunk on v4 because it didn't apply directly. Because another patch from 9 days ago modified it. I do agree that this should go elsewhere, but I don't think there's much of an issue doing it on the block side for now. That can then get killed when PCI does it. -- Jens Axboe
On Thu, Nov 13, 2025 at 01:43:09PM -0700, Jens Axboe wrote: > On 11/13/25 12:50 PM, Leon Romanovsky wrote: > > On Thu, Nov 13, 2025 at 10:45:53AM -0700, Jens Axboe wrote: > >> On 11/13/25 10:12 AM, Jens Axboe wrote: > >>> On 11/13/25 9:39 AM, Jens Axboe wrote: > >>>> > >>>> On Wed, 12 Nov 2025 21:48:03 +0200, Leon Romanovsky wrote: > >>>>> Changelog: > >>>>> v4: > >>>>> * Changed double "if" to be "else if". > >>>>> * Added missed PCI_P2PDMA_MAP_NONE case. > >>>>> v3: https://patch.msgid.link/20251027-block-with-mmio-v3-0-ac3370e1f7b7@nvidia.com > >>>>> * Encoded p2p map type in IOD flags instead of DMA attributes. > >>>>> * Removed REQ_P2PDMA flag from block layer. > >>>>> * Simplified map_phys conversion patch. > >>>>> v2: https://lore.kernel.org/all/20251020-block-with-mmio-v2-0-147e9f93d8d4@nvidia.com/ > >>>>> * Added Chirstoph's Reviewed-by tag for first patch. > >>>>> * Squashed patches > >>>>> * Stored DMA MMIO attribute in NVMe IOD flags variable instead of block layer. > >>>>> v1: https://patch.msgid.link/20251017-block-with-mmio-v1-0-3f486904db5e@nvidia.com > >>>>> * Reordered patches. > >>>>> * Dropped patch which tried to unify unmap flow. > >>>>> * Set MMIO flag separately for data and integrity payloads. > >>>>> v0: https://lore.kernel.org/all/cover.1760369219.git.leon@kernel.org/ > >>>>> > >>>>> [...] > >>>> > >>>> Applied, thanks! > >>>> > >>>> [1/2] nvme-pci: migrate to dma_map_phys instead of map_page > >>>> commit: f10000db2f7cf29d8c2ade69266bed7b51c772cb > >>>> [2/2] block-dma: properly take MMIO path > >>>> commit: 8df2745e8b23fdbe34c5b0a24607f5aaf10ed7eb > >>> > >>> And now dropped again - this doesn't boot on neither my big test box > >>> with 33 nvme drives, nor even on my local test vm. Two different archs, > >>> and very different setups. Which begs the question, how on earth was > >>> this tested, if it doesn't boot on anything I have here?! > >> > >> I took a look, and what happens here is that iter.p2pdma.map is 0 as it > >> never got set to anything. That is the same as PCI_P2PDMA_MAP_UNKNOWN, > >> and hence we just end up in a BLK_STS_RESOURCE. First of all, returning > >> BLK_STS_RESOURCE for that seems... highly suspicious. That should surely > >> be a fatal error. And secondly, this just further backs up that there's > >> ZERO testing done on this patchset at all. WTF? > >> > >> FWIW, the below makes it boot just fine, as expected, as a default zero > >> filled iter then matches the UNKNOWN case. > >> > >> > >> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c > >> index e5ca8301bb8b..4cce69226773 100644 > >> --- a/drivers/nvme/host/pci.c > >> +++ b/drivers/nvme/host/pci.c > >> @@ -1087,6 +1087,7 @@ static blk_status_t nvme_map_data(struct request *req) > >> case PCI_P2PDMA_MAP_THRU_HOST_BRIDGE: > >> iod->flags |= IOD_DATA_MMIO; > >> break; > >> + case PCI_P2PDMA_MAP_UNKNOWN: > >> case PCI_P2PDMA_MAP_NONE: > >> break; > >> default: > >> @@ -1122,6 +1123,7 @@ static blk_status_t nvme_pci_setup_meta_iter(struct request *req) > >> case PCI_P2PDMA_MAP_THRU_HOST_BRIDGE: > >> iod->flags |= IOD_META_MMIO; > >> break; > >> + case PCI_P2PDMA_MAP_UNKNOWN: > >> case PCI_P2PDMA_MAP_NONE: > >> break; > >> default: > > > > Sorry for troubles. > > > > Can you please squash this fixup instead? > > diff --git a/block/blk-mq-dma.c b/block/blk-mq-dma.c > > index 98554929507a..807048644f2e 100644 > > --- a/block/blk-mq-dma.c > > +++ b/block/blk-mq-dma.c > > @@ -172,6 +172,7 @@ static bool blk_dma_map_iter_start(struct request *req, struct device *dma_dev, > > > > memset(&iter->p2pdma, 0, sizeof(iter->p2pdma)); > > iter->status = BLK_STS_OK; > > + iter->p2pdma.map = PCI_P2PDMA_MAP_NONE; > > > > /* > > * Grab the first segment ASAP because we'll need it to check for P2P > > Please send out a v5, and then also base it on the current tree. I had > to hand apply one hunk on v4 because it didn't apply directly. Because > another patch from 9 days ago modified it. Thanks, will do.
On Thu, Nov 13, 2025 at 09:50:08PM +0200, Leon Romanovsky wrote: > Can you please squash this fixup instead? I think you should change pci_p2pdma_state() to always initialize the map type instead of making the caller do it.
On Thu, Nov 13, 2025, at 22:03, Keith Busch wrote: > On Thu, Nov 13, 2025 at 09:50:08PM +0200, Leon Romanovsky wrote: >> Can you please squash this fixup instead? > > I think you should change pci_p2pdma_state() to always initialize the > map type instead of making the caller do it. Yes, i just was afraid to add another subsystem into the mix.
On Thu, Nov 13, 2025, at 22:13, Leon Romanovsky wrote: > On Thu, Nov 13, 2025, at 22:03, Keith Busch wrote: >> On Thu, Nov 13, 2025 at 09:50:08PM +0200, Leon Romanovsky wrote: >>> Can you please squash this fixup instead? >> >> I think you should change pci_p2pdma_state() to always initialize the >> map type instead of making the caller do it. > > Yes, i just was afraid to add another subsystem into the mix. Another solution is to call to pci_p2pdma_state() again instead of checking iter.p2pdma.map in nvme. Which option do you prefer? Thanks
© 2016 - 2026 Red Hat, Inc.