md treats an unsupported P2PDMA transfer to a member as success, and
nvme-rdma retries one forever. Whether peer memory can be DMA-mapped
depends on the PCIe topology between the two devices, and since v6.17
the failure completes as BLK_STS_INVAL: retryable for multipath and
deliberately ignored by md for member failures, so mirrors silently
diverge.
v1-v3 restored BLK_STS_TARGET for these failures and tagged P2PDMA
bios in md. Logan Gunthorpe proposed a dedicated BLK_STS_P2PDMA
instead, so no consumer needs to know whether an I/O was P2P, and
wrote the core patches; v4 is built on them. Patches 1, 5, 7 and 8
are his and keep his authorship (7 and 8 with completion-handler
additions).
1 block: add BLK_STS_P2PDMA (stable, v6.17)
2 md: keep REQ_NOMERGE on P2PDMA bios
3 md/raid1: serialize non-write-behind writes on CollisionCheck
rdevs (pre-existing bug patch 4 would widen; stable)
4 md/raid1: no write-behind for P2PDMA bios
5 md/raid1,raid10: factor out raid1_write_error() helper
6 md/raid1,raid10: keep REQ_NOMERGE on narrow_write_error()
retry clones
7 md/raid1,raid10: skip futile retries on P2PDMA mapping
failures
8 md/raid1,raid10: set IO_BLOCKED in case of BLK_STS_P2PDMA
9 nvme-rdma: return BLK_STS_P2PDMA (stable, v7.1; needs patch 1)
Routing: patch 1 block, 2-8 md, 9 nvme; 7-9 use the status patch 1
adds. Patch 1 alone already fixes the divergence (md stops ignoring
the failure) and the multipath retry storm; the later patches only
refine the fallout, so the series bisects safely. The md patches
fix a v7.2-rc1 regression (02666132403a) and carry no stable tags.
Tested with the QEMU raid1/raid10 error matrix and a ConnectX-4 Lx.
After patch 1, block-layer error injection accepts status=P2PDMA, so
the md completion paths can be exercised on any member with no P2P
hardware. The patches were developed with AI assistance (see the
Assisted-by trailers); all code was human-reviewed and tested.
Against v7.2-rc4 (Logan's base).
Changes in v4:
- new BLK_STS_P2PDMA (Logan) replaces restoring BLK_STS_TARGET; the
R1BIO_P2PDMA/R10BIO_P2PDMA state bits are gone and the md changes
shrink to completion-time status tests
- blk_dma_map_iter_start() gets the explicit
PCI_P2PDMA_MAP_NOT_SUPPORTED case (Logan's patch; v3 kept
default: only)
- write completions record the whole failed range as bad blocks
instead of narrow_write_error()'s per-block retries; v3's coarse
retry is gone -- the new status is deterministic, nothing to
re-attempt
- read path: bi_status is evaluated before bio_put()
Link: https://lore.kernel.org/linux-raid/20260718162547.448892-1-mykola@meshstor.io/ [v1]
Link: https://lore.kernel.org/linux-raid/20260719105327.864949-1-mykola@meshstor.io/ [v2]
Link: https://lore.kernel.org/linux-raid/20260721174502.111503-1-mykola@meshstor.io/ [v3]
Link: https://lore.kernel.org/linux-raid/9f846c85-f427-433a-b87d-8e9e3a4406e7@deltatee.com/ [Logan's proposal]
Logan Gunthorpe (4):
block: add BLK_STS_P2PDMA for unsupported peer-to-peer transfers
md/raid1,raid10: factor out raid1_write_error() helper
md/raid1,raid10: skip futile retries on P2PDMA mapping failures
md/raid1,raid10: set IO_BLOCKED in case of BLK_STS_P2PDMA
Mykola Marzhan (5):
md: ensure REQ_NOMERGE is set on P2PDMA bios
md/raid1: serialize non-write-behind writes on CollisionCheck rdevs
md/raid1: don't use write-behind for P2PDMA bios
md/raid1,raid10: keep REQ_NOMERGE on narrow_write_error() retry clones
nvme-rdma: return BLK_STS_P2PDMA for unsupported P2P transfers
block/blk-core.c | 3 +++
block/blk-mq-dma.c | 3 ++-
drivers/md/md.c | 10 ++++++--
drivers/md/md.h | 15 ++++++++++++
drivers/md/raid1-10.c | 16 +++++++++++++
drivers/md/raid1.c | 48 +++++++++++++++++++++++----------------
drivers/md/raid10.c | 26 ++++++++++++---------
drivers/nvme/host/rdma.c | 38 ++++++++++++++++++-------------
include/linux/blk_types.h | 10 ++++++++
9 files changed, 120 insertions(+), 49 deletions(-)
base-commit: 1590cf0329716306e948a8fc29f1d3ee87d3989f
--
2.52.0