From nobody Tue Jun 23 01:23:34 2026 Received: from sender4-op-o15.zoho.com (sender4-op-o15.zoho.com [136.143.188.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 006B93382CB for ; Sun, 21 Jun 2026 13:03:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.15 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782046998; cv=pass; b=IfDxvbuiOslO3QwAbsLNrxcKmweo2O0lOkl0dDlazBUHwS9klhWcczi1TuUHkruFGvIq/QODgH5Q4JX6cm1QhmdLQKeRqvrHvKnsVcfCKGumY3mGsdCaKCeGrMqlVDG8+N64EMm2Crl3ZdHSuU5A8szcJSM1SIw6tRLVyk2+83w= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782046998; c=relaxed/simple; bh=BObJGRZzmtRxVfVzdFlLpRgDXZJViibwtMW5khQwtNY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=d+p/ZAIT4ZyYOaL7zlWXOywyheMHSThU1c6Q4ivFqnDnGdDVdZMsEUN1+BGGFunX+uAKKfDHQXGVQgQZkB2hjeolQJMgfsKhw4dyzT2kffwosT1clV3U3SisGocW8NmKMSz2boOZPrvlLN5KG4JwGQqafB9aljJ4UtTAphrRIsE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty; spf=pass smtp.mailfrom=linux.beauty; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b=k+jt0Rnn; arc=pass smtp.client-ip=136.143.188.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.beauty Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b="k+jt0Rnn" ARC-Seal: i=1; a=rsa-sha256; t=1782046988; cv=none; d=zohomail.com; s=zohoarc; b=FwjkGTc6hYYIgB9wwvkVKiUrXYQmqd9TgZwNDcg9I3ftcB1vTmiNW6RZVjJVK+cvZsvqRTGOM1PqcqX0F2x80xmUrmU/rnjusRC8X4qADO6EvhT8OoMxQwD/aYe1l3gw18V1w73/2NmzEO+/pPsNxynzcpnpcwUIkEkFs9xA2oM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782046988; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=Kkm6b8v9HrmRxgveYsXGJWG4c5oEBQ3malO2AScRBME=; b=WSwjrjm9SXUuh4K8H3gNf72vApLE5G9bqLvRFGI/V0aJ+68fMM+DDHau+MMQQ3ELnqn9fCaOkn6yXXdnwRr74OMqTqadyw4aviuemmv47vzgoEPp3SpMAs7ygxeOD20zNl37d4CfDHgV31DTq5JxgibFWHpvMSDxHqhXunbBFI4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=linux.beauty; spf=pass smtp.mailfrom=me@linux.beauty; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1782046988; s=zmail; d=linux.beauty; i=me@linux.beauty; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Message-Id:Reply-To; bh=Kkm6b8v9HrmRxgveYsXGJWG4c5oEBQ3malO2AScRBME=; b=k+jt0RnnZHpeMTck8WP3eoK+qJPoaxDXW4+cfJIDj3MSpoLZgOAKaTD7HLrc8KuS w/25Trckyg2V9iI251BcN/q/OyePYCiyHMg7aQVtuHb4DkvuoRkQwnplolE5m79oZ6z uc8q2SabJ0xwJNosUhWp/arhpCRfYe6IwI3ZayXQ= Received: by mx.zohomail.com with SMTPS id 1782046985587100.80729899755318; Sun, 21 Jun 2026 06:03:05 -0700 (PDT) From: Li Chen To: Pankaj Gupta , Dan Williams , Vishal Verma , Dave Jiang , Ira Weiny , Alison Schofield , virtualization@lists.linux.dev, nvdimm@lists.linux.dev Cc: linux-kernel@vger.kernel.org, Li Chen Subject: [PATCH v6 01/12] nvdimm: preserve flush callback errors Date: Sun, 21 Jun 2026 21:02:32 +0800 Message-ID: <20260621130246.2973254-2-me@linux.beauty> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260621130246.2973254-1-me@linux.beauty> References: <20260621130246.2973254-1-me@linux.beauty> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Content-Type: text/plain; charset="utf-8" nvdimm_flush() currently converts any non-zero provider flush error to -EIO. That loses useful errno values from provider callbacks. A local virtio-pmem mkfs sanity test showed the masking clearly: wipefs: /dev/pmem0: cannot flush modified buffers: Input/output error mkfs.ext4: Input/output error while writing out and closing file system nd_region region0: dbg: nvdimm_flush rc=3D-5 The virtio-pmem callback can return -ENOMEM when async_pmem_flush() fails to allocate a child flush bio, but nvdimm_flush() hides that as -EIO before pmem_submit_bio() converts it to a block status. Return the provider callback error directly. The generic flush path still returns 0, and pmem_submit_bio() already handles errno-to-blk_status conversion for bio completion. Signed-off-by: Li Chen --- v3->v4: - New patch. drivers/nvdimm/region_devs.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c index e35c2e18518f0..0cd96503c0596 100644 --- a/drivers/nvdimm/region_devs.c +++ b/drivers/nvdimm/region_devs.c @@ -1114,10 +1114,8 @@ int nvdimm_flush(struct nd_region *nd_region, struct= bio *bio) =20 if (!nd_region->flush) rc =3D generic_nvdimm_flush(nd_region); - else { - if (nd_region->flush(nd_region, bio)) - rc =3D -EIO; - } + else + rc =3D nd_region->flush(nd_region, bio); =20 return rc; } --=20 2.52.0 From nobody Tue Jun 23 01:23:34 2026 Received: from sender4-op-o15.zoho.com (sender4-op-o15.zoho.com [136.143.188.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 157051D5174 for ; Sun, 21 Jun 2026 13:03:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.15 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047003; cv=pass; b=NHkwyyXVv9WeqK0MWtRjPG4jSYia8AgipcJeuXkpGOtGAcuz9DoTZuKWJLZANjAUaKAloK5Ul2r5NmFwQefIV0lOV81ilOWQa9qZfL+dQecEkG0NtzWIOyq2QinBfFGxPm8pH6LG/AxA0p1MyW9c32Lr0KjH2bJ+qFj0hBQn8/k= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047003; c=relaxed/simple; bh=yNGrEJlXC/3Kj2L50w8PBQwDVtyNG/aExJYIOATyaIM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=G0WF4DmfmWyAyaqJYGLOcBxIV45cv4kLtg51ysykGk4K4WzoNdJNU2R3SRQ6B4t/LGwkib2UoHkmCwGhtysi9lZakiDHzsU4v3CruNJCRV/QBFnVyOwtyC4UHeQMAGKYRCiNaeMqfFxR2zVUsZrJG/bI5p/lpOepG/gQ88NyciU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty; spf=pass smtp.mailfrom=linux.beauty; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b=Ky0hFX24; arc=pass smtp.client-ip=136.143.188.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.beauty Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b="Ky0hFX24" ARC-Seal: i=1; a=rsa-sha256; t=1782046992; cv=none; d=zohomail.com; s=zohoarc; b=UJVQ/eeoSQUxw+Rkm3csEyxOpJ4G20yx880ZLVfX8X1gcmPeR1Xhee3dSD80I28TUFz54zA5SIk4zRF9Mv5USJkhceX2hq5x6fENQu7T9hTxBUVKxmr4FI+qy6LFNhI9UeQIkX77h/wJ+vXk8ifN1leF8d7ZIOhSTA75uS4JT/s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782046992; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=l+TairmX9egfA9LqMO7TzDhHCGZIpaTe4WqZKhyGKds=; b=lAyelh7nWepCHQziXdcpBWwyxSg6WRfeddmhoDp4j9vuZ9X5jg65LB/a9v8aZ50KDeuGqsoAe0CxfJ+ehPSSbxD7RfmExzWWD7bXhcW7CDgHK2Zzvw/zXgoPdCu5gb1F8+9gftTIcZeca4i3I/8C1YXjZcFSGKynPEw1anNOR90= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=linux.beauty; spf=pass smtp.mailfrom=me@linux.beauty; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1782046992; s=zmail; d=linux.beauty; i=me@linux.beauty; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Message-Id:Reply-To; bh=l+TairmX9egfA9LqMO7TzDhHCGZIpaTe4WqZKhyGKds=; b=Ky0hFX24OAgnKFHKLCuhmowMj4REFAr9ynAnCywtiNnitG1l+95Z+fW4xi79sNLn Q0sN4dpzoZ2KYYHSybuP0Gy3xIPgBdYt4cyniGu/XdMedQN5SUQK888EIZ1SvPoUrKk LFuzMPfg74n4Ik3PSdLWnSLE7fO7hvQaWewNSPBI= Received: by mx.zohomail.com with SMTPS id 1782046989260929.7539967088986; Sun, 21 Jun 2026 06:03:09 -0700 (PDT) From: Li Chen To: Pankaj Gupta , Dan Williams , Vishal Verma , Dave Jiang , Ira Weiny , Alison Schofield , virtualization@lists.linux.dev, nvdimm@lists.linux.dev Cc: linux-kernel@vger.kernel.org, Li Chen Subject: [PATCH v6 02/12] nvdimm: pmem: keep PREFLUSH before data writes Date: Sun, 21 Jun 2026 21:02:33 +0800 Message-ID: <20260621130246.2973254-3-me@linux.beauty> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260621130246.2973254-1-me@linux.beauty> References: <20260621130246.2973254-1-me@linux.beauty> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Content-Type: text/plain; charset="utf-8" pmem_submit_bio() records a REQ_PREFLUSH error, but continues to copy the bio data and can later overwrite the error with a successful REQ_FUA flush. That lets data writes run after a failed preflush and can complete the bio successfully despite the failed ordering barrier. Run the REQ_PREFLUSH flush synchronously before touching the bio data and complete the bio with the flush error if it fails. Keep asynchronous flush chaining for REQ_FUA. At that point, data copy has completed and the parent bio can wait for the chained flush bio. Signed-off-by: Li Chen --- Changes in v5: - New patch. drivers/nvdimm/pmem.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index 92c67fbbc1c85..05d3de33e2706 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -208,8 +208,14 @@ static void pmem_submit_bio(struct bio *bio) struct pmem_device *pmem =3D bio->bi_bdev->bd_disk->private_data; struct nd_region *nd_region =3D to_region(pmem); =20 - if (bio->bi_opf & REQ_PREFLUSH) - ret =3D nvdimm_flush(nd_region, bio); + if (bio->bi_opf & REQ_PREFLUSH) { + ret =3D nvdimm_flush(nd_region, NULL); + if (ret) { + bio->bi_status =3D errno_to_blk_status(ret); + bio_endio(bio); + return; + } + } =20 do_acct =3D blk_queue_io_stat(bio->bi_bdev->bd_disk->queue); if (do_acct) @@ -229,7 +235,7 @@ static void pmem_submit_bio(struct bio *bio) if (do_acct) bio_end_io_acct(bio, start); =20 - if (bio->bi_opf & REQ_FUA) + if ((bio->bi_opf & REQ_FUA) && !bio->bi_status) ret =3D nvdimm_flush(nd_region, bio); =20 if (ret) --=20 2.52.0 From nobody Tue Jun 23 01:23:34 2026 Received: from sender4-op-o15.zoho.com (sender4-op-o15.zoho.com [136.143.188.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2987D1D5174 for ; Sun, 21 Jun 2026 13:03:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.15 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047008; cv=pass; b=GfFrLY2CUbJ3SE3GWI4v9SXM8PJh0yA5TAYemoLfzl8MF9L6qwucFkEyg6EBnP0IFY2fA6AjV1WbTFmmIi6XzOePziAR4qB9/ntOYvi81/3YutdSqdznWa7ye2JysHlxbcoVx3nMYNqe4vGBpLS34OGluyf9M9JDjVTZfoWEvIg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047008; c=relaxed/simple; bh=itjLrkKJEW81RURIxDECsGtTLBhbeI004y7GPmhHinc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ncryzE9KfT0IIL2zGAiJuo2glCDQV2ZjwrfbsKrMMLCVv3f6WUaDMDnTrRYiSI259WXkoYc+GGm/njYk9W6KbK1AnAZ1J33ke8JEXGDJenmzvjYBEU4wkwm0Vc/+irqT4cZicvhhHSFsS3RNxfTyAGNQUFMnolVa9YD9WLaMa3A= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty; spf=pass smtp.mailfrom=linux.beauty; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b=i4EOrt9z; arc=pass smtp.client-ip=136.143.188.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.beauty Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b="i4EOrt9z" ARC-Seal: i=1; a=rsa-sha256; t=1782046996; cv=none; d=zohomail.com; s=zohoarc; b=YDIit72m4ticVGIAgvj14rN8bamQK7mkEwYY0VamG9eXaKwuHPHhbNf8jO1otq08XxmaccpQK5t0IXn76O2qWH2an/ZXKkQHyZomJ/qo29LsZMOOyffVj5kmbOUmJGlyhE4fRgAw7SotOd9tyr3sNA5rRhy7w/y2gzFcWpZB8Qg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782046996; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=c0EOLBoih0cc+8cnP+Tzr7COxL+v2rD15YCFnpCQMeg=; b=P5nsVLzpjoFEAV1jpyShma37Cm/s2PayE5pzX8wrhmEqdMj86uWLdv0f+c+JSNlIFDT4CxXL4QYYaQfbujkFmSaYCmlrmce8ifY+Ov1DaeoENo/zVTi/FGEbDsWT0xcT2fQ5A+Y9653Zd48LbgpAb6ohatPh5sJuOJZXs+ep4PU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=linux.beauty; spf=pass smtp.mailfrom=me@linux.beauty; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1782046996; s=zmail; d=linux.beauty; i=me@linux.beauty; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Message-Id:Reply-To; bh=c0EOLBoih0cc+8cnP+Tzr7COxL+v2rD15YCFnpCQMeg=; b=i4EOrt9zfwk3FQH38nO2rWjQ1fHF/8lYPI/Cgz4CgJRU49Sax6afXjZ3tN2SQcHS UdUslO8hV40hjDC4FYUVr/NpXQR1agoBCGAYSLpgbMtqsYkaNF4gWxUl6FTEaywVNp+ UokEd9QyDR0SQssNmDsClbR+ClBKbS42qVfB3eXY= Received: by mx.zohomail.com with SMTPS id 1782046992618983.5404623082977; Sun, 21 Jun 2026 06:03:12 -0700 (PDT) From: Li Chen To: Pankaj Gupta , Dan Williams , Vishal Verma , Dave Jiang , Ira Weiny , Alison Schofield , virtualization@lists.linux.dev, nvdimm@lists.linux.dev Cc: linux-kernel@vger.kernel.org, Li Chen Subject: [PATCH v6 03/12] nvdimm: pmem: guard data loop for dataless bios Date: Sun, 21 Jun 2026 21:02:34 +0800 Message-ID: <20260621130246.2973254-4-me@linux.beauty> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260621130246.2973254-1-me@linux.beauty> References: <20260621130246.2973254-1-me@linux.beauty> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Content-Type: text/plain; charset="utf-8" pmem_submit_bio() handles flush-only bios before and after the data loop. Keep dataless bios out of bio_for_each_segment() so the data path only walks bios that actually carry bvec data. Signed-off-by: Li Chen --- Changes in v6: - New patch. drivers/nvdimm/pmem.c | 36 +++++++++++++++++++++--------------- 1 file changed, 21 insertions(+), 15 deletions(-) diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index 05d3de33e2706..82ee1ddb3a445 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -217,23 +217,29 @@ static void pmem_submit_bio(struct bio *bio) } } =20 - do_acct =3D blk_queue_io_stat(bio->bi_bdev->bd_disk->queue); - if (do_acct) - start =3D bio_start_io_acct(bio); - bio_for_each_segment(bvec, bio, iter) { - if (op_is_write(bio_op(bio))) - rc =3D pmem_do_write(pmem, bvec.bv_page, bvec.bv_offset, - iter.bi_sector, bvec.bv_len); - else - rc =3D pmem_do_read(pmem, bvec.bv_page, bvec.bv_offset, - iter.bi_sector, bvec.bv_len); - if (rc) { - bio->bi_status =3D rc; - break; + if (bio_has_data(bio)) { + do_acct =3D blk_queue_io_stat(bio->bi_bdev->bd_disk->queue); + if (do_acct) + start =3D bio_start_io_acct(bio); + bio_for_each_segment(bvec, bio, iter) { + if (op_is_write(bio_op(bio))) + rc =3D pmem_do_write(pmem, bvec.bv_page, + bvec.bv_offset, + iter.bi_sector, + bvec.bv_len); + else + rc =3D pmem_do_read(pmem, bvec.bv_page, + bvec.bv_offset, + iter.bi_sector, + bvec.bv_len); + if (rc) { + bio->bi_status =3D rc; + break; + } } + if (do_acct) + bio_end_io_acct(bio, start); } - if (do_acct) - bio_end_io_acct(bio, start); =20 if ((bio->bi_opf & REQ_FUA) && !bio->bi_status) ret =3D nvdimm_flush(nd_region, bio); --=20 2.52.0 From nobody Tue Jun 23 01:23:34 2026 Received: from sender4-op-o15.zoho.com (sender4-op-o15.zoho.com [136.143.188.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E9559364EB8 for ; Sun, 21 Jun 2026 13:03:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.15 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047013; cv=pass; b=PZOewakzXEJgjk8WxEYwLeJp96p0ylOJPQkXjARWYPxltC5q0h56WcWE7iZRnTpliI2i2GKlIrNNcvIKDvDJzHNqwLFNA4OoC2WYEOxpq4ePSRnGC7FHLEOX5+CQvfbcZjNBdD3Sb0jGgAy5gW4UPmLVVM3s7c7pLu3NYdO5/4U= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047013; c=relaxed/simple; bh=hyJUYuMM3yNKIj/7gM8TyqSviP72o38Bz2OQhycgbZw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=D3ATlkeMTywDImtKZMs4TqnxzbbyTqAXOA+VqS8fmicYCNxd+JavF+0yIxS455IqJm+It1BdTCYp6T/uPD2d4ONCHvF8Wad3A2LKYZwRIGno83uG/yeYVISyAQinFaLapkXLcXwuQuNV+uXyJIJjSQgwW2kzZ/Akv0i+KV6+2Go= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty; spf=pass smtp.mailfrom=linux.beauty; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b=ZSzmJW7M; arc=pass smtp.client-ip=136.143.188.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.beauty Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b="ZSzmJW7M" ARC-Seal: i=1; a=rsa-sha256; t=1782046998; cv=none; d=zohomail.com; s=zohoarc; b=NfF1pGePgsZnut26MfgdjhbBWAbislLX2xHm++Vl1m012jp28gTJFqlkHRJl1FhRKVAbkMfh+TZx6U7RB2Oq8LXXKTtjswu8ax9H6wr+s6ABBN55u+Ng3qsDe4ZJZB7q4S05WMjKp2Bp16zystlIRRl58m50+qD2AbAUpKdV4Ac= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782046998; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=FQJiXUAaMOJcZLDmmijVIF15/BdHvIXbJtlsM+hDaqQ=; b=MAk0Aa/dY+0dOhAh81AdVf17/boVnuCrQu7/KFL5Gers5quqXPYkJbfT1zSy9v6NChEujHkxZCgWrHfC5xVchmFp27bmYXmCZwdIIHw6HB81Xgnmib++WEFfRhujEgYcW1WrBAlkKktt0FBF8y2M4nvJZqcaczUH8h7MNd23lPY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=linux.beauty; spf=pass smtp.mailfrom=me@linux.beauty; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1782046998; s=zmail; d=linux.beauty; i=me@linux.beauty; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Message-Id:Reply-To; bh=FQJiXUAaMOJcZLDmmijVIF15/BdHvIXbJtlsM+hDaqQ=; b=ZSzmJW7MyPmJ84hjBV4vHsZISAGKGgk4KeuvSaCH2sYQFgiqLaDa7J2HSitA+3Sc IX9+I6wopkdNDSou+5Ez9di46XnJrOUznz87s6lEm7S+0ZhDrjodIHVdRsFgbfnZar+ CdITMXmZdcLgx15vZc8ODUbPiU5nmmCYYRezgSiE= Received: by mx.zohomail.com with SMTPS id 1782046996030630.2157956865668; Sun, 21 Jun 2026 06:03:16 -0700 (PDT) From: Li Chen To: Pankaj Gupta , Dan Williams , Vishal Verma , Dave Jiang , Ira Weiny , Alison Schofield , virtualization@lists.linux.dev, nvdimm@lists.linux.dev Cc: linux-kernel@vger.kernel.org, Li Chen Subject: [PATCH v6 04/12] nvdimm: virtio_pmem: stop allocating child flush bio Date: Sun, 21 Jun 2026 21:02:35 +0800 Message-ID: <20260621130246.2973254-5-me@linux.beauty> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260621130246.2973254-1-me@linux.beauty> References: <20260621130246.2973254-1-me@linux.beauty> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Content-Type: text/plain; charset="utf-8" pmem_submit_bio() passes the parent bio to nvdimm_flush() for REQ_FUA. For virtio-pmem this makes async_pmem_flush() allocate and submit a child PREFLUSH bio chained to the parent. That child allocation is in the block submit path. Making it blocking with GFP_NOIO can consume the same global bio mempool that submit_bio() uses, while making it GFP_ATOMIC can fail under pressure. A forced failure of the child allocation produced: virtio_pmem: forcing child bio allocation failure for test Buffer I/O error on dev pmem0, logical block 0, lost sync page write EXT4-fs (pmem0): I/O error while writing superblock EXT4-fs (pmem0): mount failed Avoid the child bio completely. Flush FUA synchronously, like REQ_PREFLUSH, then complete the parent after the flush. Since no child bio can be created, async_pmem_flush() now only issues the virtio flush and preserves negative errno values. Signed-off-by: Li Chen --- Changes in v6: - Replace the child bio allocation fix with synchronous FUA flushing. drivers/nvdimm/nd_virtio.c | 22 ++++------------------ drivers/nvdimm/pmem.c | 2 +- 2 files changed, 5 insertions(+), 19 deletions(-) diff --git a/drivers/nvdimm/nd_virtio.c b/drivers/nvdimm/nd_virtio.c index 4176046627beb..4b2e9c47af0f5 100644 --- a/drivers/nvdimm/nd_virtio.c +++ b/drivers/nvdimm/nd_virtio.c @@ -110,27 +110,13 @@ static int virtio_pmem_flush(struct nd_region *nd_reg= ion) /* The asynchronous flush callback function */ int async_pmem_flush(struct nd_region *nd_region, struct bio *bio) { - /* - * Create child bio for asynchronous flush and chain with - * parent bio. Otherwise directly call nd_region flush. - */ - if (bio && bio->bi_iter.bi_sector !=3D -1) { - struct bio *child =3D bio_alloc(bio->bi_bdev, 0, - REQ_OP_WRITE | REQ_PREFLUSH, - GFP_ATOMIC); + int err; =20 - if (!child) - return -ENOMEM; - bio_clone_blkg_association(child, bio); - child->bi_iter.bi_sector =3D -1; - bio_chain(child, bio); - submit_bio(child); - return 0; - } - if (virtio_pmem_flush(nd_region)) + err =3D virtio_pmem_flush(nd_region); + if (err > 0) return -EIO; =20 - return 0; + return err; }; EXPORT_SYMBOL_GPL(async_pmem_flush); MODULE_DESCRIPTION("Virtio Persistent Memory Driver"); diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index 82ee1ddb3a445..058d2739c95a1 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -242,7 +242,7 @@ static void pmem_submit_bio(struct bio *bio) } =20 if ((bio->bi_opf & REQ_FUA) && !bio->bi_status) - ret =3D nvdimm_flush(nd_region, bio); + ret =3D nvdimm_flush(nd_region, NULL); =20 if (ret) bio->bi_status =3D errno_to_blk_status(ret); --=20 2.52.0 From nobody Tue Jun 23 01:23:34 2026 Received: from sender4-op-o15.zoho.com (sender4-op-o15.zoho.com [136.143.188.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 11AC43382CB for ; Sun, 21 Jun 2026 13:03:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.15 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047018; cv=pass; b=dUSF72cQvPHPUGcJQnfk3NIi8rbHMvQwxyWiu6YBIOvxL9jRr49MKiJJpKwhkbSGE8yQxxoMmE+Thj5fgIt+AvlS1wfBHxcnck7qiJ8PjombQUx3LcteUYlfBQTDX8Lk4t+RH9/Cwrsiw2o+Sn7Wk7eGJapjPeBO/CiLS3K2yHU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047018; c=relaxed/simple; bh=ewbtKz2hdslCFWnjQTPY4AnRMcpsvvM+zeJ55wqDaQw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IVBn2PGnimZcjuUux5pkKWLyuGo980i8vHtDQPT/UEeCpiCZ/htETN9Q17sX66TEx6KYmUBnniM2AnvWh5vyv0R++Z4RbxHDzVkpCcecgNJ1zyLDbmpxsFSdRYPPlkIatDpE6gGFKvAMu6RA4d6K6Ji0rKBdT4bsBJgNyuq4a7c= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty; spf=pass smtp.mailfrom=linux.beauty; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b=UyY9byqd; arc=pass smtp.client-ip=136.143.188.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.beauty Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b="UyY9byqd" ARC-Seal: i=1; a=rsa-sha256; t=1782047002; cv=none; d=zohomail.com; s=zohoarc; b=MxAYZL6+AXhvOvX5N5/u3vCCr9I9aX6oFOAWrO5i3jHw51QJFhebcx1iW1p0mqGQV3XeI6X2CkVTcsSrhlvKNoOvn8JjI4VCzgTa0pobvA9SUvdNbKoiI2vSZq8CDZzIFOnVTqEzHJ4b3rxfDTc8jqGKJam89YzCwDUypTMh1+w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782047002; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=W+o+IKys+lU/ZxFNZxi3pNHAmf/FkNcd7IUh/VLtXLU=; b=JbSZg7FTX4uyoANvOZApavJ49Jeqe+gH4umI9MTO4ybfNxyCJmYr4Lzksm6JJQy9s61yr/jxupLB4aZRWops+lEKN3DCOvUAiX9wKmaV3jpJSnRTW+zA9D61ZsApKHTzMKqJf09TbFxEnLYQ68lGuZeGzZcjA2g4zApXcDMjBk4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=linux.beauty; spf=pass smtp.mailfrom=me@linux.beauty; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1782047002; s=zmail; d=linux.beauty; i=me@linux.beauty; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Message-Id:Reply-To; bh=W+o+IKys+lU/ZxFNZxi3pNHAmf/FkNcd7IUh/VLtXLU=; b=UyY9byqdcuyB30IuQ4ltnwxAEZmGtNo3xN9LDYF0vaqgG/xVdbAWihEG0j+ai5jh Fg5spON+uD1FgC7I0uAcTkVHD+ad9XIf1LTJBZJfsX/rhD16n6MW0f5iUg63wVrhjxd Fz5BEwj4fEBVFJQZfJO/D+Cpel9NHnbYw0x48oyQ= Received: by mx.zohomail.com with SMTPS id 1782046999373828.3231573844629; Sun, 21 Jun 2026 06:03:19 -0700 (PDT) From: Li Chen To: Pankaj Gupta , Dan Williams , Vishal Verma , Dave Jiang , Ira Weiny , Alison Schofield , virtualization@lists.linux.dev, nvdimm@lists.linux.dev Cc: linux-kernel@vger.kernel.org, Li Chen Subject: [PATCH v6 05/12] nvdimm: virtio_pmem: use GFP_NOIO for flush requests Date: Sun, 21 Jun 2026 21:02:36 +0800 Message-ID: <20260621130246.2973254-6-me@linux.beauty> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260621130246.2973254-1-me@linux.beauty> References: <20260621130246.2973254-1-me@linux.beauty> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Content-Type: text/plain; charset="utf-8" virtio_pmem_flush() can run from pmem_submit_bio() while filesystem IO is waiting on the flush completion. The request object allocation can sleep, but it should not enter filesystem or block IO reclaim from this flush path. Use GFP_NOIO for the request allocation. The virtqueue descriptor allocation still uses GFP_ATOMIC because it runs under pmem_lock. Signed-off-by: Li Chen --- Changes in v6: - New patch; keep GFP_NOIO only for the virtio-pmem request allocation. drivers/nvdimm/nd_virtio.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/nvdimm/nd_virtio.c b/drivers/nvdimm/nd_virtio.c index 4b2e9c47af0f5..91ca144607531 100644 --- a/drivers/nvdimm/nd_virtio.c +++ b/drivers/nvdimm/nd_virtio.c @@ -55,7 +55,7 @@ static int virtio_pmem_flush(struct nd_region *nd_region) return -EIO; } =20 - req_data =3D kmalloc_obj(*req_data); + req_data =3D kmalloc_obj(*req_data, GFP_NOIO); if (!req_data) return -ENOMEM; =20 --=20 2.52.0 From nobody Tue Jun 23 01:23:34 2026 Received: from sender4-op-o15.zoho.com (sender4-op-o15.zoho.com [136.143.188.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1ECA73655FD for ; Sun, 21 Jun 2026 13:03:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.15 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047023; cv=pass; b=SECHka2pozm8FwsI0/nEct1e+oLgTbM4D4UyERUUzgO1uzXGdcBLpiQkNwcbRaoHLSUsqojMHNAC7+g+lg6BkT7f0Of7oGOYXEsFnmlg2bgEkggj2g8+TnfqFjRs20ClFxmq7p6QQvuowG2p1b4YdXO4VKKV8Zi+epPx9zViEkQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047023; c=relaxed/simple; bh=q/GRfoSlw5dIy2Aus/xfpx0LMjWlGNXaHPaEwSblvUE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QILydnEKJcZyyALMRblKI3eFhOQIuJLNhAvXcYuB7mjveN0T4mSDDOVQO387OwBqZZYDWOzoHMoyi9Y7xCkY2SbMwLYCEa1v14tFxQADykMuntijS7NHWIMB9am3H0nmRwLr6iUhUfcWFESgaJznuJTOaAQ7rJ+R447NPI01VDU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty; spf=pass smtp.mailfrom=linux.beauty; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b=BnBZErVk; arc=pass smtp.client-ip=136.143.188.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.beauty Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b="BnBZErVk" ARC-Seal: i=1; a=rsa-sha256; t=1782047006; cv=none; d=zohomail.com; s=zohoarc; b=fAVjLuq8Ruy8DtKDTxh4TXjOIz82y/rghiiiIWpKYkhO2YbYonBasBVliq9TuoE9mmSekw/lHyYJbJUKCioU1xJlhNO8U58dwNbJbNJvtWcIC+3O8GZL/HiTrd7cgxRcceilpz0GPbxbzKtni8J6YujjCppxAKd99sd+0ydTuug= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782047006; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=7/P+RfQlAxUphbROAQccTU2vS2PxCRT9XWwAjXeYZMY=; b=FIFNfm9jjnBHkYV4gVC0kELekgCxD7knT7v6fsiiLuYxGlGr/XsMcnYGmMm7f+Rp1Wu676mZHxdnC9lAPK5Ri8Aa6BtV0eBpm+L8cvDfH9CSrv5x9M0HNldyc1W9BQOK0epmO8DUI/L/jqDyD0Yso87xuWSg3wc9TD6xQZpQT/Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=linux.beauty; spf=pass smtp.mailfrom=me@linux.beauty; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1782047006; s=zmail; d=linux.beauty; i=me@linux.beauty; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Message-Id:Reply-To; bh=7/P+RfQlAxUphbROAQccTU2vS2PxCRT9XWwAjXeYZMY=; b=BnBZErVkfu4TSWy/xSByAym1XakIduIdJA5WYb0IjWYZKrHfreimaZIJnDSjI5uw 80t4iggeoetaAHiisLfbFJpfam3MtNGmYpCXDTjwCg800uae8JDNwOSFpgdig2lfzXd msMv7mpJJHzebAsUrzNrQXPMzmdPW1ol+HfET3N8= Received: by mx.zohomail.com with SMTPS id 1782047002784416.09041512376143; Sun, 21 Jun 2026 06:03:22 -0700 (PDT) From: Li Chen To: Pankaj Gupta , Dan Williams , Vishal Verma , Dave Jiang , Ira Weiny , Alison Schofield , virtualization@lists.linux.dev, nvdimm@lists.linux.dev Cc: linux-kernel@vger.kernel.org, Li Chen Subject: [PATCH v6 06/12] nvdimm: virtio_pmem: always wake -ENOSPC waiters Date: Sun, 21 Jun 2026 21:02:37 +0800 Message-ID: <20260621130246.2973254-7-me@linux.beauty> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260621130246.2973254-1-me@linux.beauty> References: <20260621130246.2973254-1-me@linux.beauty> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Content-Type: text/plain; charset="utf-8" virtio_pmem_host_ack() reclaims virtqueue descriptors with virtqueue_get_buf(). The -ENOSPC waiter wakeup is tied to completing the returned token. If token completion is skipped for any reason, reclaimed descriptors may not wake a waiter and the submitter may sleep forever waiting for a free slot. Always wake one -ENOSPC waiter for each virtqueue completion before touching the returned token. Signed-off-by: Li Chen --- v2->v3: - Split out the waiter wakeup ordering change from READ_ONCE()/WRITE_ONCE() updates (now patch 4/7), per Pankaj's suggestion. v3->v4: - Rebased onto v7.1-rc7 and renumbered after the flush error patches. drivers/nvdimm/nd_virtio.c | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/drivers/nvdimm/nd_virtio.c b/drivers/nvdimm/nd_virtio.c index 91ca144607531..8ed4d6b3a9284 100644 --- a/drivers/nvdimm/nd_virtio.c +++ b/drivers/nvdimm/nd_virtio.c @@ -9,26 +9,33 @@ #include "virtio_pmem.h" #include "nd.h" =20 +static void virtio_pmem_wake_one_waiter(struct virtio_pmem *vpmem) +{ + struct virtio_pmem_request *req_buf; + + if (list_empty(&vpmem->req_list)) + return; + + req_buf =3D list_first_entry(&vpmem->req_list, + struct virtio_pmem_request, list); + req_buf->wq_buf_avail =3D true; + wake_up(&req_buf->wq_buf); + list_del(&req_buf->list); +} + /* The interrupt handler */ void virtio_pmem_host_ack(struct virtqueue *vq) { struct virtio_pmem *vpmem =3D vq->vdev->priv; - struct virtio_pmem_request *req_data, *req_buf; + struct virtio_pmem_request *req_data; unsigned long flags; unsigned int len; =20 spin_lock_irqsave(&vpmem->pmem_lock, flags); while ((req_data =3D virtqueue_get_buf(vq, &len)) !=3D NULL) { + virtio_pmem_wake_one_waiter(vpmem); req_data->done =3D true; wake_up(&req_data->host_acked); - - if (!list_empty(&vpmem->req_list)) { - req_buf =3D list_first_entry(&vpmem->req_list, - struct virtio_pmem_request, list); - req_buf->wq_buf_avail =3D true; - wake_up(&req_buf->wq_buf); - list_del(&req_buf->list); - } } spin_unlock_irqrestore(&vpmem->pmem_lock, flags); } --=20 2.52.0 From nobody Tue Jun 23 01:23:34 2026 Received: from sender4-op-o15.zoho.com (sender4-op-o15.zoho.com [136.143.188.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F3B8B3403F1 for ; Sun, 21 Jun 2026 13:03:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.15 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047028; cv=pass; b=PCpAknI7t+gbFuUFNdjnJEHLUuCugonKk0ahCS4G2HuF5ozs0CT79PJ093irf+q9E/0aHs4AE1Uxa8ltY2GjQReLYxDr3bWrWF7A/oeXzBf0jPfIIUTYrCrbgYkullL3Olvb0Tro08xPyWC2JiRw+syY3Zk3LI5tLQ9M9G5nbkU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047028; c=relaxed/simple; bh=VMwCRHdy9IMRfHKnX1xRM4eNLkPvOY+0DdfmzkYIW3s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tbfprOCL3dsOWeRxL1FQu7j/bT09W8p04bsBu4A5fY//kmqtm68x4WY2XsVRhLnFDRtwNdj11A6FmlQ1Frh2IJ2x2xA2auoCZmrUVy3jbnS9R8HSbiPawREIClFafiCS8lfOPktWq5mbHOs4MZEVAGk+7C3a13o2hG+IBOYCqqo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty; spf=pass smtp.mailfrom=linux.beauty; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b=BjTE4sF5; arc=pass smtp.client-ip=136.143.188.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.beauty Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b="BjTE4sF5" ARC-Seal: i=1; a=rsa-sha256; t=1782047008; cv=none; d=zohomail.com; s=zohoarc; b=mz4tRTN10/3PKKrwaoXU2xNxwnvgf/xuSRoT3toUqckbakNR3T+KVXViv1xO+R1LEReFk6TwRAc1XW4NlpYP5W7lbqgQEtA4xYLrfAQJn0sQhSzNNpXgBCha9ev6VPL00B0biXEbSkw8rrEqnVVEqGAD48NsuAk22w/SyWV2VCk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782047008; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=CYPFoc7dqOuYPjj6y5CQXL5AQ+M6t8Hqk3YI7BStlXY=; b=P286Q8Y2fyF8axLR0sazro70FjEvZ1o8Xt+on8vTo0E3K42pizG7DM2JTRygS//nWIlP6PHSQ9CBzSR5CiL3LHPI3rwFD4Udbr1IzDlnPoceCHSYOsuWZa04Us1LiH+GzQynXCujTG59PCWSmYFF5FCW1q/5HzO6MvtHQQq7UyQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=linux.beauty; spf=pass smtp.mailfrom=me@linux.beauty; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1782047008; s=zmail; d=linux.beauty; i=me@linux.beauty; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Message-Id:Reply-To; bh=CYPFoc7dqOuYPjj6y5CQXL5AQ+M6t8Hqk3YI7BStlXY=; b=BjTE4sF58owwPt5mcxqhtyEOzUph1iU4BQSkKcYjxYjhr7QvE6fqNI+wyl+XbzAD 7zgvuwzDae75A7szdnlQAX+fcYrbqgVr289R9cKyagoJ4rftCCIHDDh22zo+XpwJxST NB3ZM6Nal8hqiN8tvJ3cWuPJdAUM6Ja+qFaNDM9g= Received: by mx.zohomail.com with SMTPS id 1782047006327302.10929894680703; Sun, 21 Jun 2026 06:03:26 -0700 (PDT) From: Li Chen To: Pankaj Gupta , Dan Williams , Vishal Verma , Dave Jiang , Ira Weiny , Alison Schofield , virtualization@lists.linux.dev, nvdimm@lists.linux.dev Cc: linux-kernel@vger.kernel.org, Li Chen Subject: [PATCH v6 07/12] nvdimm: virtio_pmem: use READ_ONCE()/WRITE_ONCE() for wait flags Date: Sun, 21 Jun 2026 21:02:38 +0800 Message-ID: <20260621130246.2973254-8-me@linux.beauty> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260621130246.2973254-1-me@linux.beauty> References: <20260621130246.2973254-1-me@linux.beauty> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Content-Type: text/plain; charset="utf-8" Use READ_ONCE()/WRITE_ONCE() for the wait_event() flags (done and wq_buf_avail). They are observed by waiters without pmem_lock, so make the accesses explicit single loads/stores and avoid compiler reordering/caching across the wait/wake paths. Signed-off-by: Li Chen --- v2->v3: - Split out READ_ONCE()/WRITE_ONCE() updates from patch 3/7 (no functional change intended). v3->v4: - Rebased onto v7.1-rc7 and renumbered after the flush error patches. drivers/nvdimm/nd_virtio.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/drivers/nvdimm/nd_virtio.c b/drivers/nvdimm/nd_virtio.c index 8ed4d6b3a9284..da829e9f4bdff 100644 --- a/drivers/nvdimm/nd_virtio.c +++ b/drivers/nvdimm/nd_virtio.c @@ -18,9 +18,9 @@ static void virtio_pmem_wake_one_waiter(struct virtio_pme= m *vpmem) =20 req_buf =3D list_first_entry(&vpmem->req_list, struct virtio_pmem_request, list); - req_buf->wq_buf_avail =3D true; + list_del_init(&req_buf->list); + WRITE_ONCE(req_buf->wq_buf_avail, true); wake_up(&req_buf->wq_buf); - list_del(&req_buf->list); } =20 /* The interrupt handler */ @@ -34,7 +34,7 @@ void virtio_pmem_host_ack(struct virtqueue *vq) spin_lock_irqsave(&vpmem->pmem_lock, flags); while ((req_data =3D virtqueue_get_buf(vq, &len)) !=3D NULL) { virtio_pmem_wake_one_waiter(vpmem); - req_data->done =3D true; + WRITE_ONCE(req_data->done, true); wake_up(&req_data->host_acked); } spin_unlock_irqrestore(&vpmem->pmem_lock, flags); @@ -66,7 +66,7 @@ static int virtio_pmem_flush(struct nd_region *nd_region) if (!req_data) return -ENOMEM; =20 - req_data->done =3D false; + WRITE_ONCE(req_data->done, false); init_waitqueue_head(&req_data->host_acked); init_waitqueue_head(&req_data->wq_buf); INIT_LIST_HEAD(&req_data->list); @@ -87,12 +87,12 @@ static int virtio_pmem_flush(struct nd_region *nd_regio= n) GFP_ATOMIC)) =3D=3D -ENOSPC) { =20 dev_info(&vdev->dev, "failed to send command to virtio pmem device, no f= ree slots in the virtqueue\n"); - req_data->wq_buf_avail =3D false; + WRITE_ONCE(req_data->wq_buf_avail, false); list_add_tail(&req_data->list, &vpmem->req_list); spin_unlock_irqrestore(&vpmem->pmem_lock, flags); =20 /* A host response results in "host_ack" getting called */ - wait_event(req_data->wq_buf, req_data->wq_buf_avail); + wait_event(req_data->wq_buf, READ_ONCE(req_data->wq_buf_avail)); spin_lock_irqsave(&vpmem->pmem_lock, flags); } err1 =3D virtqueue_kick(vpmem->req_vq); @@ -106,7 +106,7 @@ static int virtio_pmem_flush(struct nd_region *nd_regio= n) err =3D -EIO; } else { /* A host response results in "host_ack" getting called */ - wait_event(req_data->host_acked, req_data->done); + wait_event(req_data->host_acked, READ_ONCE(req_data->done)); err =3D le32_to_cpu(req_data->resp.ret); } =20 --=20 2.52.0 From nobody Tue Jun 23 01:23:34 2026 Received: from sender4-op-o15.zoho.com (sender4-op-o15.zoho.com [136.143.188.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CD9B2364EB8; Sun, 21 Jun 2026 13:03:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.15 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047033; cv=pass; b=HagsUEmWJu8KC9VIeDQUWBW4/t8vLOZq+f9c+bXGAj6lqf+cihBfqch0GAmM2SRjFOOSHCOCot0i7rQoyl+c0/qtqZx3+lfTMTdRBBN2LeV2HeVYYuqnZbLhzK7M+UrO9/KArZlZmraTfHSWiOrePsBjvjiFU20lj4HNgRgBUQ0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047033; c=relaxed/simple; bh=7PrAuHyBeAQBenZW4uSwuarZHrLxay6iu9szZXVdzzk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AdpjZJsouROY8/MIflFSiZ1psGdaVYeC7THuyTeF1JB0aojcTR1jrZBbULdFUKiMjwZPIYura1g8Z4CnEJ0mRZFDzFKVZgTcSL9P0nmApqOyKIZQq4/HdgP2wNARLL60Y2pdpCZspwC9nPER3YZswtBX3nl044hlftwZ8VtmJsc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty; spf=pass smtp.mailfrom=linux.beauty; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b=RhFuP8cw; arc=pass smtp.client-ip=136.143.188.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.beauty Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b="RhFuP8cw" ARC-Seal: i=1; a=rsa-sha256; t=1782047012; cv=none; d=zohomail.com; s=zohoarc; b=eOAd2mp+W7gVNnMPPBFafakukPmkQ7Bg69HT11sLRyM0a1NDDes2Uq+4fx7JCCIu0Do78u8LBcPXzLPlcULn98d7C3QJ30gYibD/PxrUPYK7qUntUqow4iHfZxktL7fCSYvYqERvqMTbFUaxwduPUmfEcwJCy+/ZEM+47m4moR4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782047012; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=YUNRo1JqI1ERS6Bx8k5dKxete2j3i20QxBJxNki2PEo=; b=E0tiiDN7z2Zc6Jr5MTLzScog1S7qHAZc8gfJpSGcu18UqPlYXMTgEvbp2nRKjn/sVQ1DZEMD0Ea1Wq+Qo2ti5LT5c/Y/3qyi9rIcY8j6sm8yZdw9ffwYQkdL+sLpYXKaFCfRl0jt3XDd3vBzeZusUXnxUaMx5mVHxl3dt/yHpbU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=linux.beauty; spf=pass smtp.mailfrom=me@linux.beauty; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1782047012; s=zmail; d=linux.beauty; i=me@linux.beauty; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Message-Id:Reply-To; bh=YUNRo1JqI1ERS6Bx8k5dKxete2j3i20QxBJxNki2PEo=; b=RhFuP8cwcLBNM8vpj4CyZo6sJliD4PQmP4aYwdHUB06+pVaBwUnWjbb00VrPpaA1 x9c0t04EpsWQSNhuB4bsi8D4Ic4+fYBXD1c9z/do3dG2YLNSDA3eSg/ETOPbWcoUM3i RGSF+JDenZ9Dsf+YTbzMl/7OqxeoWTPkYWhCAdiU= Received: by mx.zohomail.com with SMTPS id 1782047010077356.34651500703205; Sun, 21 Jun 2026 06:03:30 -0700 (PDT) From: Li Chen To: Pankaj Gupta , Dan Williams , Vishal Verma , Dave Jiang , Ira Weiny , Alison Schofield , virtualization@lists.linux.dev, nvdimm@lists.linux.dev Cc: linux-kernel@vger.kernel.org, stable@vger.kernel.org, Li Chen Subject: [PATCH v6 08/12] nvdimm: virtio_pmem: refcount requests for token lifetime Date: Sun, 21 Jun 2026 21:02:39 +0800 Message-ID: <20260621130246.2973254-9-me@linux.beauty> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260621130246.2973254-1-me@linux.beauty> References: <20260621130246.2973254-1-me@linux.beauty> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Content-Type: text/plain; charset="utf-8" KASAN reports slab-use-after-free in __wake_up_common(): BUG: KASAN: slab-use-after-free in __wake_up_common+0x114/0x160 Read of size 8 at addr ffff88810fdcb710 by task swapper/0/0 CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 6.19.0-next-20260220-00006-g1eae5f204ec3 #4 PREEMPT(full) Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.17.0-2-2 04/01/2014 Call Trace: dump_stack_lvl+0x6d/0xb0 print_report+0x170/0x4e2 ? __pfx__raw_spin_lock_irqsave+0x10/0x10 ? __virt_addr_valid+0x1dc/0x380 kasan_report+0xbc/0xf0 ? __wake_up_common+0x114/0x160 ? __wake_up_common+0x114/0x160 __wake_up_common+0x114/0x160 ? __pfx__raw_spin_lock_irqsave+0x10/0x10 __wake_up+0x36/0x60 virtio_pmem_host_ack+0x11d/0x3b0 ? sched_balance_domains+0x29f/0xb00 ? __pfx_virtio_pmem_host_ack+0x10/0x10 ? _raw_spin_lock_irqsave+0x98/0x100 ? __pfx__raw_spin_lock_irqsave+0x10/0x10 vring_interrupt+0x1c9/0x5e0 ? __pfx_vp_interrupt+0x10/0x10 vp_vring_interrupt+0x87/0x100 ? __pfx_vp_interrupt+0x10/0x10 __handle_irq_event_percpu+0x17f/0x550 ? __pfx__raw_spin_lock+0x10/0x10 handle_irq_event+0xab/0x1c0 handle_fasteoi_irq+0x276/0xae0 __common_interrupt+0x65/0x130 common_interrupt+0x78/0xa0 virtio_pmem_host_ack() wakes a request that has already been freed by the submitter. This happens when the request token is still reachable via the virtqueue, but virtio_pmem_flush() returns and frees it. Fix the token lifetime by refcounting struct virtio_pmem_request. virtio_pmem_flush() holds a submitter reference, and the virtqueue holds an extra reference once the request is queued. The completion path drops the virtqueue reference, and the submitter drops its reference before returning. Fixes: 6e84200c0a29 ("virtio-pmem: Add virtio pmem driver") Cc: stable@vger.kernel.org Signed-off-by: Li Chen --- v2->v3: - Add raw KASAN report to the patch description. - Drop timestamps from the embedded report. v3->v4: - Rebased onto v7.1-rc7 and renumbered after the flush error patches. drivers/nvdimm/nd_virtio.c | 34 +++++++++++++++++++++++++++++----- drivers/nvdimm/virtio_pmem.h | 2 ++ 2 files changed, 31 insertions(+), 5 deletions(-) diff --git a/drivers/nvdimm/nd_virtio.c b/drivers/nvdimm/nd_virtio.c index da829e9f4bdff..7b6761adf28bc 100644 --- a/drivers/nvdimm/nd_virtio.c +++ b/drivers/nvdimm/nd_virtio.c @@ -9,6 +9,14 @@ #include "virtio_pmem.h" #include "nd.h" =20 +static void virtio_pmem_req_release(struct kref *kref) +{ + struct virtio_pmem_request *req; + + req =3D container_of(kref, struct virtio_pmem_request, kref); + kfree(req); +} + static void virtio_pmem_wake_one_waiter(struct virtio_pmem *vpmem) { struct virtio_pmem_request *req_buf; @@ -36,6 +44,7 @@ void virtio_pmem_host_ack(struct virtqueue *vq) virtio_pmem_wake_one_waiter(vpmem); WRITE_ONCE(req_data->done, true); wake_up(&req_data->host_acked); + kref_put(&req_data->kref, virtio_pmem_req_release); } spin_unlock_irqrestore(&vpmem->pmem_lock, flags); } @@ -66,6 +75,7 @@ static int virtio_pmem_flush(struct nd_region *nd_region) if (!req_data) return -ENOMEM; =20 + kref_init(&req_data->kref); WRITE_ONCE(req_data->done, false); init_waitqueue_head(&req_data->host_acked); init_waitqueue_head(&req_data->wq_buf); @@ -83,10 +93,23 @@ static int virtio_pmem_flush(struct nd_region *nd_regio= n) * to req_list and wait for host_ack to wake us up when free * slots are available. */ - while ((err =3D virtqueue_add_sgs(vpmem->req_vq, sgs, 1, 1, req_data, - GFP_ATOMIC)) =3D=3D -ENOSPC) { - - dev_info(&vdev->dev, "failed to send command to virtio pmem device, no f= ree slots in the virtqueue\n"); + for (;;) { + err =3D virtqueue_add_sgs(vpmem->req_vq, sgs, 1, 1, req_data, + GFP_ATOMIC); + if (!err) { + /* + * Take the virtqueue reference while @pmem_lock is + * held so completion cannot run concurrently. + */ + kref_get(&req_data->kref); + break; + } + + if (err !=3D -ENOSPC) + break; + + dev_info_ratelimited(&vdev->dev, + "failed to send command to virtio pmem device, no free slots in t= he virtqueue\n"); WRITE_ONCE(req_data->wq_buf_avail, false); list_add_tail(&req_data->list, &vpmem->req_list); spin_unlock_irqrestore(&vpmem->pmem_lock, flags); @@ -95,6 +118,7 @@ static int virtio_pmem_flush(struct nd_region *nd_region) wait_event(req_data->wq_buf, READ_ONCE(req_data->wq_buf_avail)); spin_lock_irqsave(&vpmem->pmem_lock, flags); } + err1 =3D virtqueue_kick(vpmem->req_vq); spin_unlock_irqrestore(&vpmem->pmem_lock, flags); /* @@ -110,7 +134,7 @@ static int virtio_pmem_flush(struct nd_region *nd_regio= n) err =3D le32_to_cpu(req_data->resp.ret); } =20 - kfree(req_data); + kref_put(&req_data->kref, virtio_pmem_req_release); return err; }; =20 diff --git a/drivers/nvdimm/virtio_pmem.h b/drivers/nvdimm/virtio_pmem.h index f72cf17f9518f..1017e498c9b4c 100644 --- a/drivers/nvdimm/virtio_pmem.h +++ b/drivers/nvdimm/virtio_pmem.h @@ -12,11 +12,13 @@ =20 #include #include +#include #include #include #include =20 struct virtio_pmem_request { + struct kref kref; struct virtio_pmem_req req; struct virtio_pmem_resp resp; =20 --=20 2.52.0 From nobody Tue Jun 23 01:23:34 2026 Received: from sender4-op-o15.zoho.com (sender4-op-o15.zoho.com [136.143.188.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C019C36657D for ; Sun, 21 Jun 2026 13:03:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.15 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047037; cv=pass; b=SViEtCkpE2momU///oiGQijmNmcdq5UH/s7wPNJ204hqnAnbHz+NMPN9ZY3xfz21JoUdwM+9eURiYYMkSIl8lYhddBrtz0T7gw7RfQVJjaIrNcblfusEvaXdBXk6/YDu/Ho75+3VidVaCemNgrHt5lTo1A9bidiBqBCPdD3MZ6s= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047037; c=relaxed/simple; bh=7M9kmKBZp9rRu0tA2yaME8tBtzESVFG0IxssR+73ZAI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tMOPtPpyg+xotjh4DlxOYL025b0loeDHmgqOnXD++BRBkUu6lMAULiKx2VgjvBxunIOmOxuzeu1JOYVhc7IQJPiWR9KOaF8EzRmFppbMEpOXH3fpm+A8mV4I+S0VyyVIuJbhtq2ETU6u/3QHrg93BFS1upF9iVQ8XJzuG2I9Hdo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty; spf=pass smtp.mailfrom=linux.beauty; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b=EFmv781H; arc=pass smtp.client-ip=136.143.188.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.beauty Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b="EFmv781H" ARC-Seal: i=1; a=rsa-sha256; t=1782047016; cv=none; d=zohomail.com; s=zohoarc; b=f9E9qvTYWBNIBDkXm6fgw6tSiaHY/GxLNBJ03Uj9UiPyJPbXyjlv9phHJEYJvsBiLbg82GgVS5wJTZXmM0SKEzkYOpeG5XGID2DVA2Az+cxuXpBLIhJBeu4UKFWrr+7+OBwE4Od8ymL+orH+F9TtOdtm4BIvDqC0KkZGVSJjlrU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782047016; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=LZLBhHbmtjpySzZTarBL23J3mQcNrZwkvgLBIJ78RGM=; b=bAlad01sDqx1GLmgptCqPp1NLsmH+y41m3zan5lt70cBWIFgxgYg/mkrIcuIufj7JKkYLHqgT9lUzY0Pu4vPRlZkxA7UBYAiyOSm4tqSt52GD3dFAt2iMZ5dad7g9R2JARjxq6v1eY5mDi1QcOAdDGrlxVwY86NHsJ7SlpMwioA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=linux.beauty; spf=pass smtp.mailfrom=me@linux.beauty; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1782047016; s=zmail; d=linux.beauty; i=me@linux.beauty; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Message-Id:Reply-To; bh=LZLBhHbmtjpySzZTarBL23J3mQcNrZwkvgLBIJ78RGM=; b=EFmv781HFfJZxTBIGc5tZUGUme4fRFChURsdzCnruo146STntdNbJDjOoLv5fwW7 vtM5gyRbD2Xnwn4ydJD/c5ZUe5rExCE2vlWfhPCpxUxlXiEz8YlZmhIedOfcg4LBfPR S7VUGj5aFDr5Y51D1uvCRSXM7T7mHquUEi+1b6sQ= Received: by mx.zohomail.com with SMTPS id 1782047013626864.9869101777588; Sun, 21 Jun 2026 06:03:33 -0700 (PDT) From: Li Chen To: Pankaj Gupta , Dan Williams , Vishal Verma , Dave Jiang , Ira Weiny , Alison Schofield , virtualization@lists.linux.dev, nvdimm@lists.linux.dev Cc: linux-kernel@vger.kernel.org, Li Chen Subject: [PATCH v6 09/12] nvdimm: virtio_pmem: publish done with release/acquire Date: Sun, 21 Jun 2026 21:02:40 +0800 Message-ID: <20260621130246.2973254-10-me@linux.beauty> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260621130246.2973254-1-me@linux.beauty> References: <20260621130246.2973254-1-me@linux.beauty> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Content-Type: text/plain; charset="utf-8" virtio_pmem_host_ack() publishes the device response by setting done and waking the submitter. The submitter reads resp.ret after wait_event() observes done. Use smp_store_release() on done and smp_load_acquire() in the wait condition so the response read is ordered after completion. Signed-off-by: Li Chen --- Changes in v6: - New patch. drivers/nvdimm/nd_virtio.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/drivers/nvdimm/nd_virtio.c b/drivers/nvdimm/nd_virtio.c index 7b6761adf28bc..35d36bd36a526 100644 --- a/drivers/nvdimm/nd_virtio.c +++ b/drivers/nvdimm/nd_virtio.c @@ -17,6 +17,19 @@ static void virtio_pmem_req_release(struct kref *kref) kfree(req); } =20 +static void virtio_pmem_signal_done(struct virtio_pmem_request *req) +{ + /* Pairs with smp_load_acquire() in virtio_pmem_req_done(). */ + smp_store_release(&req->done, true); + wake_up(&req->host_acked); +} + +static bool virtio_pmem_req_done(struct virtio_pmem_request *req) +{ + /* Pairs with smp_store_release() in virtio_pmem_signal_done(). */ + return smp_load_acquire(&req->done); +} + static void virtio_pmem_wake_one_waiter(struct virtio_pmem *vpmem) { struct virtio_pmem_request *req_buf; @@ -42,8 +55,7 @@ void virtio_pmem_host_ack(struct virtqueue *vq) spin_lock_irqsave(&vpmem->pmem_lock, flags); while ((req_data =3D virtqueue_get_buf(vq, &len)) !=3D NULL) { virtio_pmem_wake_one_waiter(vpmem); - WRITE_ONCE(req_data->done, true); - wake_up(&req_data->host_acked); + virtio_pmem_signal_done(req_data); kref_put(&req_data->kref, virtio_pmem_req_release); } spin_unlock_irqrestore(&vpmem->pmem_lock, flags); @@ -130,7 +142,8 @@ static int virtio_pmem_flush(struct nd_region *nd_regio= n) err =3D -EIO; } else { /* A host response results in "host_ack" getting called */ - wait_event(req_data->host_acked, READ_ONCE(req_data->done)); + wait_event(req_data->host_acked, + virtio_pmem_req_done(req_data)); err =3D le32_to_cpu(req_data->resp.ret); } =20 --=20 2.52.0 From nobody Tue Jun 23 01:23:34 2026 Received: from sender4-op-o15.zoho.com (sender4-op-o15.zoho.com [136.143.188.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7405E367282 for ; Sun, 21 Jun 2026 13:04:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.15 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047042; cv=pass; b=VrkYrGLF7NvMbIHRNYPGlL46ALRek1F1Ia7WVD+cT2gW5EY90m3gcBmftRm9GISLTkyZtI0/m04ozV1F4nSZdPatsrg9PeO1MQeHxwcCRMdBjOKa5NYvtOYcV2KEoLgwmhm3j3C+JlupIWrRuEjb88ZlCP5SwX6f9oJMDq6Jj7U= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047042; c=relaxed/simple; bh=AuZG6eLVxtRjnPbYcuX0f0Nm13o7MKcCVWJRxui5OnI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FvXDrl6TL+3si4bZyo4fWe27bGOUx09pp4I+cV0L1Z9FeNGPPlXAKnhsL7QUwn2d3gzaoqBHcFMdmA7c5J+TLsS6hG8lobwNGrankcS9zuvBXgTCTR5tIcZbEF65fXIfkfmXQxMdsxpgL6ZXBW2e997+dMp47pgsLyLSzI5hZHc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty; spf=pass smtp.mailfrom=linux.beauty; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b=ek8+MBuj; arc=pass smtp.client-ip=136.143.188.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.beauty Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b="ek8+MBuj" ARC-Seal: i=1; a=rsa-sha256; t=1782047020; cv=none; d=zohomail.com; s=zohoarc; b=bfaJ9l7nNjq+fyqJ7vWYfioo80HaaNE6oCdLgYWe5PQ6HN1R338bVXRgVj0ObM9SAw3njsy6HaktJOOPvmQ36EchKPvysT6aOa8RIVvRc+Q2oeKACIuL6ht0PQGiIcnvCiCEX3uKUw/FwA8EkGom0Vd/nue+M2I0HmR8BBnFo7o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782047020; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=MEsva7hJtbMtZ8GDT7P31U3EHKfjcwNJzPN0mOdKvuQ=; b=Q0v1Z0EPoeOBBRVAkFPKlnWbKTf6wtvTryHRwnNhD2QWE8aHU2xIvPJi0xndoNKpP6c3GYm3eJZry7Oi+rkOu5mhHJI/hZI8aOsHQV6U8rllq1DOQDuY3GpvzKv/Yk8GLVw/dV72rpG/hxJRvHSEEPuSrfHPTDZgKwuyW/iUm+w= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=linux.beauty; spf=pass smtp.mailfrom=me@linux.beauty; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1782047020; s=zmail; d=linux.beauty; i=me@linux.beauty; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Message-Id:Reply-To; bh=MEsva7hJtbMtZ8GDT7P31U3EHKfjcwNJzPN0mOdKvuQ=; b=ek8+MBujHE+ItXMDGilx144xNRG0Uin7RXIFskcKk80FN4AzQRFvZFH+KZrtRgOG HzotoXnFF2nhh9u1R8730EKjekj9Tps1C7P2vX+5+4MsFbQr5TXgyKJvICdE9fVj4OQ 49wo26f6lVC9VjVaI5eWukBl2eiYE4wHqXt0f/iI= Received: by mx.zohomail.com with SMTPS id 1782047017371275.4613699500153; Sun, 21 Jun 2026 06:03:37 -0700 (PDT) From: Li Chen To: Pankaj Gupta , Dan Williams , Vishal Verma , Dave Jiang , Ira Weiny , Alison Schofield , virtualization@lists.linux.dev, nvdimm@lists.linux.dev Cc: linux-kernel@vger.kernel.org, Li Chen Subject: [PATCH v6 10/12] nvdimm: virtio_pmem: isolate DMA request buffers Date: Sun, 21 Jun 2026 21:02:41 +0800 Message-ID: <20260621130246.2973254-11-me@linux.beauty> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260621130246.2973254-1-me@linux.beauty> References: <20260621130246.2973254-1-me@linux.beauty> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Content-Type: text/plain; charset="utf-8" The virtio-pmem request object stores wait queues, flags, and list pointers next to buffers mapped for virtqueue DMA. The response buffer is mapped DMA_FROM_DEVICE, so non-coherent DMA invalidation must not share a cache line with CPU-owned fields. Keep the request buffer outside the DMA-from-device group and wrap only the response buffer with __dma_from_device_group_begin/end. Signed-off-by: Li Chen --- Changes in v6: - New patch. drivers/nvdimm/virtio_pmem.h | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/nvdimm/virtio_pmem.h b/drivers/nvdimm/virtio_pmem.h index 1017e498c9b4c..23bff40249c1b 100644 --- a/drivers/nvdimm/virtio_pmem.h +++ b/drivers/nvdimm/virtio_pmem.h @@ -10,6 +10,7 @@ #ifndef _LINUX_VIRTIO_PMEM_H #define _LINUX_VIRTIO_PMEM_H =20 +#include #include #include #include @@ -19,8 +20,6 @@ =20 struct virtio_pmem_request { struct kref kref; - struct virtio_pmem_req req; - struct virtio_pmem_resp resp; =20 /* Wait queue to process deferred work after ack from host */ wait_queue_head_t host_acked; @@ -30,6 +29,11 @@ struct virtio_pmem_request { wait_queue_head_t wq_buf; bool wq_buf_avail; struct list_head list; + + struct virtio_pmem_req req; + __dma_from_device_group_begin(resp); + struct virtio_pmem_resp resp; + __dma_from_device_group_end(resp); }; =20 struct virtio_pmem { --=20 2.52.0 From nobody Tue Jun 23 01:23:34 2026 Received: from sender4-op-o15.zoho.com (sender4-op-o15.zoho.com [136.143.188.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B26C368D6D for ; Sun, 21 Jun 2026 13:04:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.15 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047047; cv=pass; b=g1IJ0hZrAa+J7x8Vda/1lgkkYGxA5US2+R2CWrLHR+KtZ7SkJ14knRod7li5XMgmukJP9t3NrNbfF2uwW3VP0rLbA9DvUHxiN7dOAgAMldCDlCj+UcnRt3e5HChJ1XZFkbQg7zN8JgDHOSh3vveIVhmnr4p8n3zwiN0CgcX887o= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047047; c=relaxed/simple; bh=H/nNiBO6U7u4CqbY9F0Hbyz60Ya4J0OQzrwsizH01gM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=su8lm9pWL8BXT/MdTERK5eA5LUxoS2MxeoUdTNha9xSzBH1zBnzims3tGNBpniiqAzvBg7WYXwpHG70Ul44u3A38IN8Vk4ct1LSKL+jgejDlqOTN4FsaYQp6HDFm2ZPMP+/L9zvppb9DboJJkrWeVkEtCBZO5y07jgk96u4QjG4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty; spf=pass smtp.mailfrom=linux.beauty; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b=oIvIg09i; arc=pass smtp.client-ip=136.143.188.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.beauty Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b="oIvIg09i" ARC-Seal: i=1; a=rsa-sha256; t=1782047022; cv=none; d=zohomail.com; s=zohoarc; b=iz3Apb/Z2yhuisnKHCdJgZKQqOT0Sn3Fx0CWRdkKHm9n5uhLYERixVNsX+LfoI3ikdEG5uT94XXognSF2/TN9vHsGYVAZMN4U60x8P8HOd000TAQdmU30JQt2xq3sBg6Fj49tlG5secKc5X/m/pNeA5l9O0DiRCKpNft9RRfsdM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782047022; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=xYqJhk551PY84Z5EkDORmsyNfqwhCUkqdlmRz3F67IQ=; b=IQ86AvVgUKgibRTRpqO+kazfiTIUK0+vdlVGLGrjE180cDsfRSbjk8y4KjmkB1QWqkhBDgt4D6j1Ez9k8G9DnAg1p7rxWMN5NHlo+3xF0j/qNVH6wmrS0cTGAZguNIbNy1v+mHM7RNA1P04j6gLVFGsI5Xo8hvzgigll3wM3nHI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=linux.beauty; spf=pass smtp.mailfrom=me@linux.beauty; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1782047022; s=zmail; d=linux.beauty; i=me@linux.beauty; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Message-Id:Reply-To; bh=xYqJhk551PY84Z5EkDORmsyNfqwhCUkqdlmRz3F67IQ=; b=oIvIg09i6WpbQvxfBk4MpZa9xP7xTf9q47eH9FJFS2RmXv/xRVwE+Vzy2egEfJko SqxX9cJy36EdpYoYupAivLLPeoVhvx2XiyRH4iAeBMzmGVwAiH+xebgxSR7ZOV08sgL ecs+FXMBqW3VrKoTiUtq2AYTQgNDNLbelmTwZCg4= Received: by mx.zohomail.com with SMTPS id 1782047020733430.6074913642492; Sun, 21 Jun 2026 06:03:40 -0700 (PDT) From: Li Chen To: Pankaj Gupta , Dan Williams , Vishal Verma , Dave Jiang , Ira Weiny , Alison Schofield , virtualization@lists.linux.dev, nvdimm@lists.linux.dev Cc: linux-kernel@vger.kernel.org, Li Chen Subject: [PATCH v6 11/12] nvdimm: virtio_pmem: converge broken virtqueue to -EIO Date: Sun, 21 Jun 2026 21:02:42 +0800 Message-ID: <20260621130246.2973254-12-me@linux.beauty> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260621130246.2973254-1-me@linux.beauty> References: <20260621130246.2973254-1-me@linux.beauty> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Content-Type: text/plain; charset="utf-8" dmesg reports virtqueue failure and device reset: virtio_pmem virtio2: failed to send command to virtio pmem device, no free slots in the virtqueue virtio_pmem virtio2: virtio pmem device needs a reset virtio_pmem_flush() can wait for a free virtqueue descriptor (-ENOSPC). It can also wait for host completion. If the request virtqueue breaks, those waiters may never make progress. One example is notify failure from virtqueue_kick(). Track a device-level broken state and converge the failure to -EIO. New requests fail fast, -ENOSPC waiters are unlinked and woken, and the currently submitted request is woken so its host_acked waiter can return without waiting forever for host completion. Completed requests are forced to report an error after the queue is marked broken. Do not detach unused buffers from an active virtqueue. Runtime broken-queue handling only stops new submissions and wakes local waiters. Removal resets the device first. It then drains request tokens. After that, the device no longer owns the buffers when the virtqueue reference is dropped. Closes: https://lore.kernel.org/r/202512250116.ewtzlD0g-lkp@intel.com/ Signed-off-by: Li Chen --- Changes in v6: - Wake the in-flight host-completion waiter when marking the queue broken. - Track req_inflight and clear it on completion/drain paths. - Return -EIO if the queue breaks before a host response is observed. Changes in v5: - Split broken marking from token draining. - Do not call virtqueue_detach_unused_buf() on an active queue. - Reset the device before draining tokens in remove(). - Do not let the host-completion wait return only because the device is marked broken. v2->v3: - Add raw dmesg excerpt to the patch description. - Drop timestamps from the embedded dmesg. - Fold the CONFIG_VIRTIO_PMEM=3Dm export fix into this patch. v3->v4: - Rebased onto v7.1-rc7 and renumbered after the flush error patches. - Use kmalloc_obj(*req_data) at the allocation site to match current nvdimm code. drivers/nvdimm/nd_virtio.c | 117 +++++++++++++++++++++++++++++++---- drivers/nvdimm/virtio_pmem.c | 15 ++++- drivers/nvdimm/virtio_pmem.h | 8 +++ 3 files changed, 127 insertions(+), 13 deletions(-) diff --git a/drivers/nvdimm/nd_virtio.c b/drivers/nvdimm/nd_virtio.c index 35d36bd36a526..fb9391ebc46e7 100644 --- a/drivers/nvdimm/nd_virtio.c +++ b/drivers/nvdimm/nd_virtio.c @@ -30,6 +30,12 @@ static bool virtio_pmem_req_done(struct virtio_pmem_requ= est *req) return smp_load_acquire(&req->done); } =20 +static void virtio_pmem_complete_err(struct virtio_pmem_request *req) +{ + req->resp.ret =3D cpu_to_le32(1); + virtio_pmem_signal_done(req); +} + static void virtio_pmem_wake_one_waiter(struct virtio_pmem *vpmem) { struct virtio_pmem_request *req_buf; @@ -44,6 +50,63 @@ static void virtio_pmem_wake_one_waiter(struct virtio_pm= em *vpmem) wake_up(&req_buf->wq_buf); } =20 +static void virtio_pmem_wake_all_waiters(struct virtio_pmem *vpmem) +{ + struct virtio_pmem_request *req, *tmp; + + list_for_each_entry_safe(req, tmp, &vpmem->req_list, list) { + list_del_init(&req->list); + WRITE_ONCE(req->wq_buf_avail, true); + wake_up(&req->wq_buf); + } +} + +static void virtio_pmem_clear_inflight(struct virtio_pmem *vpmem, + struct virtio_pmem_request *req) +{ + if (vpmem->req_inflight =3D=3D req) + vpmem->req_inflight =3D NULL; +} + +static void virtio_pmem_wake_inflight(struct virtio_pmem *vpmem) +{ + struct virtio_pmem_request *req =3D vpmem->req_inflight; + + if (req) + wake_up(&req->host_acked); +} + +void virtio_pmem_mark_broken(struct virtio_pmem *vpmem) +{ + if (!READ_ONCE(vpmem->broken)) { + WRITE_ONCE(vpmem->broken, true); + dev_err_once(&vpmem->vdev->dev, "virtqueue is broken\n"); + } + + virtio_pmem_wake_inflight(vpmem); + virtio_pmem_wake_all_waiters(vpmem); +} +EXPORT_SYMBOL_GPL(virtio_pmem_mark_broken); + +void virtio_pmem_drain(struct virtio_pmem *vpmem) +{ + struct virtio_pmem_request *req; + unsigned int len; + + while ((req =3D virtqueue_get_buf(vpmem->req_vq, &len)) !=3D NULL) { + virtio_pmem_clear_inflight(vpmem, req); + virtio_pmem_complete_err(req); + kref_put(&req->kref, virtio_pmem_req_release); + } + + while ((req =3D virtqueue_detach_unused_buf(vpmem->req_vq)) !=3D NULL) { + virtio_pmem_clear_inflight(vpmem, req); + virtio_pmem_complete_err(req); + kref_put(&req->kref, virtio_pmem_req_release); + } +} +EXPORT_SYMBOL_GPL(virtio_pmem_drain); + /* The interrupt handler */ void virtio_pmem_host_ack(struct virtqueue *vq) { @@ -54,8 +117,12 @@ void virtio_pmem_host_ack(struct virtqueue *vq) =20 spin_lock_irqsave(&vpmem->pmem_lock, flags); while ((req_data =3D virtqueue_get_buf(vq, &len)) !=3D NULL) { + virtio_pmem_clear_inflight(vpmem, req_data); virtio_pmem_wake_one_waiter(vpmem); - virtio_pmem_signal_done(req_data); + if (READ_ONCE(vpmem->broken)) + virtio_pmem_complete_err(req_data); + else + virtio_pmem_signal_done(req_data); kref_put(&req_data->kref, virtio_pmem_req_release); } spin_unlock_irqrestore(&vpmem->pmem_lock, flags); @@ -83,6 +150,9 @@ static int virtio_pmem_flush(struct nd_region *nd_region) return -EIO; } =20 + if (READ_ONCE(vpmem->broken)) + return -EIO; + req_data =3D kmalloc_obj(*req_data, GFP_NOIO); if (!req_data) return -ENOMEM; @@ -99,13 +169,18 @@ static int virtio_pmem_flush(struct nd_region *nd_regi= on) sgs[1] =3D &ret; =20 spin_lock_irqsave(&vpmem->pmem_lock, flags); - /* - * If virtqueue_add_sgs returns -ENOSPC then req_vq virtual - * queue does not have free descriptor. We add the request - * to req_list and wait for host_ack to wake us up when free - * slots are available. - */ + /* + * If virtqueue_add_sgs returns -ENOSPC then req_vq virtual + * queue does not have free descriptor. We add the request + * to req_list and wait for host_ack to wake us up when free + * slots are available. + */ for (;;) { + if (READ_ONCE(vpmem->broken)) { + err =3D -EIO; + break; + } + err =3D virtqueue_add_sgs(vpmem->req_vq, sgs, 1, 1, req_data, GFP_ATOMIC); if (!err) { @@ -114,6 +189,7 @@ static int virtio_pmem_flush(struct nd_region *nd_regio= n) * held so completion cannot run concurrently. */ kref_get(&req_data->kref); + vpmem->req_inflight =3D req_data; break; } =20 @@ -127,24 +203,41 @@ static int virtio_pmem_flush(struct nd_region *nd_reg= ion) spin_unlock_irqrestore(&vpmem->pmem_lock, flags); =20 /* A host response results in "host_ack" getting called */ - wait_event(req_data->wq_buf, READ_ONCE(req_data->wq_buf_avail)); + wait_event(req_data->wq_buf, + READ_ONCE(req_data->wq_buf_avail) || + READ_ONCE(vpmem->broken)); spin_lock_irqsave(&vpmem->pmem_lock, flags); + + if (READ_ONCE(vpmem->broken)) + break; } =20 - err1 =3D virtqueue_kick(vpmem->req_vq); + if (err =3D=3D -EIO || virtqueue_is_broken(vpmem->req_vq)) + virtio_pmem_mark_broken(vpmem); + + err1 =3D true; + if (!err && !READ_ONCE(vpmem->broken)) { + err1 =3D virtqueue_kick(vpmem->req_vq); + if (!err1) + virtio_pmem_mark_broken(vpmem); + } spin_unlock_irqrestore(&vpmem->pmem_lock, flags); /* * virtqueue_add_sgs failed with error different than -ENOSPC, we can't * do anything about that. */ - if (err || !err1) { + if (READ_ONCE(vpmem->broken) || err || !err1) { dev_info(&vdev->dev, "failed to send command to virtio pmem device\n"); err =3D -EIO; } else { /* A host response results in "host_ack" getting called */ wait_event(req_data->host_acked, - virtio_pmem_req_done(req_data)); - err =3D le32_to_cpu(req_data->resp.ret); + virtio_pmem_req_done(req_data) || + READ_ONCE(vpmem->broken)); + if (virtio_pmem_req_done(req_data)) + err =3D le32_to_cpu(req_data->resp.ret); + else + err =3D -EIO; } =20 kref_put(&req_data->kref, virtio_pmem_req_release); diff --git a/drivers/nvdimm/virtio_pmem.c b/drivers/nvdimm/virtio_pmem.c index 77b1966619059..b272e9279ef23 100644 --- a/drivers/nvdimm/virtio_pmem.c +++ b/drivers/nvdimm/virtio_pmem.c @@ -25,6 +25,8 @@ static int init_vq(struct virtio_pmem *vpmem) =20 spin_lock_init(&vpmem->pmem_lock); INIT_LIST_HEAD(&vpmem->req_list); + vpmem->req_inflight =3D NULL; + WRITE_ONCE(vpmem->broken, false); =20 return 0; }; @@ -138,10 +140,21 @@ static int virtio_pmem_probe(struct virtio_device *vd= ev) static void virtio_pmem_remove(struct virtio_device *vdev) { struct nvdimm_bus *nvdimm_bus =3D dev_get_drvdata(&vdev->dev); + struct virtio_pmem *vpmem =3D vdev->priv; + unsigned long flags; + + spin_lock_irqsave(&vpmem->pmem_lock, flags); + virtio_pmem_mark_broken(vpmem); + spin_unlock_irqrestore(&vpmem->pmem_lock, flags); + + virtio_reset_device(vdev); + + spin_lock_irqsave(&vpmem->pmem_lock, flags); + virtio_pmem_drain(vpmem); + spin_unlock_irqrestore(&vpmem->pmem_lock, flags); =20 nvdimm_bus_unregister(nvdimm_bus); vdev->config->del_vqs(vdev); - virtio_reset_device(vdev); } =20 static int virtio_pmem_freeze(struct virtio_device *vdev) diff --git a/drivers/nvdimm/virtio_pmem.h b/drivers/nvdimm/virtio_pmem.h index 23bff40249c1b..bc7de2b328985 100644 --- a/drivers/nvdimm/virtio_pmem.h +++ b/drivers/nvdimm/virtio_pmem.h @@ -52,6 +52,12 @@ struct virtio_pmem { /* List to store deferred work if virtqueue is full */ struct list_head req_list; =20 + /* Request currently owned by the virtqueue. */ + struct virtio_pmem_request *req_inflight; + + /* Fail fast and wake waiters if the request virtqueue is broken. */ + bool broken; + /* Synchronize virtqueue data */ spinlock_t pmem_lock; =20 @@ -61,5 +67,7 @@ struct virtio_pmem { }; =20 void virtio_pmem_host_ack(struct virtqueue *vq); +void virtio_pmem_mark_broken(struct virtio_pmem *vpmem); +void virtio_pmem_drain(struct virtio_pmem *vpmem); int async_pmem_flush(struct nd_region *nd_region, struct bio *bio); #endif --=20 2.52.0 From nobody Tue Jun 23 01:23:34 2026 Received: from sender4-op-o15.zoho.com (sender4-op-o15.zoho.com [136.143.188.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 472F0367B72 for ; Sun, 21 Jun 2026 13:04:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.15 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047052; cv=pass; b=iYC/9xRHrhxxhSfPsVpxISrx5LRCdND2wn5MmTez9BOPGT6cf+kadRr3oUv2Cl1u4r5qgQaNNm7reMHXi/5XIdP7DAu6tYjg/HsZExL2FJCZ/hal1m/3EqCJrtkMf5Hc43JouqgbySb5PMkyF5uaP9CtY0IKFNZXH3RviDuZ90I= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782047052; c=relaxed/simple; bh=6A9bWkf8F7xrUAnVlnOMf683ur53Qgpma7ZseFnzL+4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Wlsm+haRH3VVerobvxYlqPzKkcpB1slZ0Z+EhSxeGBaC/DKXh0FDhTA2QgVGOKgMaBrsiMH6Hqlx6RRqksmiiwyQyg1p9bq6YKH9je9TI8Yr9Gxziy0nyp/+WpiTHF4JDSeDMlgGKgH9/60YnoR1LDg2Bucs5oAx7wQZ9/KLX1k= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty; spf=pass smtp.mailfrom=linux.beauty; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b=EG3c/+KY; arc=pass smtp.client-ip=136.143.188.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.beauty Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.beauty Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b="EG3c/+KY" ARC-Seal: i=1; a=rsa-sha256; t=1782047026; cv=none; d=zohomail.com; s=zohoarc; b=gnmhKO7qHPV2bZVslIlMV9Qkik4X581pkAUerMN9G6E8eooGvcRJu1TpGeBoEu7mFxJ2a49iKbHCI9HPKV4r0fbwg9PzlxAR2e2Lrspu75BbQ0CAF4BN7A78y7s68W6h2xTKroIjnCs2elKjuQDmGQjZhPWvTaGo+hKLMbMxCIE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782047026; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=+WTLNJnsHO7DCEChsDnpLBkOw/OLyz+oCtQwOwb1Nvo=; b=BW5SevhwVLkI1gZmXuObn9topz2pUxEYTlUkgrTtgoYGLVWKIAWB9lSv/kXSLkaP1DueHLV0JXf+wvbWxUT105uGv9OVyzfZqYGoZft45jq/d1nttc7qgr7Ds0z9Ea5zOrMVIgKp9x/CTyU2PlQfhtzD61dLxh5Xhjrd4hBTXjQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=linux.beauty; spf=pass smtp.mailfrom=me@linux.beauty; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1782047026; s=zmail; d=linux.beauty; i=me@linux.beauty; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Message-Id:Reply-To; bh=+WTLNJnsHO7DCEChsDnpLBkOw/OLyz+oCtQwOwb1Nvo=; b=EG3c/+KYzqyCPkV1fWwGBj2aDC405wJTIJhIJVGYAO/O6vn3hhaL24+zbWW/gsjp 7eW7EKu9gq4xnLN5gPNGktzkEyBfL4v3raaSZY7X9NRnfOWRSmveEVt7N/UrCJdaccA 4p47FD1LdP9iueHkj5U8hI9WUmKLe4hHwBfGKN30= Received: by mx.zohomail.com with SMTPS id 1782047024089538.331592221489; Sun, 21 Jun 2026 06:03:44 -0700 (PDT) From: Li Chen To: Pankaj Gupta , Dan Williams , Vishal Verma , Dave Jiang , Ira Weiny , Alison Schofield , virtualization@lists.linux.dev, nvdimm@lists.linux.dev Cc: linux-kernel@vger.kernel.org, Li Chen Subject: [PATCH v6 12/12] nvdimm: virtio_pmem: drain requests in freeze Date: Sun, 21 Jun 2026 21:02:43 +0800 Message-ID: <20260621130246.2973254-13-me@linux.beauty> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260621130246.2973254-1-me@linux.beauty> References: <20260621130246.2973254-1-me@linux.beauty> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Content-Type: text/plain; charset="utf-8" virtio_pmem_freeze() currently deletes virtqueues and resets the device without waking threads waiting for a virtqueue descriptor or a host completion. Mark the request virtqueue broken before reset. This makes new submissions fail fast and lets -ENOSPC waiters leave the wait list. Reset the device before draining used and unused request tokens, then delete the virtqueues. This wakes waiters with -EIO. It also keeps the detach call on a quiesced device. Clear req_vq after del_vqs(), and make drain tolerate a NULL queue, so remove after freeze does not dereference a stale virtqueue pointer. Signed-off-by: Li Chen --- Changes in v6: - Clear req_vq after del_vqs() and make drain tolerate a NULL queue. Changes in v5: - Reset the device before draining used and unused request tokens. - Use the split broken-marking and post-reset drain helpers. v2->v3: - No change. v3->v4: - Rebased onto v7.1-rc7 and renumbered after the flush error patches. drivers/nvdimm/nd_virtio.c | 3 +++ drivers/nvdimm/virtio_pmem.c | 36 +++++++++++++++++++++++++++++++----- 2 files changed, 34 insertions(+), 5 deletions(-) diff --git a/drivers/nvdimm/nd_virtio.c b/drivers/nvdimm/nd_virtio.c index fb9391ebc46e7..ce4032dc07628 100644 --- a/drivers/nvdimm/nd_virtio.c +++ b/drivers/nvdimm/nd_virtio.c @@ -93,6 +93,9 @@ void virtio_pmem_drain(struct virtio_pmem *vpmem) struct virtio_pmem_request *req; unsigned int len; =20 + if (!vpmem->req_vq) + return; + while ((req =3D virtqueue_get_buf(vpmem->req_vq, &len)) !=3D NULL) { virtio_pmem_clear_inflight(vpmem, req); virtio_pmem_complete_err(req); diff --git a/drivers/nvdimm/virtio_pmem.c b/drivers/nvdimm/virtio_pmem.c index b272e9279ef23..fef792f725db2 100644 --- a/drivers/nvdimm/virtio_pmem.c +++ b/drivers/nvdimm/virtio_pmem.c @@ -17,11 +17,16 @@ static struct virtio_device_id id_table[] =3D { /* Initialize virt queue */ static int init_vq(struct virtio_pmem *vpmem) { + int err; + /* single vq */ vpmem->req_vq =3D virtio_find_single_vq(vpmem->vdev, virtio_pmem_host_ack, "flush_queue"); - if (IS_ERR(vpmem->req_vq)) - return PTR_ERR(vpmem->req_vq); + if (IS_ERR(vpmem->req_vq)) { + err =3D PTR_ERR(vpmem->req_vq); + vpmem->req_vq =3D NULL; + return err; + } =20 spin_lock_init(&vpmem->pmem_lock); INIT_LIST_HEAD(&vpmem->req_list); @@ -31,6 +36,15 @@ static int init_vq(struct virtio_pmem *vpmem) return 0; }; =20 +static void virtio_pmem_del_vqs(struct virtio_pmem *vpmem) +{ + if (!vpmem->req_vq) + return; + + vpmem->vdev->config->del_vqs(vpmem->vdev); + vpmem->req_vq =3D NULL; +} + static int virtio_pmem_validate(struct virtio_device *vdev) { struct virtio_shm_region shm_reg; @@ -132,7 +146,7 @@ static int virtio_pmem_probe(struct virtio_device *vdev) virtio_reset_device(vdev); nvdimm_bus_unregister(vpmem->nvdimm_bus); out_vq: - vdev->config->del_vqs(vdev); + virtio_pmem_del_vqs(vpmem); out_err: return err; } @@ -154,14 +168,26 @@ static void virtio_pmem_remove(struct virtio_device *= vdev) spin_unlock_irqrestore(&vpmem->pmem_lock, flags); =20 nvdimm_bus_unregister(nvdimm_bus); - vdev->config->del_vqs(vdev); + virtio_pmem_del_vqs(vpmem); } =20 static int virtio_pmem_freeze(struct virtio_device *vdev) { - vdev->config->del_vqs(vdev); + struct virtio_pmem *vpmem =3D vdev->priv; + unsigned long flags; + + spin_lock_irqsave(&vpmem->pmem_lock, flags); + virtio_pmem_mark_broken(vpmem); + spin_unlock_irqrestore(&vpmem->pmem_lock, flags); + virtio_reset_device(vdev); =20 + spin_lock_irqsave(&vpmem->pmem_lock, flags); + virtio_pmem_drain(vpmem); + spin_unlock_irqrestore(&vpmem->pmem_lock, flags); + + virtio_pmem_del_vqs(vpmem); + return 0; } =20 --=20 2.52.0