From nobody Sun Feb 8 02:26:28 2026 Received: from sender4-pp-f112.zoho.com (sender4-pp-f112.zoho.com [136.143.188.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0A5962EA731 for ; Sat, 20 Dec 2025 08:35:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.112 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766219704; cv=pass; b=FlpPuR2ng6C9i2n8aVB/MCqPZksJD0Rs3v5/ktgdzqT4LJxdDMVEVHCLE9tGie1frfKYpBMx5i5UGYE/pHqmOI51YJNoi9V89dQy0YXn3pCIQezXpHeqiQKcYySgqnjPpElC/wge0a1De50RUPhSg6LyupVUb6zYpo1O7suyAIM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766219704; c=relaxed/simple; bh=UpBQjzOv2zWJm0MiE40rFexes1ffQA8ZykCSkFpddhA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=N/oM4doityU/tJ0ce4LAKgu18n/MdYPkSQ20vFUmGAeKY6+dcfVBXq54S0Pl31kTAyKjnFY6jOZCXWvwPgRwtJbNiaS88P67JquXDloAMhXKsgp7AYBDdwsKaLiqMaO2sZQxKKo1Iy0M3WntQlvbiIK3sEmMrUI6SkwRUoHDti0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=linux.beauty; spf=pass smtp.mailfrom=linux.beauty; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b=sgu8AE7P; arc=pass smtp.client-ip=136.143.188.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=linux.beauty Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.beauty Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b="sgu8AE7P" ARC-Seal: i=1; a=rsa-sha256; t=1766219694; cv=none; d=zohomail.com; s=zohoarc; b=F6xkMlVw2L1Bipdj2Ma32EA2LTgqTXvZ9e4y8LkQnmUQzoRpP9XeaOniTte9zppx2b1SXQmsvag0iZtNVHlw5zwxAIuonCIVpwJbYCTUKjPgQEae0tjBpO0CudhMSXBGLU4RbX567L9G3sHVT18Vcj/UqadXHEB5IKDQPCD8K1w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1766219694; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=5InwQq38W8dhaMny4s4eL6hBKvOJdW7iE44wBOaPYsE=; b=PRVrZb0ZhZh11g9TpfV147HpUCr0r8iCh4OTOiJaolNvv/ezoPmOanHsYBXBpzxr95aBGOeVYeVRcRDwxyHPBKfLtaF6RvLWUwrxiJYuiyPCFBSW5FBgC3GPsTpHW0jIFeGD/Tkg7VvkPeLF/+PEOvrgUs70fWKAma30ywTIkCg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=linux.beauty; spf=pass smtp.mailfrom=me@linux.beauty; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1766219694; s=zmail; d=linux.beauty; i=me@linux.beauty; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Message-Id:Reply-To; bh=5InwQq38W8dhaMny4s4eL6hBKvOJdW7iE44wBOaPYsE=; b=sgu8AE7PCzvD6PyY8+lKT2wTsvgH8kM5EHNDkkmDpypDyXArHO0sWLnBdQ5UWJ7d tPa8nw5vNv+lykg23+U8Focltd2xfdj+1h5Qg4yBcS1Xkw3Yq/LEY9kFZ15/SzFb87Q QgapBF8KNi864CVJCKxwmBFLzOmaUi9e1u4ZV9cM= Received: by mx.zohomail.com with SMTPS id 1766219692437326.47273936198314; Sat, 20 Dec 2025 00:34:52 -0800 (PST) From: Li Chen To: Pankaj Gupta , Dan Williams , Vishal Verma , Dave Jiang , Ira Weiny , virtualization@lists.linux.dev, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org Cc: Li Chen Subject: [PATCH 1/4] nvdimm: virtio_pmem: always wake -ENOSPC waiters Date: Sat, 20 Dec 2025 16:34:37 +0800 Message-ID: <20251220083441.313737-2-me@linux.beauty> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251220083441.313737-1-me@linux.beauty> References: <20251220083441.313737-1-me@linux.beauty> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Content-Type: text/plain; charset="utf-8" virtio_pmem_host_ack() reclaims virtqueue descriptors with virtqueue_get_buf(). The -ENOSPC waiter wakeup is tied to completing the returned token. If token completion is skipped for any reason, reclaimed descriptors may not wake a waiter and the submitter may sleep forever waiting for a free slot. Always wake one -ENOSPC waiter for each virtqueue completion before touching the returned token. Use READ_ONCE()/WRITE_ONCE() for the wait_event() flags (done and wq_buf_avail). They are observed by waiters without pmem_lock, so make the accesses explicit single loads/stores and avoid compiler reordering/caching across the wait/wake paths. Signed-off-by: Li Chen --- drivers/nvdimm/nd_virtio.c | 35 +++++++++++++++++++++-------------- 1 file changed, 21 insertions(+), 14 deletions(-) diff --git a/drivers/nvdimm/nd_virtio.c b/drivers/nvdimm/nd_virtio.c index c3f07be4aa22..6f9890361d0b 100644 --- a/drivers/nvdimm/nd_virtio.c +++ b/drivers/nvdimm/nd_virtio.c @@ -9,26 +9,33 @@ #include "virtio_pmem.h" #include "nd.h" =20 +static void virtio_pmem_wake_one_waiter(struct virtio_pmem *vpmem) +{ + struct virtio_pmem_request *req_buf; + + if (list_empty(&vpmem->req_list)) + return; + + req_buf =3D list_first_entry(&vpmem->req_list, + struct virtio_pmem_request, list); + list_del_init(&req_buf->list); + WRITE_ONCE(req_buf->wq_buf_avail, true); + wake_up(&req_buf->wq_buf); +} + /* The interrupt handler */ void virtio_pmem_host_ack(struct virtqueue *vq) { struct virtio_pmem *vpmem =3D vq->vdev->priv; - struct virtio_pmem_request *req_data, *req_buf; + struct virtio_pmem_request *req_data; unsigned long flags; unsigned int len; =20 spin_lock_irqsave(&vpmem->pmem_lock, flags); while ((req_data =3D virtqueue_get_buf(vq, &len)) !=3D NULL) { - req_data->done =3D true; + virtio_pmem_wake_one_waiter(vpmem); + WRITE_ONCE(req_data->done, true); wake_up(&req_data->host_acked); - - if (!list_empty(&vpmem->req_list)) { - req_buf =3D list_first_entry(&vpmem->req_list, - struct virtio_pmem_request, list); - req_buf->wq_buf_avail =3D true; - wake_up(&req_buf->wq_buf); - list_del(&req_buf->list); - } } spin_unlock_irqrestore(&vpmem->pmem_lock, flags); } @@ -58,7 +65,7 @@ static int virtio_pmem_flush(struct nd_region *nd_region) if (!req_data) return -ENOMEM; =20 - req_data->done =3D false; + WRITE_ONCE(req_data->done, false); init_waitqueue_head(&req_data->host_acked); init_waitqueue_head(&req_data->wq_buf); INIT_LIST_HEAD(&req_data->list); @@ -79,12 +86,12 @@ static int virtio_pmem_flush(struct nd_region *nd_regio= n) GFP_ATOMIC)) =3D=3D -ENOSPC) { =20 dev_info(&vdev->dev, "failed to send command to virtio pmem device, no f= ree slots in the virtqueue\n"); - req_data->wq_buf_avail =3D false; + WRITE_ONCE(req_data->wq_buf_avail, false); list_add_tail(&req_data->list, &vpmem->req_list); spin_unlock_irqrestore(&vpmem->pmem_lock, flags); =20 /* A host response results in "host_ack" getting called */ - wait_event(req_data->wq_buf, req_data->wq_buf_avail); + wait_event(req_data->wq_buf, READ_ONCE(req_data->wq_buf_avail)); spin_lock_irqsave(&vpmem->pmem_lock, flags); } err1 =3D virtqueue_kick(vpmem->req_vq); @@ -98,7 +105,7 @@ static int virtio_pmem_flush(struct nd_region *nd_region) err =3D -EIO; } else { /* A host response results in "host_ack" getting called */ - wait_event(req_data->host_acked, req_data->done); + wait_event(req_data->host_acked, READ_ONCE(req_data->done)); err =3D le32_to_cpu(req_data->resp.ret); } =20 --=20 2.51.0 From nobody Sun Feb 8 02:26:28 2026 Received: from sender4-pp-f112.zoho.com (sender4-pp-f112.zoho.com [136.143.188.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4295E1FECBA for ; Sat, 20 Dec 2025 08:35:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.112 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766219711; cv=pass; b=sOaYpFTwufFWqlArRQevrWlETBF6lGAw6fcgUlT6Y5ehxWUj2pLXKgIFiPShoC9aKgOPu0/sElJkpjhB7KZyIVt0dluFKabm0wMcAwxIwQJaCLXdD3/JJNGQ+UAPWEuSYKiNYZk3/E4jQSYlEEk6h5EsHhJV2BK7ZyWhRKNWpMk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766219711; c=relaxed/simple; bh=wt+gTUC83JZ2Aq1RqnhRNClbtK0fIj9oGNFDYzmLP6c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=c+fZjJwaw97bPT8atWg9q58wEp2BdqB5TW6sieKwthFs0Z/zc6V6RJWOVFwD+pLMeSUtyJXMtpb3hg0svviZLfOHS5i+PhFL46kEiIM/yTRphhmUd78pNVk0yWNn8TFBOKJM4zxoYgyz+brZoDfTV6pTIxVcECEv+tHQ04ATK5Q= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=linux.beauty; spf=pass smtp.mailfrom=linux.beauty; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b=eANzaIgm; arc=pass smtp.client-ip=136.143.188.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=linux.beauty Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.beauty Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b="eANzaIgm" ARC-Seal: i=1; a=rsa-sha256; t=1766219698; cv=none; d=zohomail.com; s=zohoarc; b=ndbPvrkOuOd6KqefsIWqYKNj9xAq6YAJcmF9gQSSGhl3aGnCb/r7QdIhTcoA/qZZ5oRR39qIXKpAgBTZIR+46GpWvb9O1swT1zzmbnr0XSb9y1484FPQOfwftUE0WRaHv1qVg54gAW0+wHc8rmDuWdVHzeOwUMBVsw4TD2ZLjFc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1766219698; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=MLyXrglSwdiqLX4MJciWPiqlZQ2ZHYVwKNHsXrftGsY=; b=Vh+RdB5qtCPFBSkpyKWPLdG5X81UI3TNlcYFLzTfP+xse+VYlZ3KHGsO6b79r/5MlPrWjIFk2yyumOLUZ3q4CXhfn8DaR0JNWfvOA6wCYVMlG5MVP3bZWDuhY5Y1jZe0z8wjg+NRdQXYxs1KwB1N1wFHt2n8VAO6iNYRNeh1BH0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=linux.beauty; spf=pass smtp.mailfrom=me@linux.beauty; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1766219698; s=zmail; d=linux.beauty; i=me@linux.beauty; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Message-Id:Reply-To; bh=MLyXrglSwdiqLX4MJciWPiqlZQ2ZHYVwKNHsXrftGsY=; b=eANzaIgm7w1fTQCFR67PWg6vd91MfzcvcWUU84Jz7H7giijC+/Tn1jMyGSpMkbKj q1RK5+k5s57DQT+8u7HVs/qxLyr/s1SDMkxmRL0nXOJDittARnT7J8E9FKQCOkedj82 fWj8yKlIFzD4hbOfISkUXTP7l61MSWIrMpI9cDes= Received: by mx.zohomail.com with SMTPS id 1766219696031997.6480453791337; Sat, 20 Dec 2025 00:34:56 -0800 (PST) From: Li Chen To: Pankaj Gupta , Dan Williams , Vishal Verma , Dave Jiang , Ira Weiny , virtualization@lists.linux.dev, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org Cc: Li Chen Subject: [PATCH 2/4] nvdimm: virtio_pmem: refcount requests for token lifetime Date: Sat, 20 Dec 2025 16:34:38 +0800 Message-ID: <20251220083441.313737-3-me@linux.beauty> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251220083441.313737-1-me@linux.beauty> References: <20251220083441.313737-1-me@linux.beauty> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Content-Type: text/plain; charset="utf-8" KASAN reports a slab use-after-free from virtio_pmem_host_ack(). It happens when it wakes a request that has already been freed by the submitter. This happens when the request token is still reachable via the virtqueue, but virtio_pmem_flush() returns and frees it. Fix the token lifetime by refcounting struct virtio_pmem_request. virtio_pmem_flush() holds a submitter reference, and the virtqueue holds an extra reference once the request is queued. The completion path drops the virtqueue reference, and the submitter drops its reference before returning. Signed-off-by: Li Chen --- drivers/nvdimm/nd_virtio.c | 34 +++++++++++++++++++++++++++++----- drivers/nvdimm/virtio_pmem.h | 2 ++ 2 files changed, 31 insertions(+), 5 deletions(-) diff --git a/drivers/nvdimm/nd_virtio.c b/drivers/nvdimm/nd_virtio.c index 6f9890361d0b..d0385d4646f2 100644 --- a/drivers/nvdimm/nd_virtio.c +++ b/drivers/nvdimm/nd_virtio.c @@ -9,6 +9,14 @@ #include "virtio_pmem.h" #include "nd.h" =20 +static void virtio_pmem_req_release(struct kref *kref) +{ + struct virtio_pmem_request *req; + + req =3D container_of(kref, struct virtio_pmem_request, kref); + kfree(req); +} + static void virtio_pmem_wake_one_waiter(struct virtio_pmem *vpmem) { struct virtio_pmem_request *req_buf; @@ -36,6 +44,7 @@ void virtio_pmem_host_ack(struct virtqueue *vq) virtio_pmem_wake_one_waiter(vpmem); WRITE_ONCE(req_data->done, true); wake_up(&req_data->host_acked); + kref_put(&req_data->kref, virtio_pmem_req_release); } spin_unlock_irqrestore(&vpmem->pmem_lock, flags); } @@ -65,6 +74,7 @@ static int virtio_pmem_flush(struct nd_region *nd_region) if (!req_data) return -ENOMEM; =20 + kref_init(&req_data->kref); WRITE_ONCE(req_data->done, false); init_waitqueue_head(&req_data->host_acked); init_waitqueue_head(&req_data->wq_buf); @@ -82,10 +92,23 @@ static int virtio_pmem_flush(struct nd_region *nd_regio= n) * to req_list and wait for host_ack to wake us up when free * slots are available. */ - while ((err =3D virtqueue_add_sgs(vpmem->req_vq, sgs, 1, 1, req_data, - GFP_ATOMIC)) =3D=3D -ENOSPC) { - - dev_info(&vdev->dev, "failed to send command to virtio pmem device, no f= ree slots in the virtqueue\n"); + for (;;) { + err =3D virtqueue_add_sgs(vpmem->req_vq, sgs, 1, 1, req_data, + GFP_ATOMIC); + if (!err) { + /* + * Take the virtqueue reference while @pmem_lock is + * held so completion cannot run concurrently. + */ + kref_get(&req_data->kref); + break; + } + + if (err !=3D -ENOSPC) + break; + + dev_info_ratelimited(&vdev->dev, + "failed to send command to virtio pmem device, no free slots in t= he virtqueue\n"); WRITE_ONCE(req_data->wq_buf_avail, false); list_add_tail(&req_data->list, &vpmem->req_list); spin_unlock_irqrestore(&vpmem->pmem_lock, flags); @@ -94,6 +117,7 @@ static int virtio_pmem_flush(struct nd_region *nd_region) wait_event(req_data->wq_buf, READ_ONCE(req_data->wq_buf_avail)); spin_lock_irqsave(&vpmem->pmem_lock, flags); } + err1 =3D virtqueue_kick(vpmem->req_vq); spin_unlock_irqrestore(&vpmem->pmem_lock, flags); /* @@ -109,7 +133,7 @@ static int virtio_pmem_flush(struct nd_region *nd_regio= n) err =3D le32_to_cpu(req_data->resp.ret); } =20 - kfree(req_data); + kref_put(&req_data->kref, virtio_pmem_req_release); return err; }; =20 diff --git a/drivers/nvdimm/virtio_pmem.h b/drivers/nvdimm/virtio_pmem.h index 0dddefe594c4..fc8f613f8f28 100644 --- a/drivers/nvdimm/virtio_pmem.h +++ b/drivers/nvdimm/virtio_pmem.h @@ -12,10 +12,12 @@ =20 #include #include +#include #include #include =20 struct virtio_pmem_request { + struct kref kref; struct virtio_pmem_req req; struct virtio_pmem_resp resp; =20 --=20 2.51.0 From nobody Sun Feb 8 02:26:28 2026 Received: from sender4-pp-f112.zoho.com (sender4-pp-f112.zoho.com [136.143.188.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A29432F1FFE for ; Sat, 20 Dec 2025 08:35:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.112 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766219717; cv=pass; b=OTMgbHtdZaHEWuVrFPzLUCXHBKfmGce3qr2fD+wUoFdJPfjbwc0+s4EHratiZ93W7CG9y1H0X6608ybShCj0EJxjk02aNk+8eEekXS950dO7cHYHEJHHsNnrjPC5cHDpZTRnjKAgriP56RHxX499P06vlisjoATVCF0sJK1u0Ng= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766219717; c=relaxed/simple; bh=AaLiwDtvGkY3vHnFwawR8U6bnOAwo1ETBQRXRVjdpqg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=S7PmeZGpXhIQ6oGnmbf3qnXbAi4DAGENmNwa5QYmoPURJichjcaj1289n7tMHzbfLnCU0VZQ60u5rYhc92PY5WWELHt8Pqucxyql/pK3X5kHIVa5EiidOdYksDgZcq8dW99rIA/JF0m45H0yRdleHijWx1JBjBhQaRKiCtVRdXE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=linux.beauty; spf=pass smtp.mailfrom=linux.beauty; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b=Z8XKpnca; arc=pass smtp.client-ip=136.143.188.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=linux.beauty Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.beauty Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b="Z8XKpnca" ARC-Seal: i=1; a=rsa-sha256; t=1766219702; cv=none; d=zohomail.com; s=zohoarc; b=LDMMxj+G7HdfeMgEhullogBDTbwRK+/fb9t4qtbxSxAvRsxNge9aQT6eK1Jjzb3cd8E1gEqkltjnhWD5l4lJsTW6bvPPrU7lAC5+Y2Ptobd2Io6G/t4XYdh2g2GGPel9RLijTxmirGpxUPjH6YzBewinV1H3bLkuogZiBhhfobo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1766219702; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=hcjKCO1Cd5oN9WUb2/5peAP7fBiezP8gzIOyojr9VwM=; b=EnM5BMJ2idBdPIy9HJ5HPaS2EFrDaIP01M1T81nCVWOD/pwA9dxqt/QJgVi5At1+bwMl0lBrligdqCRiUrqipmajmstO0HRsGsDPcOjyFo1A0FWAHEkWtZBEcv0Oe9ziBIbVc0wXvmxSeA2BCqIfXbNiXd8sZnBHqIN4uIvbxpo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=linux.beauty; spf=pass smtp.mailfrom=me@linux.beauty; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1766219702; s=zmail; d=linux.beauty; i=me@linux.beauty; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Message-Id:Reply-To; bh=hcjKCO1Cd5oN9WUb2/5peAP7fBiezP8gzIOyojr9VwM=; b=Z8XKpnca8MtGxA8LMH3WVwOkiiOdzgeJ5fKR6UQRfrgDbsLfAmq9KVahlL2MGqno U/KgiYu6FiANntEXFTD6rVz8XOW2nyBrq8e5paCCerslxOsDJxENVlmJ9R5MyLnW5Zj WjfHlgqazZKqYojd/fT7SXOpDSKJMNilz7cIBM58= Received: by mx.zohomail.com with SMTPS id 1766219699941897.9768922852649; Sat, 20 Dec 2025 00:34:59 -0800 (PST) From: Li Chen To: Pankaj Gupta , Dan Williams , Vishal Verma , Dave Jiang , Ira Weiny , virtualization@lists.linux.dev, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org Cc: Li Chen Subject: [PATCH 3/4] nvdimm: virtio_pmem: converge broken virtqueue to -EIO Date: Sat, 20 Dec 2025 16:34:39 +0800 Message-ID: <20251220083441.313737-4-me@linux.beauty> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251220083441.313737-1-me@linux.beauty> References: <20251220083441.313737-1-me@linux.beauty> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Content-Type: text/plain; charset="utf-8" virtio_pmem_flush() waits for either a free virtqueue descriptor (-ENOSPC) or a host completion. If the request virtqueue becomes broken (e.g. virtqueue_kick() notify failure), those waiters may never make progress. Track a device-level broken state and converge all error paths to -EIO. Fail fast for new requests, wake all -ENOSPC waiters, and drain/detach outstanding request tokens to complete them with an error. Signed-off-by: Li Chen --- drivers/nvdimm/nd_virtio.c | 73 +++++++++++++++++++++++++++++++++--- drivers/nvdimm/virtio_pmem.c | 7 ++++ drivers/nvdimm/virtio_pmem.h | 4 ++ 3 files changed, 78 insertions(+), 6 deletions(-) diff --git a/drivers/nvdimm/nd_virtio.c b/drivers/nvdimm/nd_virtio.c index d0385d4646f2..de1e3dde85eb 100644 --- a/drivers/nvdimm/nd_virtio.c +++ b/drivers/nvdimm/nd_virtio.c @@ -17,6 +17,18 @@ static void virtio_pmem_req_release(struct kref *kref) kfree(req); } =20 +static void virtio_pmem_signal_done(struct virtio_pmem_request *req) +{ + WRITE_ONCE(req->done, true); + wake_up(&req->host_acked); +} + +static void virtio_pmem_complete_err(struct virtio_pmem_request *req) +{ + req->resp.ret =3D cpu_to_le32(1); + virtio_pmem_signal_done(req); +} + static void virtio_pmem_wake_one_waiter(struct virtio_pmem *vpmem) { struct virtio_pmem_request *req_buf; @@ -31,6 +43,40 @@ static void virtio_pmem_wake_one_waiter(struct virtio_pm= em *vpmem) wake_up(&req_buf->wq_buf); } =20 +static void virtio_pmem_wake_all_waiters(struct virtio_pmem *vpmem) +{ + struct virtio_pmem_request *req, *tmp; + + list_for_each_entry_safe(req, tmp, &vpmem->req_list, list) { + WRITE_ONCE(req->wq_buf_avail, true); + wake_up(&req->wq_buf); + list_del_init(&req->list); + } +} + +void virtio_pmem_mark_broken_and_drain(struct virtio_pmem *vpmem) +{ + struct virtio_pmem_request *req; + unsigned int len; + + if (READ_ONCE(vpmem->broken)) + return; + + WRITE_ONCE(vpmem->broken, true); + dev_err_once(&vpmem->vdev->dev, "virtqueue is broken\n"); + virtio_pmem_wake_all_waiters(vpmem); + + while ((req =3D virtqueue_get_buf(vpmem->req_vq, &len)) !=3D NULL) { + virtio_pmem_complete_err(req); + kref_put(&req->kref, virtio_pmem_req_release); + } + + while ((req =3D virtqueue_detach_unused_buf(vpmem->req_vq)) !=3D NULL) { + virtio_pmem_complete_err(req); + kref_put(&req->kref, virtio_pmem_req_release); + } +} + /* The interrupt handler */ void virtio_pmem_host_ack(struct virtqueue *vq) { @@ -42,8 +88,7 @@ void virtio_pmem_host_ack(struct virtqueue *vq) spin_lock_irqsave(&vpmem->pmem_lock, flags); while ((req_data =3D virtqueue_get_buf(vq, &len)) !=3D NULL) { virtio_pmem_wake_one_waiter(vpmem); - WRITE_ONCE(req_data->done, true); - wake_up(&req_data->host_acked); + virtio_pmem_signal_done(req_data); kref_put(&req_data->kref, virtio_pmem_req_release); } spin_unlock_irqrestore(&vpmem->pmem_lock, flags); @@ -69,6 +114,9 @@ static int virtio_pmem_flush(struct nd_region *nd_region) return -EIO; } =20 + if (READ_ONCE(vpmem->broken)) + return -EIO; + might_sleep(); req_data =3D kmalloc(sizeof(*req_data), GFP_KERNEL); if (!req_data) @@ -114,22 +162,35 @@ static int virtio_pmem_flush(struct nd_region *nd_reg= ion) spin_unlock_irqrestore(&vpmem->pmem_lock, flags); =20 /* A host response results in "host_ack" getting called */ - wait_event(req_data->wq_buf, READ_ONCE(req_data->wq_buf_avail)); + wait_event(req_data->wq_buf, READ_ONCE(req_data->wq_buf_avail) || + READ_ONCE(vpmem->broken)); spin_lock_irqsave(&vpmem->pmem_lock, flags); + + if (READ_ONCE(vpmem->broken)) + break; } =20 - err1 =3D virtqueue_kick(vpmem->req_vq); + if (err =3D=3D -EIO || virtqueue_is_broken(vpmem->req_vq)) + virtio_pmem_mark_broken_and_drain(vpmem); + + err1 =3D true; + if (!err && !READ_ONCE(vpmem->broken)) { + err1 =3D virtqueue_kick(vpmem->req_vq); + if (!err1) + virtio_pmem_mark_broken_and_drain(vpmem); + } spin_unlock_irqrestore(&vpmem->pmem_lock, flags); /* * virtqueue_add_sgs failed with error different than -ENOSPC, we can't * do anything about that. */ - if (err || !err1) { + if (READ_ONCE(vpmem->broken) || err || !err1) { dev_info(&vdev->dev, "failed to send command to virtio pmem device\n"); err =3D -EIO; } else { /* A host response results in "host_ack" getting called */ - wait_event(req_data->host_acked, READ_ONCE(req_data->done)); + wait_event(req_data->host_acked, READ_ONCE(req_data->done) || + READ_ONCE(vpmem->broken)); err =3D le32_to_cpu(req_data->resp.ret); } =20 diff --git a/drivers/nvdimm/virtio_pmem.c b/drivers/nvdimm/virtio_pmem.c index 2396d19ce549..aa07328e3ff9 100644 --- a/drivers/nvdimm/virtio_pmem.c +++ b/drivers/nvdimm/virtio_pmem.c @@ -25,6 +25,7 @@ static int init_vq(struct virtio_pmem *vpmem) =20 spin_lock_init(&vpmem->pmem_lock); INIT_LIST_HEAD(&vpmem->req_list); + WRITE_ONCE(vpmem->broken, false); =20 return 0; }; @@ -137,6 +138,12 @@ static int virtio_pmem_probe(struct virtio_device *vde= v) static void virtio_pmem_remove(struct virtio_device *vdev) { struct nvdimm_bus *nvdimm_bus =3D dev_get_drvdata(&vdev->dev); + struct virtio_pmem *vpmem =3D vdev->priv; + unsigned long flags; + + spin_lock_irqsave(&vpmem->pmem_lock, flags); + virtio_pmem_mark_broken_and_drain(vpmem); + spin_unlock_irqrestore(&vpmem->pmem_lock, flags); =20 nvdimm_bus_unregister(nvdimm_bus); vdev->config->del_vqs(vdev); diff --git a/drivers/nvdimm/virtio_pmem.h b/drivers/nvdimm/virtio_pmem.h index fc8f613f8f28..49dd2e62d198 100644 --- a/drivers/nvdimm/virtio_pmem.h +++ b/drivers/nvdimm/virtio_pmem.h @@ -44,6 +44,9 @@ struct virtio_pmem { /* List to store deferred work if virtqueue is full */ struct list_head req_list; =20 + /* Fail fast and wake waiters if the request virtqueue is broken. */ + bool broken; + /* Synchronize virtqueue data */ spinlock_t pmem_lock; =20 @@ -53,5 +56,6 @@ struct virtio_pmem { }; =20 void virtio_pmem_host_ack(struct virtqueue *vq); +void virtio_pmem_mark_broken_and_drain(struct virtio_pmem *vpmem); int async_pmem_flush(struct nd_region *nd_region, struct bio *bio); #endif --=20 2.51.0 From nobody Sun Feb 8 02:26:28 2026 Received: from sender4-pp-f112.zoho.com (sender4-pp-f112.zoho.com [136.143.188.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 52E9A2F1FFE for ; Sat, 20 Dec 2025 08:35:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.112 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766219722; cv=pass; b=Zsv3UwkM1MdAuR4bj9/ZkArDraLQcSSTTuF3s4C+bA0kgF3NB9gtGf+jQyE63ucvvYlXFPOzja+LtlpyhF0dc8dhXnOC2kBB8Raa7buuSD6w8XtFyM+odtBQ7GOktV8Z061FtIHoFiyNUl574RfJOMCSSX555Wds5SdwWNC3Ri8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766219722; c=relaxed/simple; bh=xgICL+AROYOEauYrDUTbYngxQO2C4HrLy6Y3+XbcN0s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SUwGqQL/rh9ZaBBEoJLSCU9ww3TIc9VU4iDon1DOAbikS+pJ8PZl2UXifMn+AIESs6rdZlXw7i8Dzn/J7VuEsZ6gHrUrsLDW8SeLPmdLstvFXqp23tiixDl6Skf7GGWlUeGDccGMNo8aFP3ldIAjveHCiOMWQbt5Hzhe1nMeBfI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=linux.beauty; spf=pass smtp.mailfrom=linux.beauty; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b=MYUmmMhE; arc=pass smtp.client-ip=136.143.188.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=linux.beauty Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.beauty Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.beauty header.i=me@linux.beauty header.b="MYUmmMhE" ARC-Seal: i=1; a=rsa-sha256; t=1766219704; cv=none; d=zohomail.com; s=zohoarc; b=buA3kA8hIoTqkpgvAurMNsLpy6SFVAD+/z/qjmDAlEQ8MQuDPC8DZtkZnmCKxGtwcJnL7rGnfK2Vi77L9F7i8Zxt14Et6bbIesqVV5Bx8mxzH573D7Jl5NZ3ccnmntx6TYaaf5XiizpRW56zyDU0WFC6w1wnNJglb6GtNBr1cJM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1766219704; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=eYfBRITWUnnz0wYgbPwxrDTSwaVRo1CFJZSXwe3HQ70=; b=nKCt6ePpmhRKO3i4kLl0/POcdQjTVaEbg6oA4aSUhvCAw28LYl3zHZoxmu82LrcHy6786mEJENnzxr+tNHjXUCIPxMokRWKeYrHarxnM5dVESo7VZSoPD4MJx3Gh4aHZwQtu26oElhnodKnpLoGMYnFTcCii2zZBqkL1s8lVMyE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=linux.beauty; spf=pass smtp.mailfrom=me@linux.beauty; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1766219704; s=zmail; d=linux.beauty; i=me@linux.beauty; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Message-Id:Reply-To; bh=eYfBRITWUnnz0wYgbPwxrDTSwaVRo1CFJZSXwe3HQ70=; b=MYUmmMhEk1Zp+nVRueRvpy7XjgtHT0aJQSG06UG8iKv3P/YvyIRJIdXfeeH4J7iv BWPwjvJANpdr8CH3XI8ADW4rFr1Z2vKgZyb4I2WsDAyfyO66yVqR/JJeV3zz6gstPsg UMCoa11zL6y27vSuJhJpCE2ihyiLmRjaSqQs4TvA= Received: by mx.zohomail.com with SMTPS id 17662197034231022.1025851067282; Sat, 20 Dec 2025 00:35:03 -0800 (PST) From: Li Chen To: Pankaj Gupta , Dan Williams , Vishal Verma , Dave Jiang , Ira Weiny , virtualization@lists.linux.dev, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org Cc: Li Chen Subject: [PATCH 4/4] nvdimm: virtio_pmem: drain requests in freeze Date: Sat, 20 Dec 2025 16:34:40 +0800 Message-ID: <20251220083441.313737-5-me@linux.beauty> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251220083441.313737-1-me@linux.beauty> References: <20251220083441.313737-1-me@linux.beauty> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Content-Type: text/plain; charset="utf-8" virtio_pmem_freeze() deletes virtqueues and resets the device without waking threads waiting for a virtqueue descriptor or a host completion. Mark the request virtqueue broken and drain outstanding requests under pmem_lock before teardown so waiters can make progress and return -EIO. Signed-off-by: Li Chen --- drivers/nvdimm/virtio_pmem.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/nvdimm/virtio_pmem.c b/drivers/nvdimm/virtio_pmem.c index aa07328e3ff9..5c60a7b459d4 100644 --- a/drivers/nvdimm/virtio_pmem.c +++ b/drivers/nvdimm/virtio_pmem.c @@ -152,6 +152,13 @@ static void virtio_pmem_remove(struct virtio_device *v= dev) =20 static int virtio_pmem_freeze(struct virtio_device *vdev) { + struct virtio_pmem *vpmem =3D vdev->priv; + unsigned long flags; + + spin_lock_irqsave(&vpmem->pmem_lock, flags); + virtio_pmem_mark_broken_and_drain(vpmem); + spin_unlock_irqrestore(&vpmem->pmem_lock, flags); + vdev->config->del_vqs(vdev); virtio_reset_device(vdev); =20 --=20 2.51.0