From nobody Tue Sep 9 16:53:34 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58DBFC64ED8 for ; Mon, 27 Feb 2023 09:21:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230071AbjB0JVn (ORCPT ); Mon, 27 Feb 2023 04:21:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55846 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229792AbjB0JVX (ORCPT ); Mon, 27 Feb 2023 04:21:23 -0500 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C58C4C29 for ; Mon, 27 Feb 2023 01:19:33 -0800 (PST) Received: from kwepemi100025.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4PQFL6434szrSMM; Mon, 27 Feb 2023 17:18:54 +0800 (CST) Received: from DESKTOP-27KDQMV.china.huawei.com (10.174.148.223) by kwepemi100025.china.huawei.com (7.221.188.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Mon, 27 Feb 2023 17:19:00 +0800 From: "Longpeng(Mike)" To: , CC: , , , , , , , , Longpeng Subject: [PATCH v4 1/2] vdpa: support specify the pgprot of vq notification area Date: Mon, 27 Feb 2023 17:18:56 +0800 Message-ID: <20230227091857.2406-2-longpeng2@huawei.com> X-Mailer: git-send-email 2.25.0.windows.1 In-Reply-To: <20230227091857.2406-1-longpeng2@huawei.com> References: <20230227091857.2406-1-longpeng2@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.174.148.223] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To kwepemi100025.china.huawei.com (7.221.188.158) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Longpeng Adds get_vq_notification_pgprot operation to vdpa_config_ops to support specify the pgprot of vq norification area. It's an optional operation, the vdpa framework will treat the pgprot of vq notification area as noncached as default as usual. Acked-by: Jason Wang Signed-off-by: Longpeng --- drivers/vhost/vdpa.c | 4 +++- include/linux/vdpa.h | 9 +++++++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index ec32f785dfde..8b595521c958 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -1273,7 +1273,9 @@ static vm_fault_t vhost_vdpa_fault(struct vm_fault *v= mf) =20 notify =3D ops->get_vq_notification(vdpa, index); =20 - vma->vm_page_prot =3D pgprot_noncached(vma->vm_page_prot); + vma->vm_page_prot =3D ops->get_vq_notification_pgprot ? + ops->get_vq_notification_pgprot(vdpa, index, vma->vm_page_prot) : + pgprot_noncached(vma->vm_page_prot); if (remap_pfn_range(vma, vmf->address & PAGE_MASK, PFN_DOWN(notify.addr), PAGE_SIZE, vma->vm_page_prot)) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index 6d0f5e4e82c2..07fcf5e6abc8 100644 --- a/include/linux/vdpa.h +++ b/include/linux/vdpa.h @@ -169,6 +169,12 @@ struct vdpa_map_file { * @vdev: vdpa device * @idx: virtqueue index * Returns the notifcation area + * @get_vq_notification_pgprot: Get the pgprot of the vq's notification ar= ea (optional) + * @vdev: vdpa device + * @idx: virtqueue index + * @prot: original page protection value of the + * notification area + * Returns pgprot_t: the pgprot of the notification area * @get_vq_irq: Get the irq number of a virtqueue (optional, * but must implemented if require vq irq offloading) * @vdev: vdpa device @@ -305,6 +311,9 @@ struct vdpa_config_ops { struct netlink_ext_ack *extack); struct vdpa_notification_area (*get_vq_notification)(struct vdpa_device *vdev, u16 idx); + pgprot_t (*get_vq_notification_pgprot)(struct vdpa_device *vdev, + u16 idx, + pgprot_t prot); /* vq irq is not expected to be changed once DRIVER_OK is set */ int (*get_vq_irq)(struct vdpa_device *vdev, u16 idx); =20 --=20 2.23.0 From nobody Tue Sep 9 16:53:34 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7AF01C7EE2D for ; Mon, 27 Feb 2023 09:21:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229501AbjB0JVu (ORCPT ); Mon, 27 Feb 2023 04:21:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56052 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229795AbjB0JVX (ORCPT ); Mon, 27 Feb 2023 04:21:23 -0500 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8B7C75269 for ; Mon, 27 Feb 2023 01:19:33 -0800 (PST) Received: from kwepemi100025.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4PQFL662XnzrSMw; Mon, 27 Feb 2023 17:18:54 +0800 (CST) Received: from DESKTOP-27KDQMV.china.huawei.com (10.174.148.223) by kwepemi100025.china.huawei.com (7.221.188.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Mon, 27 Feb 2023 17:19:01 +0800 From: "Longpeng(Mike)" To: , CC: , , , , , , , , Longpeng Subject: [PATCH v4 2/2] vdpasim: support doorbell mapping Date: Mon, 27 Feb 2023 17:18:57 +0800 Message-ID: <20230227091857.2406-3-longpeng2@huawei.com> X-Mailer: git-send-email 2.25.0.windows.1 In-Reply-To: <20230227091857.2406-1-longpeng2@huawei.com> References: <20230227091857.2406-1-longpeng2@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.174.148.223] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To kwepemi100025.china.huawei.com (7.221.188.158) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Longpeng Support doorbell mapping for vdpasim devices, then we can test the notify passthrough feature even if there's no real hardware on hand. Allocates a dummy page which is used to emulate the notify page of the devi= ce, all VQs share the same notify register that initiated to 0xffff. A period= ic work will check whether there're requests need to process ( the value of the notify register is 0xffff or not ). We can test on QEMU with: -device vhost-vdpa-device-pci,vhostdev=3D/dev/vhost-vdpa-0,page-per-vq=3Don Signed-off-by: Longpeng Acked-by: Jason Wang --- drivers/vdpa/vdpa_sim/vdpa_sim.c | 67 ++++++++++++++++++++++++++++++++ drivers/vdpa/vdpa_sim/vdpa_sim.h | 3 ++ 2 files changed, 70 insertions(+) diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_= sim.c index cb88891b44a8..5a8c820b179f 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c @@ -39,6 +39,8 @@ MODULE_PARM_DESC(max_iotlb_entries, #define VDPASIM_QUEUE_ALIGN PAGE_SIZE #define VDPASIM_QUEUE_MAX 256 #define VDPASIM_VENDOR_ID 0 +#define VDPASIM_VRING_POLL_PERIOD 100 /* ms */ +#define VDPASIM_NOTIFY_DEFVAL 0xffff =20 static struct vdpasim *vdpa_to_sim(struct vdpa_device *vdpa) { @@ -245,6 +247,28 @@ static const struct dma_map_ops vdpasim_dma_ops =3D { static const struct vdpa_config_ops vdpasim_config_ops; static const struct vdpa_config_ops vdpasim_batch_config_ops; =20 +static void vdpasim_notify_work(struct work_struct *work) +{ + struct vdpasim *vdpasim; + u16 *val; + + vdpasim =3D container_of(work, struct vdpasim, notify_work.work); + + if (!(vdpasim->status & VIRTIO_CONFIG_S_DRIVER_OK)) + goto out; + + if (!vdpasim->running) + goto out; + + val =3D (u16 *)vdpasim->notify; + if (xchg(val, VDPASIM_NOTIFY_DEFVAL) !=3D VDPASIM_NOTIFY_DEFVAL) + schedule_work(&vdpasim->work); + +out: + schedule_delayed_work(&vdpasim->notify_work, + msecs_to_jiffies(VDPASIM_VRING_POLL_PERIOD)); +} + struct vdpasim *vdpasim_create(struct vdpasim_dev_attr *dev_attr, const struct vdpa_dev_set_config *config) { @@ -286,6 +310,13 @@ struct vdpasim *vdpasim_create(struct vdpasim_dev_attr= *dev_attr, set_dma_ops(dev, &vdpasim_dma_ops); vdpasim->vdpa.mdev =3D dev_attr->mgmt_dev; =20 + INIT_DELAYED_WORK(&vdpasim->notify_work, vdpasim_notify_work); + + vdpasim->notify =3D (u16 *)__get_free_page(GFP_KERNEL | __GFP_ZERO); + if (!vdpasim->notify) + goto err_iommu; + WRITE_ONCE(*vdpasim->notify, VDPASIM_NOTIFY_DEFVAL); + vdpasim->config =3D kzalloc(dev_attr->config_size, GFP_KERNEL); if (!vdpasim->config) goto err_iommu; @@ -320,6 +351,13 @@ struct vdpasim *vdpasim_create(struct vdpasim_dev_attr= *dev_attr, =20 vdpasim->vdpa.dma_dev =3D dev; =20 + /* + * Start periodic (every 100ms) notify work, it won't introduce + * any overhead if the device is not started. + */ + schedule_delayed_work(&vdpasim->notify_work, + msecs_to_jiffies(VDPASIM_VRING_POLL_PERIOD)); + return vdpasim; =20 err_iommu: @@ -671,11 +709,34 @@ static int vdpasim_dma_unmap(struct vdpa_device *vdpa= , unsigned int asid, return 0; } =20 +static pgprot_t vdpasim_get_vq_notification_pgprot(struct vdpa_device *vdp= a, + u16 qid, pgprot_t prot) +{ + /* + * We use normal RAM pages to emulate the vq notification area, so + * just keep the pgprot as it mmaped. + */ + return prot; +} + +static struct vdpa_notification_area +vdpasim_get_vq_notification(struct vdpa_device *vdpa, u16 qid) +{ + struct vdpasim *vdpasim =3D vdpa_to_sim(vdpa); + struct vdpa_notification_area notify; + + notify.addr =3D virt_to_phys((void *)vdpasim->notify); + notify.size =3D PAGE_SIZE; + + return notify; +} + static void vdpasim_free(struct vdpa_device *vdpa) { struct vdpasim *vdpasim =3D vdpa_to_sim(vdpa); int i; =20 + cancel_delayed_work_sync(&vdpasim->notify_work); cancel_work_sync(&vdpasim->work); =20 for (i =3D 0; i < vdpasim->dev_attr.nvqs; i++) { @@ -694,6 +755,8 @@ static void vdpasim_free(struct vdpa_device *vdpa) kfree(vdpasim->iommu); kfree(vdpasim->vqs); kfree(vdpasim->config); + if (vdpasim->notify) + free_page((unsigned long)vdpasim->notify); } =20 static const struct vdpa_config_ops vdpasim_config_ops =3D { @@ -705,6 +768,8 @@ static const struct vdpa_config_ops vdpasim_config_ops = =3D { .get_vq_ready =3D vdpasim_get_vq_ready, .set_vq_state =3D vdpasim_set_vq_state, .get_vq_state =3D vdpasim_get_vq_state, + .get_vq_notification =3D vdpasim_get_vq_notification, + .get_vq_notification_pgprot =3D vdpasim_get_vq_notification_pgprot, .get_vq_align =3D vdpasim_get_vq_align, .get_vq_group =3D vdpasim_get_vq_group, .get_device_features =3D vdpasim_get_device_features, @@ -738,6 +803,8 @@ static const struct vdpa_config_ops vdpasim_batch_confi= g_ops =3D { .get_vq_ready =3D vdpasim_get_vq_ready, .set_vq_state =3D vdpasim_set_vq_state, .get_vq_state =3D vdpasim_get_vq_state, + .get_vq_notification =3D vdpasim_get_vq_notification, + .get_vq_notification_pgprot =3D vdpasim_get_vq_notification_pgprot, .get_vq_align =3D vdpasim_get_vq_align, .get_vq_group =3D vdpasim_get_vq_group, .get_device_features =3D vdpasim_get_device_features, diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.h b/drivers/vdpa/vdpa_sim/vdpa_= sim.h index 0e78737dcc16..786e902de0b2 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.h +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.h @@ -69,6 +69,9 @@ struct vdpasim { bool running; /* spinlock to synchronize iommu table */ spinlock_t iommu_lock; + /* dummy notify page */ + u16 *notify; + struct delayed_work notify_work; }; =20 struct vdpasim *vdpasim_create(struct vdpasim_dev_attr *attr, --=20 2.23.0