From nobody Sun Dec 28 21:16:59 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5272DC10DC3 for ; Tue, 5 Dec 2023 08:35:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344837AbjLEIe4 (ORCPT ); Tue, 5 Dec 2023 03:34:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33088 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229980AbjLEIex (ORCPT ); Tue, 5 Dec 2023 03:34:53 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 586ACFA for ; Tue, 5 Dec 2023 00:35:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1701765299; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PvXGrTDaK1+8fbsQp8oZ7MPiXUC1xBJXWn44I4Pe5Yo=; b=hgJWgnAGmd8CwrnaiZfIdqpwngHHql0N0JA9LfFwNMF00wfyBxzcCm9cYoI/sy9NMR/dqk A34gq3/fIQBOmLrfmfQMWKH1Qxo8h8T9tYgS2hzfOiSyHavD7JurwTa1WIWznZtbDMehjc diNT7OtYR4VOWt2PLJjTXPWVBfR8We4= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-180-nLPidf_8NBSXCDQYWq1QDQ-1; Tue, 05 Dec 2023 03:34:57 -0500 X-MC-Unique: nLPidf_8NBSXCDQYWq1QDQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A499C1C068DD; Tue, 5 Dec 2023 08:34:57 +0000 (UTC) Received: from server.redhat.com (unknown [10.72.112.70]) by smtp.corp.redhat.com (Postfix) with ESMTP id 248002166B31; Tue, 5 Dec 2023 08:34:54 +0000 (UTC) From: Cindy Lu To: lulu@redhat.com, jasowang@redhat.com, mst@redhat.com, xieyongji@bytedance.com, linux-kernel@vger.kernel.org, maxime.coquelin@redhat.com Subject: [PATCH v3 1/7] vdpa: Move struct vdpa_vq_state to uAPI Date: Tue, 5 Dec 2023 16:34:38 +0800 Message-Id: <20231205083444.3029239-2-lulu@redhat.com> In-Reply-To: <20231205083444.3029239-1-lulu@redhat.com> References: <20231205083444.3029239-1-lulu@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.6 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Move struct vdpa_vq_state to uAPI for vduse reconnect. Userspace and kernel will use this structure to synchronize the vq status Signed-off-by: Cindy Lu --- include/linux/vdpa.h | 29 +---------------------------- include/uapi/linux/vdpa.h | 31 +++++++++++++++++++++++++++++++ 2 files changed, 32 insertions(+), 28 deletions(-) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index 43f59ef10cc9..62ce97bce934 100644 --- a/include/linux/vdpa.h +++ b/include/linux/vdpa.h @@ -8,6 +8,7 @@ #include #include #include +#include =20 /** * struct vdpa_calllback - vDPA callback definition. @@ -29,34 +30,6 @@ struct vdpa_notification_area { resource_size_t size; }; =20 -/** - * struct vdpa_vq_state_split - vDPA split virtqueue state - * @avail_index: available index - */ -struct vdpa_vq_state_split { - u16 avail_index; -}; - -/** - * struct vdpa_vq_state_packed - vDPA packed virtqueue state - * @last_avail_counter: last driver ring wrap counter observed by device - * @last_avail_idx: device available index - * @last_used_counter: device ring wrap counter - * @last_used_idx: used index - */ -struct vdpa_vq_state_packed { - u16 last_avail_counter:1; - u16 last_avail_idx:15; - u16 last_used_counter:1; - u16 last_used_idx:15; -}; - -struct vdpa_vq_state { - union { - struct vdpa_vq_state_split split; - struct vdpa_vq_state_packed packed; - }; -}; =20 struct vdpa_mgmt_dev; =20 diff --git a/include/uapi/linux/vdpa.h b/include/uapi/linux/vdpa.h index 54b649ab0f22..f4bc3589685e 100644 --- a/include/uapi/linux/vdpa.h +++ b/include/uapi/linux/vdpa.h @@ -7,6 +7,8 @@ #ifndef _UAPI_LINUX_VDPA_H_ #define _UAPI_LINUX_VDPA_H_ =20 +#include + #define VDPA_GENL_NAME "vdpa" #define VDPA_GENL_VERSION 0x1 =20 @@ -60,4 +62,33 @@ enum vdpa_attr { VDPA_ATTR_MAX, }; =20 +/** + * struct vdpa_vq_state_split - vDPA split virtqueue state + * @avail_index: available index + */ +struct vdpa_vq_state_split { + __u16 avail_index; +}; + +/** + * struct vdpa_vq_state_packed - vDPA packed virtqueue state + * @last_avail_counter: last driver ring wrap counter observed by device + * @last_avail_idx: device available index + * @last_used_counter: device ring wrap counter + * @last_used_idx: used index + */ +struct vdpa_vq_state_packed { + __u16 last_avail_counter : 1; + __u16 last_avail_idx : 15; + __u16 last_used_counter : 1; + __u16 last_used_idx : 15; +}; + +struct vdpa_vq_state { + union { + struct vdpa_vq_state_split split; + struct vdpa_vq_state_packed packed; + }; +}; + #endif --=20 2.34.3 From nobody Sun Dec 28 21:16:59 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB59FC4167B for ; Tue, 5 Dec 2023 08:35:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231774AbjLEIfD (ORCPT ); Tue, 5 Dec 2023 03:35:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33110 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229980AbjLEIe4 (ORCPT ); Tue, 5 Dec 2023 03:34:56 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 36CB3FA for ; Tue, 5 Dec 2023 00:35:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1701765302; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Go5CbagaE3XYKeSXAgFuW3lKr5JwgDqKZNKpmj7+xFw=; b=frjKNqq/4j1UeHsJBjvatzMSKl3Ef2Dd2rK2mWhccdu/iVRgEPcLFhqHVG0C1u3UQNARe0 BFQDGDAiUwe0GsTkyHj3GazIINuT72sokoiXomOqvrMbbRq1/Zl3uhEvauEsI1IPQwTAtt 92d+x+0h3qQKblxs/O599gryTK16Et8= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-88-0e4_gnNANvCgaBGDcw9MEw-1; Tue, 05 Dec 2023 03:35:01 -0500 X-MC-Unique: 0e4_gnNANvCgaBGDcw9MEw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C61FA3C0263D; Tue, 5 Dec 2023 08:35:00 +0000 (UTC) Received: from server.redhat.com (unknown [10.72.112.70]) by smtp.corp.redhat.com (Postfix) with ESMTP id 44A6D2166B31; Tue, 5 Dec 2023 08:34:57 +0000 (UTC) From: Cindy Lu To: lulu@redhat.com, jasowang@redhat.com, mst@redhat.com, xieyongji@bytedance.com, linux-kernel@vger.kernel.org, maxime.coquelin@redhat.com Subject: [PATCH v3 2/7] vduse: Add new uAPI for vduse reconnection Date: Tue, 5 Dec 2023 16:34:39 +0800 Message-Id: <20231205083444.3029239-3-lulu@redhat.com> In-Reply-To: <20231205083444.3029239-1-lulu@redhat.com> References: <20231205083444.3029239-1-lulu@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.6 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" To synchronize the information for reconnection, add a new structure struct vduse_dev_reconnect_data to save the device-related information, Add the VDUSE_RECONNCT_MMAP_SIZE for the size of mapped memory for each vq and device status. Signed-off-by: Cindy Lu --- include/uapi/linux/vduse.h | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/include/uapi/linux/vduse.h b/include/uapi/linux/vduse.h index 11bd48c72c6c..c22838247814 100644 --- a/include/uapi/linux/vduse.h +++ b/include/uapi/linux/vduse.h @@ -350,4 +350,26 @@ struct vduse_dev_response { }; }; =20 +/** + * struct vduse_dev_reconnect_data - saved the reconnect info for device + * @version; version for userspace APP + * @reconnected: indetify if this is reconnected.userspace APP needs set t= his + * to VDUSE_RECONNECT, while reconnecting.kernel will use t= his + * to indetify if this is reconnect + * @features; Device features negotiated in the last connect. + * @status; Device status in last reconnect + */ + +struct vduse_dev_reconnect_data { + __u32 version; +#define VDUSE_RECONNECT 1 +#define VDUSE_NOT_RECONNECT 0 + __u32 reconnected; + __u64 features; + __u8 status; +}; + +/* the reconnection mmap size for each VQ and dev status */ +#define VDUSE_RECONNCT_MMAP_SIZE PAGE_SIZE + #endif /* _UAPI_VDUSE_H_ */ --=20 2.34.3 From nobody Sun Dec 28 21:16:59 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1ECF2C4167B for ; Tue, 5 Dec 2023 08:35:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344855AbjLEIfG (ORCPT ); Tue, 5 Dec 2023 03:35:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33216 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344835AbjLEIfC (ORCPT ); Tue, 5 Dec 2023 03:35:02 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4DB36138 for ; Tue, 5 Dec 2023 00:35:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1701765307; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oONAYYTEgNjcC4Phz29fvyjfObj+V7u6RFkS9+bCyRY=; b=euMKehM04CKTsO86meJoF6ZcSpOiO5SlsKAUDSTv94pydl1ZNTU4+OfTb4Qb/7n/ockTRP fEACS9Ms7KJTUzoZigsSkXGQTBqAU7tS9qDl7sG3hPUr3VeJ6MKQkqPK0X2Aqdy+gl/iVi VT8YTLnzPn7wDZQpWth/Qlc4R3uonkY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-119-w8Dp9Zc-MxWUoIp4NgUW3w-1; Tue, 05 Dec 2023 03:35:04 -0500 X-MC-Unique: w8Dp9Zc-MxWUoIp4NgUW3w-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E81AB101A52A; Tue, 5 Dec 2023 08:35:03 +0000 (UTC) Received: from server.redhat.com (unknown [10.72.112.70]) by smtp.corp.redhat.com (Postfix) with ESMTP id 669D92166B31; Tue, 5 Dec 2023 08:35:01 +0000 (UTC) From: Cindy Lu To: lulu@redhat.com, jasowang@redhat.com, mst@redhat.com, xieyongji@bytedance.com, linux-kernel@vger.kernel.org, maxime.coquelin@redhat.com Subject: [PATCH v3 3/7] vduse: Add new ioctl VDUSE_DEV_GET_CONFIG Date: Tue, 5 Dec 2023 16:34:40 +0800 Message-Id: <20231205083444.3029239-4-lulu@redhat.com> In-Reply-To: <20231205083444.3029239-1-lulu@redhat.com> References: <20231205083444.3029239-1-lulu@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.6 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The ioctl VDUSE_GET_RECONNECT_INFO is used by the Userspace App to get the device configuration space. Signed-off-by: Cindy Lu --- drivers/vdpa/vdpa_user/vduse_dev.c | 21 +++++++++++++++++++++ include/uapi/linux/vduse.h | 3 +++ 2 files changed, 24 insertions(+) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vd= use_dev.c index 26b7e29cb900..dd074a7b4bc7 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -1273,6 +1273,27 @@ static long vduse_dev_ioctl(struct file *file, unsig= ned int cmd, ret =3D 0; break; } + case VDUSE_DEV_GET_CONFIG: { + struct vduse_config_data config; + unsigned long size =3D offsetof(struct vduse_config_data, buffer); + + ret =3D -EFAULT; + if (copy_from_user(&config, argp, size)) + break; + + ret =3D -EINVAL; + if (config.offset > dev->config_size || config.length =3D=3D 0 || + config.length > dev->config_size - config.offset) + break; + + if (copy_to_user(argp + size, dev->config + config.offset, + config.length)) { + ret =3D -EFAULT; + break; + } + ret =3D 0; + break; + } default: ret =3D -ENOIOCTLCMD; break; diff --git a/include/uapi/linux/vduse.h b/include/uapi/linux/vduse.h index c22838247814..5280042ced23 100644 --- a/include/uapi/linux/vduse.h +++ b/include/uapi/linux/vduse.h @@ -372,4 +372,7 @@ struct vduse_dev_reconnect_data { /* the reconnection mmap size for each VQ and dev status */ #define VDUSE_RECONNCT_MMAP_SIZE PAGE_SIZE =20 +/* get device configuration space */ +#define VDUSE_DEV_GET_CONFIG _IOR(VDUSE_BASE, 0x1b, struct vduse_config_da= ta) + #endif /* _UAPI_VDUSE_H_ */ --=20 2.34.3 From nobody Sun Dec 28 21:16:59 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0E03C46CA0 for ; Tue, 5 Dec 2023 08:36:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344886AbjLEIfl (ORCPT ); Tue, 5 Dec 2023 03:35:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41338 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234943AbjLEIfZ (ORCPT ); Tue, 5 Dec 2023 03:35:25 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 23E3CD44 for ; Tue, 5 Dec 2023 00:35:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1701765330; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vm1nuGxCCo/CoP2TcwoqlCKrZ6P/0ERKcylQR/oOFpM=; b=KziZUvKaWrOmDUUEL8hqW382o++zX/KjKNN6pNDNXQwFSjSRrxZEuttkjgEt1YY8jW0Ikk d7RSc4yVtpkcfFXZS0Yu4EiGOuvp50ec/cQXfKJAhEqpNnaEBu6pKT232ggcQiDLusA7np gYMj8fGHg6fi7QeQGU1Id1pvTR4Kg5g= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-652-vpdp_UpWMfi5ovLxqck8Uw-1; Tue, 05 Dec 2023 03:35:28 -0500 X-MC-Unique: vpdp_UpWMfi5ovLxqck8Uw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9C5D1185A780; Tue, 5 Dec 2023 08:35:28 +0000 (UTC) Received: from server.redhat.com (unknown [10.72.112.70]) by smtp.corp.redhat.com (Postfix) with ESMTP id 179543C25; Tue, 5 Dec 2023 08:35:25 +0000 (UTC) From: Cindy Lu To: lulu@redhat.com, jasowang@redhat.com, mst@redhat.com, xieyongji@bytedance.com, linux-kernel@vger.kernel.org, maxime.coquelin@redhat.com Subject: [PATCH v3 4/7] vduse: Add function to get/free the pages for reconnection Date: Tue, 5 Dec 2023 16:34:41 +0800 Message-Id: <20231205083444.3029239-5-lulu@redhat.com> In-Reply-To: <20231205083444.3029239-1-lulu@redhat.com> References: <20231205083444.3029239-1-lulu@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add the function vduse_alloc_reconnnect_info_mem and vduse_alloc_reconnnect_info_mem These functions allow vduse to allocate and free memory for reconnection information. The amount of memory allocated is (vq_num + 1) pages. Page 0 is reserved for saving the dev reconnection information, which will be maintained by the Userspace App. Pages 1 to vq_num + 1 will be used to save the reconnection information for vqs. Signed-off-by: Cindy Lu --- drivers/vdpa/vdpa_user/vduse_dev.c | 64 ++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vd= use_dev.c index dd074a7b4bc7..52ccde636406 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -41,6 +41,7 @@ #define VDUSE_IOVA_SIZE (128 * 1024 * 1024) #define VDUSE_MSG_DEFAULT_TIMEOUT 30 =20 + struct vduse_virtqueue { u16 index; u16 num_max; @@ -57,6 +58,7 @@ struct vduse_virtqueue { struct vdpa_callback cb; struct work_struct inject; struct work_struct kick; + unsigned long vdpa_reconnect_vaddr; }; =20 struct vduse_dev; @@ -106,6 +108,7 @@ struct vduse_dev { u32 vq_align; struct vduse_umem *umem; struct mutex mem_lock; + unsigned long vdpa_reconnect_vaddr; }; =20 struct vduse_dev_msg { @@ -1030,6 +1033,57 @@ static int vduse_dev_reg_umem(struct vduse_dev *dev, return ret; } =20 +static int vduse_alloc_reconnnect_info_mem(struct vduse_dev *dev) +{ + unsigned long vaddr =3D 0; + struct vduse_virtqueue *vq; + + for (int i =3D 0; i < dev->vq_num + 1; i++) { + if (i =3D=3D 0) { + vaddr =3D __get_free_pages( + GFP_KERNEL | __GFP_ZERO, + get_order(VDUSE_RECONNCT_MMAP_SIZE)); + if (vaddr =3D=3D 0) + return -ENOMEM; + dev->vdpa_reconnect_vaddr =3D vaddr; + } + + /*page 1~ vq_num + 1 save the reconnect info for vq*/ + vq =3D &dev->vqs[i]; + vaddr =3D __get_free_pages(GFP_KERNEL | __GFP_ZERO, + get_order(VDUSE_RECONNCT_MMAP_SIZE)); + if (vaddr =3D=3D 0) + return -ENOMEM; + + vq->vdpa_reconnect_vaddr =3D vaddr; + } + + return 0; +} + +static int vduse_free_reconnnect_info_mem(struct vduse_dev *dev) +{ + struct vduse_virtqueue *vq; + + for (int i =3D 0; i < dev->vq_num + 1; i++) { + if (i =3D=3D 0) { + if (dev->vdpa_reconnect_vaddr) + free_pages(dev->vdpa_reconnect_vaddr, + get_order(VDUSE_RECONNCT_MMAP_SIZE)); + dev->vdpa_reconnect_vaddr =3D 0; + } + + vq =3D &dev->vqs[i]; + + if (vq->vdpa_reconnect_vaddr) + free_pages(vq->vdpa_reconnect_vaddr, + get_order(VDUSE_RECONNCT_MMAP_SIZE)); + vq->vdpa_reconnect_vaddr =3D 0; + } + + return 0; +} + static long vduse_dev_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -1411,6 +1465,8 @@ static int vduse_destroy_dev(char *name) mutex_unlock(&dev->lock); return -EBUSY; } + vduse_free_reconnnect_info_mem(dev); + dev->connected =3D true; mutex_unlock(&dev->lock); =20 @@ -1563,9 +1619,17 @@ static int vduse_create_dev(struct vduse_dev_config = *config, ret =3D PTR_ERR(dev->dev); goto err_dev; } + + ret =3D vduse_alloc_reconnnect_info_mem(dev); + if (ret < 0) + goto err_mem; + __module_get(THIS_MODULE); =20 return 0; + +err_mem: + vduse_free_reconnnect_info_mem(dev); err_dev: idr_remove(&vduse_idr, dev->minor); err_idr: --=20 2.34.3 From nobody Sun Dec 28 21:16:59 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94FEFC4167B for ; Tue, 5 Dec 2023 08:36:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1376355AbjLEIgA (ORCPT ); Tue, 5 Dec 2023 03:36:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41326 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235458AbjLEIf2 (ORCPT ); Tue, 5 Dec 2023 03:35:28 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2D1E2185 for ; Tue, 5 Dec 2023 00:35:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1701765333; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1EF7vfXUKmZ7w1vqKhCPA032G3uF9wOvyUGxj1cxm00=; b=KVWExMcplhPWc61Dx92bdtpNXGgOQcTWjHCMZtpOHIwJ+0JC7ffV0AlU70WT0Mf7mGgEQo 2lZD38dp6kdeDJHFNZhq9TR0ieg9bG8sbs0tdgUBGajtZLDSJPiuJMKghNoJN9yg8RD2S9 GCuJEeDK7mNlQnxxfSSBG12fmVJ3ajI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-645-l76KXkNdNkaAjlgEfj5TDw-1; Tue, 05 Dec 2023 03:35:32 -0500 X-MC-Unique: l76KXkNdNkaAjlgEfj5TDw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BCDB0879843; Tue, 5 Dec 2023 08:35:31 +0000 (UTC) Received: from server.redhat.com (unknown [10.72.112.70]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3C87F3C25; Tue, 5 Dec 2023 08:35:28 +0000 (UTC) From: Cindy Lu To: lulu@redhat.com, jasowang@redhat.com, mst@redhat.com, xieyongji@bytedance.com, linux-kernel@vger.kernel.org, maxime.coquelin@redhat.com Subject: [PATCH v3 5/7] vduse: Add file operation for mmap Date: Tue, 5 Dec 2023 16:34:42 +0800 Message-Id: <20231205083444.3029239-6-lulu@redhat.com> In-Reply-To: <20231205083444.3029239-1-lulu@redhat.com> References: <20231205083444.3029239-1-lulu@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add the operation for mmap, This function will be used by the user space application to map the pages to the user space. Signed-off-by: Cindy Lu --- drivers/vdpa/vdpa_user/vduse_dev.c | 75 ++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vd= use_dev.c index 52ccde636406..f55f415629de 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -1381,6 +1381,79 @@ static struct vduse_dev *vduse_dev_get_from_minor(in= t minor) return dev; } =20 +static vm_fault_t vduse_vm_fault(struct vm_fault *vmf) +{ + struct vduse_dev *dev =3D vmf->vma->vm_file->private_data; + struct vm_area_struct *vma =3D vmf->vma; + u16 index =3D vma->vm_pgoff; + struct vduse_virtqueue *vq; + unsigned long vaddr; + + /* index 0 page reserved for vduse status*/ + if (index =3D=3D 0) { + vaddr =3D dev->vdpa_reconnect_vaddr; + } else { + /* index 1+vq_number page reserved for vduse vqs*/ + vq =3D &dev->vqs[index - 1]; + vaddr =3D vq->vdpa_reconnect_vaddr; + } + if (remap_pfn_range(vma, vmf->address & PAGE_MASK, + PFN_DOWN(virt_to_phys((void *)vaddr)), + VDUSE_RECONNCT_MMAP_SIZE, vma->vm_page_prot)) + return VM_FAULT_SIGBUS; + return VM_FAULT_NOPAGE; +} + +static const struct vm_operations_struct vduse_vm_ops =3D { + .fault =3D vduse_vm_fault, +}; + +static int vduse_dev_mmap(struct file *file, struct vm_area_struct *vma) +{ + struct vduse_dev *dev =3D file->private_data; + unsigned long vaddr =3D 0; + unsigned long index =3D vma->vm_pgoff; + struct vduse_virtqueue *vq; + + if (vma->vm_end - vma->vm_start !=3D PAGE_SIZE) + return -EINVAL; + if ((vma->vm_flags & VM_SHARED) =3D=3D 0) + return -EINVAL; + + /*check if Userspace App map the page number larger than kernel allocated= */ + if (index > dev->vq_num + 1) + return -EINVAL; + + /* index 0 page reserved for vduse status*/ + if (index =3D=3D 0) { + vaddr =3D dev->vdpa_reconnect_vaddr; + } else { + /* index 1+vq_number page reserved for vduse vqs*/ + vq =3D &dev->vqs[index - 1]; + vaddr =3D vq->vdpa_reconnect_vaddr; + } + /* Check whether the memory for the mmap was allocated by the kernel. + * If not, this device may not have been created/destroyed correctly. + */ + if (vaddr =3D=3D 0) + return -EOPNOTSUPP; + + /* check if the address is page aligned, if not, + * this address maybe damaged + */ + if (virt_to_phys((void *)vaddr) & (PAGE_SIZE - 1)) + return -EINVAL; + + /* Check if the Userspace App has mapped the correct size */ + if (vma->vm_end - vma->vm_start !=3D VDUSE_RECONNCT_MMAP_SIZE) + return -EOPNOTSUPP; + + vm_flags_set(vma, VM_DONTEXPAND); + vma->vm_ops =3D &vduse_vm_ops; + + return 0; +} + static int vduse_dev_open(struct inode *inode, struct file *file) { int ret; @@ -1413,6 +1486,8 @@ static const struct file_operations vduse_dev_fops = =3D { .unlocked_ioctl =3D vduse_dev_ioctl, .compat_ioctl =3D compat_ptr_ioctl, .llseek =3D noop_llseek, + .mmap =3D vduse_dev_mmap, + }; =20 static struct vduse_dev *vduse_dev_create(void) --=20 2.34.3 From nobody Sun Dec 28 21:16:59 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A6A9C4167B for ; Tue, 5 Dec 2023 08:36:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346337AbjLEIgI (ORCPT ); Tue, 5 Dec 2023 03:36:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41448 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235483AbjLEIfa (ORCPT ); Tue, 5 Dec 2023 03:35:30 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 702BE191 for ; Tue, 5 Dec 2023 00:35:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1701765336; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+OP1NgMKKCzWtLe7quAE0mrqWNiRYPYGvVhdKqorCyI=; b=Ii06pIdtsAxZGf3JEaT1+koVOziYStQCAt8y3NeqqCevhAZjGV6cBJLbStb5lfVkO7oOoR BBJYOFhIFEqBumHTzBWWDgBdmCMU/h8QZhixUfNv8n4Ol44Cu9Uho+22dQ5BclvtwOS7X/ kGTknM6cJ0k2edJUKQzrDMUAUEJNEJs= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-56-Ts2cfc3cOLGP9ypVf0uzsA-1; Tue, 05 Dec 2023 03:35:35 -0500 X-MC-Unique: Ts2cfc3cOLGP9ypVf0uzsA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DDC833C0263D; Tue, 5 Dec 2023 08:35:34 +0000 (UTC) Received: from server.redhat.com (unknown [10.72.112.70]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5CBBB3C27; Tue, 5 Dec 2023 08:35:32 +0000 (UTC) From: Cindy Lu To: lulu@redhat.com, jasowang@redhat.com, mst@redhat.com, xieyongji@bytedance.com, linux-kernel@vger.kernel.org, maxime.coquelin@redhat.com Subject: [PATCH v3 6/7] vduse: Update the vq_info in ioctl VDUSE_VQ_GET_INFO Date: Tue, 5 Dec 2023 16:34:43 +0800 Message-Id: <20231205083444.3029239-7-lulu@redhat.com> In-Reply-To: <20231205083444.3029239-1-lulu@redhat.com> References: <20231205083444.3029239-1-lulu@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Once the reconnect memory pages are mapped to userspace, the userspace application will update the "reconnected" bit in the "struct vduse_dev_reconnect_data". The kernel will then check this bit. If it is not set to "VDUSE_NOT_RECONNECT", it means that the application has been reconnected, and the kernel will synchronize the vq information. Signed-off-by: Cindy Lu --- drivers/vdpa/vdpa_user/vduse_dev.c | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vd= use_dev.c index f55f415629de..422f1aedebac 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -1193,6 +1193,9 @@ static long vduse_dev_ioctl(struct file *file, unsign= ed int cmd, struct vduse_vq_info vq_info; struct vduse_virtqueue *vq; u32 index; + unsigned long vaddr; + struct vduse_vq_state *vq_reconnect; + struct vduse_dev_reconnect_data *dev_reconnect; =20 ret =3D -EFAULT; if (copy_from_user(&vq_info, argp, sizeof(vq_info))) @@ -1209,6 +1212,12 @@ static long vduse_dev_ioctl(struct file *file, unsig= ned int cmd, vq_info.device_addr =3D vq->device_addr; vq_info.num =3D vq->num; =20 + vaddr =3D dev->vdpa_reconnect_vaddr; + dev_reconnect =3D (struct vduse_dev_reconnect_data *)vaddr; + + vaddr =3D vq->vdpa_reconnect_vaddr; + vq_reconnect =3D (struct vduse_vq_state *)vaddr; + if (dev->driver_features & BIT_ULL(VIRTIO_F_RING_PACKED)) { vq_info.packed.last_avail_counter =3D vq->state.packed.last_avail_counter; @@ -1218,9 +1227,22 @@ static long vduse_dev_ioctl(struct file *file, unsig= ned int cmd, vq->state.packed.last_used_counter; vq_info.packed.last_used_idx =3D vq->state.packed.last_used_idx; - } else + /*check if the vq is reconnect, if yes then update the info*/ + if (dev_reconnect->reconnected !=3D VDUSE_NOT_RECONNECT) { + vq_info.packed.last_avail_idx =3D + vq_reconnect->packed.last_avail_idx; + vq_info.packed.last_avail_counter =3D + vq_reconnect->packed.last_avail_counter; + } + } else { vq_info.split.avail_index =3D vq->state.split.avail_index; + /*check if the vq is reconnect, if yes then update the info*/ + if (dev_reconnect->reconnected !=3D VDUSE_NOT_RECONNECT) { + vq_info.split.avail_index =3D + vq_reconnect->split.avail_index; + } + } =20 vq_info.ready =3D vq->ready; =20 --=20 2.34.3 From nobody Sun Dec 28 21:16:59 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B841C10DC3 for ; Tue, 5 Dec 2023 08:36:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1376500AbjLEIgC (ORCPT ); Tue, 5 Dec 2023 03:36:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55044 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344826AbjLEIfj (ORCPT ); Tue, 5 Dec 2023 03:35:39 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D55E111F for ; Tue, 5 Dec 2023 00:35:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1701765345; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=d/3zDHOQJcTtZVsBfjBojXcizA44t39WpNjmMG4thIs=; b=UWcePgjshK04nwsr+hJFhGVvJ+waynJ/dvlcJuFPqaGAbpb9VGaF5VuxI90RW0jRogE2M+ MTQzB3ACwkt6PyfIL9WPdjqmMFkxyVykMpczKBGlcbWy5PzHTf+RP963fa4g0iAmIqqUIq imV0/zGQ/igyPQpVnuYxtV0+gzrHolw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-597-7chujVY3OOmYGcJ8vk_svw-1; Tue, 05 Dec 2023 03:35:38 -0500 X-MC-Unique: 7chujVY3OOmYGcJ8vk_svw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0AF44879843; Tue, 5 Dec 2023 08:35:38 +0000 (UTC) Received: from server.redhat.com (unknown [10.72.112.70]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7DBE03C25; Tue, 5 Dec 2023 08:35:35 +0000 (UTC) From: Cindy Lu To: lulu@redhat.com, jasowang@redhat.com, mst@redhat.com, xieyongji@bytedance.com, linux-kernel@vger.kernel.org, maxime.coquelin@redhat.com Subject: [PATCH v3 7/7] Documentation: Add reconnect process for VDUSE Date: Tue, 5 Dec 2023 16:34:44 +0800 Message-Id: <20231205083444.3029239-8-lulu@redhat.com> In-Reply-To: <20231205083444.3029239-1-lulu@redhat.com> References: <20231205083444.3029239-1-lulu@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add a document explaining the reconnect process, including what the Userspace App needs to do and how it works with the kernel. Signed-off-by: Cindy Lu --- Documentation/userspace-api/vduse.rst | 30 +++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/Documentation/userspace-api/vduse.rst b/Documentation/userspac= e-api/vduse.rst index bdb880e01132..ec66e573f6da 100644 --- a/Documentation/userspace-api/vduse.rst +++ b/Documentation/userspace-api/vduse.rst @@ -231,3 +231,33 @@ able to start the dataplane processing as follows: after the used ring is filled. =20 For more details on the uAPI, please see include/uapi/linux/vduse.h. + +HOW VDUSE devices reconnectoin works +---------------- +0. Userspace APP checks if the device /dev/vduse/vduse_name exists. + If it does not exist, need to create the instance. + If it does exist, it means this is a reconnect and the program proceeds= to step 3. + +1. Create a new VDUSE instance with ioctl(VDUSE_CREATE_DEV) on + /dev/vduse/control. + +2. When the ioctl(VDUSE_CREATE_DEV) function is called, the kernel allocat= es memory + to synchronize the reconnect information. + +3. Userspace App need to mmap the pages to userspace + + When connecting for the first time, the userspace app must save the rec= onnect + information (struct vhost_reconnect_data) in mapped pages + + If this is reconnect,If the purpose is to reconnect, the userspace appl= ication + must verify if the saved information is suitable for reconnection. Addi= tionally, + the userspace application is required to set the "reconnected" bit in t= he + "struct vhost_reconnect_data" to "VDUSE_RECONNECT". The kernel will use= this bit + to identify if the userspace application is reconnect. + +4. Start the userspace App. + userspace APP will call ioctl VDUSE_VQ_GET_INFO to sync the vq informat= ion. + Additionally, the APP must save the vq reconnect information (struct vd= pa_vq_state) + in mapped pages while running. + +5. When the Userspace App exits, it is necessary to unmap all the reconnec= t pages. --=20 2.34.3