From nobody Tue Dec 16 12:21:18 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A1C7C4167B for ; Fri, 1 Dec 2023 10:49:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378421AbjLAKtj (ORCPT ); Fri, 1 Dec 2023 05:49:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54758 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378407AbjLAKtg (ORCPT ); Fri, 1 Dec 2023 05:49:36 -0500 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2069.outbound.protection.outlook.com [40.107.243.69]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 974EC10DF; Fri, 1 Dec 2023 02:49:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=L5adrPuLaMD2fPz/ToNnO23Hpg1439ZWtLXFaEQy1pwsDRQtq8gDt5CWNdBgZJWNQyOc4GRH6wRW3gmShE2OUBYBcRoGl7hml6KKlecxtdT4yH9Ujz98a7Jdr3TLf72vN6Rt01Rk3DTq6vu30dlSGWrqteA6+jZxCdSnWDZOlvgSiIMmdBUEeJTg3anYeEkOrm3YseuOhRFrr2YYlX+bAwJDiJShphi0YxK73K+PthK18QxKwlVTOlH9okMmEpHvEyvJShGoWpmVIiv68iq/Q9piltiXFRBYAuIfBrXygBHEWma37kfRCU6fv+SGXLlUvfXj+3FcI8CzWZWPw3s+6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PLndcNIvbHaL6btQSx+smp5PpDwR7TaSywJapj3aEO8=; b=SuEeg15CurT8PI361zSeSiVWVrMbP9h40Tx/LEriwxGzG18uyRL0OaVcuIIDiDyVP3niusiuleSjslqC0E9C2aFytc4zQQsl4d6Xaqi7haxij8IZWBwEMGPm8E4vSwU60StbKeykUWFiC2ZZ+lhE14uMU2YZ6JoR0FzgBcZ1WVGgbiOHJ4BjAWGztBzvY/Fi3sK2dhDJEeO2fOro+Jvd2Xze8VwjnaZrfLif/LW0JtXmP7dLX94M5bUIP+gyFeHDbX79JbAAEttSkpGV6WZRVRSOcoezkldINzwZZ4AnXOYo0w+aQFKSqr/6JyMrR8Norz4EO1WVdOalKtB3gdIl/A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PLndcNIvbHaL6btQSx+smp5PpDwR7TaSywJapj3aEO8=; b=CrMptfkLOfF0i46iHFH1yBzyAXDe5sKzq5aU7CxkbQdLlySllZ+JInZxyL972URmiaWdd+APo7jVP8ZVVdCFV84obkbWSAa3Ssq+MrEVmZtnOTuuK5Ai7I96avI/LruO/df128/bXDD+csvqE2nu4MUgTnRCqTqP6I892q7/KN3qVNeiS4nmKNBBKILC8vwBkLSN36RKXSWuRqmqstnePgarjZ/M955wck+PKxE1JaV/eAwRQH1etTM/zQNqRJpeBM/v4RAFwiifs3A0AVsNwr1rZ1UwHOfYcIOh/p7QPQKWI14OttRnzc+yQWTUdIuFItYRxj5mcl0Bbb32qV3Qmg== Received: from MN2PR20CA0063.namprd20.prod.outlook.com (2603:10b6:208:235::32) by SJ2PR12MB8831.namprd12.prod.outlook.com (2603:10b6:a03:4d0::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.23; Fri, 1 Dec 2023 10:49:40 +0000 Received: from MN1PEPF0000ECDA.namprd02.prod.outlook.com (2603:10b6:208:235::4) by MN2PR20CA0063.outlook.office365.com (2603:10b6:208:235::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7025.28 via Frontend Transport; Fri, 1 Dec 2023 10:49:39 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by MN1PEPF0000ECDA.mail.protection.outlook.com (10.167.242.134) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.17 via Frontend Transport; Fri, 1 Dec 2023 10:49:39 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Fri, 1 Dec 2023 02:49:22 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Fri, 1 Dec 2023 02:49:22 -0800 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Fri, 1 Dec 2023 02:49:18 -0800 From: Dragos Tatulea To: "Michael S . Tsirkin" , Jason Wang , Eugenio Perez Martin , Si-Wei Liu , Saeed Mahameed , Leon Romanovsky , CC: Dragos Tatulea , , , Gal Pressman , Parav Pandit , Xuan Zhuo Subject: [PATCH mlx5-vhost 1/7] vdpa/mlx5: Expose resumable vq capability Date: Fri, 1 Dec 2023 12:48:51 +0200 Message-ID: <20231201104857.665737-2-dtatulea@nvidia.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231201104857.665737-1-dtatulea@nvidia.com> References: <20231201104857.665737-1-dtatulea@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000ECDA:EE_|SJ2PR12MB8831:EE_ X-MS-Office365-Filtering-Correlation-Id: 5d504498-a8bc-48a5-7653-08dbf25b37fd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: O7Z1MNOFn+dnL8jzlmONFP++umlm3SVksvhoychYLULUWaDc6HOCaXevpVjgJ7OvPLChbtyHGBlAjWOjJKuP41lm8UuIJmFS3WS3tyNg5k9BqmAzZF5cwxDjLiGWat33vaLalzrsjWaN+CDAhKoVyKAFVfhJbsTvfmz6GCriJLCyDB2rkpF4STq9vEVTiiJ2DBbuoQXnaZQYDpRfwDV5+vE/qHyEH5C7a+9+2hYVf8B2eUBc1t3tV0phMseFvNSouYiTxA6J6fTMYRlJlOM+cOxUo7cxNx+m7eCJQvZxUpHqOul4l9XBExv09D3rlwa4UWisVEPfZtcjne9dMijCeJEhlCLhLko5kO0nrkCb13r+9HpTRXFum9DAgN1P7LxrFoXksIOe8FkYAiRaeC/YlwAbtpfyEYG5Ti15IuwO0KqmLqJVi4mAs/kTMJtsbcoyb95iKMhshnjinuAleI91tkYLD9j4WVv6khcB7e7N/2InWz3UGLVzt49026G6Cs7T39cp+VwId3uxSw6DpOsR08fIgmGb9XLE34WtPd9T/XEWYIZZKIG2TVqBAM5zyp3BmB/+heb79ck6o2Duz8uTRMuTroF7yDutRpZ/jqoRrexdG03QqHQbZTuwMK40UryVcHO7KNIA64jvAY8ks15FIBYxtv5z6MtdpgatTW6riS8A5BERIEw4L6GoQctlo8LlenvDuTWDb8OPz4RkbBmUrA5YK+7wpiqoebAKGz+7CNIwgipRq+/ca6Yh7miMTfvm X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(346002)(376002)(396003)(136003)(230922051799003)(186009)(451199024)(82310400011)(64100799003)(1800799012)(46966006)(40470700004)(36840700001)(40460700003)(356005)(7636003)(82740400003)(86362001)(36756003)(478600001)(2906002)(6666004)(4744005)(4326008)(5660300002)(336012)(41300700001)(426003)(2616005)(40480700001)(26005)(1076003)(316002)(8936002)(8676002)(110136005)(70586007)(54906003)(47076005)(36860700001)(70206006)(83380400001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Dec 2023 10:49:39.6329 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5d504498-a8bc-48a5-7653-08dbf25b37fd X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000ECDA.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB8831 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Necessary for checking if resumable vqs are supported by the hardware. Actual support will be added in a downstream patch. Signed-off-by: Dragos Tatulea --- include/linux/mlx5/mlx5_ifc.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h index 3388007c645f..21dcfa034b7b 100644 --- a/include/linux/mlx5/mlx5_ifc.h +++ b/include/linux/mlx5/mlx5_ifc.h @@ -1235,7 +1235,8 @@ struct mlx5_ifc_virtio_emulation_cap_bits { =20 u8 reserved_at_c0[0x13]; u8 desc_group_mkey_supported[0x1]; - u8 reserved_at_d4[0xc]; + u8 freeze_to_rdy_supported[0x1]; + u8 reserved_at_d5[0xb]; =20 u8 reserved_at_e0[0x20]; =20 --=20 2.42.0 From nobody Tue Dec 16 12:21:18 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 048ABC4167B for ; Fri, 1 Dec 2023 10:49:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378429AbjLAKtm (ORCPT ); Fri, 1 Dec 2023 05:49:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54838 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378407AbjLAKtk (ORCPT ); Fri, 1 Dec 2023 05:49:40 -0500 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2053.outbound.protection.outlook.com [40.107.94.53]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C4E7B10F1; Fri, 1 Dec 2023 02:49:45 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=M1cjpykQXSsdTKf/PZUA6ZuRNGqVOyN6XIsXc8SRktZPPf3y4NpO6EK6LvDLWH5lqTqwJKmhjTmzORP0YKk5WQHy9byeaAeUMC5hNaPQrabEZzhk9PsrcR/sY3l+VZUGLnTXhy9q75bBG5V3DiUzxUg+ArZApTJfkJlbnYErIQ2sFPO1/3CwutmGafZIIWRSsOPQXZzOc1KzUf/TtxmAYy6l6Hd9gzalSsgWDiPRqfCcfM2x91GD71ncZD99WgRAAiZu5to0yaKQku5VkuKP2kW5A1u4vclLlfZvWt7kF0h4Eas5gwbuSSCVTeOQk2PyZeSpI+Qg49IfO2f4lgx5xg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=b6hDxocxpj5OzTimObjmwJdf213sHeAV07ZmX51gJo0=; b=Pp28i5A3wWb+90VhQZV4AWWVwtTJ9iyW0ESY4aF3b8FG9TPlHwZhs9Y1SIfc1t3o23yyxIkbHhMxhxNc7XT2bxmUsdtv/w7AvlyrPCQIv6OWkAqN8LYC2PJC3+tq9etBRg25u6QGpHJQZGgZ5/oh+18tUtfxXHtQ0a4MUM4K322e9VwQGw7b/BktjvzaMJNbilC06Q1oe5QrV3GC92vg6cSDdGZGkcoQewE3RpZmLforQhKkYylPf2EspWoHCUPOz0k5a5ycG/iuMTqFvXDXwyIKn3U4rGZw4Xv7ETrhJnbtV9jXf6z4UrwHX0pzqWgJEBDYSjmWv/kK4VkpQJTz3g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=b6hDxocxpj5OzTimObjmwJdf213sHeAV07ZmX51gJo0=; b=QjwzzcxdEbxoDa1DWwjep5QTohhbK3bS0rPeWcKa1HABcGIytiK/C+CjJ9Hf4vxlxAhVaiz5lM6aDQH/kT1dxDaxj+Uv31HliDjahUKEKPGwhA1/eqipqqMA68Llys3Ple9fxeb31ng5O/NqRh38pjsqZcZ+qNWB2NcNxU8VCC8PM46RVurS92L7AqrOSktnD/5fYSztdOPqbuH51k5yUeSdyyGdqCSyJNJ/v8LmEZ394XpFb0lsiBKgKzEkrzed7+wRqyC+JQkOvaxbJ3zXSs1BOnndwjgftIpSXdkaZrwrvNCOG1DSmyLRPl+mQ7xZsU1XhnO+j1WI15DuDGMyKQ== Received: from BL0PR02CA0079.namprd02.prod.outlook.com (2603:10b6:208:51::20) by PH0PR12MB8052.namprd12.prod.outlook.com (2603:10b6:510:28b::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.24; Fri, 1 Dec 2023 10:49:43 +0000 Received: from BL02EPF0001A0FA.namprd03.prod.outlook.com (2603:10b6:208:51:cafe::a7) by BL0PR02CA0079.outlook.office365.com (2603:10b6:208:51::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.23 via Frontend Transport; Fri, 1 Dec 2023 10:49:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BL02EPF0001A0FA.mail.protection.outlook.com (10.167.242.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.17 via Frontend Transport; Fri, 1 Dec 2023 10:49:42 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Fri, 1 Dec 2023 02:49:26 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Fri, 1 Dec 2023 02:49:26 -0800 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Fri, 1 Dec 2023 02:49:23 -0800 From: Dragos Tatulea To: "Michael S . Tsirkin" , Jason Wang , Eugenio Perez Martin , Si-Wei Liu , Saeed Mahameed , Leon Romanovsky , CC: Dragos Tatulea , , , Gal Pressman , Parav Pandit , Xuan Zhuo Subject: [PATCH vhost 2/7] vdpa/mlx5: Split function into locked and unlocked variants Date: Fri, 1 Dec 2023 12:48:52 +0200 Message-ID: <20231201104857.665737-3-dtatulea@nvidia.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231201104857.665737-1-dtatulea@nvidia.com> References: <20231201104857.665737-1-dtatulea@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF0001A0FA:EE_|PH0PR12MB8052:EE_ X-MS-Office365-Filtering-Correlation-Id: 10bf0976-c178-43af-b9f4-08dbf25b39eb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: OVHDmkUXYoslfIzi8AZ0BIwVHWpuw9cJXTB44HU739z5aA92R7LNE0gGSrdLXEpS0Ts3SEUoWAwCjIfpJdzjDfPhHXFP6XLQHVfuIJ3eVXYgMged7uFLgVzL7xAuWxjnFl6avcyfxTM2sc39oLR9y5nKSzJybv0EFfHwmZ7mkmztZMcaNf+myo6Ei4z8MDNjQYGvtwwAuPxWgdMC6MVQK6sItfl9ZPlaWuJDbWoHaXP9eVdn99V7hCN0QqOF7OT7YlYnlRJnEo9rs6sWtOT8RKgXy60JpKXwVu5KUBOTccIfqGTXbeukQwBXJSslnTroXt2q2RrF0CTSrgoYkoz7yTMjnpBXNw9NDaCzcbN5Oau0VH8p80HI1tYVnv9bLzSb7P0CKm1zqwr7oB1bKnWqFfbauE2H0rbFfIJmXyhor8edIyoq9Ld9S7Qkj0A7Lk7AXeLvNkVbgHIY0BKAHwKiB/UaRP+CmSFXwbbqZqi36JSMPkcT/0LvSLo6WuUhgPBgdkRszHZtp36HhrQRxhN99CWo8QrAkm9hmNCTFRFudTVxtwI+L411YiZzGFAyBrlEr++fEsWCx0iooE3bHy+8okuvSxzzn7ZAC/MsEMolqtM/oClHj4oyfLPblpDEA8Pd+dGXLlgKOoeH3CME+glMZyd7PR8kDk9x07+mwI+votry0C6MvmZlebCDPTc7jcPOnZCb4FRdkIliZkihCHObgV1Y4aR/rymw4Gg/YZXwCvfmFgirsh+RGD/zzt7BnxFm X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(396003)(376002)(346002)(39860400002)(136003)(230922051799003)(186009)(64100799003)(82310400011)(451199024)(1800799012)(40470700004)(36840700001)(46966006)(40480700001)(86362001)(40460700003)(70586007)(70206006)(7636003)(356005)(82740400003)(36756003)(83380400001)(54906003)(47076005)(426003)(36860700001)(1076003)(26005)(2616005)(6666004)(110136005)(2906002)(316002)(4326008)(8676002)(5660300002)(336012)(8936002)(478600001)(41300700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Dec 2023 10:49:42.8877 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 10bf0976-c178-43af-b9f4-08dbf25b39eb X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF0001A0FA.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB8052 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" mlx5_vdpa_destroy_mr contains more logic than _mlx5_vdpa_destroy_mr. There is no reason for this to be the case. All the logic can go into the unlocked variant. Using the unlocked version is needed in a follow-up patch. And it also makes it more consistent with mlx5_vdpa_create_mr. Signed-off-by: Dragos Tatulea Acked-by: Eugenio P=C3=A9rez --- drivers/vdpa/mlx5/core/mr.c | 31 ++++++++++++++++--------------- 1 file changed, 16 insertions(+), 15 deletions(-) diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c index 2197c46e563a..8c80d9e77935 100644 --- a/drivers/vdpa/mlx5/core/mr.c +++ b/drivers/vdpa/mlx5/core/mr.c @@ -498,32 +498,32 @@ static void destroy_user_mr(struct mlx5_vdpa_dev *mvd= ev, struct mlx5_vdpa_mr *mr =20 static void _mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, struct mlx5= _vdpa_mr *mr) { + if (!mr) + return; + if (mr->user_mr) destroy_user_mr(mvdev, mr); else destroy_dma_mr(mvdev, mr); =20 + for (int i =3D 0; i < MLX5_VDPA_NUM_AS; i++) { + if (mvdev->mr[i] =3D=3D mr) + mvdev->mr[i] =3D NULL; + } + vhost_iotlb_free(mr->iotlb); + + kfree(mr); } =20 void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_mr *mr) { - if (!mr) - return; - mutex_lock(&mvdev->mr_mtx); =20 _mlx5_vdpa_destroy_mr(mvdev, mr); =20 - for (int i =3D 0; i < MLX5_VDPA_NUM_AS; i++) { - if (mvdev->mr[i] =3D=3D mr) - mvdev->mr[i] =3D NULL; - } - mutex_unlock(&mvdev->mr_mtx); - - kfree(mr); } =20 void mlx5_vdpa_update_mr(struct mlx5_vdpa_dev *mvdev, @@ -535,10 +535,7 @@ void mlx5_vdpa_update_mr(struct mlx5_vdpa_dev *mvdev, mutex_lock(&mvdev->mr_mtx); =20 mvdev->mr[asid] =3D new_mr; - if (old_mr) { - _mlx5_vdpa_destroy_mr(mvdev, old_mr); - kfree(old_mr); - } + _mlx5_vdpa_destroy_mr(mvdev, old_mr); =20 mutex_unlock(&mvdev->mr_mtx); =20 @@ -546,8 +543,12 @@ void mlx5_vdpa_update_mr(struct mlx5_vdpa_dev *mvdev, =20 void mlx5_vdpa_destroy_mr_resources(struct mlx5_vdpa_dev *mvdev) { + mutex_lock(&mvdev->mr_mtx); + for (int i =3D 0; i < MLX5_VDPA_NUM_AS; i++) - mlx5_vdpa_destroy_mr(mvdev, mvdev->mr[i]); + _mlx5_vdpa_destroy_mr(mvdev, mvdev->mr[i]); + + mutex_unlock(&mvdev->mr_mtx); =20 prune_iotlb(mvdev->cvq.iotlb); } --=20 2.42.0 From nobody Tue Dec 16 12:21:18 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EFCE1C4167B for ; Fri, 1 Dec 2023 10:49:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378468AbjLAKtv (ORCPT ); Fri, 1 Dec 2023 05:49:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36084 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378441AbjLAKtr (ORCPT ); Fri, 1 Dec 2023 05:49:47 -0500 Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on2089.outbound.protection.outlook.com [40.107.95.89]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 793D2170E; Fri, 1 Dec 2023 02:49:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=D4v5PMou7X0UrRJFP3JtnRIa4YEALvESf0HNWNWkOD7VGNEle8gjf7s3tw7L9ndN/l5LM/ahG3ffPOTLjMEjt5vxgtkuy2WrDRVE9NtwjDj8X8iaVRBMW9M2IZq9xPaH6xxTLJh63HChabvvYmtvIKhshtfXKpb+cuidwiiojidz2nI2Mx70XqaJbt6ldDjHpetZ2Col41a4+xN91Q44+wJhVD7rzwVDGbR+in+NOoHNosrT0DtcEYRDG5bxyoHOE7auRDFBAdyrvcWA0b5WqhQklVyaGLpIgMV1mRRBKW+UoFAjkaG0MApzOo2tGYPYlCriXhNySiTN7OfTj9cHQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AbkkSRuhAnoXxmmmaWrUCRkEl/cD3hqskadOAIqvb/g=; b=SQLmG5IkCBIiUo5TNv6evMZhohvOPNdqI7LHEcsB1X9jZ/1OuYD3j4QztJDUX0R9EOmraqf5ReFaitq2qFqdENjpGsk4x8RKihGW4i2QY5IvP2OuSIzbVNEs95G8BOYm9yOzT7FV/mdumIBOT5UTuy+eRh6CyiEKgYkeB5TXqUWxSaKQmQbxTlX/0mQyBmpYzXuH3eK0T3fAh2t9ZPSp0pnKnATANw0rkx/WWr6E1yCx/nQYeX/FvnqBLftl//r6ZTyZQ7v1ZbsvZGXBt4dh2oAX3gkDdHh1YtzI8AwaEG4pleqB/Hi1U3ng5jf+fVQRvjKmusUrJgxjAzUCFnFzPg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AbkkSRuhAnoXxmmmaWrUCRkEl/cD3hqskadOAIqvb/g=; b=G1dsSuB8rtBLNM3Aa4T1+ml7mYl+Rw+J5chgMtFHjg/Kk0sYbSNaj4LSX5LJ8JGec5mjnpdVJ1Pp/ZvnqSFa0JEUtwvK2fZZ+S7TTBw9wG/eprZ6tOqD9IW2avgefe+Y4hNeW/v5bCpp/3Ebqkv22eFhDe4ka08xh1MLHxRJzj9IrOG2YddracIFYrFRk5BMGNt10TnYQZvZEqd6tgNA/nx9E5PwmF7QPRoAxIRPOhZyRba7N0VaAD1XW4se7eMtBMr9R2r/goIDfcHMKebkG3q+cBhcvSUrEnaZ/Ul+IWz6DjAfYQVA9oO5ZPq46H9YcgyXRtwPJy/eJArKUB2Igg== Received: from BL0PR02CA0103.namprd02.prod.outlook.com (2603:10b6:208:51::44) by DM4PR12MB5723.namprd12.prod.outlook.com (2603:10b6:8:5e::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Fri, 1 Dec 2023 10:49:47 +0000 Received: from BL02EPF0001A0FA.namprd03.prod.outlook.com (2603:10b6:208:51:cafe::19) by BL0PR02CA0103.outlook.office365.com (2603:10b6:208:51::44) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.22 via Frontend Transport; Fri, 1 Dec 2023 10:49:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BL02EPF0001A0FA.mail.protection.outlook.com (10.167.242.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.17 via Frontend Transport; Fri, 1 Dec 2023 10:49:47 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Fri, 1 Dec 2023 02:49:31 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Fri, 1 Dec 2023 02:49:30 -0800 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Fri, 1 Dec 2023 02:49:27 -0800 From: Dragos Tatulea To: "Michael S . Tsirkin" , Jason Wang , Eugenio Perez Martin , Si-Wei Liu , Saeed Mahameed , Leon Romanovsky , CC: Dragos Tatulea , , , Gal Pressman , Parav Pandit , Xuan Zhuo Subject: [PATCH vhost 3/7] vdpa/mlx5: Allow modifying multiple vq fields in one modify command Date: Fri, 1 Dec 2023 12:48:53 +0200 Message-ID: <20231201104857.665737-4-dtatulea@nvidia.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231201104857.665737-1-dtatulea@nvidia.com> References: <20231201104857.665737-1-dtatulea@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF0001A0FA:EE_|DM4PR12MB5723:EE_ X-MS-Office365-Filtering-Correlation-Id: 6179766c-a5ec-41c3-c62f-08dbf25b3c9c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5kJWC9zPolktG+YSeSwWTnGFVcE+I04TD1wn941VTluQLB9kPtVGetdsAUq2uTLOHmoK3cYlu4WH9I2hr2uAkb2/Pyin0rPCjLWDUtqhMBP9Ip/QzfRqmXzlkJkjZSb8r6cFvMYOpl7nHstwWpwvMH5LfQSoCOlzJEn/PlVYLbBWdwc/GW3x6ixBpOolZPrXkjVe0Zrh21F9VYJK95o6SM/h2IqLZZdIDgThSdRRgkThwgLuTS/xroZqysvFwPtopXOyZAJ8QiD6XBOPJ+Pad/ELF/t8gRUHw8j20G8PrvQmAnvAjGW2md72T8AYLa7VCekir+M1Z8lsy9A+hLr2GwXsUjW7cNP3Z3AHjfis/CEFUqPT8MLdDfXYRTRH1+W12sN7gAE5HAQ1u3u4WeBF+AoflZ5rGQ9pZ36r4BN1wprcDdvWAozLlQoPAe/JJhXv3YAtHgD/yER3XWkQubMwWkxca3895jpWmBI+HYdfNrPHu/4dzISJ7v1I6dT4L1S58z9AHAikWvwouj0FM6KjQRTdD/3VLI5Hh3FeUvCrduXfbwRkpdDXT7aBR5mTK0Sincz8dsEVcXrWft4Nqeaf9qDf26cO0/EWjZ697BMFs7JVeps6ZrKrHFcd0zMWNk+DmmxR7bKv94IScFZmFVIxrEEBhQzSqA8A3zScCGvn9b9sHYLKXdFFQsgGug/ihg4HRyjagbe6l/7pxvwEekds/WM/PnzlNK89qbwGo68vqk4Jb5dJbu93FIKSNE/jw8I+ X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(396003)(376002)(346002)(39860400002)(136003)(230922051799003)(186009)(82310400011)(64100799003)(451199024)(1800799012)(40470700004)(36840700001)(46966006)(316002)(66899024)(54906003)(70206006)(110136005)(70586007)(478600001)(40460700003)(6666004)(5660300002)(41300700001)(36756003)(2906002)(8676002)(8936002)(86362001)(336012)(426003)(2616005)(26005)(4326008)(1076003)(40480700001)(356005)(47076005)(7636003)(36860700001)(82740400003)(83380400001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Dec 2023 10:49:47.4034 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6179766c-a5ec-41c3-c62f-08dbf25b3c9c X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF0001A0FA.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5723 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add a bitmask variable that tracks hw vq field changes that are supposed to be modified on next hw vq change command. This will be useful to set multiple vq fields when resuming the vq. The state needs to remain as a parameter as it doesn't make sense to make it part of the vq struct: fw_state is updated only after a successful command. Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/net/mlx5_vnet.c | 48 +++++++++++++++++++++++++------ 1 file changed, 40 insertions(+), 8 deletions(-) diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5= _vnet.c index 12ac3397f39b..d06285e46fe2 100644 --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c @@ -120,6 +120,9 @@ struct mlx5_vdpa_virtqueue { u16 avail_idx; u16 used_idx; int fw_state; + + u64 modified_fields; + struct msi_map map; =20 /* keep last in the struct */ @@ -1181,7 +1184,19 @@ static bool is_valid_state_change(int oldstate, int = newstate) } } =20 -static int modify_virtqueue(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_v= irtqueue *mvq, int state) +static bool modifiable_virtqueue_fields(struct mlx5_vdpa_virtqueue *mvq) +{ + /* Only state is always modifiable */ + if (mvq->modified_fields & ~MLX5_VIRTQ_MODIFY_MASK_STATE) + return mvq->fw_state =3D=3D MLX5_VIRTIO_NET_Q_OBJECT_STATE_INIT || + mvq->fw_state =3D=3D MLX5_VIRTIO_NET_Q_OBJECT_STATE_SUSPEND; + + return true; +} + +static int modify_virtqueue(struct mlx5_vdpa_net *ndev, + struct mlx5_vdpa_virtqueue *mvq, + int state) { int inlen =3D MLX5_ST_SZ_BYTES(modify_virtio_net_q_in); u32 out[MLX5_ST_SZ_DW(modify_virtio_net_q_out)] =3D {}; @@ -1193,6 +1208,9 @@ static int modify_virtqueue(struct mlx5_vdpa_net *nde= v, struct mlx5_vdpa_virtque if (mvq->fw_state =3D=3D MLX5_VIRTIO_NET_Q_OBJECT_NONE) return 0; =20 + if (!modifiable_virtqueue_fields(mvq)) + return -EINVAL; + if (!is_valid_state_change(mvq->fw_state, state)) return -EINVAL; =20 @@ -1208,17 +1226,28 @@ static int modify_virtqueue(struct mlx5_vdpa_net *n= dev, struct mlx5_vdpa_virtque MLX5_SET(general_obj_in_cmd_hdr, cmd_hdr, uid, ndev->mvdev.res.uid); =20 obj_context =3D MLX5_ADDR_OF(modify_virtio_net_q_in, in, obj_context); - MLX5_SET64(virtio_net_q_object, obj_context, modify_field_select, - MLX5_VIRTQ_MODIFY_MASK_STATE); - MLX5_SET(virtio_net_q_object, obj_context, state, state); + if (mvq->modified_fields & MLX5_VIRTQ_MODIFY_MASK_STATE) + MLX5_SET(virtio_net_q_object, obj_context, state, state); + + MLX5_SET64(virtio_net_q_object, obj_context, modify_field_select, mvq->mo= dified_fields); err =3D mlx5_cmd_exec(ndev->mvdev.mdev, in, inlen, out, sizeof(out)); kfree(in); if (!err) mvq->fw_state =3D state; =20 + mvq->modified_fields =3D 0; + return err; } =20 +static int modify_virtqueue_state(struct mlx5_vdpa_net *ndev, + struct mlx5_vdpa_virtqueue *mvq, + unsigned int state) +{ + mvq->modified_fields |=3D MLX5_VIRTQ_MODIFY_MASK_STATE; + return modify_virtqueue(ndev, mvq, state); +} + static int counter_set_alloc(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_= virtqueue *mvq) { u32 in[MLX5_ST_SZ_DW(create_virtio_q_counters_in)] =3D {}; @@ -1347,7 +1376,7 @@ static int setup_vq(struct mlx5_vdpa_net *ndev, struc= t mlx5_vdpa_virtqueue *mvq) goto err_vq; =20 if (mvq->ready) { - err =3D modify_virtqueue(ndev, mvq, MLX5_VIRTIO_NET_Q_OBJECT_STATE_RDY); + err =3D modify_virtqueue_state(ndev, mvq, MLX5_VIRTIO_NET_Q_OBJECT_STATE= _RDY); if (err) { mlx5_vdpa_warn(&ndev->mvdev, "failed to modify to ready vq idx %d(%d)\n= ", idx, err); @@ -1382,7 +1411,7 @@ static void suspend_vq(struct mlx5_vdpa_net *ndev, st= ruct mlx5_vdpa_virtqueue *m if (mvq->fw_state !=3D MLX5_VIRTIO_NET_Q_OBJECT_STATE_RDY) return; =20 - if (modify_virtqueue(ndev, mvq, MLX5_VIRTIO_NET_Q_OBJECT_STATE_SUSPEND)) + if (modify_virtqueue_state(ndev, mvq, MLX5_VIRTIO_NET_Q_OBJECT_STATE_SUSP= END)) mlx5_vdpa_warn(&ndev->mvdev, "modify to suspend failed\n"); =20 if (query_virtqueue(ndev, mvq, &attr)) { @@ -1407,6 +1436,7 @@ static void teardown_vq(struct mlx5_vdpa_net *ndev, s= truct mlx5_vdpa_virtqueue * return; =20 suspend_vq(ndev, mvq); + mvq->modified_fields =3D 0; destroy_virtqueue(ndev, mvq); dealloc_vector(ndev, mvq); counter_set_dealloc(ndev, mvq); @@ -2207,7 +2237,7 @@ static void mlx5_vdpa_set_vq_ready(struct vdpa_device= *vdev, u16 idx, bool ready if (!ready) { suspend_vq(ndev, mvq); } else { - err =3D modify_virtqueue(ndev, mvq, MLX5_VIRTIO_NET_Q_OBJECT_STATE_RDY); + err =3D modify_virtqueue_state(ndev, mvq, MLX5_VIRTIO_NET_Q_OBJECT_STATE= _RDY); if (err) { mlx5_vdpa_warn(mvdev, "modify VQ %d to ready failed (%d)\n", idx, err); ready =3D false; @@ -2804,8 +2834,10 @@ static void clear_vqs_ready(struct mlx5_vdpa_net *nd= ev) { int i; =20 - for (i =3D 0; i < ndev->mvdev.max_vqs; i++) + for (i =3D 0; i < ndev->mvdev.max_vqs; i++) { ndev->vqs[i].ready =3D false; + ndev->vqs[i].modified_fields =3D 0; + } =20 ndev->mvdev.cvq.ready =3D false; } --=20 2.42.0 From nobody Tue Dec 16 12:21:18 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B632CC4167B for ; Fri, 1 Dec 2023 10:50:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378427AbjLAKuA (ORCPT ); Fri, 1 Dec 2023 05:50:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36122 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378387AbjLAKtu (ORCPT ); Fri, 1 Dec 2023 05:49:50 -0500 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2040.outbound.protection.outlook.com [40.107.237.40]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 464E41704; Fri, 1 Dec 2023 02:49:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Co24YRV2t+8sYdjvGE9HfiCMdhV8uHCUbLqu/wwXMDZ4Urmdv07Ykv9axzuFnAWgy3OTLYL0hGAVwKd4KcgWY9Pfc3qNzgNvV1bPMQGwssjaePV2Er7irgu3pHxRHL+I4FoCEPyXh01tprQFv9wnwgX2cRbUl0O6+gnM1As2a+yQgWVdjilXGvNTCzSO2//WHQG9LFs2UyPDZcRWyj0KgjB43A87nERuhc/AuEkg0nuhmJH44F4FjdpoIWNU5usc/+sRVZZQ0cybK91IIY6Bgiz1zpk6TDeEmF6LIoBU/w37dyujaC3pN27EG9fkPUXPBUA3r0LSEl9F8mn1TZYJ8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ept1Tf2SnzBMD4BLVt8Odf9sFEtXbZW1+WTxNC48gAI=; b=lx8wYttaPgSp9wKXya8KGxCVOhxk9gnTu3m29+/DsrKGAymMq1XvgdMaQzgjUxFfaYKDTMbOgjKxeTSMIWso7gDrniPvkN9Y3c8dYWxveRPl2f9qASVu9QJI91P6XFB7GzeRk6+TM2VoQOFN3cI5yOMBLxgxgedfxurAiSzMOASLQ5yulP9Xlear7bB4dVOjFJsJwbdcCJSQMqU0I1jAntx7eEs92hjtxHNqVS9mFJ+/nrG6qA22hcO21ngDmO2tosmtrlzMuns86J3KJoE19lFG55i5LFsaZdiEp1/EZ1KaGIqbC9LH2KEajA5sZhBgravdafskedJ9R5FCVbz6oA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ept1Tf2SnzBMD4BLVt8Odf9sFEtXbZW1+WTxNC48gAI=; b=OysHD+s5UJknLkE/Oi7VzC9Mgp2BW56FtV9XSpe35wkyxKYe7CYHJR4iJwgZ4CfW07++mPugZae8nl7mbWLtUU1blWTneX/d5LWqjBIMk6FQ9sQkKrecYTXTvI9fQ6zCUos74QfpTpU4QLu0Trkx1uukFErZ6X2PDdIExXycw3bxRNQ6uf5fLIKp+jiYPrWQ+kM4PG9LYVVWkQ/DvrzTiroNOjz4T6i+yVmQ6gE4dwBn+PAmDmiqic1ezczRxh7zkLXBxgTSlzEQBJX3n3KE7SQNrZUPfP4aWP/OseGABvdcCtBioHOysS9qfz/YXjaIR2wld9F2/t/UnnzvUHijOA== Received: from MN2PR20CA0060.namprd20.prod.outlook.com (2603:10b6:208:235::29) by IA1PR12MB9031.namprd12.prod.outlook.com (2603:10b6:208:3f9::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.24; Fri, 1 Dec 2023 10:49:52 +0000 Received: from MN1PEPF0000ECDA.namprd02.prod.outlook.com (2603:10b6:208:235:cafe::42) by MN2PR20CA0060.outlook.office365.com (2603:10b6:208:235::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7025.29 via Frontend Transport; Fri, 1 Dec 2023 10:49:52 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by MN1PEPF0000ECDA.mail.protection.outlook.com (10.167.242.134) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.17 via Frontend Transport; Fri, 1 Dec 2023 10:49:52 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Fri, 1 Dec 2023 02:49:34 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Fri, 1 Dec 2023 02:49:34 -0800 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Fri, 1 Dec 2023 02:49:31 -0800 From: Dragos Tatulea To: "Michael S . Tsirkin" , Jason Wang , Eugenio Perez Martin , Si-Wei Liu , Saeed Mahameed , Leon Romanovsky , CC: Dragos Tatulea , , , Gal Pressman , Parav Pandit , Xuan Zhuo Subject: [PATCH vhost 4/7] vdpa/mlx5: Introduce per vq and device resume Date: Fri, 1 Dec 2023 12:48:54 +0200 Message-ID: <20231201104857.665737-5-dtatulea@nvidia.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231201104857.665737-1-dtatulea@nvidia.com> References: <20231201104857.665737-1-dtatulea@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000ECDA:EE_|IA1PR12MB9031:EE_ X-MS-Office365-Filtering-Correlation-Id: 5d62fbca-2648-4b26-2e1f-08dbf25b3f92 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5v0hwFR4APWRLGbjDBeNs3gzx7ZIUg6F21gHalWJRNgFd5xzG3a5M21Tv9SXtLI+zI93liIwqAng1DX82Sz1Y/iksr5Xpg2ic7fqdzl5S36GnyIfvgyiuCQnR1XeEDOV2SUi6QY+zk8/j1U1PDBVO7fL4/9WZWum+a8HHerVqZEqXE1plBumtFUfRppVJ/0qQgReFMOULssEQzTc00vM/hisyf/9o68r5TS58mu4D4Q9bFWEgcX/OF4D0O/Q5/cJcnsNsCLZavp0dROlhLLA2uaqHi5t48lTnwfCnY3bK4Ivc6FS3xzFQCf9B9Hz+0RaYuErfhPRmkwDl5rw/DL+1Mh3cnNNTYM3wnAptXLDV9X5tFXjRkFWrE5D9ypjcx5IjizSjiqSNc6AfDq6i5oR3Vuqn1P+uqXbK/anDtqll8bnHDIq2XQM/+7vdFjQ8MZlA9kcIjxfEynonZTCj6xDsU+j1MN9i2wJ9dD+/aPMZnVJiGl5dotuJs+ZoyA6z5HiOGNkhSzRMKCuVthu//AQiFbmm7zgOpay0HfetG9qwWVn92tG130eWrkOr+5BurfPO4KLu1+7RzJEqKhPQwnKs4/8BpbnfmQQQHYZ/4Ypt+2f5O/i8w1H9v9BC5OAPYLIdH2oND7v+uTarnxvbnq8V4DDIRtodSutP6fL3SQx+xcDb5dMo1WSnn1o5Lwyv5qIpapb5wfh3OEb2CcQ0KZDhQ7/QqrMQ5jFQG424JQb7Rj7pmzJHPwg5e2F3/1dsB/D X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(396003)(376002)(346002)(39860400002)(136003)(230922051799003)(186009)(64100799003)(82310400011)(451199024)(1800799012)(40470700004)(36840700001)(46966006)(40480700001)(86362001)(40460700003)(70586007)(70206006)(7636003)(356005)(82740400003)(36756003)(83380400001)(54906003)(47076005)(426003)(36860700001)(1076003)(26005)(2616005)(6666004)(110136005)(2906002)(316002)(4326008)(8676002)(5660300002)(336012)(8936002)(478600001)(41300700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Dec 2023 10:49:52.3672 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5d62fbca-2648-4b26-2e1f-08dbf25b3f92 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000ECDA.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB9031 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Implement vdpa vq and device resume if capability detected. Add support for suspend -> ready state change. Signed-off-by: Dragos Tatulea Acked-by: Eugenio P=C3=A9rez --- drivers/vdpa/mlx5/net/mlx5_vnet.c | 67 +++++++++++++++++++++++++++---- 1 file changed, 60 insertions(+), 7 deletions(-) diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5= _vnet.c index d06285e46fe2..68e534cb57e2 100644 --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c @@ -1170,7 +1170,12 @@ static int query_virtqueue(struct mlx5_vdpa_net *nde= v, struct mlx5_vdpa_virtqueu return err; } =20 -static bool is_valid_state_change(int oldstate, int newstate) +static bool is_resumable(struct mlx5_vdpa_net *ndev) +{ + return ndev->mvdev.vdev.config->resume; +} + +static bool is_valid_state_change(int oldstate, int newstate, bool resumab= le) { switch (oldstate) { case MLX5_VIRTIO_NET_Q_OBJECT_STATE_INIT: @@ -1178,6 +1183,7 @@ static bool is_valid_state_change(int oldstate, int n= ewstate) case MLX5_VIRTIO_NET_Q_OBJECT_STATE_RDY: return newstate =3D=3D MLX5_VIRTIO_NET_Q_OBJECT_STATE_SUSPEND; case MLX5_VIRTIO_NET_Q_OBJECT_STATE_SUSPEND: + return resumable ? newstate =3D=3D MLX5_VIRTIO_NET_Q_OBJECT_STATE_RDY : = false; case MLX5_VIRTIO_NET_Q_OBJECT_STATE_ERR: default: return false; @@ -1200,6 +1206,7 @@ static int modify_virtqueue(struct mlx5_vdpa_net *nde= v, { int inlen =3D MLX5_ST_SZ_BYTES(modify_virtio_net_q_in); u32 out[MLX5_ST_SZ_DW(modify_virtio_net_q_out)] =3D {}; + bool state_change =3D false; void *obj_context; void *cmd_hdr; void *in; @@ -1211,9 +1218,6 @@ static int modify_virtqueue(struct mlx5_vdpa_net *nde= v, if (!modifiable_virtqueue_fields(mvq)) return -EINVAL; =20 - if (!is_valid_state_change(mvq->fw_state, state)) - return -EINVAL; - in =3D kzalloc(inlen, GFP_KERNEL); if (!in) return -ENOMEM; @@ -1226,17 +1230,29 @@ static int modify_virtqueue(struct mlx5_vdpa_net *n= dev, MLX5_SET(general_obj_in_cmd_hdr, cmd_hdr, uid, ndev->mvdev.res.uid); =20 obj_context =3D MLX5_ADDR_OF(modify_virtio_net_q_in, in, obj_context); - if (mvq->modified_fields & MLX5_VIRTQ_MODIFY_MASK_STATE) + + if (mvq->modified_fields & MLX5_VIRTQ_MODIFY_MASK_STATE) { + if (!is_valid_state_change(mvq->fw_state, state, is_resumable(ndev))) { + err =3D -EINVAL; + goto done; + } + MLX5_SET(virtio_net_q_object, obj_context, state, state); + state_change =3D true; + } =20 MLX5_SET64(virtio_net_q_object, obj_context, modify_field_select, mvq->mo= dified_fields); err =3D mlx5_cmd_exec(ndev->mvdev.mdev, in, inlen, out, sizeof(out)); - kfree(in); - if (!err) + if (err) + goto done; + + if (state_change) mvq->fw_state =3D state; =20 mvq->modified_fields =3D 0; =20 +done: + kfree(in); return err; } =20 @@ -1430,6 +1446,24 @@ static void suspend_vqs(struct mlx5_vdpa_net *ndev) suspend_vq(ndev, &ndev->vqs[i]); } =20 +static void resume_vq(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtque= ue *mvq) +{ + if (!mvq->initialized || !is_resumable(ndev)) + return; + + if (mvq->fw_state !=3D MLX5_VIRTIO_NET_Q_OBJECT_STATE_SUSPEND) + return; + + if (modify_virtqueue_state(ndev, mvq, MLX5_VIRTIO_NET_Q_OBJECT_STATE_RDY)) + mlx5_vdpa_warn(&ndev->mvdev, "modify to resume failed\n"); +} + +static void resume_vqs(struct mlx5_vdpa_net *ndev) +{ + for (int i =3D 0; i < ndev->mvdev.max_vqs; i++) + resume_vq(ndev, &ndev->vqs[i]); +} + static void teardown_vq(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtq= ueue *mvq) { if (!mvq->initialized) @@ -3256,6 +3290,21 @@ static int mlx5_vdpa_suspend(struct vdpa_device *vde= v) return 0; } =20 +static int mlx5_vdpa_resume(struct vdpa_device *vdev) +{ + struct mlx5_vdpa_dev *mvdev =3D to_mvdev(vdev); + struct mlx5_vdpa_net *ndev =3D to_mlx5_vdpa_ndev(mvdev); + + mlx5_vdpa_info(mvdev, "resuming device\n"); + + down_write(&ndev->reslock); + mvdev->suspended =3D false; + resume_vqs(ndev); + register_link_notifier(ndev); + up_write(&ndev->reslock); + return 0; +} + static int mlx5_set_group_asid(struct vdpa_device *vdev, u32 group, unsigned int asid) { @@ -3312,6 +3361,7 @@ static const struct vdpa_config_ops mlx5_vdpa_ops =3D= { .get_vq_dma_dev =3D mlx5_get_vq_dma_dev, .free =3D mlx5_vdpa_free, .suspend =3D mlx5_vdpa_suspend, + .resume =3D mlx5_vdpa_resume, /* Op disabled if not supported. */ }; =20 static int query_mtu(struct mlx5_core_dev *mdev, u16 *mtu) @@ -3683,6 +3733,9 @@ static int mlx5v_probe(struct auxiliary_device *adev, if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, desc_group_mkey_supported)) mgtdev->vdpa_ops.get_vq_desc_group =3D NULL; =20 + if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, freeze_to_rdy_supported)) + mgtdev->vdpa_ops.resume =3D NULL; + err =3D vdpa_mgmtdev_register(&mgtdev->mgtdev); if (err) goto reg_err; --=20 2.42.0 From nobody Tue Dec 16 12:21:18 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0B4BC4167B for ; Fri, 1 Dec 2023 10:50:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378438AbjLAKuD (ORCPT ); Fri, 1 Dec 2023 05:50:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36090 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378435AbjLAKtw (ORCPT ); Fri, 1 Dec 2023 05:49:52 -0500 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2056.outbound.protection.outlook.com [40.107.102.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 985EA1992; Fri, 1 Dec 2023 02:49:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DCSjNbBdIWSL63lPSQGllvT9KP4SDnkUVlEpMcHyYtBt1ufKgUox+e2XzKHh85yc8DzSHTuJymQzSwGkhkfLMyRNf3SB+HBpUO+EIhufZSzKXz6wcrR6FUw/a5N4x/zgyh1D8UT5b9GQz71XPDrrjhzMg5VeQOCTOq5I468CkMJWhqvKu5nKYitP108ZYq63HN7kxCYv4qFHiu4KHgJ/miJ0pyW1RTN31otO/BYFK2gR9kHxS7MEifg/yggvI0ndTuL11fDMnM9VNR15cbfLvItvC6UVOiBNc/JsbRik5eS0VZlLSFeAfH9/fKaUMWOM58Hw+BvuaoNd4zzwrmCZkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=51vWU5QOyBdAV9Vx5J79GCDO5Bn3tO/JvvgMCz/mhCc=; b=JbDbfd/ezK68w0YPJPwG0lbRPeHaEOJhBLIY74J+9cogE9t9rHkrfO24ep6Rt58Vb68G1BAfwNMTl5ABErMHVIHReR+I1U9sM0mjR3Unjwt7sbq096K0e2aHd1uG6j5GrL8PxJovdzmhc6S/mNtPo9ckak1QAwKpgksyu7T0eEx33ZAIhW8eZ16Ja+NelZbA0ldeSo/rmwYkmcpIdnoaNIoWh7nw2orEL+M9pClHSrS9aryvy8+vvM7Reh1iNGLdk4oE4uui6W/TkHkMkpkiBEKoc6orqO6wXE8CDBM2u3DNRojqXLTO5OCGvbaer647Y/tHq+1HJ7LBipXxrdD1cw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=51vWU5QOyBdAV9Vx5J79GCDO5Bn3tO/JvvgMCz/mhCc=; b=hEiawE8zzVq3/HYVYsLJX2IX0d1q0Xsj+5Q5L7nfpKgxv33MeYMNajF0LvbHkbIi/BIMoJ77BqBWR3err2ya1yPyGkYX7yqvo6AsXILAwvhyR1zG6EA3s21otFdd35zgWF0ugvuxgdse/x+V8SV5/me3fA6mjSurZvt7poyCG/1/EM0qemwL4UUjiSv8Q8NFF0j5zY1uBlfgemvzE2iujm2rcEEdRg9uFDXigYl3JXa7LkCDclPcH2vZGTC3xRabN1y3GyhKdTxC59GII9DRULdAANf62XDHfeqw7TF3/WrdKda5PPxlKMk/v48UhBiVkoDZntTFCbND0cGKfTaZvw== Received: from MN2PR20CA0051.namprd20.prod.outlook.com (2603:10b6:208:235::20) by IA1PR12MB6530.namprd12.prod.outlook.com (2603:10b6:208:3a5::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.24; Fri, 1 Dec 2023 10:49:55 +0000 Received: from BL02EPF0001A0FC.namprd03.prod.outlook.com (2603:10b6:208:235:cafe::5b) by MN2PR20CA0051.outlook.office365.com (2603:10b6:208:235::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7025.29 via Frontend Transport; Fri, 1 Dec 2023 10:49:55 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BL02EPF0001A0FC.mail.protection.outlook.com (10.167.242.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.17 via Frontend Transport; Fri, 1 Dec 2023 10:49:54 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Fri, 1 Dec 2023 02:49:38 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Fri, 1 Dec 2023 02:49:38 -0800 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Fri, 1 Dec 2023 02:49:35 -0800 From: Dragos Tatulea To: "Michael S . Tsirkin" , Jason Wang , Eugenio Perez Martin , Si-Wei Liu , Saeed Mahameed , Leon Romanovsky , CC: Dragos Tatulea , , , Gal Pressman , Parav Pandit , Xuan Zhuo Subject: [PATCH vhost 5/7] vdpa/mlx5: Mark vq addrs for modification in hw vq Date: Fri, 1 Dec 2023 12:48:55 +0200 Message-ID: <20231201104857.665737-6-dtatulea@nvidia.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231201104857.665737-1-dtatulea@nvidia.com> References: <20231201104857.665737-1-dtatulea@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF0001A0FC:EE_|IA1PR12MB6530:EE_ X-MS-Office365-Filtering-Correlation-Id: 59916c1e-55c0-48a5-a3e6-08dbf25b40fe X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ZY0Mty1JnBPkuhhNRBmn4igHXCBvA1Q13wMSRhmZGpx1b1vZ2iYNgESd1tdBcRMX961UWO1C5+QncQQq+SowmI+Jl2z9Nwe3qG+89G1kb05gKarInk44GRaaYLxRRaY7jHXTCxThi28mXp0lZCuukXGBXzsEDVsOPdxvKCSiZHzbt6iEhLlpns92hfsZ09EW7lK8OA1CMptKvRZAyfO+fFTIvIs6foGSzzgJeyoBfeBUc52srzaNUopeO+GOpkuDc8b1fos09qmNFGeSj2ASLB6MD6g0+6n4gdro2IppYeP3BPa7/Rt2CnlVtbxC5973RAnipJHLVbI5sNv8I9ATHicP8Sq8J1HLGLEWY8ChkdKb+3HPAe3RBTQoFBmP6rLLq9IfnVmg7y/uonYRUhJb9gqkDaYStk8d5Fb51/95ow9Hxfl08nQiKUndAx9/0csYmzeQ4ZteahU3HoFV5o+l1BhtH1UCSJ9Ey6YqyXgZ0hO7DIEDHGrgDzNmSZIXr06ioi97TLSpv6G1Ymsf2lJRg7ovUz126scT8PwDDYP/oNeqIqMa2ImZQMyXBajsZI3bTwESHB7SjE+yZYaOU6zevbHK4fe3rYTpuXgnB3LchI1P69KnrN4GQk4xOFWs417rSVIwcZmrZHGqk9VVlot5XlNt6CNabz554+eBlLPN3EYgYAcw1TTg0ll0Nl/X+6+4wZLk8snQCBQq7h85MAkqsPl6a/lUrQPaVvHDFp0eHG/mUZQfa6uKgdj2/NUU6cV9 X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(136003)(39860400002)(346002)(376002)(396003)(230922051799003)(1800799012)(186009)(64100799003)(82310400011)(451199024)(46966006)(40470700004)(36840700001)(40460700003)(86362001)(26005)(1076003)(6666004)(2616005)(47076005)(36860700001)(426003)(336012)(5660300002)(8936002)(41300700001)(478600001)(8676002)(2906002)(110136005)(4326008)(316002)(70586007)(70206006)(54906003)(36756003)(356005)(7636003)(82740400003)(83380400001)(40480700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Dec 2023 10:49:54.7401 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 59916c1e-55c0-48a5-a3e6-08dbf25b40fe X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF0001A0FC.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6530 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Addresses get set by .set_vq_address. hw vq addresses will be updated on next modify_virtqueue. Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/net/mlx5_vnet.c | 9 +++++++++ include/linux/mlx5/mlx5_ifc_vdpa.h | 1 + 2 files changed, 10 insertions(+) diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5= _vnet.c index 68e534cb57e2..2277daf4814f 100644 --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c @@ -1209,6 +1209,7 @@ static int modify_virtqueue(struct mlx5_vdpa_net *nde= v, bool state_change =3D false; void *obj_context; void *cmd_hdr; + void *vq_ctx; void *in; int err; =20 @@ -1230,6 +1231,7 @@ static int modify_virtqueue(struct mlx5_vdpa_net *nde= v, MLX5_SET(general_obj_in_cmd_hdr, cmd_hdr, uid, ndev->mvdev.res.uid); =20 obj_context =3D MLX5_ADDR_OF(modify_virtio_net_q_in, in, obj_context); + vq_ctx =3D MLX5_ADDR_OF(virtio_net_q_object, obj_context, virtio_q_contex= t); =20 if (mvq->modified_fields & MLX5_VIRTQ_MODIFY_MASK_STATE) { if (!is_valid_state_change(mvq->fw_state, state, is_resumable(ndev))) { @@ -1241,6 +1243,12 @@ static int modify_virtqueue(struct mlx5_vdpa_net *nd= ev, state_change =3D true; } =20 + if (mvq->modified_fields & MLX5_VIRTQ_MODIFY_MASK_VIRTIO_Q_ADDRS) { + MLX5_SET64(virtio_q, vq_ctx, desc_addr, mvq->desc_addr); + MLX5_SET64(virtio_q, vq_ctx, used_addr, mvq->device_addr); + MLX5_SET64(virtio_q, vq_ctx, available_addr, mvq->driver_addr); + } + MLX5_SET64(virtio_net_q_object, obj_context, modify_field_select, mvq->mo= dified_fields); err =3D mlx5_cmd_exec(ndev->mvdev.mdev, in, inlen, out, sizeof(out)); if (err) @@ -2202,6 +2210,7 @@ static int mlx5_vdpa_set_vq_address(struct vdpa_devic= e *vdev, u16 idx, u64 desc_ mvq->desc_addr =3D desc_area; mvq->device_addr =3D device_area; mvq->driver_addr =3D driver_area; + mvq->modified_fields |=3D MLX5_VIRTQ_MODIFY_MASK_VIRTIO_Q_ADDRS; return 0; } =20 diff --git a/include/linux/mlx5/mlx5_ifc_vdpa.h b/include/linux/mlx5/mlx5_i= fc_vdpa.h index b86d51a855f6..9594ac405740 100644 --- a/include/linux/mlx5/mlx5_ifc_vdpa.h +++ b/include/linux/mlx5/mlx5_ifc_vdpa.h @@ -145,6 +145,7 @@ enum { MLX5_VIRTQ_MODIFY_MASK_STATE =3D (u64)1 << 0, MLX5_VIRTQ_MODIFY_MASK_DIRTY_BITMAP_PARAMS =3D (u64)1 << 3, MLX5_VIRTQ_MODIFY_MASK_DIRTY_BITMAP_DUMP_ENABLE =3D (u64)1 << 4, + MLX5_VIRTQ_MODIFY_MASK_VIRTIO_Q_ADDRS =3D (u64)1 << 6, MLX5_VIRTQ_MODIFY_MASK_DESC_GROUP_MKEY =3D (u64)1 << 14, }; =20 --=20 2.42.0 From nobody Tue Dec 16 12:21:18 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A03B0C4167B for ; Fri, 1 Dec 2023 10:50:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378422AbjLAKuJ (ORCPT ); Fri, 1 Dec 2023 05:50:09 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36122 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378458AbjLAKt6 (ORCPT ); Fri, 1 Dec 2023 05:49:58 -0500 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2056.outbound.protection.outlook.com [40.107.94.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B15FD19B9; Fri, 1 Dec 2023 02:50:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kVgVl/8bxgtTRh/JNNMtm860s3UpzYQncEPNEJ5yogX6z3hfK7D8oVrc89BYGb0KNMuQaPHvVEUT4KuWc9N5d0QWn3c5ubZY0qpbm8I6QKSdJbF5KiE6yheCBphQaRUj22dWXfI57klViMiGpdrpQiPQQ5vp3yqzUIWjNUvK0Isd/ykh9mPh1Ur0g/b6GqEfS0cdsrwJFA9cm1voOpvIytZ45V6jQXc7nF5Pby5Bn52gZ5JlBT0VqEN7gv8t0F2F3vJMGc3o44MLbRjh3/Pa+/970RpnbQEQqr/zfrkcp+2fcbrTQR0IjlpAAA4SWUWhRtD2XFtNf/ONByNteOWXYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=C9Jspx+SLwQ9ATSuVOd0UT8zDB4SlGNgjSwZ95Ao924=; b=LIQUsMWMdq7XRqdEcMJ4qac9psotUy6TT+BSGys4zCwLeXtJcQ9GyMbn7bjNLjCjHC6QlGChkLL+WyPlEV75ZvOIs3i6HQAGnMTv5Zg4HAC3Nza9Zv2d3dpKa/D4gklFdi+5Rns8kps9e3vu5EImWbMH+E5KUlMBOgZ62vX3mB2Ot3OG1k8XD5kcwO11XHF5g1wOWZFI6Ug94wjSbxORR4KHTiQtoyJaezWspU/SyJuuAChLY+lW9XvetYUXfxd/aWVelQGmCAasLd7msUp7L7UqDQP3yono5Vw717vf4Lqa8Gr5xfki2jCOmYqIIoq80S1U+vgzzYhpS9oMoviNaw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=C9Jspx+SLwQ9ATSuVOd0UT8zDB4SlGNgjSwZ95Ao924=; b=hrU2cjWNT+MmvzkBYduT0vd4+p29uT49FvTfmpEx2db6avlh2epeNf4L+VlUcWrAbtsbPL9IqWKXHeEQf5ujc5ZqT4VT3onZvU1ml89O7a7zsGNk8UHqonF+aSRswNR3wV3Ud1c+xwLyqRU2xcnnqY1ky/KKosxVIHe84VadRivXIrnfIsMQkPlbmATIJL2MJD8HxGf/IgU1qQ76zxzGsgyDBNT2vHZUQFd8uOMr2filzIP3faMH5Scu+D5rL8/5+HD2xQoTTq+6jLTV0Nhq+MWdbIZPV1hoJIoMUnQIMQ7bexaqjZ2A85UCXzm0pa7P4tuBaTNFZppdU6XPO06+2A== Received: from BL0PR05CA0008.namprd05.prod.outlook.com (2603:10b6:208:91::18) by IA1PR12MB8405.namprd12.prod.outlook.com (2603:10b6:208:3d8::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.29; Fri, 1 Dec 2023 10:49:58 +0000 Received: from BL02EPF0001A0FC.namprd03.prod.outlook.com (2603:10b6:208:91:cafe::ad) by BL0PR05CA0008.outlook.office365.com (2603:10b6:208:91::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.17 via Frontend Transport; Fri, 1 Dec 2023 10:49:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BL02EPF0001A0FC.mail.protection.outlook.com (10.167.242.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.17 via Frontend Transport; Fri, 1 Dec 2023 10:49:58 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Fri, 1 Dec 2023 02:49:42 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Fri, 1 Dec 2023 02:49:41 -0800 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Fri, 1 Dec 2023 02:49:38 -0800 From: Dragos Tatulea To: "Michael S . Tsirkin" , Jason Wang , Eugenio Perez Martin , Si-Wei Liu , Saeed Mahameed , Leon Romanovsky , CC: Dragos Tatulea , , , Gal Pressman , Parav Pandit , Xuan Zhuo Subject: [PATCH vhost 6/7] vdpa/mlx5: Mark vq state for modification in hw vq Date: Fri, 1 Dec 2023 12:48:56 +0200 Message-ID: <20231201104857.665737-7-dtatulea@nvidia.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231201104857.665737-1-dtatulea@nvidia.com> References: <20231201104857.665737-1-dtatulea@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF0001A0FC:EE_|IA1PR12MB8405:EE_ X-MS-Office365-Filtering-Correlation-Id: 9b5aea7f-6dc4-4869-02e8-08dbf25b4323 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 8/ZMmQWXbsByNyNUq/tb5s3+MlTT5VWcOvWiN7bbKrun9WZTH7jJUyZ02hxdCR7AIly8fETyWjuC7P1LvbMi1IEMGoj+ZdNouqkEHEN31wWB5XksN7nJUHCL43P1uy1h1hI3KlEiwCc15JVnadOQ4hSvurI1WfHYuSIzP2rrl3oFrdJ7otTkxI+DixCucgTiUH2Yd5nUG6w+QWVIh3KWpzeZbzFUuAhZ9SQ4aK8W9nbddf0d8bDytKhwq9Yct9VIbqgSWVVXrigH/frK92Hpe1jThKRGJjuHHh8jKSVm7NICRYc8QTHC/orqIht3SJOmjYHIj1T112s6enHg3ErPNxbfK23Df0Fu+u92tliCHSFPFDfRH80c78bU7f/ZDEvA1IurMU1C9lLbbWkCOmG6nj6c9OSLOCQZmlKfuK8SReep7Ok5e5wWOm5JTTyvu+1mWsf5fvu8uGRRQyPIKxC4no//Vy7RpYZhXxHk8zEBrx9S8+dfKBY+piSRx8m41ta3USevdJPCsypNHGha48diLzZ3POpHk2Of+7xac8xg/RBK9o03GrWGNw/oTxs1c0tRHSfvsvXFMAWTtIaN5yuamwe7+rnCPOxcIRrh6mSjlAsyB7wPOmONRr7J2Jc3FB1KyRDNNFOeMWfQM7+XiluuNVUnOWVVdif7aNYC2ZNS8bAxhvW7o/7wMVOcUeOHJhZzqpeXqo+qlj5C0RpbuXU2yeSV6nalRw79uVY5u2HGRw36+rEzZZpHnGu1tgO35owm X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(396003)(376002)(136003)(39860400002)(346002)(230922051799003)(1800799012)(451199024)(64100799003)(82310400011)(186009)(36840700001)(40470700004)(46966006)(40460700003)(426003)(336012)(82740400003)(36756003)(86362001)(7636003)(356005)(47076005)(36860700001)(110136005)(70586007)(70206006)(4326008)(8676002)(8936002)(54906003)(316002)(40480700001)(5660300002)(2906002)(2616005)(1076003)(478600001)(26005)(6666004)(41300700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Dec 2023 10:49:58.3495 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9b5aea7f-6dc4-4869-02e8-08dbf25b4323 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF0001A0FC.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8405 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" .set_vq_state will set the indices and mark the fields to be modified in the hw vq. Signed-off-by: Dragos Tatulea Acked-by: Eugenio P=C3=A9rez --- drivers/vdpa/mlx5/net/mlx5_vnet.c | 8 ++++++++ include/linux/mlx5/mlx5_ifc_vdpa.h | 2 ++ 2 files changed, 10 insertions(+) diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5= _vnet.c index 2277daf4814f..6325aef045e2 100644 --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c @@ -1249,6 +1249,12 @@ static int modify_virtqueue(struct mlx5_vdpa_net *nd= ev, MLX5_SET64(virtio_q, vq_ctx, available_addr, mvq->driver_addr); } =20 + if (mvq->modified_fields & MLX5_VIRTQ_MODIFY_MASK_VIRTIO_Q_AVAIL_IDX) + MLX5_SET(virtio_net_q_object, obj_context, hw_available_index, mvq->avai= l_idx); + + if (mvq->modified_fields & MLX5_VIRTQ_MODIFY_MASK_VIRTIO_Q_USED_IDX) + MLX5_SET(virtio_net_q_object, obj_context, hw_used_index, mvq->used_idx); + MLX5_SET64(virtio_net_q_object, obj_context, modify_field_select, mvq->mo= dified_fields); err =3D mlx5_cmd_exec(ndev->mvdev.mdev, in, inlen, out, sizeof(out)); if (err) @@ -2328,6 +2334,8 @@ static int mlx5_vdpa_set_vq_state(struct vdpa_device = *vdev, u16 idx, =20 mvq->used_idx =3D state->split.avail_index; mvq->avail_idx =3D state->split.avail_index; + mvq->modified_fields |=3D MLX5_VIRTQ_MODIFY_MASK_VIRTIO_Q_AVAIL_IDX | + MLX5_VIRTQ_MODIFY_MASK_VIRTIO_Q_USED_IDX; return 0; } =20 diff --git a/include/linux/mlx5/mlx5_ifc_vdpa.h b/include/linux/mlx5/mlx5_i= fc_vdpa.h index 9594ac405740..32e712106e68 100644 --- a/include/linux/mlx5/mlx5_ifc_vdpa.h +++ b/include/linux/mlx5/mlx5_ifc_vdpa.h @@ -146,6 +146,8 @@ enum { MLX5_VIRTQ_MODIFY_MASK_DIRTY_BITMAP_PARAMS =3D (u64)1 << 3, MLX5_VIRTQ_MODIFY_MASK_DIRTY_BITMAP_DUMP_ENABLE =3D (u64)1 << 4, MLX5_VIRTQ_MODIFY_MASK_VIRTIO_Q_ADDRS =3D (u64)1 << 6, + MLX5_VIRTQ_MODIFY_MASK_VIRTIO_Q_AVAIL_IDX =3D (u64)1 << 7, + MLX5_VIRTQ_MODIFY_MASK_VIRTIO_Q_USED_IDX =3D (u64)1 << 8, MLX5_VIRTQ_MODIFY_MASK_DESC_GROUP_MKEY =3D (u64)1 << 14, }; =20 --=20 2.42.0 From nobody Tue Dec 16 12:21:18 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 233A4C4167B for ; Fri, 1 Dec 2023 10:50:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378424AbjLAKuW (ORCPT ); Fri, 1 Dec 2023 05:50:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40302 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378455AbjLAKuC (ORCPT ); Fri, 1 Dec 2023 05:50:02 -0500 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2069.outbound.protection.outlook.com [40.107.223.69]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4B961170F; Fri, 1 Dec 2023 02:50:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=g1Uy7AtnTX6+PsciGjfrnHsJYljcQ5zMsampAL/IEZRbsLh0i1WlmZj1TxaG7awjDrTF91Fx8QiaIA6XLKHOaCj4a5bkx7sLnIuoF5qY5rNL38ghbZgNTO+yOqaM/YvXMicOhLXMWBsctdccHJtPQt2IwiQHGSGh7LIciy9cVSey5jTci93zkgVkmPW1E2OHKtCSHVHQ8wfPugqJzqJh/tLtWnFww6OcLE4A8puCBxKWDHI96W7EGd5OILmuZoAWCoaCSDaL+phIvfra2h1ovtnca668bFmtrK2+/erxx5QLq1R53Qd2ChjvDSxmTT3jk9z6QjO+SD+V/mMSsZ/enA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rgPfObaWb0RRVtn7uMUlOSz5sZ2GrOBEfteGvXPSkDE=; b=X4fbsM+xtkZiImMcyyP9s2EpszpiGXpEctzk1RMWSSMfzlpjBTeCBwnlOjAR9ay4VgQnUp0tTf+oIrE4k4XFM4HG1Tk3HTNLbqiDJL//YsC7XxE2Qc1dHGXxhCYVi0njGCWzxRk9+Blxn6NXumiuwlBNbqxFjjhC/XoZjwvG1RHWoq9iqzHfDvAiI+ZsXT/NaqFriVV2+Mv/u1HpDqJJw0qCJRGDqPjVxhZ8aY7qpTGr6PvaRF9R/E7rXCBUJlnl16+kQjACIpjzEvqNCQ8kHYCTDseapF2Ga+Mv/2qj8Ue1L/KKboDQ64UWb3+bQdpSfWKW9I87MMche8C7raWOqw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rgPfObaWb0RRVtn7uMUlOSz5sZ2GrOBEfteGvXPSkDE=; b=qBGTlKRS5E0coY0PgEelMwcPLj3AjMVFcM+r+eet38YWBSQqkOoBecZezl3BHcD3SnMk8He46F+t5hNb0nqrqnM8TUAdzb3zZUQ7rswXMpcTFnJLL0xJlgTxoD2IQ349X2cEOsPMf31vu6UkVbK9Cb/2voqH5ZiYnBblIBCUwmY9IvAFlo0jbxztyOmVz37LV7e4H8cIyMAfdtm2YHhCls2r9BH3euhQM+uAHtBZoXXtep8gArvdrW8LAoBYEoFC7j/E4zqc1gUB55DN9h+42ydRAfnBgZHLnrYQLmBQjPzIT9pgTrsPYew2ELo425cHg03Cni/pU+HMhyLF9p5TVg== Received: from MN2PR02CA0001.namprd02.prod.outlook.com (2603:10b6:208:fc::14) by PH7PR12MB8122.namprd12.prod.outlook.com (2603:10b6:510:2b7::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Fri, 1 Dec 2023 10:50:02 +0000 Received: from BL02EPF0001A0FF.namprd03.prod.outlook.com (2603:10b6:208:fc:cafe::a8) by MN2PR02CA0001.outlook.office365.com (2603:10b6:208:fc::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.24 via Frontend Transport; Fri, 1 Dec 2023 10:50:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BL02EPF0001A0FF.mail.protection.outlook.com (10.167.242.106) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.17 via Frontend Transport; Fri, 1 Dec 2023 10:50:01 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Fri, 1 Dec 2023 02:49:45 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Fri, 1 Dec 2023 02:49:45 -0800 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Fri, 1 Dec 2023 02:49:42 -0800 From: Dragos Tatulea To: "Michael S . Tsirkin" , Jason Wang , Eugenio Perez Martin , Si-Wei Liu , Saeed Mahameed , Leon Romanovsky , CC: Dragos Tatulea , , , Gal Pressman , Parav Pandit , Xuan Zhuo Subject: [PATCH vhost 7/7] vdpa/mlx5: Use vq suspend/resume during .set_map Date: Fri, 1 Dec 2023 12:48:57 +0200 Message-ID: <20231201104857.665737-8-dtatulea@nvidia.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231201104857.665737-1-dtatulea@nvidia.com> References: <20231201104857.665737-1-dtatulea@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF0001A0FF:EE_|PH7PR12MB8122:EE_ X-MS-Office365-Filtering-Correlation-Id: eb842aa8-2c05-4d55-cd2c-08dbf25b4524 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lP7yC7mm+hhX2Ew99I4UXE2wLk15YRdAZrewYfe3DLydqEO6VgPeWXFcVn+9wbg1pSpW0REvWWRav/5KjR5HQyipDEXFc1166caUBUu9LZBb/8ZqCnlStQI6KYuaHdhSRFeF+Uqg1RoTbG+sQesyBJ5iApTLU5LVzMCDd3RAf4C38arbbELHsVbmZuNG9OQgNPZyV2h9+3ZW1kPIQSYv3IXZI2uvm2IKFk054ZD+bZJk3jKZyaB74/NhH4eJmLqvC9CH6BmXJKRQ1vkhVV7MNNzHT51TzpJVmYmGxjcah394OmSWtYDFZyZmgf9fMy8/AVBrGPrWsQnAAQxVAMJPcdMW7L5/g4YmjiJazN8q6UMpopAtkyxFmWfgSxqgl14j9D/4rx7KJcPraBXol6Le1Kw8T1UV6HoEbTTNCsMKar7NGvopWwU/adexJDli1P8aLOx7ogEsWQcEudufCemc1rAS4/b0E1lQzWsJdwZJNfQFJQ2vlt4ET31pSkdBq42D8rGeG46R62EyvtCUrhnOVKdFzMFDnw/z9MpXRacjo/kjc7VdXroD36rAynOD7pJhvLVfAfZq/UieLeu6MkLFBHJ252hzbTvb/jt+22+zz2en3oF9GS2DEjc+4VpU7Qq+PawSxd58oQ9+zDxefX1N/g0bfnh8HJLB2O3RuxBQmMBYMAHTaDuTj00gUCy9fMHdIF5vXBdo+d0RnoM4fv7eAQOP9sifU+QjsqGp9vJe5/XKIF/TSdro15q6ti3ohIRagD5sM1iUO2OIt0UvxrQNUg== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(376002)(346002)(136003)(396003)(230922051799003)(1800799012)(186009)(82310400011)(64100799003)(451199024)(36840700001)(46966006)(40470700004)(1076003)(26005)(2616005)(478600001)(6666004)(47076005)(336012)(426003)(83380400001)(15650500001)(5660300002)(41300700001)(2906002)(110136005)(70586007)(36860700001)(54906003)(70206006)(4326008)(8676002)(8936002)(316002)(7636003)(82740400003)(356005)(86362001)(36756003)(40480700001)(40460700003)(142923001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Dec 2023 10:50:01.7011 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: eb842aa8-2c05-4d55-cd2c-08dbf25b4524 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF0001A0FF.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB8122 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Instead of tearing down and setting up vq resources, use vq suspend/resume during .set_map to speed things up a bit. The vq mr is updated with the new mapping while the vqs are suspended. If the device doesn't support resumable vqs, do the old teardown and setup dance. Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/net/mlx5_vnet.c | 46 ++++++++++++++++++++++++------ include/linux/mlx5/mlx5_ifc_vdpa.h | 1 + 2 files changed, 39 insertions(+), 8 deletions(-) diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5= _vnet.c index 6325aef045e2..eb0ac3d9c9d2 100644 --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c @@ -1206,6 +1206,7 @@ static int modify_virtqueue(struct mlx5_vdpa_net *nde= v, { int inlen =3D MLX5_ST_SZ_BYTES(modify_virtio_net_q_in); u32 out[MLX5_ST_SZ_DW(modify_virtio_net_q_out)] =3D {}; + struct mlx5_vdpa_dev *mvdev =3D &ndev->mvdev; bool state_change =3D false; void *obj_context; void *cmd_hdr; @@ -1255,6 +1256,24 @@ static int modify_virtqueue(struct mlx5_vdpa_net *nd= ev, if (mvq->modified_fields & MLX5_VIRTQ_MODIFY_MASK_VIRTIO_Q_USED_IDX) MLX5_SET(virtio_net_q_object, obj_context, hw_used_index, mvq->used_idx); =20 + if (mvq->modified_fields & MLX5_VIRTQ_MODIFY_MASK_VIRTIO_Q_MKEY) { + struct mlx5_vdpa_mr *mr =3D mvdev->mr[mvdev->group2asid[MLX5_VDPA_DATAVQ= _GROUP]]; + + if (mr) + MLX5_SET(virtio_q, vq_ctx, virtio_q_mkey, mr->mkey); + else + mvq->modified_fields &=3D ~MLX5_VIRTQ_MODIFY_MASK_VIRTIO_Q_MKEY; + } + + if (mvq->modified_fields & MLX5_VIRTQ_MODIFY_MASK_DESC_GROUP_MKEY) { + struct mlx5_vdpa_mr *mr =3D mvdev->mr[mvdev->group2asid[MLX5_VDPA_DATAVQ= _DESC_GROUP]]; + + if (mr && MLX5_CAP_DEV_VDPA_EMULATION(mvdev->mdev, desc_group_mkey_suppo= rted)) + MLX5_SET(virtio_q, vq_ctx, desc_group_mkey, mr->mkey); + else + mvq->modified_fields &=3D ~MLX5_VIRTQ_MODIFY_MASK_DESC_GROUP_MKEY; + } + MLX5_SET64(virtio_net_q_object, obj_context, modify_field_select, mvq->mo= dified_fields); err =3D mlx5_cmd_exec(ndev->mvdev.mdev, in, inlen, out, sizeof(out)); if (err) @@ -2784,24 +2803,35 @@ static int mlx5_vdpa_change_map(struct mlx5_vdpa_de= v *mvdev, unsigned int asid) { struct mlx5_vdpa_net *ndev =3D to_mlx5_vdpa_ndev(mvdev); + bool teardown =3D !is_resumable(ndev); int err; =20 suspend_vqs(ndev); - err =3D save_channels_info(ndev); - if (err) - return err; + if (teardown) { + err =3D save_channels_info(ndev); + if (err) + return err; =20 - teardown_driver(ndev); + teardown_driver(ndev); + } =20 mlx5_vdpa_update_mr(mvdev, new_mr, asid); =20 + for (int i =3D 0; i < ndev->cur_num_vqs; i++) + ndev->vqs[i].modified_fields |=3D MLX5_VIRTQ_MODIFY_MASK_VIRTIO_Q_MKEY | + MLX5_VIRTQ_MODIFY_MASK_DESC_GROUP_MKEY; + if (!(mvdev->status & VIRTIO_CONFIG_S_DRIVER_OK) || mvdev->suspended) return 0; =20 - restore_channels_info(ndev); - err =3D setup_driver(mvdev); - if (err) - return err; + if (teardown) { + restore_channels_info(ndev); + err =3D setup_driver(mvdev); + if (err) + return err; + } + + resume_vqs(ndev); =20 return 0; } diff --git a/include/linux/mlx5/mlx5_ifc_vdpa.h b/include/linux/mlx5/mlx5_i= fc_vdpa.h index 32e712106e68..40371c916cf9 100644 --- a/include/linux/mlx5/mlx5_ifc_vdpa.h +++ b/include/linux/mlx5/mlx5_ifc_vdpa.h @@ -148,6 +148,7 @@ enum { MLX5_VIRTQ_MODIFY_MASK_VIRTIO_Q_ADDRS =3D (u64)1 << 6, MLX5_VIRTQ_MODIFY_MASK_VIRTIO_Q_AVAIL_IDX =3D (u64)1 << 7, MLX5_VIRTQ_MODIFY_MASK_VIRTIO_Q_USED_IDX =3D (u64)1 << 8, + MLX5_VIRTQ_MODIFY_MASK_VIRTIO_Q_MKEY =3D (u64)1 << 11, MLX5_VIRTQ_MODIFY_MASK_DESC_GROUP_MKEY =3D (u64)1 << 14, }; =20 --=20 2.42.0