From nobody Sun Feb 8 12:58:10 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97595C04E69 for ; Wed, 2 Aug 2023 17:13:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232101AbjHBRN0 (ORCPT ); Wed, 2 Aug 2023 13:13:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38460 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232014AbjHBRNR (ORCPT ); Wed, 2 Aug 2023 13:13:17 -0400 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2056.outbound.protection.outlook.com [40.107.96.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 803C31734 for ; Wed, 2 Aug 2023 10:13:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IavSKqK64YjUSvBo4hps72Hs1aAbpsJDk2lYlUu5hk1AndYdMQ6CRs9yDTY0KHELrKo1t+dSnJVL+ggH9t7D6Nsbp/J9xWhwWSu+ri0QrnOpy5JaqcqF7uJLqCiAi4JcRXkl+zlj1AwcZd2zMHPJIH9W2Q646LmNe7BNwQkNfe7KsHbm0XHuJ2OsnYVVc6RgaKqOqHa9NQBgrm5llg+0oJrApwNxmbfmqOOcSJqSy30CfoGl0nxwF/PH32Uyh6HgOlbM2OrIWLeqkSUEe2dTzSCaIcyDPC5or5Vi89iDAZaeFBsS4jVi8PwTr6beR1fK7IJ4exlosyxxJoxkkWVd7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Z3o64W6AxgCasG8EB+Q2EH9SyaH2V1AMN/IpgxvfuVs=; b=Hb6Dtycl6qkai1ebLP7oi+5Y8Ry0K9NPaqUEujnDgeF9ya7eXREAukAcrmYHIoSKWYtsYFc9DSbuiV4I2e5jDSrMQ4DOTea5B3I9Rg85Z1XCIj3zagLRHNr+khUJmnabkrqO466IQj20ruwulfo45WGHxV1jzbXTvUhTX2bMuinAPVxfiFFUcosB6+36jWp4rceilYwTZeListn+b88/yiZ6poJetYbEwUDDalS6wvQwVmHe8zWUqvIN7RkpXEW+eEVnD3+fJUrl9quYQxAAWdlVvoDlSuXORnPxXnZIOMbfKkBxL1+rF5vqctGLCxgPKo5vS14WE8SwmUUeWBy6mQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Z3o64W6AxgCasG8EB+Q2EH9SyaH2V1AMN/IpgxvfuVs=; b=cvCPZyaCfaccG4etB2GxvLGzW7+Lk1cgP0eHYtKczFv1vvUb3nmbTvyu7uzU13SfZtOANUMlMnakdDizy2XCgT1/Qdv8POug8z2l2bth8VaG04T85axBquZ2lIMtG2Vv7Ddy1biV8XC0X0CmCWLJ31402wCXzHE8VyFe+LtnbVNn7cIzVdFtLMg1V/VgFI7NM1L3SCCsPmqSM2BdFgieO7WAyQP/QqEZDvZs6pYktlJekU1keoyMtce089clFVEYm5Q3y+3CSf7GHZKpl1lQ17Q4lqzUwoC7JOg1TaKh9tAZ55KZ6rIvcDC846/K63IiyvNVsokEj/6e7pK8zFdsgg== Received: from MW4PR03CA0096.namprd03.prod.outlook.com (2603:10b6:303:b7::11) by IA0PR12MB7555.namprd12.prod.outlook.com (2603:10b6:208:43d::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.45; Wed, 2 Aug 2023 17:13:08 +0000 Received: from MWH0EPF000971E6.namprd02.prod.outlook.com (2603:10b6:303:b7:cafe::72) by MW4PR03CA0096.outlook.office365.com (2603:10b6:303:b7::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.19 via Frontend Transport; Wed, 2 Aug 2023 17:13:07 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by MWH0EPF000971E6.mail.protection.outlook.com (10.167.243.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.20 via Frontend Transport; Wed, 2 Aug 2023 17:13:07 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Wed, 2 Aug 2023 10:12:53 -0700 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Wed, 2 Aug 2023 10:12:53 -0700 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.986.37 via Frontend Transport; Wed, 2 Aug 2023 10:12:50 -0700 From: Dragos Tatulea To: "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo CC: Dragos Tatulea , =?UTF-8?q?Eugenio=20P=C3=A9rez?= , Gal Pressman , , Subject: [PATCH 1/2] vdpa/mlx5: Fix mr->initialized semantics Date: Wed, 2 Aug 2023 20:12:18 +0300 Message-ID: <20230802171231.11001-3-dtatulea@nvidia.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230802171231.11001-1-dtatulea@nvidia.com> References: <20230802171231.11001-1-dtatulea@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000971E6:EE_|IA0PR12MB7555:EE_ X-MS-Office365-Filtering-Correlation-Id: 25d91cf9-007c-40ad-25b5-08db937bbddb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3ebzthq04FOkvko+bkA4hJChFsnc701BWXCqTZ7KAlO+zHhcFSd6zjT0c7euDAf7YyyZkZbY2AErdRGkcRmhtqrB5fBaTEYocUxWobBa69ad2Z4VQIVITWnpBTg3dDznbP5/5wbXfMYruZryzcy6fSKhloN0Qakc4FcXTKBdFKDI2rzW4KYzD9nZY2dXoSzps20Ny7mbSTxTyk2awKssEecvSxdXTMAYKzL4NO78gNn4hgIs0P6zFyUkuQNklqiavplQ7LBMzmEsBbTtXt9p0Lo3HBIet5dlPUf7Nwl/XSgVfxXWjyq8EGS9U7PYS9GzSYiDCU07JQu8ztSn12aajbc4YIagvSftCVQ/Tl4Nyqc2dF6ZV9XfR3OS0O+cKci9SNAbtuYxICn9/++p2ZVazW+Nl6XGxKo4pIcd1PxE0R+u1/12BMcramAWhwHi/GdIO9hiT83lco0Kalo7m5FK88KFkcIy1eHr5SoCAlxqZH8/khWM5zWpodsH/BmltGtmphoGHd7JgKAI0Fe5kCaKwlwCiJ5HKIfnQxG5LgqG6cfqztYEAwPnKe+l82egTF3CBFY0nwx52vnARH4BXv1rxdZDGQvEGu6wZt7q4l1dSMoG6U41LxgIh8k0ADWy55MgekFxjk7LEKKRdnF04J+yEup2KNVzhaNZVOnJ6BdSUF3zkVpcCykTM/Intial52DoSBU2oKMkEAccl9qPr1v28IUL7k0Y1z1pRjRrRZs13e/EAyFbwtlPEwoG8ylPDz2P X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230028)(4636009)(396003)(346002)(136003)(376002)(39860400002)(451199021)(82310400008)(36840700001)(46966006)(40470700004)(36756003)(2616005)(66574015)(426003)(5660300002)(8936002)(186003)(36860700001)(8676002)(47076005)(83380400001)(26005)(336012)(478600001)(4326008)(70206006)(54906003)(110136005)(316002)(6666004)(41300700001)(70586007)(86362001)(40480700001)(40460700003)(1076003)(2906002)(356005)(7636003)(82740400003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Aug 2023 17:13:07.7010 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 25d91cf9-007c-40ad-25b5-08db937bbddb X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000971E6.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB7555 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The mr->initialized flag is shared between the control vq and data vq part of the mr init/uninit. But if the control vq and data vq get placed in different ASIDs, it can happen that initializing the control vq will prevent the data vq mr from being initialized. This patch consolidates the control and data vq init parts into their own init functions. The mr->initialized will now be used for the data vq only. The control vq currently doesn't need a flag. The uninitializing part is also taken care of: mlx5_vdpa_destroy_mr got split into data and control vq functions which are now also ASID aware. Fixes: 8fcd20c30704 ("vdpa/mlx5: Support different address spaces for contr= ol and data") Signed-off-by: Dragos Tatulea Reviewed-by: Eugenio P=C3=A9rez Reviewed-by: Gal Pressman Acked-by: Jason Wang --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1 + drivers/vdpa/mlx5/core/mr.c | 97 +++++++++++++++++++++--------- 2 files changed, 71 insertions(+), 27 deletions(-) diff --git a/drivers/vdpa/mlx5/core/mlx5_vdpa.h b/drivers/vdpa/mlx5/core/ml= x5_vdpa.h index 25fc4120b618..a0420be5059f 100644 --- a/drivers/vdpa/mlx5/core/mlx5_vdpa.h +++ b/drivers/vdpa/mlx5/core/mlx5_vdpa.h @@ -31,6 +31,7 @@ struct mlx5_vdpa_mr { struct list_head head; unsigned long num_directs; unsigned long num_klms; + /* state of dvq mr */ bool initialized; =20 /* serialize mkey creation and destruction */ diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c index 03e543229791..4ae14a248a4b 100644 --- a/drivers/vdpa/mlx5/core/mr.c +++ b/drivers/vdpa/mlx5/core/mr.c @@ -489,60 +489,103 @@ static void destroy_user_mr(struct mlx5_vdpa_dev *mv= dev, struct mlx5_vdpa_mr *mr } } =20 -void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev) +static void _mlx5_vdpa_destroy_cvq_mr(struct mlx5_vdpa_dev *mvdev, unsigne= d int asid) +{ + if (mvdev->group2asid[MLX5_VDPA_CVQ_GROUP] !=3D asid) + return; + + prune_iotlb(mvdev); +} + +static void _mlx5_vdpa_destroy_dvq_mr(struct mlx5_vdpa_dev *mvdev, unsigne= d int asid) { struct mlx5_vdpa_mr *mr =3D &mvdev->mr; =20 - mutex_lock(&mr->mkey_mtx); + if (mvdev->group2asid[MLX5_VDPA_DATAVQ_GROUP] !=3D asid) + return; + if (!mr->initialized) - goto out; + return; =20 - prune_iotlb(mvdev); if (mr->user_mr) destroy_user_mr(mvdev, mr); else destroy_dma_mr(mvdev, mr); =20 mr->initialized =3D false; -out: +} + +static void mlx5_vdpa_destroy_mr_asid(struct mlx5_vdpa_dev *mvdev, unsigne= d int asid) +{ + struct mlx5_vdpa_mr *mr =3D &mvdev->mr; + + mutex_lock(&mr->mkey_mtx); + + _mlx5_vdpa_destroy_dvq_mr(mvdev, asid); + _mlx5_vdpa_destroy_cvq_mr(mvdev, asid); + mutex_unlock(&mr->mkey_mtx); } =20 -static int _mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, - struct vhost_iotlb *iotlb, unsigned int asid) +void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev) +{ + mlx5_vdpa_destroy_mr_asid(mvdev, mvdev->group2asid[MLX5_VDPA_CVQ_GROUP]); + mlx5_vdpa_destroy_mr_asid(mvdev, mvdev->group2asid[MLX5_VDPA_DATAVQ_GROUP= ]); +} + +static int _mlx5_vdpa_create_cvq_mr(struct mlx5_vdpa_dev *mvdev, + struct vhost_iotlb *iotlb, + unsigned int asid) +{ + if (mvdev->group2asid[MLX5_VDPA_CVQ_GROUP] !=3D asid) + return 0; + + return dup_iotlb(mvdev, iotlb); +} + +static int _mlx5_vdpa_create_dvq_mr(struct mlx5_vdpa_dev *mvdev, + struct vhost_iotlb *iotlb, + unsigned int asid) { struct mlx5_vdpa_mr *mr =3D &mvdev->mr; int err; =20 - if (mr->initialized) + if (mvdev->group2asid[MLX5_VDPA_DATAVQ_GROUP] !=3D asid) return 0; =20 - if (mvdev->group2asid[MLX5_VDPA_DATAVQ_GROUP] =3D=3D asid) { - if (iotlb) - err =3D create_user_mr(mvdev, iotlb); - else - err =3D create_dma_mr(mvdev, mr); + if (mr->initialized) + return 0; =20 - if (err) - return err; - } + if (iotlb) + err =3D create_user_mr(mvdev, iotlb); + else + err =3D create_dma_mr(mvdev, mr); =20 - if (mvdev->group2asid[MLX5_VDPA_CVQ_GROUP] =3D=3D asid) { - err =3D dup_iotlb(mvdev, iotlb); - if (err) - goto out_err; - } + if (err) + return err; =20 mr->initialized =3D true; + + return 0; +} + +static int _mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, + struct vhost_iotlb *iotlb, unsigned int asid) +{ + int err; + + err =3D _mlx5_vdpa_create_dvq_mr(mvdev, iotlb, asid); + if (err) + return err; + + err =3D _mlx5_vdpa_create_cvq_mr(mvdev, iotlb, asid); + if (err) + goto out_err; + return 0; =20 out_err: - if (mvdev->group2asid[MLX5_VDPA_DATAVQ_GROUP] =3D=3D asid) { - if (iotlb) - destroy_user_mr(mvdev, mr); - else - destroy_dma_mr(mvdev, mr); - } + _mlx5_vdpa_destroy_dvq_mr(mvdev, asid); =20 return err; } --=20 2.41.0 From nobody Sun Feb 8 12:58:10 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E22AC001DF for ; Wed, 2 Aug 2023 17:14:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232223AbjHBROk (ORCPT ); Wed, 2 Aug 2023 13:14:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39886 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232155AbjHBROe (ORCPT ); Wed, 2 Aug 2023 13:14:34 -0400 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on20602.outbound.protection.outlook.com [IPv6:2a01:111:f400:7e89::602]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE7AC2D70 for ; Wed, 2 Aug 2023 10:14:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dttdudDqvQi2+hQJ3NaHb3YGylDNMZojjRe2U0NPXfIvG2qkANSrM4cbMC4RD2ZrimPa4b9CJtJhfJbn5v9s4GwMlaWJFGPE9E8D6uoD9tovU4N6qiaHMKO4xF/4aZMBGV82A8wYM3aV2wEOmddmDL7ymRSUPWsRmJnbq6WhLVATEKEg1eeqhQHynKNmD75PN5Ck6LWSFHoyYqA8HNvxbhDW+SMIIUuMzeREsuhK7M+TLowFjKHX3M/LgeSSzjUMONdpNWhboacnbKeUfwCNAZggdtdF2uOMWo6D0AGxRPOcju6InimzZKX5zHtagjteWteKIVB+yFl8rlQ3uIuGZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+UyKQTMFX1mut5/H2U/CfaadEbA4WrB66IyJaD16oIQ=; b=i+vd9N1fZ1EQsezdHymW+cek/BKhzeOZRTQurxApvnAbqJfT4msC1q7TiEjai2PEvw9XRV1eLiyomRaGcW1kaIIF1BrbpW1dUtvdXRzZ8+ZqwkrpYyJkL18UOmXff72cxsm7Q6YJLU7jy17H2eJ0AMGpyVqRDc0hZTlbmN7GR0iwsDDpNLyLV34srULRodmwb6KdFfc5AFW48VL2BhSpudmXViWm+U9sXh8Hf7N1H7LEiMZJRHirIUvIsCf1gki8TzpeqP8rLQvrdI5vaXSY1nJbHGHxiO++CsITodUR0eT/kC2rUmDoTNqSPo5EqxelDAFg+j4Y02+A59/s1virTA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+UyKQTMFX1mut5/H2U/CfaadEbA4WrB66IyJaD16oIQ=; b=dcLxT6iex1FoDKFrue6uu8T040Pt884AeBGzzd2BUFR3RaCpcGmQ5sH4+usmjW0G8Y3mUOvgSB5wRMO/iY4x26aqQcddXsSYRXnj/99tt7Gm/Heim41wTa9FuLTK6ojd3i/KqSxze+qloQHUAEJxT2H/E6ZiOOsGCFI+7jJkXmdRViOR/coovt97eWSF2u/1FU0/TAoJkywccnLvjZyHMuYvVIwjfnlTkWKEd6E28UVMIPwSfQAWz42WdVNzC0QRrp6me2e9vZsKrSKSJcl+FgNwX3QrSP3oCK270bQknNq5+SPhMM4gvZqqFvXiZ00kBlsCTruHo6nqk7k7IKd2qg== Received: from MW4P220CA0005.NAMP220.PROD.OUTLOOK.COM (2603:10b6:303:115::10) by BL1PR12MB5144.namprd12.prod.outlook.com (2603:10b6:208:316::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.19; Wed, 2 Aug 2023 17:13:18 +0000 Received: from MWH0EPF000971E8.namprd02.prod.outlook.com (2603:10b6:303:115:cafe::b8) by MW4P220CA0005.outlook.office365.com (2603:10b6:303:115::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.19 via Frontend Transport; Wed, 2 Aug 2023 17:13:18 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by MWH0EPF000971E8.mail.protection.outlook.com (10.167.243.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.19 via Frontend Transport; Wed, 2 Aug 2023 17:13:18 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Wed, 2 Aug 2023 10:13:05 -0700 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Wed, 2 Aug 2023 10:13:04 -0700 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.986.37 via Frontend Transport; Wed, 2 Aug 2023 10:13:03 -0700 From: Dragos Tatulea To: "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo CC: =?UTF-8?q?Eugenio=20P=C3=A9rez?= , Gal Pressman , , Subject: [PATCH 2/2] vdpa/mlx5: Delete control vq iotlb in destroy_mr only when necessary Date: Wed, 2 Aug 2023 20:12:20 +0300 Message-ID: <20230802171231.11001-5-dtatulea@nvidia.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230802171231.11001-1-dtatulea@nvidia.com> References: <20230802171231.11001-1-dtatulea@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000971E8:EE_|BL1PR12MB5144:EE_ X-MS-Office365-Filtering-Correlation-Id: 0868c003-c92f-467d-cb1b-08db937bc43e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cdONbFcQg/XDn/bSE8yzrOwELtztIb22l3ZbDOlzuDXq53xarGoMujSj/nHqWWXg/iFrAUAKZuMGNWqfC7OTBl2sw3VUwDjmFpBJf9Ao4+ajmw8J2GRoi+KA4OiSvsGskJ3VxSFuF2BYNj5YEcSmwl+DUNNdAXj5g8p8P0EGIBEJ4Gh3GSOeObrnzxNnrNkxKnndqgqpsnIaYZwzdWwOfnHEGwTDeqKs6/TrVv5Q/UISlITGwhtR6+HZAkZvXEcjqTduZXiebyZPxX+UcHY+yxPdQ6Y1XP2mydC8qfYGOiLXLPXOjnDe9rfKjqMu6+4yGYkyMrl6UxZcVJ73t0UkIOllXTk7EzByLEqWgW4ks7+zFOjtWvKDYlRK0IJzh4gyaZ7byTC5bcgOtA52oXk9BM0WUW2kSmFYZCybMxNFjdMtAlc88Qo0djSLAUTyMTSStXhKrEfJgZO7MDPoqZwOsZ/PXktTtwvYexGq0eP+IFrHKzOr8IKMtQqq3luZcHuKx3ykJ7MQ6GcCcGy1yUIb/NffXlQ8C2lFaNOKhQkAm/4bnCGcQtdJKlT1FdrL4mFCFXVoR+KiCfbbdvH91HXVIt2Fas6gLCdXYeI7dHxU2HzhrRmTsogaYU0xD6VafHyzoiGUHrE9663M9C/70yXAHiX7qLAXKpE1+o+ERXa09aNUXS2pI22HrGDOi9jE4QFbIJ/F0dPzMX68CYra/GFD+xb/gKip5YUMs7KeixYJl4Siz9sPCgMUObX8xFILmEFh X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230028)(4636009)(136003)(39860400002)(346002)(376002)(396003)(82310400008)(451199021)(46966006)(36840700001)(40470700004)(4326008)(40460700003)(426003)(66574015)(70206006)(70586007)(186003)(2616005)(356005)(7636003)(82740400003)(336012)(2906002)(26005)(1076003)(83380400001)(36860700001)(47076005)(54906003)(110136005)(40480700001)(86362001)(478600001)(36756003)(6666004)(41300700001)(8676002)(8936002)(5660300002)(316002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Aug 2023 17:13:18.4204 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0868c003-c92f-467d-cb1b-08db937bc43e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000971E8.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5144 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Eugenio P=C3=A9rez mlx5_vdpa_destroy_mr can be called from .set_map with data ASID after the control virtqueue ASID iotlb has been populated. The control vq iotlb must not be cleared, since it will not be populated again. So call the ASID aware destroy function which makes sure that the right vq resource is destroyed. Fixes: 8fcd20c30704 ("vdpa/mlx5: Support different address spaces for contr= ol and data") Signed-off-by: Eugenio P=C3=A9rez Reviewed-by: Gal Pressman Acked-by: Jason Wang --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1 + drivers/vdpa/mlx5/core/mr.c | 2 +- drivers/vdpa/mlx5/net/mlx5_vnet.c | 4 ++-- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/vdpa/mlx5/core/mlx5_vdpa.h b/drivers/vdpa/mlx5/core/ml= x5_vdpa.h index a0420be5059f..b53420e874ac 100644 --- a/drivers/vdpa/mlx5/core/mlx5_vdpa.h +++ b/drivers/vdpa/mlx5/core/mlx5_vdpa.h @@ -122,6 +122,7 @@ int mlx5_vdpa_handle_set_map(struct mlx5_vdpa_dev *mvde= v, struct vhost_iotlb *io int mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *i= otlb, unsigned int asid); void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev); +void mlx5_vdpa_destroy_mr_asid(struct mlx5_vdpa_dev *mvdev, unsigned int a= sid); =20 #define mlx5_vdpa_warn(__dev, format, ...) = \ dev_warn((__dev)->mdev->device, "%s:%d:(pid %d) warning: " format, __func= __, __LINE__, \ diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c index 4ae14a248a4b..5a1971fcd87b 100644 --- a/drivers/vdpa/mlx5/core/mr.c +++ b/drivers/vdpa/mlx5/core/mr.c @@ -515,7 +515,7 @@ static void _mlx5_vdpa_destroy_dvq_mr(struct mlx5_vdpa_= dev *mvdev, unsigned int mr->initialized =3D false; } =20 -static void mlx5_vdpa_destroy_mr_asid(struct mlx5_vdpa_dev *mvdev, unsigne= d int asid) +void mlx5_vdpa_destroy_mr_asid(struct mlx5_vdpa_dev *mvdev, unsigned int a= sid) { struct mlx5_vdpa_mr *mr =3D &mvdev->mr; =20 diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5= _vnet.c index 9138ef2fb2c8..61c10ba5e3f5 100644 --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c @@ -2636,7 +2636,7 @@ static int mlx5_vdpa_change_map(struct mlx5_vdpa_dev = *mvdev, goto err_mr; =20 teardown_driver(ndev); - mlx5_vdpa_destroy_mr(mvdev); + mlx5_vdpa_destroy_mr_asid(mvdev, asid); err =3D mlx5_vdpa_create_mr(mvdev, iotlb, asid); if (err) goto err_mr; @@ -2652,7 +2652,7 @@ static int mlx5_vdpa_change_map(struct mlx5_vdpa_dev = *mvdev, return 0; =20 err_setup: - mlx5_vdpa_destroy_mr(mvdev); + mlx5_vdpa_destroy_mr_asid(mvdev, asid); err_mr: return err; } --=20 2.41.0