From nobody Fri Nov 29 18:38:24 2024 Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on2056.outbound.protection.outlook.com [40.107.95.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DBF71158A1F for ; Mon, 16 Sep 2024 17:18:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.95.56 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726507134; cv=fail; b=Q3mqNr5axJd0qI8wWRt/2TGBOei1QLw8g6xyoQVUMQn76vwu4hfDOitcq65yNfh202Bvn4AuEAkfNDhJ36s1Q/RZ09PSod3wUyCr8sWaC3Cb58TRCp4tI0hKHwR/vjg1tAT2n8QmrKjdlDdF0J5MwO79zjEqbQY30enXOpbhyOM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726507134; c=relaxed/simple; bh=rz7SwFwq7jjensCmduRsfPqJ7WSerRxdZTOASySlwmA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=deAmv5lKAj/miUFeRkc5iCoNDzi4Qsj/6/d/UB5CTdwYNbCybozYxvdNHqkU+RDNyiJCQdXS67zJGi//n3yeVFRzqsEYQbxFv7EbRPFlM+8GgTh2rNrfFN8SpkQVsuLSQ/xCCbq74wYOTCqtF7EqcqUjZSFMhbxjb56OJbEBQX4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=hGt2KyAY; arc=fail smtp.client-ip=40.107.95.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="hGt2KyAY" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=JbB3xQG+91imme3bmSsNWxdAyIg/DZdUYdZsJ7yeuIRiqvBecyOWce6q93a271Ea6spoyrypM2bn56mbMtd2kENB3T1E5UnFQ2RQslpkGOEwYg/jD46dK+edWqMcP67e0KwcrTaEjG4f9xu4cy1RtyW8BurVnkFfkEL3u3ZAni0bGNgg5FuzcjvfJhnqDTTC4yvyvcFuubR+SjfCEe+4vZaouq8+zf8K9ewvlnb9ighwB54Yp1CWvpkh4IOOdJLC7plgCtkY0o8tkz7WsqGaEFc9Au1MuqdnlX8U96yQnUG6aw5dpAqepwU2DmOnUbjKCdrpWb+lhpXBjLIAHVHsHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LXb5vzhKyLudYQpsLsRnz4NHfmQ4HexomRt2R1L1KLI=; b=p4mwNh+NJ1Gw+knc9/WUCYGy2D3Wh6ydEWRSefvllBA778XrKP2hGscV2eZdkguSBWiaDKZ1XHVF+MQY+vfUfYsmjqVXXtmxJfcwAPkgZ2/QkGINi+yBNc7E+yhSxtDTBuz5vuic7rxv8CUuZ0PTIncqWXH1Go2p8fejKMe1vL3RuXbvJVQuQT7nowCfMkvkXE3pyXfuVfuXlFZDeYG17VPfNBEXjBFfF2KwKAFqgyd4nY5oNpFKHBlwfWB+EKMAId2ste3QtpLuTMNjS269Mg4aMM0cD2iQLgChGQuInTOcguYpl5DuNOV4tobEeJPQt1JXCbKwcb7sM+HV0PnI6Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LXb5vzhKyLudYQpsLsRnz4NHfmQ4HexomRt2R1L1KLI=; b=hGt2KyAYY+gOjeehYfupBAB2RtxlPVSsxg0We/+37P5sBR1XyW7WsYhGrCTzaLwTuCDhGvEtubm9vZU5UY3CEltiSm/jZ9W6d3dfKxra/QtsmVJILvH+e3IUtszXgJDZhvIqlt2dtSgywpLr1BVxuAHgPQxDeV67BuhYc3XforQ= Received: from CH5PR05CA0002.namprd05.prod.outlook.com (2603:10b6:610:1f0::8) by DS7PR12MB8273.namprd12.prod.outlook.com (2603:10b6:8:ed::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7962.23; Mon, 16 Sep 2024 17:18:49 +0000 Received: from CH2PEPF00000142.namprd02.prod.outlook.com (2603:10b6:610:1f0:cafe::82) by CH5PR05CA0002.outlook.office365.com (2603:10b6:610:1f0::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7982.15 via Frontend Transport; Mon, 16 Sep 2024 17:18:49 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CH2PEPF00000142.mail.protection.outlook.com (10.167.244.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7918.13 via Frontend Transport; Mon, 16 Sep 2024 17:18:49 +0000 Received: from purico-ed03host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 16 Sep 2024 12:18:32 -0500 From: Suravee Suthikulpanit To: , CC: , , , , , , , , , Suravee Suthikulpanit Subject: [PATCH v4 1/6] iommu/amd: Disable AMD IOMMU if CMPXCHG16B feature is not supported Date: Mon, 16 Sep 2024 17:18:00 +0000 Message-ID: <20240916171805.324292-2-suravee.suthikulpanit@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240916171805.324292-1-suravee.suthikulpanit@amd.com> References: <20240916171805.324292-1-suravee.suthikulpanit@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF00000142:EE_|DS7PR12MB8273:EE_ X-MS-Office365-Filtering-Correlation-Id: a250491f-c684-4a22-e4b2-08dcd673a175 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|376014|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?URDTV6kLhhoDzq7T+EPreU9RW6sCngIphtlwJmCrPbSjlpPnzykq1uMB6qpL?= =?us-ascii?Q?1Kxi/msY2BXUvQudy4KASlROAtiE4uIOdBL7XJB9+YzVlJidHZtphvALnqF0?= =?us-ascii?Q?N+UZ7jnc7BwC/RsH/8whvS99EBaah1FSqi7DmbT2e4oFDCdwfgyF4XkNREH4?= =?us-ascii?Q?PhLeHIUo/y0FOXys6l3OvfGxLxNHZgt3c00qP9fPWalQbs0JY4WCm8yPmznZ?= =?us-ascii?Q?B52elnS24EgBxW+ye9mn+5Ai0Zm2MMyenDBZOHzkMe0F4Ut0dBce1EXpPK3h?= =?us-ascii?Q?4AuFd5X3YNez3kSYd/lByyjUqOhSBwdU8e5GJ/ub7tvphKoZHxhf0UNSwO6E?= =?us-ascii?Q?pK1vUaFiBkomixsytxAPKcgJ4YZt7ktWpUSJAEvBiR5UF34aOEwKu9DRhFRR?= =?us-ascii?Q?JW4kHIGZpZDzbA0ECl7onw4HgmI1HcYYO1JcLKRuDRAOuggP7mcbbyP0AN2p?= =?us-ascii?Q?ITSC2ElZb1X6XtbZdJOPqcn/fDg1VnTxWLWyKRLb6P6vfkIiOLSeKzTIoK8d?= =?us-ascii?Q?JQigYchzes0nf4KyaOVT63nMUU4xqkA2PUSk1TMU23ZCnYKp1F23oSnLMhhC?= =?us-ascii?Q?dCAs2v6e7cpwWc8Kk5dHHsUX1HiW8Y0sEbTXfiwy0jyXP8FpHpL1GXdMz6q6?= =?us-ascii?Q?FzoaUi+mj+8qz3jF9O7g7/nikXq022byl4mReaq7F8pTlR8knWGF/qDui4Yg?= =?us-ascii?Q?SptwX+4MKe+mWbXXAxLrAVbIVnzB4x7K5S3aXnqqM2B/X8C/LRfP8Qb8qXdB?= =?us-ascii?Q?doJiYg5mN4WnGgKMuoY2UuFRIDfwBx6ux4Wgmnwq4rZ27NHL3V+DkYlIo4Wc?= =?us-ascii?Q?TCpadkwwADBLx285nfrmkvTmud+RAQ4eI7nOBE4pXd7atIMTcUXS5j0bB+J9?= =?us-ascii?Q?comM8RshNvt8wztGqolLOOV0WWj3cqF+rPYoDqLfDZ8iM/wWB80i5U60kusD?= =?us-ascii?Q?/Co5e36KqA7x1TBe9lawOunynB5pb8HYqeQCOcIHOYV8EnNE99XODYiGJ+0R?= =?us-ascii?Q?3aaPyKjMRwFKBOL+DnRHZUrA0+n1mR+WnZ1C9gjIWEVbJ2nj5hdV6BqeBnCS?= =?us-ascii?Q?IRYQRKGwGHXw/UeLuNYdsrWTKpUER0+qPrsFDfvdGusg3TCjHevzxA0JgNwM?= =?us-ascii?Q?dfW3YtXQkWxAPPe8y+nnq9ixQ+gzvGeDHzx92aCbdX9nB7DdoCM04WAYR72Z?= =?us-ascii?Q?kggAPVWAX3BhIbN1KdXRmVcOB/l55JkDXVGD0Pv7Hxcl26JDYBLd3ypDFymS?= =?us-ascii?Q?jfTnilnXqZGljS0+vThh6hN3KS1xbwRBiM/F5X1fQgZPIdfUIW/BEUGKtW6n?= =?us-ascii?Q?9wxelOCnc87x+1+Qz0QLew9alhwcgwziXjhKAMcjChxVUymMki1DQFmPaBl9?= =?us-ascii?Q?EdofVfrnReVddmv3ZaYx5szgG6ZGqa+HC4d/kLVRUgHeMuiJAmsAYwJQ6B2e?= =?us-ascii?Q?XPEU0IJxDg8Q8mviHCjCE+TYFeHP8Zsq?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(376014)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2024 17:18:49.7427 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a250491f-c684-4a22-e4b2-08dcd673a175 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF00000142.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB8273 Content-Type: text/plain; charset="utf-8" According to the AMD IOMMU spec, IOMMU hardware reads the entire DTE in a single 256-bit transaction. It is recommended to update DTE using 128-bit operation followed by an INVALIDATE_DEVTAB_ENTYRY command when the IV=3D1b or V=3D1b before the change. According to the AMD BIOS and Kernel Developer's Guide (BDKG) dated back to family 10h Processor [1], which is the first introduction of AMD IOMMU, AMD processor always has CPUID Fn0000_0001_ECX[CMPXCHG16B]=3D1. Therefore, it is safe to assume cmpxchg128 is available with all AMD processor w/ IOMMU. In addition, the CMPXCHG16B feature has already been checked separately before enabling the GA, XT, and GAM modes. Consolidate the detection logic, and fail the IOMMU initialization if the feature is not supported. [1] https://www.amd.com/content/dam/amd/en/documents/archived-tech-docs/pro= grammer-references/31116.pdf Reviewed-by: Jason Gunthorpe Suggested-by: Jason Gunthorpe Signed-off-by: Suravee Suthikulpanit --- drivers/iommu/amd/init.c | 23 +++++++++-------------- 1 file changed, 9 insertions(+), 14 deletions(-) diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c index 6b15ce09e78d..983c09898a10 100644 --- a/drivers/iommu/amd/init.c +++ b/drivers/iommu/amd/init.c @@ -1762,13 +1762,8 @@ static int __init init_iommu_one(struct amd_iommu *i= ommu, struct ivhd_header *h, else iommu->mmio_phys_end =3D MMIO_CNTR_CONF_OFFSET; =20 - /* - * Note: GA (128-bit IRTE) mode requires cmpxchg16b supports. - * GAM also requires GA mode. Therefore, we need to - * check cmpxchg16b support before enabling it. - */ - if (!boot_cpu_has(X86_FEATURE_CX16) || - ((h->efr_attr & (0x1 << IOMMU_FEAT_GASUP_SHIFT)) =3D=3D 0)) + /* GAM requires GA mode. */ + if ((h->efr_attr & (0x1 << IOMMU_FEAT_GASUP_SHIFT)) =3D=3D 0) amd_iommu_guest_ir =3D AMD_IOMMU_GUEST_IR_LEGACY; break; case 0x11: @@ -1778,13 +1773,8 @@ static int __init init_iommu_one(struct amd_iommu *i= ommu, struct ivhd_header *h, else iommu->mmio_phys_end =3D MMIO_CNTR_CONF_OFFSET; =20 - /* - * Note: GA (128-bit IRTE) mode requires cmpxchg16b supports. - * XT, GAM also requires GA mode. Therefore, we need to - * check cmpxchg16b support before enabling them. - */ - if (!boot_cpu_has(X86_FEATURE_CX16) || - ((h->efr_reg & (0x1 << IOMMU_EFR_GASUP_SHIFT)) =3D=3D 0)) { + /* XT and GAM require GA mode. */ + if ((h->efr_reg & (0x1 << IOMMU_EFR_GASUP_SHIFT)) =3D=3D 0) { amd_iommu_guest_ir =3D AMD_IOMMU_GUEST_IR_LEGACY; break; } @@ -3049,6 +3039,11 @@ static int __init early_amd_iommu_init(void) return -EINVAL; } =20 + if (!boot_cpu_has(X86_FEATURE_CX16)) { + pr_err("Failed to initialize. The CMPXCHG16B feature is required.\n"); + return -EINVAL; + } + /* * Validate checksum here so we don't need to do it when * we actually parse the table --=20 2.34.1 From nobody Fri Nov 29 18:38:24 2024 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2084.outbound.protection.outlook.com [40.107.243.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F2DE315B555 for ; Mon, 16 Sep 2024 17:18:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.243.84 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726507139; cv=fail; b=MPjD+++n9qINb87BDmancyOFG9xLm9FbVQR8G3WXYzziYneKghFJJ1EdFsnlGf3TP+J3PnLuEooR547GyIO6AnLq4B30iV/uNCJEQmQnTX23/yX8/lFFZO+jX/5R4ucTcxuViycihkAuME7Prg2lHSUm3FZJCvEqCwhdajr2ao8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726507139; c=relaxed/simple; bh=jCym6XIl70oeG3zG2jzJ0kFb8+77CpGDycvBLtxJBG0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=GpALcbp6MTCs4eEHBfn2X6N1li92pd+Lshwp0gtkk1w7aakFlsjecXRiBtuchl5xjYgi7uKBO+FMm2qwc/IzlqFLu4de0NTAG4KiPb7AomgGJJlyTHYHIzedHm8guX/UBDyx+Fex+KrTgxAE5QUhKdJr1E71lLXZM1d0D9BzXI8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=sQRKsr8+; arc=fail smtp.client-ip=40.107.243.84 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="sQRKsr8+" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=jyh1jttJGFYDy1uzL2zDIzDEJWaCVa5ZA2UJzdmf9sorfiWUEvycpAXFJZkNuGoCVN4RyWyIq8odna5U7ThFE0AnecjxrkBPnVtPMY9ueLuWzzkVUcWapbJVdEcO9rTwgo8052MGbZqCxR2qALLdZnymHKMQ8y7RUaHkJdI4+Vi5YCAh6v088IgfdnH2hedrOMdnhLAPEY9akU46nKur8QBfNTtGwuzABmnTwLyY935qgf+EBE7OFanwjc2/t/OcGoHATgPMdqV1NdqSQF04eI1C7jYRyPSYtKT32QOqSHL/jbMS1FWrLUnKlXk5OShq9uuYBJ61OrC4tO2QJxfVrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Jkk5/mSCoS8oC5kLklT101UQZIx71yL2BaNoEUV4cbQ=; b=X95fgBCHvahHz9dFGboq2qJBKNqcBnbgM8DJJbx2ylcw9HJvPp9Y4qyGKxyE8okXndHMf7UE/IhLam58xYeC4RXi9ZoORSEk4jqDBBbMjHA3++2B97hDZv7cXYk5/sV+4qRNFVI5tVchDa32nI1gwWyQfWxO+SvgaH2raflVdjgWXjllfX6T+uj4jWMlGY1V0bGPYsvwPXIVjUuyCfaIpskgmeJk3e4ctdVsHJPbQ8kxhfdTBWcWvNm8DEI7cAZX1i9z3eKOQ0QG5nQ0ne7y0kAyDOzJHxQHltEfU37q5Lxvl9EXvIcKvKmlrt9c7aR+141ngqBBZ4NXAota3C65PA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Jkk5/mSCoS8oC5kLklT101UQZIx71yL2BaNoEUV4cbQ=; b=sQRKsr8+o2/7MMiAjHuywggbX9J0vyqZZQikHEMYjKdQtSktHAhhCsf6YmRd6Rh03G8XVi5hL8TLSVYjMEy3hZAYHhn8TXRjrVnAD9f7bA4QfUMME2oQoGAUkmIt+4ILc8AY0SU+WwKEWUstcI62sOzOLkBPCZfZTvdyoPREedU= Received: from CH5PR05CA0022.namprd05.prod.outlook.com (2603:10b6:610:1f0::14) by SJ2PR12MB7823.namprd12.prod.outlook.com (2603:10b6:a03:4c9::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7962.23; Mon, 16 Sep 2024 17:18:50 +0000 Received: from CH2PEPF00000142.namprd02.prod.outlook.com (2603:10b6:610:1f0:cafe::aa) by CH5PR05CA0022.outlook.office365.com (2603:10b6:610:1f0::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7982.15 via Frontend Transport; Mon, 16 Sep 2024 17:18:50 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CH2PEPF00000142.mail.protection.outlook.com (10.167.244.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7918.13 via Frontend Transport; Mon, 16 Sep 2024 17:18:50 +0000 Received: from purico-ed03host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 16 Sep 2024 12:18:36 -0500 From: Suravee Suthikulpanit To: , CC: , , , , , , , , , Suravee Suthikulpanit Subject: [PATCH v4 2/6] iommu/amd: Introduce helper function to update 256-bit DTE Date: Mon, 16 Sep 2024 17:18:01 +0000 Message-ID: <20240916171805.324292-3-suravee.suthikulpanit@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240916171805.324292-1-suravee.suthikulpanit@amd.com> References: <20240916171805.324292-1-suravee.suthikulpanit@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF00000142:EE_|SJ2PR12MB7823:EE_ X-MS-Office365-Filtering-Correlation-Id: 078c0cab-7ef3-471c-a6b3-08dcd673a1b8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|376014|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?x4U2IWH5+MxhDFeeVi70k5i2r8GqxchbNTCH5dC9Tx+b5a72y208/bhtJIOT?= =?us-ascii?Q?0sPm6uKVh8aIhAV405G+bsrh1hQk5EfT1kNDS+UIoAkRp8ZpJVZiCcyRqcBQ?= =?us-ascii?Q?9YsQ1MeyBkwyi2uxUkKGzEoHw8WIbAbrzfp6sA4bEWC2FC6WKuorXVxhAqu2?= =?us-ascii?Q?tfSRipf37vCaLwZLl3p8sDYs9VbToMwke/G3honBNq8IqvvXrv43FwOm6hdd?= =?us-ascii?Q?1wyw8jlIb+sCsS3RYeP4OJNTmSuErTxRJrpwPZXjdfbKYN8b/AOGUBer0xmd?= =?us-ascii?Q?OrhPt8ctFpJ0joQO0lAN789rWpe604BESt8Tsdu69UxPw5FuWnJE5l2Op8NT?= =?us-ascii?Q?AatLKIK6/XRirQ8QkmWYTL7C+tmli0ZloJZr+iHKos+TRQAW+jrSjUhi3sDg?= =?us-ascii?Q?RkUt7nCwtYKQhW26XPc/U4gLLgSjR1zK1aS++O1oCiDkfhLfMVOcC2Xb06Wk?= =?us-ascii?Q?hcG98LmogCC+Q518KtIs1oAzML067n8WNvUOuhZROvJpf18Cg9WDiMRoG5dv?= =?us-ascii?Q?XmJbCgUtxAapsK3LXgEjGu9PcUOdg4cifdlAin2UkfXRDePFc0IhmbE+LlGf?= =?us-ascii?Q?/Krba32U5vt7uw0W4mbv6q63xRwHAGIsqumt/ZHne5TwjMqpPJ8Bin8HJ6Yw?= =?us-ascii?Q?Hphsj6ae0U/lKyp4r6oVlPs8p+HRuoIMAUM5kGvhdvGOV6HBC13ntoExFe67?= =?us-ascii?Q?T66HdwSPfrwI7xrEq1ukP8YJu8T5nxUw8zfVPSKV2FqGTZsPzlhwPxr0tJ3J?= =?us-ascii?Q?sRbK3/idCuGyTf/orAvEvzNDEA5JRNfig4UQhJNvzqq6YSv21t1/wqBkeGh3?= =?us-ascii?Q?08WxCw/AL12l0Bv4pZ2w10T/odYs5zGw3pM8za5Fp6xXDSNsJhoH/aGUPA8j?= =?us-ascii?Q?VmPhPyFs/rfXgQPmPRB4JBahJlmzPo9g6b1hmhwdT1fuAAeof+bumgFc29aw?= =?us-ascii?Q?/LXSlK4hJvvjg+FQ3Mz0pM83MmPmk0NFJNngIUp+GJR3feWKrO0B7ZigHl2m?= =?us-ascii?Q?p9BOhzraBL8jYNMaRJcL869Rvap78MVommM2NdG8yLWFXNvZrPhfaSvXDNHt?= =?us-ascii?Q?flyELSB0kRX58I/EMJXQgOwC2e3PDOaDeHupfs51/eGxXYT8rNEtDkJLby4r?= =?us-ascii?Q?pX5EoXi30il158d3xfBp38Rf7qeZs7AU1eHfS3OiA1uIldJfKBGKmK3mVA/a?= =?us-ascii?Q?+SRNkTZexdxuxEwhydEelMj03YK/m9YTfJGWkGIrV0MFPZ5OQmCNGh8MT3Lr?= =?us-ascii?Q?CBpN2raqe0o3ZkICTLgAzu0yYECtXq17HuOilSeqQpyDEH3wYnXGI7v1EbTq?= =?us-ascii?Q?Jx1gMgy9XimG/QD36bmVoEpyshoVj5poZjXfE+QqMGzF9c8Eo2c0cUKgZ9XS?= =?us-ascii?Q?18WJF8mv2pQliLsYqYvf20k2dhdTE8MyY1vwIqOIV3jgm79vKyH1+iUF6tVO?= =?us-ascii?Q?iBAAo4psjBjfSvNLd2rgx8/PeyMrdhNT?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(376014)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2024 17:18:50.1802 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 078c0cab-7ef3-471c-a6b3-08dcd673a1b8 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF00000142.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7823 Content-Type: text/plain; charset="utf-8" The current implementation does not follow 128-bit write requirement to update DTE as specified in the AMD I/O Virtualization Techonology (IOMMU) Specification. Therefore, modify the struct dev_table_entry to contain union of u128 data array, and introduce a helper functions update_dte256() to update DTE using two 128-bit cmpxchg operations to update 256-bit DTE with the modified structure, and take into account the DTE[V, GV] bits when programming the DTE to ensure proper order of DTE programming and flushing. In addition, introduce a per-DTE spin_lock struct dev_data.dte_lock to provide synchronization when updating the DTE to prevent cmpxchg128 failure. Suggested-by: Jason Gunthorpe Signed-off-by: Suravee Suthikulpanit Reviewed-by: Jason Gunthorpe --- drivers/iommu/amd/amd_iommu.h | 2 + drivers/iommu/amd/amd_iommu_types.h | 8 ++- drivers/iommu/amd/iommu.c | 96 +++++++++++++++++++++++++++++ 3 files changed, 105 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/amd/amd_iommu.h b/drivers/iommu/amd/amd_iommu.h index 52e18b5f99fd..14a153c7bc12 100644 --- a/drivers/iommu/amd/amd_iommu.h +++ b/drivers/iommu/amd/amd_iommu.h @@ -28,6 +28,8 @@ void iommu_feature_enable(struct amd_iommu *iommu, u8 bit= ); void *__init iommu_alloc_4k_pages(struct amd_iommu *iommu, gfp_t gfp, size_t size); =20 +int iommu_flush_sync_dte(struct amd_iommu *iommu, u16 devid); + #ifdef CONFIG_AMD_IOMMU_DEBUGFS void amd_iommu_debugfs_setup(struct amd_iommu *iommu); #else diff --git a/drivers/iommu/amd/amd_iommu_types.h b/drivers/iommu/amd/amd_io= mmu_types.h index c9f9a598eb82..fea7544f8c55 100644 --- a/drivers/iommu/amd/amd_iommu_types.h +++ b/drivers/iommu/amd/amd_iommu_types.h @@ -427,6 +427,8 @@ =20 #define GCR3_VALID 0x01ULL =20 +#define DTE_INTR_MASK (~GENMASK_ULL(55, 52)) + #define IOMMU_PAGE_MASK (((1ULL << 52) - 1) & ~0xfffULL) #define IOMMU_PTE_PRESENT(pte) ((pte) & IOMMU_PTE_PR) #define IOMMU_PTE_DIRTY(pte) ((pte) & IOMMU_PTE_HD) @@ -833,6 +835,7 @@ struct devid_map { struct iommu_dev_data { /*Protect against attach/detach races */ spinlock_t lock; + spinlock_t dte_lock; /* DTE lock for 256-bit access */ =20 struct list_head list; /* For domain->dev_list */ struct llist_node dev_data_list; /* For global dev_data_list */ @@ -883,7 +886,10 @@ extern struct amd_iommu *amd_iommus[MAX_IOMMUS]; * Structure defining one entry in the device table */ struct dev_table_entry { - u64 data[4]; + union { + u64 data[4]; + u128 data128[2]; + }; }; =20 /* diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index 87c5385ce3f2..48a721d10f06 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -85,6 +85,91 @@ static void set_dte_entry(struct amd_iommu *iommu, * *************************************************************************= ***/ =20 +static void write_upper(struct dev_table_entry *ptr, struct dev_table_entr= y *new) +{ + struct dev_table_entry old =3D {}; + + do { + old.data128[1] =3D ptr->data128[1]; + new->data[2] &=3D ~DTE_INTR_MASK; + new->data[2] |=3D (old.data[2] & DTE_INTR_MASK); + } while (!try_cmpxchg128(&ptr->data128[1], &old.data128[1], new->data128[= 1])); +} + +static void write_lower(struct dev_table_entry *ptr, struct dev_table_entr= y *new) +{ + struct dev_table_entry old =3D {}; + + do { + old.data128[0] =3D ptr->data128[0]; + } while (!try_cmpxchg128(&ptr->data128[0], &old.data128[0], new->data128[= 0])); +} + +/* + * Note: + * IOMMU reads the entire Device Table entry in a single 256-bit transacti= on + * but the driver is programming DTE using 2 128-bit cmpxchg. So, the driv= er + * need to ensure the following: + * - DTE[V|GV] bit is being written last when setting. + * - DTE[V|GV] bit is being written first when clearing. + * + * This function is used only by code, which updates DMA translation part = of the DTE. + * So, only consider control bits related to DMA when updating the entry. + */ +static void update_dte256(struct amd_iommu *iommu, struct iommu_dev_data *= dev_data, + struct dev_table_entry *new) +{ + struct dev_table_entry *dev_table =3D get_dev_table(iommu); + struct dev_table_entry *ptr =3D &dev_table[dev_data->devid]; + + spin_lock(&dev_data->dte_lock); + + if (!(ptr->data[0] & DTE_FLAG_V)) { + /* Existing DTE is not valid. */ + write_upper(ptr, new); + write_lower(ptr, new); + iommu_flush_sync_dte(iommu, dev_data->devid); + } else if (!(new->data[0] & DTE_FLAG_V)) { + /* Existing DTE is valid. New DTE is not valid. */ + write_lower(ptr, new); + write_upper(ptr, new); + iommu_flush_sync_dte(iommu, dev_data->devid); + } else { + /* Existing & new DTEs are valid. */ + if (!FIELD_GET(DTE_FLAG_GV, ptr->data[0])) { + /* Existing DTE has no guest page table. */ + write_upper(ptr, new); + write_lower(ptr, new); + iommu_flush_sync_dte(iommu, dev_data->devid); + } else if (!FIELD_GET(DTE_FLAG_GV, new->data[0])) { + /* + * Existing DTE has guest page table, + * new DTE has no guest page table, + */ + write_lower(ptr, new); + write_upper(ptr, new); + iommu_flush_sync_dte(iommu, dev_data->devid); + } else { + /* + * Existing DTE has guest page table, + * new DTE has guest page table. + */ + struct dev_table_entry clear =3D {}; + + /* First disable DTE */ + write_lower(ptr, &clear); + iommu_flush_sync_dte(iommu, dev_data->devid); + + /* Then update DTE */ + write_upper(ptr, new); + write_lower(ptr, new); + iommu_flush_sync_dte(iommu, dev_data->devid); + } + } + + spin_unlock(&dev_data->dte_lock); +} + static inline bool pdom_is_v2_pgtbl_mode(struct protection_domain *pdom) { return (pdom && (pdom->pd_mode =3D=3D PD_MODE_V2)); @@ -205,6 +290,7 @@ static struct iommu_dev_data *alloc_dev_data(struct amd= _iommu *iommu, u16 devid) return NULL; =20 spin_lock_init(&dev_data->lock); + spin_lock_init(&dev_data->dte_lock); dev_data->devid =3D devid; ratelimit_default_init(&dev_data->rs); =20 @@ -1256,6 +1342,16 @@ static int iommu_flush_dte(struct amd_iommu *iommu, = u16 devid) return iommu_queue_command(iommu, &cmd); } =20 +int iommu_flush_sync_dte(struct amd_iommu *iommu, u16 devid) +{ + int ret; + + ret =3D iommu_flush_dte(iommu, devid); + if (!ret) + iommu_completion_wait(iommu); + return ret; +} + static void amd_iommu_flush_dte_all(struct amd_iommu *iommu) { u32 devid; --=20 2.34.1 From nobody Fri Nov 29 18:38:24 2024 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2043.outbound.protection.outlook.com [40.107.94.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 47D6815AD95 for ; Mon, 16 Sep 2024 17:18:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.43 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726507137; cv=fail; b=WuoUEenYXa9AcvdcILEI7RVJWaSLgyDV6QV7iq1nO7fSw7qBqqkeWsXkjNNIzaE+wZ8rmphtJ0h7ciJ0fawRCVqFiLICzhXi5xI/S/sSe1iFK8xmzgsa9/nGdxfK32tnUPU1J421dJ1SDPcF22MFocxBE4XJxvaMlKaAe9bi8Fk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726507137; c=relaxed/simple; bh=lBG5nv5dXJxp4+mm3F5Z5R0IeJuDDRuvRUue5lugTvU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=cpZKBLOhNFRNy+qof58qMk7RGePUgwdtcl//LXgCqB6SiJm09yrpjicEvVAyqPsaYlXZCjApr9mGOW7L4AFTL7m5R0RUS83TgzsgPL7uczXdg45zILo42J27ef6Ep6emmQYtX652AwUzqHRPGMwEMgkWj/lAeXLiqxyb9wVyyiQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=YvfTonTZ; arc=fail smtp.client-ip=40.107.94.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="YvfTonTZ" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=h16iQQ2rKUFrVxjJO07l6YQGUmVAeVpkQotPH55nJAqM+38wr8jj5iyR4eU4RYEJD2GxQbqfCcSzNOASRYwCNEF95UudJuLtxYXzoJgVcwl41aGGwHyWohks2UNRbXcMBDbASHPnErKr5htKcn6bCmNtVmSTZVf3e4T/QwrLcnSNteW2IGR00b+LjGzbloIzbjIZt3b/eZcs03HoGhnSHGkSlzhMweDHz13x9/Es3S++zdqDxUb21HoXdHS0bpSsIihn0sOgwIsUUCEUhciAwId/AN9yTmlgSqL+TkwACvy7wPBAMF2e0IC7PHUFyH3iuzszH1OQS7Y5vASB5IWFzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JwHUomeIAedLsmsRqoWtjxl5qDnhh8DUgivQYgI3BNE=; b=gUPJq8+unfA2odqm1k0HLWztD0AOe+pGOdsiDgnEUtaBOuaL65tWhR3YTbYFfV23O0G86MOUb6w8YW0twMooOjTDtVmAAOASXMCnFVZ0DorCJgvm9UQQKk1Y9o5bWiIUoCqevcMY2bxdBkZXa6hxhY3gTPseLOnHI1TibMEsgLTUwFtCTOKBTtiQM0r58XQg4bcS6/Z9/esur7Ot+tSXKPLVQeVl36rCKmdipwYVg51K4HdDraRffh9/6kisVAJ6eRjaOdp4+hEywwi9zIeSWytuqZLpx+ASoZbGkgSw8NK0yWx6njB3mw3YvIzbTLM95WNjkaoU5RxHkm/pb1SiaQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JwHUomeIAedLsmsRqoWtjxl5qDnhh8DUgivQYgI3BNE=; b=YvfTonTZpZ1DFP50uzjxI8i5l+w99aI3g8XiyVZwb1RRdwCzmUVQX7ccWJIyvMdvPTFZ3DlOkKU5j76X+iQXZL745hcDpyiVzhbpsXXoO8KsjhmzCDvC87scQD2nEpCcqolywLElcoCcfkGiU5ReROAgWQP7lBGNZe1xGPEK4Fw= Received: from CH5PR05CA0007.namprd05.prod.outlook.com (2603:10b6:610:1f0::12) by LV3PR12MB9267.namprd12.prod.outlook.com (2603:10b6:408:211::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7962.22; Mon, 16 Sep 2024 17:18:52 +0000 Received: from CH2PEPF00000142.namprd02.prod.outlook.com (2603:10b6:610:1f0:cafe::53) by CH5PR05CA0007.outlook.office365.com (2603:10b6:610:1f0::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7982.15 via Frontend Transport; Mon, 16 Sep 2024 17:18:52 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CH2PEPF00000142.mail.protection.outlook.com (10.167.244.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7918.13 via Frontend Transport; Mon, 16 Sep 2024 17:18:52 +0000 Received: from purico-ed03host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 16 Sep 2024 12:18:40 -0500 From: Suravee Suthikulpanit To: , CC: , , , , , , , , , Suravee Suthikulpanit Subject: [PATCH v4 3/6] iommu/amd: Modify set_dte_entry() to use 256-bit DTE helpers Date: Mon, 16 Sep 2024 17:18:02 +0000 Message-ID: <20240916171805.324292-4-suravee.suthikulpanit@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240916171805.324292-1-suravee.suthikulpanit@amd.com> References: <20240916171805.324292-1-suravee.suthikulpanit@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF00000142:EE_|LV3PR12MB9267:EE_ X-MS-Office365-Filtering-Correlation-Id: 541d8675-2775-4c95-adb3-08dcd673a2e2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|36860700013|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?yQdxJMw5gqvVKmZdmYLYG+V+pk+vT9iJwyIkA9rjuzH9JetsqB1I4/AzsAtc?= =?us-ascii?Q?5aIQBV2YEW13Az3dbrJ9xW9oJV+hp9BLlhtHWnxqLSDKelzcSoQ2TKDR7Fh2?= =?us-ascii?Q?qsTwdFXR79ghY90CVXJnL+oiUeBw4uglM+qFMIBl3OY9F8BqfPQrYHf6sOcl?= =?us-ascii?Q?rWhWY8pxMv76xeqKHetGA1hnfimPOgs1Lms1+PhdUgCN4BGg93Zv05m0JWm0?= =?us-ascii?Q?i4gagG0xgJTqO+HBZryeEh0T/XlkJrVl6r5niwVmVhsiEb7yX0wDlK1Z1q4T?= =?us-ascii?Q?iK6/9uIp1XFs9n0WSifcj374YNODV7zPIxEReV9BCrvYYxNERR/qgve3t8a0?= =?us-ascii?Q?4TiHu1PZNs2O03BFFzlrVzC8BBy70Jrv1SLAqmOrwOmJVtY8Z6Gq0ABnbWj5?= =?us-ascii?Q?oqbmvlx9F/mwbVWZoDl6FGxp9qWq+6HlczyCr7T3XLf1lEknf2+sB7prg9bZ?= =?us-ascii?Q?TM74s2+V8y21n2OdyG6D6tBUADSPC+ccrzG/vwScZrpcxglGMbV3kbmTnETc?= =?us-ascii?Q?58Ld3zpV8RdQl9sRL/TWDjPa65JSfFAW1BAzq288cMLHjIq8ZEFVmkdxOnnI?= =?us-ascii?Q?/zbRf6tl8FVmglitQTTtKme6ewLFlP/yKt8y+X7ssXVoz8mn2C5s8b+AIwwB?= =?us-ascii?Q?5xYkbSssUWPE1uhX/AA5r4JXC3mU4EjgtklWSnfauChvHSuyW4s/CjTvkwNJ?= =?us-ascii?Q?ary0pup2XtvWSXwv8VZyF8ijL7LmEWDMtIz2CnrubET0b7NZStwc/spJfE8B?= =?us-ascii?Q?ZNCI6DsFU7fmw/oRn907SynWfe4Edxg33orit4LxF2UC9kECGHM4AoSb0xSz?= =?us-ascii?Q?g2HepUEzT+sxnGtp/zHElFRsc/7cggyDRncPZr7P83hdeQrXeZhL7RjlhpFs?= =?us-ascii?Q?Ygl0cMAO/6TfEwdjn9d9iL0OqSMcZdxVEce94r3ZioDgfGQENeV36w6cvNV9?= =?us-ascii?Q?92awVft2TZXZoXx4vmFhV9/v7CSAblm0ToAoL+XF+gKkzs4F62ryRJyjXLhJ?= =?us-ascii?Q?PXb/XZ1sGb1sl40kKhWqCiEBzjhlPEaaimhZe+RzSYrzmVO/TpwEWA0UNWpj?= =?us-ascii?Q?woP53yMuFJF/u9auVyLgsfkq+tOpkxE1zWivzUKqQfLymTvwm1GXi/2GtMtp?= =?us-ascii?Q?jVrN+Cw9ljDSuF/aDZ0OygV/g+me4SCNabGCGbEO6h2wGHQJVC4OlcolUAgl?= =?us-ascii?Q?oSqedeedgh9T4oEDDV4sLTBt9nG4t0hahBMHxp0oKf5eqdyYpjhwVGPyNXgv?= =?us-ascii?Q?yXdB+LctnJmAzU9IW76cxl59MRT04F4GPTMq0DLFaJeCJX/OLO0I3FKuaL9f?= =?us-ascii?Q?IUvD7rG9jbX8xOqp6qDh4o1ptn9j/X90pj0mdq6OWoRybtv55cFkZFqdwAiq?= =?us-ascii?Q?nv6JwHwYMkZxjMRb3/pRIcV30/CQHhtx9B9wYWlrpjlhZe9R2kuBJp7fMMMS?= =?us-ascii?Q?Ra2SpTyDukOYbDVnVl2/Hx1T1T90GjX8?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(36860700013)(376014);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2024 17:18:52.1334 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 541d8675-2775-4c95-adb3-08dcd673a2e2 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF00000142.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR12MB9267 Content-Type: text/plain; charset="utf-8" Also, the set_dte_entry() is used to program several DTE fields (e.g. stage1 table, stage2 table, domain id, and etc.), which is difficult to keep track with current implementation. Therefore, separate logic for setting up the GCR3 Table Root Pointer, GIOV, GV, GLX, and GuestPagingMode into another helper function set_dte_gcr3_table(). Signed-off-by: Suravee Suthikulpanit --- drivers/iommu/amd/iommu.c | 117 +++++++++++++++++++++----------------- 1 file changed, 65 insertions(+), 52 deletions(-) diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index 48a721d10f06..12f27061680d 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -1947,17 +1947,58 @@ int amd_iommu_clear_gcr3(struct iommu_dev_data *dev= _data, ioasid_t pasid) return ret; } =20 +static void set_dte_gcr3_table(struct amd_iommu *iommu, + struct iommu_dev_data *dev_data, + struct dev_table_entry *target) +{ + struct gcr3_tbl_info *gcr3_info =3D &dev_data->gcr3_info; + u64 tmp, gcr3; + + if (!gcr3_info->gcr3_tbl) + return; + + pr_debug("%s: devid=3D%#x, glx=3D%#x, gcr3_tbl=3D%#llx\n", + __func__, dev_data->devid, gcr3_info->glx, + (unsigned long long)gcr3_info->gcr3_tbl); + + tmp =3D gcr3_info->glx; + target->data[0] |=3D (tmp & DTE_GLX_MASK) << DTE_GLX_SHIFT; + if (pdom_is_v2_pgtbl_mode(dev_data->domain)) + target->data[0] |=3D DTE_FLAG_GIOV; + target->data[0] |=3D DTE_FLAG_GV; + + /* First mask out possible old values for GCR3 table */ + tmp =3D DTE_GCR3_VAL_A(~0ULL) << DTE_GCR3_SHIFT_A; + target->data[0] &=3D ~tmp; + tmp =3D DTE_GCR3_VAL_B(~0ULL) << DTE_GCR3_SHIFT_B; + tmp |=3D DTE_GCR3_VAL_C(~0ULL) << DTE_GCR3_SHIFT_C; + target->data[1] &=3D ~tmp; + + gcr3 =3D iommu_virt_to_phys(gcr3_info->gcr3_tbl); + + /* Encode GCR3 table into DTE */ + tmp =3D DTE_GCR3_VAL_A(gcr3) << DTE_GCR3_SHIFT_A; + target->data[0] |=3D tmp; + tmp =3D DTE_GCR3_VAL_B(gcr3) << DTE_GCR3_SHIFT_B; + tmp |=3D DTE_GCR3_VAL_C(gcr3) << DTE_GCR3_SHIFT_C; + target->data[1] |=3D tmp; + + /* Mask out old values for GuestPagingMode */ + target->data[2] &=3D ~(0x3ULL << DTE_GPT_LEVEL_SHIFT); + /* Guest page table can only support 4 and 5 levels */ + if (amd_iommu_gpt_level =3D=3D PAGE_MODE_5_LEVEL) + target->data[2] |=3D ((u64)GUEST_PGTABLE_5_LEVEL << DTE_GPT_LEVEL_SHIFT); +} + static void set_dte_entry(struct amd_iommu *iommu, struct iommu_dev_data *dev_data) { - u64 pte_root =3D 0; - u64 flags =3D 0; - u32 old_domid; - u16 devid =3D dev_data->devid; u16 domid; + u32 old_domid; + struct dev_table_entry new =3D {}; struct protection_domain *domain =3D dev_data->domain; - struct dev_table_entry *dev_table =3D get_dev_table(iommu); struct gcr3_tbl_info *gcr3_info =3D &dev_data->gcr3_info; + struct dev_table_entry *dte =3D &get_dev_table(iommu)[dev_data->devid]; =20 if (gcr3_info && gcr3_info->gcr3_tbl) domid =3D dev_data->gcr3_info.domid; @@ -1965,72 +2006,44 @@ static void set_dte_entry(struct amd_iommu *iommu, domid =3D domain->id; =20 if (domain->iop.mode !=3D PAGE_MODE_NONE) - pte_root =3D iommu_virt_to_phys(domain->iop.root); + new.data[0] =3D iommu_virt_to_phys(domain->iop.root); =20 - pte_root |=3D (domain->iop.mode & DEV_ENTRY_MODE_MASK) + new.data[0] |=3D (domain->iop.mode & DEV_ENTRY_MODE_MASK) << DEV_ENTRY_MODE_SHIFT; =20 - pte_root |=3D DTE_FLAG_IR | DTE_FLAG_IW | DTE_FLAG_V; + new.data[0] |=3D DTE_FLAG_IR | DTE_FLAG_IW | DTE_FLAG_V; =20 /* * When SNP is enabled, Only set TV bit when IOMMU * page translation is in use. */ if (!amd_iommu_snp_en || (domid !=3D 0)) - pte_root |=3D DTE_FLAG_TV; - - flags =3D dev_table[devid].data[1]; - - if (dev_data->ats_enabled) - flags |=3D DTE_FLAG_IOTLB; + new.data[0] |=3D DTE_FLAG_TV; =20 if (dev_data->ppr) - pte_root |=3D 1ULL << DEV_ENTRY_PPR; + new.data[0] |=3D 1ULL << DEV_ENTRY_PPR; =20 if (domain->dirty_tracking) - pte_root |=3D DTE_FLAG_HAD; - - if (gcr3_info && gcr3_info->gcr3_tbl) { - u64 gcr3 =3D iommu_virt_to_phys(gcr3_info->gcr3_tbl); - u64 glx =3D gcr3_info->glx; - u64 tmp; - - pte_root |=3D DTE_FLAG_GV; - pte_root |=3D (glx & DTE_GLX_MASK) << DTE_GLX_SHIFT; - - /* First mask out possible old values for GCR3 table */ - tmp =3D DTE_GCR3_VAL_B(~0ULL) << DTE_GCR3_SHIFT_B; - flags &=3D ~tmp; + new.data[0] |=3D DTE_FLAG_HAD; =20 - tmp =3D DTE_GCR3_VAL_C(~0ULL) << DTE_GCR3_SHIFT_C; - flags &=3D ~tmp; - - /* Encode GCR3 table into DTE */ - tmp =3D DTE_GCR3_VAL_A(gcr3) << DTE_GCR3_SHIFT_A; - pte_root |=3D tmp; - - tmp =3D DTE_GCR3_VAL_B(gcr3) << DTE_GCR3_SHIFT_B; - flags |=3D tmp; + if (dev_data->ats_enabled) + new.data[1] |=3D DTE_FLAG_IOTLB; + else + new.data[1] &=3D ~DTE_FLAG_IOTLB; =20 - tmp =3D DTE_GCR3_VAL_C(gcr3) << DTE_GCR3_SHIFT_C; - flags |=3D tmp; + old_domid =3D new.data[1] & DEV_DOMID_MASK; + new.data[1] &=3D ~DEV_DOMID_MASK; + new.data[1] |=3D domid; =20 - if (amd_iommu_gpt_level =3D=3D PAGE_MODE_5_LEVEL) { - dev_table[devid].data[2] |=3D - ((u64)GUEST_PGTABLE_5_LEVEL << DTE_GPT_LEVEL_SHIFT); - } + /* Need to preserve DTE[96:106] */ + new.data[1] |=3D dte->data[1] & DTE_FLAG_MASK; =20 - /* GIOV is supported with V2 page table mode only */ - if (pdom_is_v2_pgtbl_mode(domain)) - pte_root |=3D DTE_FLAG_GIOV; - } + /* Need to preserve interrupt remapping information in DTE[128:255] */ + new.data128[1] =3D dte->data128[1]; =20 - flags &=3D ~DEV_DOMID_MASK; - flags |=3D domid; + set_dte_gcr3_table(iommu, dev_data, &new); =20 - old_domid =3D dev_table[devid].data[1] & DEV_DOMID_MASK; - dev_table[devid].data[1] =3D flags; - dev_table[devid].data[0] =3D pte_root; + update_dte256(iommu, dev_data, &new); =20 /* * A kdump kernel might be replacing a domain ID that was copied from --=20 2.34.1 From nobody Fri Nov 29 18:38:24 2024 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2046.outbound.protection.outlook.com [40.107.244.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE44215B12B for ; Mon, 16 Sep 2024 17:18:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.244.46 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726507138; cv=fail; b=mQ0d49aMTdHAKCAM13lmejFCx3TlDJ8Pqeqe5Yl1tIiFlzH3x3GMTMX+mzWhNt1JLtyjjoRr0Mb7XGHExQNxvGxf1zWRdYbDTdtrnImnr2asFQX8+7GdLUHjXgwdSmv+wBsne4ap4TAjOhPqmci+I3rOQuYb58Gel7x1h+lATIc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726507138; c=relaxed/simple; bh=G6UUyeSCkNw80t9xzliFkoGty2fU/uwd4o2ZdUkm/wg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=dNEmQBqYwRPl/MP8LDlpexnwOCbBLJ6NKgMj3JUxO7tKgFM2unV7pGi2y+6l/7LPnYHYIS8G7+Ultk259CO/pPwrDGGIK6AQcgjHVksPFnjH/YkIJMywGDCoqhwSvfCDZ/w/drjbv6A8JPeEmEtWqYyDxq57P0QJZ4gV/enPn2o= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=havEVw3b; arc=fail smtp.client-ip=40.107.244.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="havEVw3b" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ieeLwsJd1C61qD+FXr37od45CiUOKLpKgliV3TMqaJAqKg0PU8AUGRfO49nOzwsjSpJJzkSKpNM6mLo0DBm1CS7O6cSIwmH1hT/JK02czkOmF8t1hyBmoM+KGq0XwO5Ia3DE9KMGy4TsFm+EeHMnFj9vk5Umq1datX5U4EuYlrGlBKOyCdiQ0825F5N6Dl6fh7hFH872Xlu7Kpw+5HdhLbzblPzWFhHxfli21i02GORs7gR7ukzqM4JjsvKtQFSuqB0Nml/Eny/0d/SX6xKiDBpwDz6/NKbqr4J2yX/FLe7e+/F2N/gaXwBibhCJcLiWhutFl4W4sfTxgpFv5mKD2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VcpEGhJxV04khh2/dgkIIhtdTMkFChedRdm0jsGV4yY=; b=C2JMeax1x7eo0tloFvOCBQ0zrKG8pV92IZuwGiBwWEWCjmZ+yFYzqPgj4cbcmPWyH9DAGmcR0mFiGbw0NlaLDztAPmcvNMXyu4uodoQ7yzTRt0pMvZQ/xCp/pCpE3n+T6F/qlpZF7nUmrS+txzM9MWqQ/NW1L8TJXi1O3791/Q0BNYQhnbZPXkRkgdad5+zAX2alxROxCnQX0a3TiYPAHzAgWCZibChUmG8XMW80ZuTyPfCZpdJMhm6olsu0w3iZ2StR1XrYqb9tMIwpKj793ryWQZgr15anOxNCFjYMt6KBziFtfHHJwAnO+6M9gPYZbqgU+Nl7rszRRuJ9XyTuiA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VcpEGhJxV04khh2/dgkIIhtdTMkFChedRdm0jsGV4yY=; b=havEVw3b7Q2qSLJ6zay30gOLpo0dg1839xc18esK7U8J9szsuvIXikGL3VoVbh3WF+40HpZqG9A3nV/qhiRk+PdMd1Ksxb51LmHx2qhRVvO6WCK3DNd7H9c0N5T25PAJJ/YYh74MJxM/HcS+JwtMfvmGZyKFzTIUV4bPgvcp6hk= Received: from CH5PR05CA0006.namprd05.prod.outlook.com (2603:10b6:610:1f0::11) by DM4PR12MB6158.namprd12.prod.outlook.com (2603:10b6:8:a9::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7962.23; Mon, 16 Sep 2024 17:18:52 +0000 Received: from CH2PEPF00000142.namprd02.prod.outlook.com (2603:10b6:610:1f0:cafe::6c) by CH5PR05CA0006.outlook.office365.com (2603:10b6:610:1f0::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7982.14 via Frontend Transport; Mon, 16 Sep 2024 17:18:52 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CH2PEPF00000142.mail.protection.outlook.com (10.167.244.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7918.13 via Frontend Transport; Mon, 16 Sep 2024 17:18:52 +0000 Received: from purico-ed03host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 16 Sep 2024 12:18:44 -0500 From: Suravee Suthikulpanit To: , CC: , , , , , , , , , Suravee Suthikulpanit Subject: [PATCH v4 4/6] iommu/amd: Introduce helper function get_dte256() Date: Mon, 16 Sep 2024 17:18:03 +0000 Message-ID: <20240916171805.324292-5-suravee.suthikulpanit@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240916171805.324292-1-suravee.suthikulpanit@amd.com> References: <20240916171805.324292-1-suravee.suthikulpanit@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF00000142:EE_|DM4PR12MB6158:EE_ X-MS-Office365-Filtering-Correlation-Id: ca74e4c4-bcad-458e-fdc2-08dcd673a32e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?wlIVg+6sNvV0Hy/pdoLySlw/rPCb03Fag104fnEYWGOI7NsE5Qgs2ULnt6mg?= =?us-ascii?Q?O75YtPcn/PU8Hy1Aq+ey8cl3jXro6LpN1oXxPnv6eI7f4i97b1SoOOAfrDx7?= =?us-ascii?Q?7Z9R3TNzvh89CJnv+mJfLCOEr5vVClgrL5nzWaRZNmpi2Pe5lrZ3yt5K+jIj?= =?us-ascii?Q?ETHO+TbcwSHKBZ8xR64ae0rZnQSUuLj+P17qDHPFWXkXVqbmVisG35VCX6qn?= =?us-ascii?Q?WbegH7l+HyWLC1vL1v9B9bFXD29k0K+3eFCEIpSTYji9LuBEjWKVQpFzc+tA?= =?us-ascii?Q?A9z7UNYg/OyDhEJp/drUnjOEvhWlJwGQ5G3PGuDOhVyq9xFmbw1Dal2nhH8z?= =?us-ascii?Q?Xg5CKo2gVEQYtfsqetQ2V+ItSEayM2hAl8TIm5XK0Lpv7x7f7bmm/GgB9M+P?= =?us-ascii?Q?qffyXWwkUe5hvem3wT/Lqw3j5uXbLIl2J9obzptchyekvQBMYQUY/JwS0SbK?= =?us-ascii?Q?BSp0XKujqqFFmuCqfol6pe29FNDI3raounX2yS/vLh1VTaouYg+kwqAJ58pZ?= =?us-ascii?Q?AWkaYwIq/czZOc7bgZ94kUaPtLH/0p6iVDQuKKDnAsvrx7198SLS5KkdcrSD?= =?us-ascii?Q?3qa1PGYMkbfB6/jPmeZLTvLriVbKut7P2FP2v6yhlRi2q5DkyPm4uDeQPd1+?= =?us-ascii?Q?W216CR2eCZ8kkfmDWs2PoaBHEMvKvE0QdsWJsCSfOIHCypQFPUx87F1F3ctF?= =?us-ascii?Q?j5ZqJGeZSaXfv+oHW07+aUCKg//nKMK6NMIfLKc1A0a4y0Vs//yn9raMgG+t?= =?us-ascii?Q?MK+4Rfj2AXuhTCL8xlGUmC7VUjXqIi2dUYjnVVjIZHwRLDIpfqLmWQuM4mmA?= =?us-ascii?Q?HBr2VwyOkGA5CM8GvrVZX3F/B9VjH5YgqCOGLKK6T+zXMUaWCFZZehLIMgSz?= =?us-ascii?Q?UzGJlZFMKn5xrCpKDoopLqd3YbFGirKpkHZED2pYpqlQwWg6FFAaPfPQZSVb?= =?us-ascii?Q?oCEZx3Yq7z/4UZzxuv2081+SfFhjLCzW/od6Tzst5eCOWAIU0UDlEOlC/PQ8?= =?us-ascii?Q?0j/DJJEIan2ItBBgmjMt8l+6gYUe7W/i+/cs5NPXGaeRHf7+DUVnmLx0DIV0?= =?us-ascii?Q?aO85n93d0KYqLc4YuEja3s0bOLuxSe3sjr2E2aNAn5Z22Xmxg9gnW3SjbcNw?= =?us-ascii?Q?TI44cmjRT7Xm9xwHKaMXPNjSQoiwBNmyAgQNUxH8DE8fPpQobzezTqrvKLwq?= =?us-ascii?Q?HgKMFqJ1occ3spOOr/DXDta/ibHsAKMfbUruJ21AuH4930vyPT/PFFOIUDQm?= =?us-ascii?Q?Z+Fv/7xB3l5DEPSPzejobIT2vG/7/cDHdMs2dMHF4GVlmA+su3+NhCYjyrQW?= =?us-ascii?Q?U9JgRFWVQFHYRj3rIgUisWvOljJI+ZDrBaAzJ1st2b/pa0OMdbNz59MDW9nM?= =?us-ascii?Q?YoB1cGxW/4afWVkxPGT4Gk/9rCafuyF3l+KYpV+mKfvlpgMasu0cWZPhfkdl?= =?us-ascii?Q?OP0i0aUv53g0tNLtY3ro7Cr7iFuVDMYL?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(1800799024)(36860700013)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2024 17:18:52.6334 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ca74e4c4-bcad-458e-fdc2-08dcd673a32e X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF00000142.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6158 Content-Type: text/plain; charset="utf-8" And use it in clone_alias() along with update_dte256(). Also use get_dte256() in dump_dte_entry(). Signed-off-by: Suravee Suthikulpanit Reviewed-by: Jason Gunthorpe --- drivers/iommu/amd/iommu.c | 49 ++++++++++++++++++++++++++++++++------- 1 file changed, 40 insertions(+), 9 deletions(-) diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index 12f27061680d..676742d6f19a 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -170,6 +170,20 @@ static void update_dte256(struct amd_iommu *iommu, str= uct iommu_dev_data *dev_da spin_unlock(&dev_data->dte_lock); } =20 +static void get_dte256(struct amd_iommu *iommu, struct iommu_dev_data *dev= _data, + struct dev_table_entry *dte) +{ + struct dev_table_entry *ptr; + struct dev_table_entry *dev_table =3D get_dev_table(iommu); + + ptr =3D &dev_table[dev_data->devid]; + + spin_lock(&dev_data->dte_lock); + dte->data128[0] =3D ptr->data128[0]; + dte->data128[1] =3D ptr->data128[1]; + spin_unlock(&dev_data->dte_lock); +} + static inline bool pdom_is_v2_pgtbl_mode(struct protection_domain *pdom) { return (pdom && (pdom->pd_mode =3D=3D PD_MODE_V2)); @@ -318,9 +332,11 @@ static struct iommu_dev_data *search_dev_data(struct a= md_iommu *iommu, u16 devid =20 static int clone_alias(struct pci_dev *pdev, u16 alias, void *data) { + struct dev_table_entry new; struct amd_iommu *iommu; - struct dev_table_entry *dev_table; + struct iommu_dev_data *dev_data, *alias_data; u16 devid =3D pci_dev_id(pdev); + int ret =3D 0; =20 if (devid =3D=3D alias) return 0; @@ -329,13 +345,25 @@ static int clone_alias(struct pci_dev *pdev, u16 alia= s, void *data) if (!iommu) return 0; =20 - amd_iommu_set_rlookup_table(iommu, alias); - dev_table =3D get_dev_table(iommu); - memcpy(dev_table[alias].data, - dev_table[devid].data, - sizeof(dev_table[alias].data)); + /* Copy the data from pdev */ + dev_data =3D dev_iommu_priv_get(&pdev->dev); + if (!dev_data) { + ret =3D -EINVAL; + goto out; + } + get_dte256(iommu, dev_data, &new); =20 - return 0; + /* Setup alias */ + alias_data =3D search_dev_data(iommu, alias); + if (!alias_data) { + ret =3D -EINVAL; + goto out; + } + update_dte256(iommu, alias_data, &new); + + amd_iommu_set_rlookup_table(iommu, alias); +out: + return ret; } =20 static void clone_aliases(struct amd_iommu *iommu, struct device *dev) @@ -669,10 +697,13 @@ static void amd_iommu_uninit_device(struct device *de= v) static void dump_dte_entry(struct amd_iommu *iommu, u16 devid) { int i; - struct dev_table_entry *dev_table =3D get_dev_table(iommu); + struct dev_table_entry dte; + struct iommu_dev_data *dev_data =3D find_dev_data(iommu, devid); + + get_dte256(iommu, dev_data, &dte); =20 for (i =3D 0; i < 4; ++i) - pr_err("DTE[%d]: %016llx\n", i, dev_table[devid].data[i]); + pr_err("DTE[%d]: %016llx\n", i, dte.data[i]); } =20 static void dump_command(unsigned long phys_addr) --=20 2.34.1 From nobody Fri Nov 29 18:38:24 2024 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2080.outbound.protection.outlook.com [40.107.100.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C499515B119 for ; Mon, 16 Sep 2024 17:18:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.100.80 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726507140; cv=fail; b=ayjnU/B3CseYN6lBtYxjaDepGmv1VAJkGMjOU8xuhVn1LDfUoCAzJXgCrxMT1VeHYecjzzsdC7zDNiB94zo/EkXyOAKnovgsukNcmLRg8NmyeqUOR7tamM3vTIXGKYH5UqFdy/j/fHDhCRNY2saKc6DwAFc9lY9YMCgBg5GX5+U= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726507140; c=relaxed/simple; bh=s35jOulRGrwqzVJcbZy/8Zb1F8G92E8RxmT3RSdoZzo=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ekQ+vm9TsV6SygeeefBYbsLL+SzF6/oSyZUHGUMrEN4DMqH/sB3wB5TOtw+wmIEABzMX1ZFEPLnvmfH2byeA9Hw5ZhFJijeiUONaQJl0D5M8w8L1jCPz1MxU8stiwBFCGuwUPYX0P8uxSUY1RFT/4fxngubBp7W+Bv6sv8lfy+o= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=lPc+ZdzU; arc=fail smtp.client-ip=40.107.100.80 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="lPc+ZdzU" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=qFVVWG/7YpIt3kRZ49LyYKhmhgiF01J7HD5zcS+cYaLQ9uH6/7F+hWqNbN4WZXrqb5qRitaIKvXQZtXC5dOgE7ziPFJBShgnaTT9ecvhF9P54aFW/vdBuVOXexYYD6r8oY/4csB8vwAssCXSwbuFI5nI9IBFa3WTK1AGWl2RUcBm/HcWowp9sx3S6/694K/i0vfmzJtMZwFtT8/VUr+XIhAKWIPlTvOud9mTEJtPj0R8V00s2kHGYXRzNTLe+Osr/dR6AdIoRAZMfwnGQ4aQamMhbvLgOQXJOZlpcakS/EbvE9pScYNsT2A3J9p0VqB69okxlJXrw1YKaAcQ4MVO2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0nCcHvGrUZ7EKHp6Hu+0sJMtSsGWG+1DosCD07UdrEo=; b=Q4+9yWAPvFzE19ZHTUyaijXHCZO6apwSZ63TvZeHmQmX3gBLd0sexwH4qd8TQBhbXfzdBRUWQjzAclX/ETRGO/2jcM0WXLFobIzrtwrFJ5NJjfv9qFPr/f8Iu5Oi6ReTuquqKXw5Hc7F9VnHLErX54YbnMLtQ7qu0iSe7t/O1j9rhUa1g6wN41JKu6DoL48PE0O9EgbQ1fYHsIb21u/TE+vCHxr5YGGMprfKj/19Izx5HLckFU+chw2t8K/BPqNqKedUa2NRZ4HuadWmV8C4CqYMl28UZd0sUsJyAvIdOFu45BTYr5OaK4b2QOkEFm3ZKzPalgtPLsNlBy2IOA56hw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0nCcHvGrUZ7EKHp6Hu+0sJMtSsGWG+1DosCD07UdrEo=; b=lPc+ZdzUJ48YvAvPggpBDsOE+znDwgMW78hyT3xgzuW47xCNxdtAn1y4MPJPw4Eu9YkqcbFUH9GtjQYTz15yQnUfnMtOgA/0yWcBMuE9b/wi9vimK9Zxiuk9FFPsDpl3tOaOvh2ucRFKmgRdkUmj8Ut/yIvkmE+1Dy5h7n8GIMc= Received: from CH5PR05CA0013.namprd05.prod.outlook.com (2603:10b6:610:1f0::26) by SN7PR12MB7811.namprd12.prod.outlook.com (2603:10b6:806:34f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7962.24; Mon, 16 Sep 2024 17:18:53 +0000 Received: from CH2PEPF00000142.namprd02.prod.outlook.com (2603:10b6:610:1f0:cafe::10) by CH5PR05CA0013.outlook.office365.com (2603:10b6:610:1f0::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7982.14 via Frontend Transport; Mon, 16 Sep 2024 17:18:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CH2PEPF00000142.mail.protection.outlook.com (10.167.244.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7918.13 via Frontend Transport; Mon, 16 Sep 2024 17:18:53 +0000 Received: from purico-ed03host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 16 Sep 2024 12:18:47 -0500 From: Suravee Suthikulpanit To: , CC: , , , , , , , , , Suravee Suthikulpanit Subject: [PATCH v4 5/6] iommu/amd: Modify clear_dte_entry() to avoid in-place update Date: Mon, 16 Sep 2024 17:18:04 +0000 Message-ID: <20240916171805.324292-6-suravee.suthikulpanit@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240916171805.324292-1-suravee.suthikulpanit@amd.com> References: <20240916171805.324292-1-suravee.suthikulpanit@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF00000142:EE_|SN7PR12MB7811:EE_ X-MS-Office365-Filtering-Correlation-Id: 427ea22c-d552-47f2-e28e-08dcd673a378 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|82310400026|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?j672u1TvlYLh92WmUnHLhs1Dpqi9E9lHdJ3M5y8pFLaoLZL4uoAAeZ7IzeJf?= =?us-ascii?Q?95LTM+1sPkPkyIQRB8Vmk4dEkjrbaGNx2AgFG8R9BzJMM2Ss14lfSQTTI/di?= =?us-ascii?Q?tPMFs3wRG9lRrWmmCN2RkAOuXJKSBGHrEjzoqB07YuaWRb2a8IZd+GtHa6nq?= =?us-ascii?Q?C/4CED9H4HsSZJZy9dhoM+wKrfGHhM/S72ZswFjbi217nMyn+im1Io4KokFj?= =?us-ascii?Q?IcWmIQZ1nmldUpw4sYjMNd/H3wWq075lLEwgSHecGugwI6leFJiYkU/d0QM0?= =?us-ascii?Q?9yJfGkM5sRcFTjgI6nDh8Hh+dLjRGBoMb3JPByASuwg5NjrZB/K1mimCzA1Z?= =?us-ascii?Q?va7qj/QKZZFiGur2HKr9HG4JCrfVhMM17RgXx82clT3PK3mP7mQIEj62WmN8?= =?us-ascii?Q?aQlYo6SS3NZJMt6EHFGIrmrrCS7w1vtusW5K80ihHlI+PDTtXmBa6Uy3BjLB?= =?us-ascii?Q?A5enrDM/se3tNBZQvFWx47FCCYl1l+iVr31dplmhkqPv6BqX9eSlC65t4C51?= =?us-ascii?Q?BpA4c9cSzvgEEjxDyHZzIeMnU7FiTYhO+KR/Y1A6bpU0FcRD+CK/c4R65O9x?= =?us-ascii?Q?yEbXFNIJGCB2aY9W5ioW4c3qR8T52MmmlX7vSCtunZFXzNAd23wiQBJzk2/3?= =?us-ascii?Q?+wlM3oBtVE7UyvNojahQzsHAT9NqN7XL1nzj82CnHnf61CH+cwLp7MxKmqth?= =?us-ascii?Q?ez+SQLki77wJs2NCmGF9PdEwRbjBozvCyRWCcoMiKKqqVFOGOlM3Unj7KoMP?= =?us-ascii?Q?Ontf3xuGE3n+rb6fKRAt3xylDo3jAMX52IeCqSKrsI26uc2Zxsjs8eskDWUX?= =?us-ascii?Q?gqFyIZ9nvwhv7m1ac8SD79mCtqmgjAhTaZ+S03PjJS9e+qzqEar5CV6bBbCN?= =?us-ascii?Q?V0p4zC/KwwhgzlceHrvIJMJEdKgFoRQURcL+Xm1gFJm23ZZASbOi8dXiXu3W?= =?us-ascii?Q?AAzat0j9mJrJ1DYLRorut4g+VAv6a4uR2G9pStlz4IGdvUFOLm+mhETESPQc?= =?us-ascii?Q?8+84XHulFY9ngKeAsTFbFsOCLWEIvZIwiXn7qgi2aKexXBA8uqMMMX4APNiz?= =?us-ascii?Q?idrO6nipP/Ez10N6lnA3iI3OSyAkQyvA0LJtZ8GIG/LadtynDoAq48I0pvbf?= =?us-ascii?Q?jikJI5k1ekn+9k6ZKFDDTz+2c2GFBNVg4zbX+agss6eaaCrhXIrOvdF4QN6P?= =?us-ascii?Q?ZqwnGsjJ6hXexo5/IavMjf8b8e49YKgUTnKlQdJqmMsEe/Dxq9OgdiuJ5whS?= =?us-ascii?Q?DMp5OeUWOAMgjxva2XNpaURxand1NfktUBGzB1vMtYIZiNKjzyUPFB+Djr5J?= =?us-ascii?Q?IZ/gEYlFUlyLssWWlrADuoY8ge755ZYV89khAZh62P4aEm7Erp4/24ZjEieF?= =?us-ascii?Q?CwXvfNfTfoOyExgQezM90uAoPS7Xc2rKTr8E3iK3nGW799bgfxDOG0tyt7LC?= =?us-ascii?Q?gDJ/CVa7p/ImAIIxkSv4PnEIEB3+iZHk?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(82310400026)(376014)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2024 17:18:53.1021 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 427ea22c-d552-47f2-e28e-08dcd673a378 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF00000142.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7811 Content-Type: text/plain; charset="utf-8" Lock DTE and copy value to a temporary storage before update using cmpxchg128. Also, refactor the function to simplify logic for applying erratum 63. Signed-off-by: Suravee Suthikulpanit --- drivers/iommu/amd/amd_iommu_types.h | 2 ++ drivers/iommu/amd/iommu.c | 28 ++++++++++++++++++++-------- 2 files changed, 22 insertions(+), 8 deletions(-) diff --git a/drivers/iommu/amd/amd_iommu_types.h b/drivers/iommu/amd/amd_io= mmu_types.h index fea7544f8c55..db3ee094a144 100644 --- a/drivers/iommu/amd/amd_iommu_types.h +++ b/drivers/iommu/amd/amd_iommu_types.h @@ -425,6 +425,8 @@ =20 #define DTE_GPT_LEVEL_SHIFT 54 =20 +#define DTE_SYSMGT_MASK GENMASK_ULL(41, 40) + #define GCR3_VALID 0x01ULL =20 #define DTE_INTR_MASK (~GENMASK_ULL(55, 52)) diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index 676742d6f19a..2df679eb61c9 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -2086,19 +2086,31 @@ static void set_dte_entry(struct amd_iommu *iommu, } } =20 -static void clear_dte_entry(struct amd_iommu *iommu, u16 devid) +static void clear_dte_entry(struct amd_iommu *iommu, struct iommu_dev_data= *dev_data) { - struct dev_table_entry *dev_table =3D get_dev_table(iommu); + struct dev_table_entry new; + struct dev_table_entry *dte =3D &get_dev_table(iommu)[dev_data->devid]; + + /* + * Need to preserve DTE[96:106] because certain fields are + * programmed using value in IVRS table from early init phase. + */ + new.data[0] =3D DTE_FLAG_V; =20 - /* remove entry from the device table seen by the hardware */ - dev_table[devid].data[0] =3D DTE_FLAG_V; + /* Apply erratum 63 */ + if (FIELD_GET(DTE_SYSMGT_MASK, dte->data[1]) =3D=3D 0x01) + new.data[0] |=3D BIT_ULL(DEV_ENTRY_IW); =20 if (!amd_iommu_snp_en) - dev_table[devid].data[0] |=3D DTE_FLAG_TV; + new.data[0] |=3D DTE_FLAG_TV; + + /* Need to preserve DTE[96:106] */ + new.data[1] =3D dte->data[1] & DTE_FLAG_MASK; =20 - dev_table[devid].data[1] &=3D DTE_FLAG_MASK; + /* Need to preserve interrupt remapping information in DTE[128:255] */ + new.data128[1] =3D dte->data128[1]; =20 - amd_iommu_apply_erratum_63(iommu, devid); + update_dte256(iommu, dev_data, &new); } =20 /* Update and flush DTE for the given device */ @@ -2109,7 +2121,7 @@ void amd_iommu_dev_update_dte(struct iommu_dev_data *= dev_data, bool set) if (set) set_dte_entry(iommu, dev_data); else - clear_dte_entry(iommu, dev_data->devid); + clear_dte_entry(iommu, dev_data); =20 clone_aliases(iommu, dev_data->dev); device_flush_dte(dev_data); --=20 2.34.1 From nobody Fri Nov 29 18:38:24 2024 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2049.outbound.protection.outlook.com [40.107.223.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A8C271591F0 for ; Mon, 16 Sep 2024 17:19:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.49 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726507174; cv=fail; b=NawWwmoZnlwFWBO5k0vmTt2I9ZPFch1GDKBhXBuZWOKEL9IcVIGPCCvCe9CvWSUdMVWgqDJyUMLf1njSwTeUczlmykR/NyiTmfMe9STinkgNSQfaTTICGmbX+sOUjt9LKBSpt9L5p/NoYUlcxro9rWYuX5L3Ise7dl3orT7a1NA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726507174; c=relaxed/simple; bh=c8s5quovYTwGOd9TcYkHidZZ9gIp6+84+frXQ6a9R/g=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=MtUzahzoc/XXOIKmCQHPL0mMxLqDrq0CPbflVblZ8mcHpAiOzu//7SssvffItQaK9DArKqs07XCgIrpA5WtJ5C8IaA4VKHcecUriWPtf20KrY4SYEBAaTacB12uJdFe6w8uWj7IurBmX0DLWMUHwHj55r7Tgqce3xSP5IMW8WYo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=yh4qMJKy; arc=fail smtp.client-ip=40.107.223.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="yh4qMJKy" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=D0P8SeB33yZiJuuD2AzY/SQazX8c1b3+oCR8LSSt1auXAhs0OVKpzeT0mMGorWZw6Y7T5jvubObax6A9et9HlndJ4Y7CoTQSyUc011tTcSQBFJNUtB3Y7Y7r6bAddfWnvq7a+RaI3KhwOL/vG3xYMJzHDNlNsERpAYK1aykaVWCeDCkO6+ANCXl+BDejo81JvpzeRkgktUnkskAKRHNfivxyDyHQFkLw/aKIbLTScHFR6wRKgor36afNi6BBeyM3C0fCcAelyVHWovl64oPfbDqic7dIpO6i9Yt6Kl1p8VqaDi5tmhL96J+3mmkPdAwdWBxaF2KUQcmuYu02+H9VEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pHLTDxlXXS8CyTxVxgeZKLdPirx8ZMJSKoD1ADJpnao=; b=BRFHqYncBzJolLru+lTvHWo3TN4ysvAO12ZeVZGDHleIQeTHg9UQVCXeGjdXGfkKcRPBLFRDkqCQP5aGU41xZJbOUNeiUXSTJfq7vRJ8/I5Pi7rQYWh+4y00oNN45mxz9ddU5G5C0zFz53cCjByQVokjH/EIqmMDM35Dn2PEQMBqoSlmRCsmcEFpPepnIWXVMw84Kq+P6gkfIdfQnZvrx/pMVy5iYTf7gAoJFKvUGLQmL2TYbs3zJVz1zyLkh7dTJzk31+pYdsRlPveHyPf+E13fNkVt4LAhR0DxYmMnv+oA5+Hc2h6XQSugyqT7i+YLTWpZjGF6sXFAhspuQ6vjYw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pHLTDxlXXS8CyTxVxgeZKLdPirx8ZMJSKoD1ADJpnao=; b=yh4qMJKy+jBp7E9NJiO0A8JX6QbWzHM7Q5EqZTNA57JeqEKfSGqIb/kXRW4nxr+TWYcXaBuhBk+dyUv9PyL26NPSPUHJN80iLx9RZZXhBZBJ9Pf7GfM82x4rbbKiqxgWKyBDQSRsTCbzC9/NXIJVxJ8E1WZucEGueEthVGCjdwQ= Received: from CH0PR03CA0096.namprd03.prod.outlook.com (2603:10b6:610:cd::11) by SJ2PR12MB8979.namprd12.prod.outlook.com (2603:10b6:a03:548::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7962.23; Mon, 16 Sep 2024 17:19:29 +0000 Received: from CH2PEPF0000013D.namprd02.prod.outlook.com (2603:10b6:610:cd:cafe::7a) by CH0PR03CA0096.outlook.office365.com (2603:10b6:610:cd::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7939.26 via Frontend Transport; Mon, 16 Sep 2024 17:19:29 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CH2PEPF0000013D.mail.protection.outlook.com (10.167.244.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7918.13 via Frontend Transport; Mon, 16 Sep 2024 17:19:29 +0000 Received: from purico-ed03host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 16 Sep 2024 12:18:51 -0500 From: Suravee Suthikulpanit To: , CC: , , , , , , , , , Suravee Suthikulpanit Subject: [PATCH v4 6/6] iommu/amd: Lock DTE before updating the entry with WRITE_ONCE() Date: Mon, 16 Sep 2024 17:18:05 +0000 Message-ID: <20240916171805.324292-7-suravee.suthikulpanit@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240916171805.324292-1-suravee.suthikulpanit@amd.com> References: <20240916171805.324292-1-suravee.suthikulpanit@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF0000013D:EE_|SJ2PR12MB8979:EE_ X-MS-Office365-Filtering-Correlation-Id: aaebb6fb-5e70-4baf-dc00-08dcd673b945 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|376014|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?YMc+ZTV46/pqg7LgQEKmGoeVI+WDAG47dmUqQU08BrSSCaILOpP8DSmhZeus?= =?us-ascii?Q?sVHAyptt4Wuv8QmiCIEZ5BkX/hzT4cPAAWEaaCh2Ii6c4cGzA+069sZ435/K?= =?us-ascii?Q?yG0uOj4mzwEXTyfK1RbaMW89paNfbikDDALNuvSetrgGCdTShzvvZKJJhbNH?= =?us-ascii?Q?z8hw3L2DBY49NvxEQJPvEp65/x2HMa5Vl2Sqsp5Uqv2SLSlaqLqQFbpwPXzW?= =?us-ascii?Q?o29qyyidtsQq6Ph+padFmreeNTW1hT1P/Za5SDPGxh8vKPnvuGIEXunXniyl?= =?us-ascii?Q?0osQRtYuoMOizvJjMA8LkZQahjd4ymOVytH8Wc1OWkDRNmNegaAqTF6hcs0E?= =?us-ascii?Q?oeF6hYwXHRIBq1i1h58sZcpHWW/PAaJE7faAb/qc8+Tqw/mYT1oUrcYuhoZq?= =?us-ascii?Q?tPwsLkJL1sN2v1R0GCQM0B0WGO4YrX2RB6AuKiJsZv9xGSGomokJ3msmz198?= =?us-ascii?Q?vuvBmaMuPAlP8546nT+AM0BxVVmXDAhsfPP3wWNGxhEPILLafXptdBzzabkB?= =?us-ascii?Q?qMwjNZtdl2PcQy7rTd42WyqtkUg1w6t56+AcXTU8eoy9Wt5611JcoX/buaWZ?= =?us-ascii?Q?YriFw01d4kMAgZz7Om84sS+CLDUJ1F/wyKZ0rv7roskGJXr7hYFJyoJ3rOFN?= =?us-ascii?Q?Kf4WFVIn+OhywO+kQtpb9fMiwbU9eV+CtBPVOqk4Za361MKlyKNInSb//ist?= =?us-ascii?Q?cOPpEHAfgdqW030yvE4xqFCyyMWe/am0V86lfhR5HKeubk21byeoWZYTImFd?= =?us-ascii?Q?jvNN3+QSw+qJ9O9fUBsFMZlKlaxYws04l5KrZyBv+iO8bsn1n1TJky047KHh?= =?us-ascii?Q?lMThOOzvoNwqVU4DF0m9YxQA3aIPI9fLTTdH3kSIVJ+JOOYqlCxKZ0E5MvjP?= =?us-ascii?Q?a4kkMaJaQ21DAiwb3QMJO7l8I9iC53suxF2bWKIcbibRk8T35WL9og9+zCAO?= =?us-ascii?Q?rsxU5nbAI929kRNnY31Fq05kx6afd6E2NgWV6fTj0Acr6363ozwEKsH4Uzy3?= =?us-ascii?Q?+gnFIRYfCRi67Dpe09KOkaxvarxBWF5RkKcTAv/qATKwVDXKq953mglNChaH?= =?us-ascii?Q?rlmV0yyr1hOOZb0qj0nOkNyUf49tWgZjpoerUy4OEN2mcH0nBfjrD+AIkl+J?= =?us-ascii?Q?NBJ5jlHg2HIXReOVmAtnGBLTUbMA96K3+Z8hwOM+1//BWclMorD2OlyNluHt?= =?us-ascii?Q?Hbwu6qTkpPooA5KdX5EVptRiZFoOxKwjbKibZsvWQeqZAAiFuBoP7tgmL1WF?= =?us-ascii?Q?hqUR0cLXPjYTwe7cyafmYeoTI4lfpyafvrnKFRjjE6REsFmuo9n2WfdW9mpg?= =?us-ascii?Q?oU/CxWAfFy5qtCzFSTUIvIYE+SoRGerM0BaR8OtRKXQNgQV6wPCV1NCKKaX6?= =?us-ascii?Q?sKU/iCqFUB3L+lpcG5hgQ1tITJBcCtgrFFUvkEuNYzcuX6+s7WH2YoBUpVHJ?= =?us-ascii?Q?pLJCp7ikJ9mu4qCLcG0Y2DIZ0K+E+Ep8?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(376014)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2024 17:19:29.6932 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: aaebb6fb-5e70-4baf-dc00-08dcd673b945 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF0000013D.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB8979 Content-Type: text/plain; charset="utf-8" When updating only within a 64-bit tuple of a DTE, just lock the DTE and use WRITE_ONCE() because it is writing to memory read back by HW. Suggested-by: Jason Gunthorpe Signed-off-by: Suravee Suthikulpanit Reviewed-by: Jason Gunthorpe --- drivers/iommu/amd/iommu.c | 42 ++++++++++++++++++++++----------------- 1 file changed, 24 insertions(+), 18 deletions(-) diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index 2df679eb61c9..d69b0d41e93f 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -2813,12 +2813,12 @@ static int amd_iommu_set_dirty_tracking(struct iomm= u_domain *domain, bool enable) { struct protection_domain *pdomain =3D to_pdomain(domain); - struct dev_table_entry *dev_table; + struct dev_table_entry *dte; struct iommu_dev_data *dev_data; bool domain_flush =3D false; struct amd_iommu *iommu; unsigned long flags; - u64 pte_root; + u64 new; =20 spin_lock_irqsave(&pdomain->lock, flags); if (!(pdomain->dirty_tracking ^ enable)) { @@ -2827,16 +2827,15 @@ static int amd_iommu_set_dirty_tracking(struct iomm= u_domain *domain, } =20 list_for_each_entry(dev_data, &pdomain->dev_list, list) { + spin_lock(&dev_data->dte_lock); iommu =3D get_amd_iommu_from_dev_data(dev_data); - - dev_table =3D get_dev_table(iommu); - pte_root =3D dev_table[dev_data->devid].data[0]; - - pte_root =3D (enable ? pte_root | DTE_FLAG_HAD : - pte_root & ~DTE_FLAG_HAD); + dte =3D &get_dev_table(iommu)[dev_data->devid]; + new =3D dte->data[0]; + new =3D (enable ? new | DTE_FLAG_HAD : new & ~DTE_FLAG_HAD); + WRITE_ONCE(dte->data[0], new); + spin_unlock(&dev_data->dte_lock); =20 /* Flush device DTE */ - dev_table[dev_data->devid].data[0] =3D pte_root; device_flush_dte(dev_data); domain_flush =3D true; } @@ -3102,17 +3101,24 @@ static void iommu_flush_irt_and_complete(struct amd= _iommu *iommu, u16 devid) static void set_dte_irq_entry(struct amd_iommu *iommu, u16 devid, struct irq_remap_table *table) { - u64 dte; - struct dev_table_entry *dev_table =3D get_dev_table(iommu); + u64 new; + struct dev_table_entry *dte =3D &get_dev_table(iommu)[devid]; + struct iommu_dev_data *dev_data =3D search_dev_data(iommu, devid); + + if (dev_data) + spin_lock(&dev_data->dte_lock); + + new =3D dte->data[2]; + new &=3D ~DTE_IRQ_PHYS_ADDR_MASK; + new |=3D iommu_virt_to_phys(table->table); + new |=3D DTE_IRQ_REMAP_INTCTL; + new |=3D DTE_INTTABLEN; + new |=3D DTE_IRQ_REMAP_ENABLE; =20 - dte =3D dev_table[devid].data[2]; - dte &=3D ~DTE_IRQ_PHYS_ADDR_MASK; - dte |=3D iommu_virt_to_phys(table->table); - dte |=3D DTE_IRQ_REMAP_INTCTL; - dte |=3D DTE_INTTABLEN; - dte |=3D DTE_IRQ_REMAP_ENABLE; + WRITE_ONCE(dte->data[2], new); =20 - dev_table[devid].data[2] =3D dte; + if (dev_data) + spin_unlock(&dev_data->dte_lock); } =20 static struct irq_remap_table *get_irq_table(struct amd_iommu *iommu, u16 = devid) --=20 2.34.1