From nobody Thu Nov 28 00:48:39 2024 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2057.outbound.protection.outlook.com [40.107.96.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6322A54F95 for ; Mon, 7 Oct 2024 04:14:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.96.57 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728274485; cv=fail; b=s15CiHF0jhovUMmdY4xUX6+IK4C+r2Izfy+Wylog8taod/1PquVlXt4XUUlNF0fTc4qeHgUCUdZXXTnz0olD+ThY8qTTQ9JrMF0Ws5zy1ulLkXEyhV2ArjtvSVbOV8ISXRu/2irG7Dw8Wb3hyskpjVl3+CZzUkV+ud1zgIC6xkc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728274485; c=relaxed/simple; bh=pGrLOTxeFfFR2fjty59gpiijJCuJzMPtba93B78LXFI=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=gUMHZK5U4Q4L1Ya4MoQEkIbFxMD4jtOBoKeXebrnWEvVtxjUJX6O1OvKQzov+xCSm2UWGJrT5cZ97WaGzZa5i8gaaXPgh4jYFk/fJoKUwEcOY4nMDEZH9XXZrmn7ueXdE2Enk0f2w6OTDiSkHEvo/4/6ZPQXgeEqZg6Wdg4TMmw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=z5AYgt3H; arc=fail smtp.client-ip=40.107.96.57 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="z5AYgt3H" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=XzeYWcWZyh2bhXBULlOCmF887J7AMZIYibiA/PtqZIqeWbsObVc4lVlyLftJsNUsF4Ynf1IsA9ucEly6vMlKEjkgjNmDYLD+jWx9PiR/I0xR1tsy/wNOcNcpo0PIwoKEtxMlG8NpHYbmaagpaeVY46Q/pmHDyvyMRWFdGUoaerPmD4OrgAXhz+CgUrOoO0m/Epzkyouwvv9Nmf+qDisRHcJvBAsFQ+GU950SmRXtRCAhUE/a6eIYVPQU+nytCLiiwWuaOqpaKIJZIMv9OAkVeAeRf5SK/8/OhLyqYhZKYiLi0siJe0RVNyNM4aqikN2lj0c41qwxLkzIaX75XuFOJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ZpbWI1xzF6L38oGp9dj+MXXb5pGLq+T1sJiZdbCd1Cc=; b=lbSlQk8jL5uIYcHa2jvc1wy0y5sHXQ0p8h+6cuSsWTN5LPCmUSAGEJSNAch1/OI5zYvoq2t6biKiZK2RVICAo7M+XYINsEJ6/acxAujvXEuAzosxfH2dgNY40CTwjHYsftbkKpVVx/Gxbm6Khy1QfuTkwLf+EzAd+KxSlwMQE4svCW2TXRJr3rbFshOUai/n/IKTODYiJupNr2qLtUmXPDvwWzy6CaoJRnoaFAFti1s/RhjFUSe4QVPXMpSMz4lDCctO+9HN9xxspl0GQ0xN1n5jIvCkfNMEIPaz6TGm8XvRtN72y5iL+ENRpD2g1itKwA0hHwcVyIBiB0zrcaIU0A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZpbWI1xzF6L38oGp9dj+MXXb5pGLq+T1sJiZdbCd1Cc=; b=z5AYgt3HBD3aWXYN7m0Z9J452mQzm3fGbLmLFHaWMYn4oQO51nVHfpQrwx4YVguvVEaFZObznWBxL6joHdrSUEeisvDCCK/+k+EIxw+MJDwrLDhFSSwo0cO/mSTB3dh69xFETDg+cYwOPBYrGpOM3kszCY1PFsx0TrWqHtYBHzQ= Received: from SJ0PR13CA0081.namprd13.prod.outlook.com (2603:10b6:a03:2c4::26) by PH7PR12MB7967.namprd12.prod.outlook.com (2603:10b6:510:273::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8026.22; Mon, 7 Oct 2024 04:14:41 +0000 Received: from SJ1PEPF00002315.namprd03.prod.outlook.com (2603:10b6:a03:2c4:cafe::a0) by SJ0PR13CA0081.outlook.office365.com (2603:10b6:a03:2c4::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8048.13 via Frontend Transport; Mon, 7 Oct 2024 04:14:41 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by SJ1PEPF00002315.mail.protection.outlook.com (10.167.242.169) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8048.13 via Frontend Transport; Mon, 7 Oct 2024 04:14:40 +0000 Received: from purico-ed03host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Sun, 6 Oct 2024 23:14:37 -0500 From: Suravee Suthikulpanit To: , CC: , , , , , , , , , Suravee Suthikulpanit Subject: [PATCH v5 1/6] iommu/amd: Disable AMD IOMMU if CMPXCHG16B feature is not supported Date: Mon, 7 Oct 2024 04:13:48 +0000 Message-ID: <20241007041353.4756-2-suravee.suthikulpanit@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241007041353.4756-1-suravee.suthikulpanit@amd.com> References: <20241007041353.4756-1-suravee.suthikulpanit@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF00002315:EE_|PH7PR12MB7967:EE_ X-MS-Office365-Filtering-Correlation-Id: 4c9968ee-db59-42ce-234e-08dce68690e2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|376014|36860700013; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?7ngriZtJ5oRVNOcYMNSOAwNrUR4pA+iVCUybdLclZPXngaB29q2UKQ0CHSNH?= =?us-ascii?Q?TiYOytANOWA21LxCfIIBTnznmAnrJCaMzj7x82vR4KRX7dqht45JhH7sn24G?= =?us-ascii?Q?diZpAjfBYmUMeGxhZhCYHXjlIsUMnEvUsOj5XD5CAptDbUnehsWVcq6TlN8H?= =?us-ascii?Q?3bBU627lFFU2YHj6tOO9btsPzUp6Ie4gV8pMGkkYFt1Bq5Sa/ux57+CpUR34?= =?us-ascii?Q?AXxvt6+EcRsgoQPTgAWO9ss41oJ+cMt6x+vFuSJDNSO6kwaGPCxOzzrHFBf6?= =?us-ascii?Q?G8iANJCevIl26f4vvVkxPxzPhc+j867AbVX1cWbCSrb9m6oDyjjMgpJVZAmn?= =?us-ascii?Q?Nt6c2Rzfcs4CEMmkNeuSFeX8sn+TtWOhW8ZLmasnJcWVGDQAC5rFwrVIRne/?= =?us-ascii?Q?PY2Y9Opcv1jVG4d7KQHbQI5pnID67Ir/C2ukD+9N7Da+bHQ29Lvtp0ZT5U+O?= =?us-ascii?Q?+Zvi/u8djkzsQ1dILqvs8ENzd2OjpgH4Yn3BXBvoZxXRGk1uYxCnO8YxzKrG?= =?us-ascii?Q?q40NprmwWaOg311s96SD9Et7b+fdaQCqqQ6ld5SgDZitKTpKEDx9kec+ZC1a?= =?us-ascii?Q?iLSoeYE9lPBA4D96aheofCXDNGbWDXDkTBchDeNGNZS9WasSXWnV8vZNeaSb?= =?us-ascii?Q?PNRJmKeERDC/nUfwRTniKckyP5Vd3xHfi/K9L5N5fyyAdnh4pPGQE3h3fveC?= =?us-ascii?Q?7A4HX8jJds0j1QQQAWVcny3Zc2HFVCp6LyBYOzWGGpHzEA7Lo40g25OQDAgw?= =?us-ascii?Q?96s1T11jFdZD3L2tdG6aZ4aInHG9WDcfgWOIY5EU7i42QAWpbYk5gAL69Eqd?= =?us-ascii?Q?hL5uW9N3L0mRv/jCYwashdwLpE/894vTQoH+BwSrJUASfD4Q7vei2K7RL+07?= =?us-ascii?Q?EkgBiKemqkvMSq36dDV7Y8ZguImSzSMAoYFDYy9N8uAgGT+r54vB6Xcd0l3f?= =?us-ascii?Q?Kff2dMwtHNWn5VYMay/Joni2kBJqQkgyUbPER7NfYf1PVB37ZxbFC7a6RwKj?= =?us-ascii?Q?hLJrvIm/deXZRA2C+UYw9YJk2EEdyq9nqOg/uOp4uD5NAmagczvF1pfiGdLj?= =?us-ascii?Q?liOr6aU3rUjpAyBVhE76lUoWOJWIF8Pn43+VkhVpmDTghJo75hs0ydne7mVd?= =?us-ascii?Q?H21Hr7u3v0D6ZKwDkl8UqUVkMw6IJoDB8P35etP1UUwFKoN1Yovzihjz/tkf?= =?us-ascii?Q?Hwi4/MqLPPev0w71utqJ1RHsfz5QbpC5cbu8qOQAUmhNkk+0ulz06S5bwGKl?= =?us-ascii?Q?hljOvaMlzhiM2fQWQgBf8s24b48LGNVVbds+X0JfTSf+GyofnSzSx+lWYZJM?= =?us-ascii?Q?p0jxBiuoqFsD5Lwz5IxHXInA+NbAGF/fjHfdeoUySEnxTB6aCcCtQfuz00HN?= =?us-ascii?Q?MQy0kGeVNlW/AjAHXJqjMFt6CfxL?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(376014)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Oct 2024 04:14:40.9106 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4c9968ee-db59-42ce-234e-08dce68690e2 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF00002315.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7967 Content-Type: text/plain; charset="utf-8" According to the AMD IOMMU spec, IOMMU hardware reads the entire DTE in a single 256-bit transaction. It is recommended to update DTE using 128-bit operation followed by an INVALIDATE_DEVTAB_ENTYRY command when the IV=3D1b or V=3D1b before the change. According to the AMD BIOS and Kernel Developer's Guide (BDKG) dated back to family 10h Processor [1], which is the first introduction of AMD IOMMU, AMD processor always has CPUID Fn0000_0001_ECX[CMPXCHG16B]=3D1. Therefore, it is safe to assume cmpxchg128 is available with all AMD processor w/ IOMMU. In addition, the CMPXCHG16B feature has already been checked separately before enabling the GA, XT, and GAM modes. Consolidate the detection logic, and fail the IOMMU initialization if the feature is not supported. [1] https://www.amd.com/content/dam/amd/en/documents/archived-tech-docs/pro= grammer-references/31116.pdf Reviewed-by: Jason Gunthorpe Suggested-by: Jason Gunthorpe Signed-off-by: Suravee Suthikulpanit --- drivers/iommu/amd/init.c | 23 +++++++++-------------- 1 file changed, 9 insertions(+), 14 deletions(-) diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c index 43131c3a2172..a1a0bd398c14 100644 --- a/drivers/iommu/amd/init.c +++ b/drivers/iommu/amd/init.c @@ -1764,13 +1764,8 @@ static int __init init_iommu_one(struct amd_iommu *i= ommu, struct ivhd_header *h, else iommu->mmio_phys_end =3D MMIO_CNTR_CONF_OFFSET; =20 - /* - * Note: GA (128-bit IRTE) mode requires cmpxchg16b supports. - * GAM also requires GA mode. Therefore, we need to - * check cmpxchg16b support before enabling it. - */ - if (!boot_cpu_has(X86_FEATURE_CX16) || - ((h->efr_attr & (0x1 << IOMMU_FEAT_GASUP_SHIFT)) =3D=3D 0)) + /* GAM requires GA mode. */ + if ((h->efr_attr & (0x1 << IOMMU_FEAT_GASUP_SHIFT)) =3D=3D 0) amd_iommu_guest_ir =3D AMD_IOMMU_GUEST_IR_LEGACY; break; case 0x11: @@ -1780,13 +1775,8 @@ static int __init init_iommu_one(struct amd_iommu *i= ommu, struct ivhd_header *h, else iommu->mmio_phys_end =3D MMIO_CNTR_CONF_OFFSET; =20 - /* - * Note: GA (128-bit IRTE) mode requires cmpxchg16b supports. - * XT, GAM also requires GA mode. Therefore, we need to - * check cmpxchg16b support before enabling them. - */ - if (!boot_cpu_has(X86_FEATURE_CX16) || - ((h->efr_reg & (0x1 << IOMMU_EFR_GASUP_SHIFT)) =3D=3D 0)) { + /* XT and GAM require GA mode. */ + if ((h->efr_reg & (0x1 << IOMMU_EFR_GASUP_SHIFT)) =3D=3D 0) { amd_iommu_guest_ir =3D AMD_IOMMU_GUEST_IR_LEGACY; break; } @@ -3051,6 +3041,11 @@ static int __init early_amd_iommu_init(void) return -EINVAL; } =20 + if (!boot_cpu_has(X86_FEATURE_CX16)) { + pr_err("Failed to initialize. The CMPXCHG16B feature is required.\n"); + return -EINVAL; + } + /* * Validate checksum here so we don't need to do it when * we actually parse the table --=20 2.34.1 From nobody Thu Nov 28 00:48:39 2024 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2071.outbound.protection.outlook.com [40.107.243.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0FF31136E09 for ; Mon, 7 Oct 2024 04:14:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.243.71 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728274491; cv=fail; b=nn7BkwmZDXfSSo0fh4V0tGaX5cR7edCBkYxVAOq464FUFITejiKkE/bygdLLx9+0MGxIpvbXzCzgAvHsiWEv6BJjgxK1iq6OAg45d76JgAjrmN1LsM8UoXENQkLy7AXjROHef6DESPT3ri3GWR3rIRIQiPsZFOUb3J8zX3EWa74= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728274491; c=relaxed/simple; bh=BkaK2l6BPIj9YMoWUZuCtgo0Pb/xGgExvoOx0dPPxnw=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=omZi211R+WS728I1jIEtoXtnIFC4xYGJY2qUJGZVoY/aJA5yBLPbGiR3W098d8o+tQMcyYgauauQZ3y1FcUKglLVBXTi6suFeS8I03Mcxa9UDYH6SfAA6rSCvABRSsuVzquoesFpicQt0Q/ZCEJHyKQZ5V17iR1qVO3s/9UFCYo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=WnV4ik6X; arc=fail smtp.client-ip=40.107.243.71 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="WnV4ik6X" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=vpQ2pRwfjkeNJnI18cmtYPZRwH0FMrL3FHK6Bf3/houRx+E0miaAF+KDtU7wd0SDAM9OZVglFN6WihwHTp2WXJc4oDCNg2jigJ85u0X5IQGmp60ywb/TIZNTWj5c2N//HSRNvZhr0saLgzwm9clu+BCHCG9OVdKYAq8JiSzdb4LfvbtEIkL92S1/TpdzoH0skq/T4QFaMBf/Jo3Atf+f8mcMTAYl5Z7m4Iyvd0hOKPTb0xwNTjzmwfm6r00Xk3+H1OvP5lwtDrx91KavPXZAB5DZ75b/WgfSL2pg9HjfDnjt1vqsyYuxmSr1cP3RY80UFFWsZDgmIIVoDIlqJZs31Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vP9GYGlo2SZhQ7Sccnv/33eRsH8yXo0gRaTl5X79j28=; b=FxWY3Nj1ZvL8WQ0YiQCJP2dZhzm0KT6I3a21cRnHAMhY8JblEWslCtYH0hhCumFNWJ92BcAHnK9imGSAR0jJITgvyi/AlXk6+1zKmLA/zYIZi6VTdoL2Tm6UKkzn01WmzAJKdHZUmS9r2oqWvqmzvbVPN4cKl5UuwLy2L226GJOGbdY+9lPTa2WAywzPUBvqeZ5/kR1lNmGq5YZp4PahB0bRLywaL79e7Qn8F29BUKnr2PuTCdq2fyH4OQIVCV6Wkgaw6TYZqGHvbY51nnDYgjwJZCiX8EVrDuwjBXA3bVlY7RW41XvRHfoMzkFrpZ2gtV7m4YJfMFRXFZWOrqZX+A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vP9GYGlo2SZhQ7Sccnv/33eRsH8yXo0gRaTl5X79j28=; b=WnV4ik6XTx5/UZJcl4ZQWtE6ZaxX4L8HmlxvFYvu/A3bvWg7qm0S1rGP+qDSH7CMatzbnzJaYYgucsxynj9A3pnhAgG4W0lLOLipgGjieOB5dUGQUZr9vmsPA9hShWxLbc/U5eJ7teG3/iLc6zZ298G1mpl3mEjFcyHCSHCreG0= Received: from MW4PR03CA0257.namprd03.prod.outlook.com (2603:10b6:303:b4::22) by MN0PR12MB6002.namprd12.prod.outlook.com (2603:10b6:208:37e::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8026.20; Mon, 7 Oct 2024 04:14:45 +0000 Received: from SJ1PEPF00002311.namprd03.prod.outlook.com (2603:10b6:303:b4:cafe::9b) by MW4PR03CA0257.outlook.office365.com (2603:10b6:303:b4::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8026.23 via Frontend Transport; Mon, 7 Oct 2024 04:14:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by SJ1PEPF00002311.mail.protection.outlook.com (10.167.242.165) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8048.13 via Frontend Transport; Mon, 7 Oct 2024 04:14:44 +0000 Received: from purico-ed03host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Sun, 6 Oct 2024 23:14:40 -0500 From: Suravee Suthikulpanit To: , CC: , , , , , , , , , Suravee Suthikulpanit Subject: [PATCH v5 2/6] iommu/amd: Introduce helper function to update 256-bit DTE Date: Mon, 7 Oct 2024 04:13:49 +0000 Message-ID: <20241007041353.4756-3-suravee.suthikulpanit@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241007041353.4756-1-suravee.suthikulpanit@amd.com> References: <20241007041353.4756-1-suravee.suthikulpanit@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF00002311:EE_|MN0PR12MB6002:EE_ X-MS-Office365-Filtering-Correlation-Id: 07ddd047-bd1f-42b1-32fa-08dce686934a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|82310400026|36860700013; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?q+APdi9x6rY7M6+wUzD4NOw3q5A7bnKxBtlgb8RfLGa0+DoBf6qZbZiq8EJJ?= =?us-ascii?Q?NsniJRyQuzQAt9V5Y+hC8P0M6T68g1vOjUvVVPWnXj9vrR9jDeswat9xk14k?= =?us-ascii?Q?mQhS7JxDTtc8BTFvKEgHwq+QxIyX8bjaJe4lX5RvNg6zwH7reWwLvauOLDzc?= =?us-ascii?Q?yCE274Q2+khQhdtGb5bEomdCJbVvkJ3khNrIBkWo/4AY11Ugru8zxHBfC+hP?= =?us-ascii?Q?VZ8N15tCyacPRb2yoZ0Ql6K38uD+uigKs9wD5ODdHsGdhLvJfMEgtGYg7OQq?= =?us-ascii?Q?xthJZKsHRpy7jk9ih181aSWgJAu3Moln8zs+YhyVMyBCTPGWaMpQs/wiT5fC?= =?us-ascii?Q?GrxwoTRtC7YAU4+0hQknyVAErSPf5IHCxwHzr4MwT54KhM7BUR0JqzlZDpJM?= =?us-ascii?Q?rP0JekijEa1p2buFsSe8BTaA9Lyzk/tcesiI5Sc58ewnmbpmhHiURnzi/vN8?= =?us-ascii?Q?vEkLKY2dbEQhy0TzKGATNB6sCQmlpXAa/FBOSAEwTnJ6SMA2XTXHt3MC5M0c?= =?us-ascii?Q?NptR49iQQenvLB2bE3L74mSGaJdR3ijPXm0zipeJOOal6nniKwVszQGM7mdc?= =?us-ascii?Q?d9f4bvTI/9YRVHzk5jeOEb7sw/fYbjoPKCJ+4VfkM2cV16RSG7Cb+36FDTjc?= =?us-ascii?Q?d2zZfFH/w5u1RW0JJf6Ts5ZStmP6Awgjol7CoQwbkqaMevVmfVZc6w7VGcew?= =?us-ascii?Q?rK8xMGbl6NxgdNlXrewZLUjRGPYx5D5LBau9QDKFl3e4DHgQKH59QfrheCap?= =?us-ascii?Q?x67nRL/tNzuxZkjAQqxUmEmPPYCFzo7AJw0ajK8stFlRvGKDqJLFpmDsIc8l?= =?us-ascii?Q?oL4YERy5N1Wdldu8biE3ZiNwK9ThNyAWjITejh6SlD/uB/TSvsVtwXCntLuu?= =?us-ascii?Q?AqBShcaifdKv0OJbWt1I8XMtMRUvSL42WdG4KA5+ZyZLM88vEFUyMMLBTRD0?= =?us-ascii?Q?Y9bFMIgktfgVCvqZY518YNzrwgtjRpL+xp9HCZ7zGHztj8JdBS/aFVAvQeG1?= =?us-ascii?Q?PEd5w9qU7uOI0aWCRKMNQzQEbWIujLktTS++7b6YmvqaVo91N+PD/It+dcF6?= =?us-ascii?Q?WgI6qKeP+lw4y85Wcverj2XuSM2UdRUUS9lRlnYGtxaLZIEmb54yxxUmNcdI?= =?us-ascii?Q?7tZMmqAnTk0Wi1Gt8uOfz72SxJ4FW9Crhg12YUbCb8aaaRCuHn66UtCndVvV?= =?us-ascii?Q?EWlZXYHNEDBSRbAngWuk+IQ28urTNFLEtuuP2S1dO8f4reJZjnRZ9Ge8RF4s?= =?us-ascii?Q?mDOf6pTpm/mKL8hsrLXSVw3X6z1lBaS7KjOSQAzQy6HdRSmpW8DaoOmPQ6Mj?= =?us-ascii?Q?xRVZszBOsivISVfmvt2sbDbchJP4Y5L0lnhlvirUgPsJb73icD7i9rYwgJi8?= =?us-ascii?Q?j7oXfSaKyrjFbN7/tt7PDOzsB2Hn?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(376014)(82310400026)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Oct 2024 04:14:44.9481 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 07ddd047-bd1f-42b1-32fa-08dce686934a X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF00002311.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6002 Content-Type: text/plain; charset="utf-8" The current implementation does not follow 128-bit write requirement to update DTE as specified in the AMD I/O Virtualization Techonology (IOMMU) Specification. Therefore, modify the struct dev_table_entry to contain union of u128 data array, and introduce a helper functions update_dte256() to update DTE using two 128-bit cmpxchg operations to update 256-bit DTE with the modified structure, and take into account the DTE[V, GV] bits when programming the DTE to ensure proper order of DTE programming and flushing. In addition, introduce a per-DTE spin_lock struct dev_data.dte_lock to provide synchronization when updating the DTE to prevent cmpxchg128 failure. Suggested-by: Jason Gunthorpe Reviewed-by: Jason Gunthorpe Signed-off-by: Suravee Suthikulpanit --- drivers/iommu/amd/amd_iommu_types.h | 13 +++- drivers/iommu/amd/iommu.c | 114 ++++++++++++++++++++++++++++ 2 files changed, 126 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/amd/amd_iommu_types.h b/drivers/iommu/amd/amd_io= mmu_types.h index 601fb4ee6900..91f802be7898 100644 --- a/drivers/iommu/amd/amd_iommu_types.h +++ b/drivers/iommu/amd/amd_iommu_types.h @@ -425,9 +425,16 @@ #define DTE_GCR3_SHIFT_C 43 =20 #define DTE_GPT_LEVEL_SHIFT 54 +#define DTE_GPT_LEVEL_MASK GENMASK_ULL(55, 54) =20 #define GCR3_VALID 0x01ULL =20 +/* DTE[128:179] | DTE[184:191] */ +#define DTE_DATA2_INTR_MASK ~GENMASK_ULL(55, 52) + +/* DTE[180:181] */ +#define DTE_DATA2_RESV_MASK GENMASK_ULL(53, 52) + #define IOMMU_PAGE_MASK (((1ULL << 52) - 1) & ~0xfffULL) #define IOMMU_PTE_PRESENT(pte) ((pte) & IOMMU_PTE_PR) #define IOMMU_PTE_DIRTY(pte) ((pte) & IOMMU_PTE_HD) @@ -832,6 +839,7 @@ struct devid_map { struct iommu_dev_data { /*Protect against attach/detach races */ spinlock_t lock; + spinlock_t dte_lock; /* DTE lock for 256-bit access */ =20 struct list_head list; /* For domain->dev_list */ struct llist_node dev_data_list; /* For global dev_data_list */ @@ -882,7 +890,10 @@ extern struct amd_iommu *amd_iommus[MAX_IOMMUS]; * Structure defining one entry in the device table */ struct dev_table_entry { - u64 data[4]; + union { + u64 data[4]; + u128 data128[2]; + }; }; =20 /* diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index 8364cd6fa47d..deb19af48a3e 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -77,12 +77,116 @@ static void detach_device(struct device *dev); static void set_dte_entry(struct amd_iommu *iommu, struct iommu_dev_data *dev_data); =20 +static void iommu_flush_dte_sync(struct amd_iommu *iommu, u16 devid); + /*************************************************************************= *** * * Helper functions * *************************************************************************= ***/ =20 +static void write_dte_upper128(struct dev_table_entry *ptr, struct dev_tab= le_entry *new) +{ + struct dev_table_entry old =3D {}; + + do { + old.data128[1] =3D ptr->data128[1]; + new->data[2] &=3D ~DTE_DATA2_INTR_MASK; + new->data[2] |=3D old.data[2] & (DTE_DATA2_INTR_MASK | DTE_DATA2_RESV_MA= SK); + } while (!try_cmpxchg128(&ptr->data128[1], &old.data128[1], new->data128[= 1])); +} + +static void write_dte_lower128(struct dev_table_entry *ptr, struct dev_tab= le_entry *new) +{ + struct dev_table_entry old =3D {}; + + /* + * Need to preserve DTE[96:106], which can be set by information in IVRS = table. + * See set_dev_entry_from_acpi(). + */ + new->data[1] |=3D ptr->data[1] & DTE_FLAG_MASK; + + do { + old.data128[0] =3D ptr->data128[0]; + } while (!try_cmpxchg128(&ptr->data128[0], &old.data128[0], new->data128[= 0])); +} + +/* + * Note: + * IOMMU reads the entire Device Table entry in a single 256-bit transacti= on + * but the driver is programming DTE using 2 128-bit cmpxchg. So, the driv= er + * need to ensure the following: + * - DTE[V|GV] bit is being written last when setting. + * - DTE[V|GV] bit is being written first when clearing. + * + * This function is used only by code, which updates DMA translation part = of the DTE. + * So, only consider control bits related to DMA when updating the entry. + */ +static void update_dte256(struct amd_iommu *iommu, struct iommu_dev_data *= dev_data, + struct dev_table_entry *new) +{ + struct dev_table_entry *dev_table =3D get_dev_table(iommu); + struct dev_table_entry *ptr =3D &dev_table[dev_data->devid]; + + spin_lock(&dev_data->dte_lock); + + if (!(ptr->data[0] & DTE_FLAG_V)) { + /* Existing DTE is not valid. */ + write_dte_upper128(ptr, new); + write_dte_lower128(ptr, new); + iommu_flush_dte_sync(iommu, dev_data->devid); + } else if (!(new->data[0] & DTE_FLAG_V)) { + /* Existing DTE is valid. New DTE is not valid. */ + write_dte_lower128(ptr, new); + write_dte_upper128(ptr, new); + iommu_flush_dte_sync(iommu, dev_data->devid); + } else if (!FIELD_GET(DTE_FLAG_GV, ptr->data[0])) { + /* + * Both DTEs are valid. + * Existing DTE has no guest page table. + */ + write_dte_upper128(ptr, new); + write_dte_lower128(ptr, new); + iommu_flush_dte_sync(iommu, dev_data->devid); + } else if (!FIELD_GET(DTE_FLAG_GV, new->data[0])) { + /* + * Both DTEs are valid. + * Existing DTE has guest page table, + * new DTE has no guest page table, + */ + write_dte_lower128(ptr, new); + write_dte_upper128(ptr, new); + iommu_flush_dte_sync(iommu, dev_data->devid); + } else if (FIELD_GET(DTE_GPT_LEVEL_MASK, ptr->data[2]) !=3D + FIELD_GET(DTE_GPT_LEVEL_MASK, new->data[2])) { + /* + * Both DTEs are valid and have guest page table, + * but have different number of levels. So, we need + * to upadte both upper and lower 128-bit value, which + * require disabling and flushing. + */ + struct dev_table_entry clear =3D {}; + + /* First disable DTE */ + write_dte_lower128(ptr, &clear); + iommu_flush_dte_sync(iommu, dev_data->devid); + + /* Then update DTE */ + write_dte_upper128(ptr, new); + write_dte_lower128(ptr, new); + iommu_flush_dte_sync(iommu, dev_data->devid); + } else { + /* + * Both DTEs are valid and have guest page table, + * and same number of levels. We just need to only + * update the lower 128-bit. So no need to disable DTE. + */ + write_dte_lower128(ptr, new); + } + + spin_unlock(&dev_data->dte_lock); +} + static inline bool pdom_is_v2_pgtbl_mode(struct protection_domain *pdom) { return (pdom && (pdom->pd_mode =3D=3D PD_MODE_V2)); @@ -203,6 +307,7 @@ static struct iommu_dev_data *alloc_dev_data(struct amd= _iommu *iommu, u16 devid) return NULL; =20 spin_lock_init(&dev_data->lock); + spin_lock_init(&dev_data->dte_lock); dev_data->devid =3D devid; ratelimit_default_init(&dev_data->rs); =20 @@ -1272,6 +1377,15 @@ static int iommu_flush_dte(struct amd_iommu *iommu, = u16 devid) return iommu_queue_command(iommu, &cmd); } =20 +static void iommu_flush_dte_sync(struct amd_iommu *iommu, u16 devid) +{ + int ret; + + ret =3D iommu_flush_dte(iommu, devid); + if (!ret) + iommu_completion_wait(iommu); +} + static void amd_iommu_flush_dte_all(struct amd_iommu *iommu) { u32 devid; --=20 2.34.1 From nobody Thu Nov 28 00:48:39 2024 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2059.outbound.protection.outlook.com [40.107.102.59]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4235813B787 for ; Mon, 7 Oct 2024 04:15:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.102.59 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728274509; cv=fail; b=N2Pz5MJ3NVQG/MWgIsKMSGZd56sHynZMp8YZ9gcVApdOC3FLbM5CDrmZ30L9+Wc0WvgqCaImReTAeDoBydbpW7WKXF7tnHhbeb+hf1mbI2r5/lrSE6QeT9fgSyZ3Y7cfanC3FBWoBcW14YRmty0D5EIQcgSdO4wLmOdmyUdM9po= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728274509; c=relaxed/simple; bh=eOXA4F/b2Eev34tu7u0fjoTg4Y00fZMQW4WtpFOhxto=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ohvv6s8vv7+C4kKPvwhifH8MyiglElrc8OAVidDT/bp2Q5v5j3yej6UjoWDdj99pw56Rtntnj/O1X4T+dq33pWMS/vpFbtIO91Wz9yWfT34NjjcxRBztlTXlCDd5uSk723oNch1yDZCwJp4A4B35BRRWvwirIf6HU/Ir1dvNYVM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=GX5lj0yh; arc=fail smtp.client-ip=40.107.102.59 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="GX5lj0yh" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=wOvMaa/rvnnC9Q+krDTB1zKPiH5v7RDfnipWtJBM5m5pidXhXgD1TZjeQe7GHJ/e578bskYVraXNGAS1HqcWrr6WVN2tFqQ+XoRwrpNchY4kjhAFHQ1HH5O7zdy5izARRi16AbpJKoHh+c5jHvtPS68hrWUo4F0XOZhLcR3oFrCgwzs35no1f9sYnx+h3LSR+z4m7Kv/TeoJpHp49Owk2U3WmcL15EXIwesMGjDzxiIuvYrutSbl2UZ7Ka5ePMnBoaZ8nKYedhoo87erVMT0m5tkaPcEjCUmZT5mtAUuzstSZpt81Tz9DYGlOccUAXv5fkU/Sjqjt2/W3fC23cY5NA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EZf3+QOnlSReo0Pvb5i2i4P9ALO5gVk7fvIq/C7nies=; b=AT1oCzddBfLZ3k500pw9kQWspJsdO8TbBqb3CCaT+V6Ef9ZE7++F48mih0ocOtCUVdVGXlh/+5kB5NKcTEiGQGo85R9bbCm+cqV0+LPIqNe5J0BaWTNElCnDjOOE9jnrxqnsBaJx086YwMrSfwTEacMWqUxdCWmexgY++ZZaKyZm0hsBNcSwWUWYZvvPvvTRAw0anVtT5w4FYlRZlOdX1WQ5gw3yxlqVTXJ/srAPFxxB9teD9kItiPYUwqNmqFJbl4QutWxIHS9dUQxTn5bQAJIWGgRTl2iVeq1zNU5Jg97WTrJLFUi3FPgsbtTsvkkFXrlVlRT5XgUBFyuWrTuOWQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EZf3+QOnlSReo0Pvb5i2i4P9ALO5gVk7fvIq/C7nies=; b=GX5lj0yhNg7Rhkybfn1h26E1uAqRTNTkNpi9k7mBYuFtTCSfxdxAuhZmMEJUsK7epGq9BZjT82WQg4PhAnMThssBsSXO0c+526FiCIAEYMaDlF3d8wEdeZR57j6lNLIQG32AboiKsbHFoqXb09kyXWLZzqvUy2hleImESaXUwY4= Received: from SJ0PR03CA0278.namprd03.prod.outlook.com (2603:10b6:a03:39e::13) by SN7PR12MB7226.namprd12.prod.outlook.com (2603:10b6:806:2a9::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8026.22; Mon, 7 Oct 2024 04:15:01 +0000 Received: from SJ1PEPF00002312.namprd03.prod.outlook.com (2603:10b6:a03:39e:cafe::e) by SJ0PR03CA0278.outlook.office365.com (2603:10b6:a03:39e::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8026.20 via Frontend Transport; Mon, 7 Oct 2024 04:15:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by SJ1PEPF00002312.mail.protection.outlook.com (10.167.242.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8048.13 via Frontend Transport; Mon, 7 Oct 2024 04:15:01 +0000 Received: from purico-ed03host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Sun, 6 Oct 2024 23:14:44 -0500 From: Suravee Suthikulpanit To: , CC: , , , , , , , , , Suravee Suthikulpanit Subject: [PATCH v5 3/6] iommu/amd: Modify set_dte_entry() to use 256-bit DTE helpers Date: Mon, 7 Oct 2024 04:13:50 +0000 Message-ID: <20241007041353.4756-4-suravee.suthikulpanit@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241007041353.4756-1-suravee.suthikulpanit@amd.com> References: <20241007041353.4756-1-suravee.suthikulpanit@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF00002312:EE_|SN7PR12MB7226:EE_ X-MS-Office365-Filtering-Correlation-Id: 743b9111-ee36-490a-ae87-08dce6869d29 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700013|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?o/LBflCCatT2Lfjx+DjnLMkrIwtd7DXesG9fDYnKANrmWk6bU0CFtD3f7R9b?= =?us-ascii?Q?1kCL+hhmQG3Rm4bd8Fd3Mlt6UPBqM1601yy8gV/DT3tgfzT+cAu/SHcYGguI?= =?us-ascii?Q?hflRmzvVUyO8vGJUIwyNJlVfRgF8yfwRjD5IroYiEdNjqN0ZGqBM3RA/ePw2?= =?us-ascii?Q?YtmFQO+v5a0kYpxI8fhhq9o0DkhHmBhgWO4D/zlaGDurbdIriKcT1e6I1MQ7?= =?us-ascii?Q?lJqPJxFo6hJlAZqfG59/OFOXugitBqrbGWt3BdclXYujjNMRnAdAhQGedw6H?= =?us-ascii?Q?UeOPLE9RQL8YUZzwE2/gQesLYkJzcET+HHMRqy/zu8L47gjPz3yPEKzX9N8z?= =?us-ascii?Q?Uy6ziuFQ+lCJY/J3AXCmU60kTgS9FgSjWXcipYHIfZYWpjlJrDRXCmqAoZxQ?= =?us-ascii?Q?EwKE6ybua3LLasWA6R8SeJmNm0mTQxZTHOCRrTnVUwKsfaCd0paSFn+1taUo?= =?us-ascii?Q?w1IqTbpDsTb6o20Zeo2BtA208Y32Je6x1bCSDjLmmIu31kFlca+Or9CtBvXt?= =?us-ascii?Q?YfAb3+t2yRykXeEe72EFxIGV4h9kAE9gL9jSO88pF+Es7+bXi51GZJ3HKVBW?= =?us-ascii?Q?zt5ay142gc40PQhZ3HYliJDdACmOApfkGtJmsTr+rNI4c88ZZRTRZnMdaiyY?= =?us-ascii?Q?OMrrwgGVVaVGDYOrIjoMvZEahRA1CNp6f03QOorLemQ/WFsPw8/hXDaRzkOx?= =?us-ascii?Q?DvTs/tLEbyNzUZRISUGjru6dwLIN+fgiIWYbxdl45Z7rGDQ4veao6jEWRrW5?= =?us-ascii?Q?ldhYoIP76TFb7BTwrsFjJWhrVNXon19Eomw5nZx73tmS/kixM/lYhX8rKpFV?= =?us-ascii?Q?yDlTWh7GcjZ4trZwsneu21QLtD7+8GJvwrnTTzf+q+GQURnAdbjwpoerrCLX?= =?us-ascii?Q?owi9POa8+qnVHudOxWn39Fj8kB16Lyolg5uM+oYtAPjtGP9A+9DNdGeUiZ17?= =?us-ascii?Q?iizlFbDCzdcDfuX4RN5GzzAZgF4Wv4YfEPLoY0figMpyDcVZOYL353hUAEut?= =?us-ascii?Q?QPyA4QBkKuJmAjVhGcFLNf+LS10/vtqatHMkdYbDta3h/6Np9eMfVRQm7H5J?= =?us-ascii?Q?EEg0Qe9LfJDr9zJPu+t0pTXtuEcMHHrFmDjsVc3hVg8DWHAcrZ2QEvmr4PBu?= =?us-ascii?Q?eYkx4LmFZjc6VdKbEtYORP3TtbA9PzxpkQuVa1yn2fle+UZxbiBWpi1lnbhm?= =?us-ascii?Q?oOpRMcen4rgrgYKQiyjF5QyaQHBUJaeacoOdQf4J2S61QYQVwcfcJWLaDRa7?= =?us-ascii?Q?az2FO4D/9JPL92yL/AQe5DkuGRvDPFpeJvrLXz+dGpnBOJ9mb+BhxDrufb8s?= =?us-ascii?Q?77CPTaWvSGhjIjD05yKgHjYdGqAwl8WjcU9Wn3f7zMhJezVBGiWWofsyzwVw?= =?us-ascii?Q?rwfB6upsvAmurxjt7FXOdj9nRckn?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(36860700013)(376014)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Oct 2024 04:15:01.4028 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 743b9111-ee36-490a-ae87-08dce6869d29 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF00002312.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7226 Content-Type: text/plain; charset="utf-8" Also, the set_dte_entry() is used to program several DTE fields (e.g. stage1 table, stage2 table, domain id, and etc.), which is difficult to keep track with current implementation. Therefore, separate logic for clearing DTE (i.e. make_clear_dte) and another function for setting up the GCR3 Table Root Pointer, GIOV, GV, GLX, and GuestPagingMode into another function set_dte_gcr3_table(). Signed-off-by: Suravee Suthikulpanit --- drivers/iommu/amd/amd_iommu_types.h | 2 + drivers/iommu/amd/iommu.c | 132 ++++++++++++++++------------ 2 files changed, 80 insertions(+), 54 deletions(-) diff --git a/drivers/iommu/amd/amd_iommu_types.h b/drivers/iommu/amd/amd_io= mmu_types.h index 91f802be7898..74aca19725cc 100644 --- a/drivers/iommu/amd/amd_iommu_types.h +++ b/drivers/iommu/amd/amd_iommu_types.h @@ -427,6 +427,8 @@ #define DTE_GPT_LEVEL_SHIFT 54 #define DTE_GPT_LEVEL_MASK GENMASK_ULL(55, 54) =20 +#define DTE_SYSMGT_MASK GENMASK_ULL(41, 40) + #define GCR3_VALID 0x01ULL =20 /* DTE[128:179] | DTE[184:191] */ diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index deb19af48a3e..6fa2f5bb5156 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -1956,90 +1956,114 @@ int amd_iommu_clear_gcr3(struct iommu_dev_data *de= v_data, ioasid_t pasid) return ret; } =20 +static void make_clear_dte(struct amd_iommu *iommu, struct dev_table_entry= *dte, + struct dev_table_entry *new) +{ + new->data[0] =3D DTE_FLAG_V; + + /* Apply erratum 63 */ + if (FIELD_GET(DTE_SYSMGT_MASK, dte->data[1]) =3D=3D 0x01) + new->data[0] |=3D BIT_ULL(DEV_ENTRY_IW); + + if (!amd_iommu_snp_en) + new->data[0] |=3D DTE_FLAG_TV; + + /* Need to preserve interrupt remapping information in DTE[128:255] */ + new->data128[1] =3D dte->data128[1]; + + /* Mask out old values for GuestPagingMode */ + new->data[2] &=3D ~(0x3ULL << DTE_GPT_LEVEL_SHIFT); +} + +/* + * Note: + * The old value for GCR3 table and GPT have been cleared from caller. + */ +static void set_dte_gcr3_table(struct amd_iommu *iommu, + struct iommu_dev_data *dev_data, + struct dev_table_entry *target) +{ + struct gcr3_tbl_info *gcr3_info =3D &dev_data->gcr3_info; + u64 tmp, gcr3; + + if (!gcr3_info->gcr3_tbl) + return; + + pr_debug("%s: devid=3D%#x, glx=3D%#x, gcr3_tbl=3D%#llx\n", + __func__, dev_data->devid, gcr3_info->glx, + (unsigned long long)gcr3_info->gcr3_tbl); + + tmp =3D gcr3_info->glx; + target->data[0] |=3D (tmp & DTE_GLX_MASK) << DTE_GLX_SHIFT; + if (pdom_is_v2_pgtbl_mode(dev_data->domain)) + target->data[0] |=3D DTE_FLAG_GIOV; + target->data[0] |=3D DTE_FLAG_GV; + + + gcr3 =3D iommu_virt_to_phys(gcr3_info->gcr3_tbl); + + /* Encode GCR3 table into DTE */ + tmp =3D DTE_GCR3_VAL_A(gcr3) << DTE_GCR3_SHIFT_A; + target->data[0] |=3D tmp; + tmp =3D DTE_GCR3_VAL_B(gcr3) << DTE_GCR3_SHIFT_B; + tmp |=3D DTE_GCR3_VAL_C(gcr3) << DTE_GCR3_SHIFT_C; + target->data[1] |=3D tmp; + + /* Guest page table can only support 4 and 5 levels */ + if (amd_iommu_gpt_level =3D=3D PAGE_MODE_5_LEVEL) + target->data[2] |=3D ((u64)GUEST_PGTABLE_5_LEVEL << DTE_GPT_LEVEL_SHIFT); +} + static void set_dte_entry(struct amd_iommu *iommu, struct iommu_dev_data *dev_data) { - u64 pte_root =3D 0; - u64 flags =3D 0; - u32 old_domid; - u16 devid =3D dev_data->devid; u16 domid; + u32 old_domid; + struct dev_table_entry new =3D {}; struct protection_domain *domain =3D dev_data->domain; - struct dev_table_entry *dev_table =3D get_dev_table(iommu); struct gcr3_tbl_info *gcr3_info =3D &dev_data->gcr3_info; + struct dev_table_entry *dte =3D &get_dev_table(iommu)[dev_data->devid]; =20 if (gcr3_info && gcr3_info->gcr3_tbl) domid =3D dev_data->gcr3_info.domid; else domid =3D domain->id; =20 + make_clear_dte(iommu, dte, &new); + if (domain->iop.mode !=3D PAGE_MODE_NONE) - pte_root =3D iommu_virt_to_phys(domain->iop.root); + new.data[0] =3D iommu_virt_to_phys(domain->iop.root); =20 - pte_root |=3D (domain->iop.mode & DEV_ENTRY_MODE_MASK) + new.data[0] |=3D (domain->iop.mode & DEV_ENTRY_MODE_MASK) << DEV_ENTRY_MODE_SHIFT; =20 - pte_root |=3D DTE_FLAG_IR | DTE_FLAG_IW | DTE_FLAG_V; + new.data[0] |=3D DTE_FLAG_IR | DTE_FLAG_IW | DTE_FLAG_V; =20 /* * When SNP is enabled, Only set TV bit when IOMMU * page translation is in use. */ if (!amd_iommu_snp_en || (domid !=3D 0)) - pte_root |=3D DTE_FLAG_TV; - - flags =3D dev_table[devid].data[1]; - - if (dev_data->ats_enabled) - flags |=3D DTE_FLAG_IOTLB; + new.data[0] |=3D DTE_FLAG_TV; =20 if (dev_data->ppr) - pte_root |=3D 1ULL << DEV_ENTRY_PPR; + new.data[0] |=3D 1ULL << DEV_ENTRY_PPR; =20 if (domain->dirty_tracking) - pte_root |=3D DTE_FLAG_HAD; - - if (gcr3_info && gcr3_info->gcr3_tbl) { - u64 gcr3 =3D iommu_virt_to_phys(gcr3_info->gcr3_tbl); - u64 glx =3D gcr3_info->glx; - u64 tmp; + new.data[0] |=3D DTE_FLAG_HAD; =20 - pte_root |=3D DTE_FLAG_GV; - pte_root |=3D (glx & DTE_GLX_MASK) << DTE_GLX_SHIFT; - - /* First mask out possible old values for GCR3 table */ - tmp =3D DTE_GCR3_VAL_B(~0ULL) << DTE_GCR3_SHIFT_B; - flags &=3D ~tmp; - - tmp =3D DTE_GCR3_VAL_C(~0ULL) << DTE_GCR3_SHIFT_C; - flags &=3D ~tmp; - - /* Encode GCR3 table into DTE */ - tmp =3D DTE_GCR3_VAL_A(gcr3) << DTE_GCR3_SHIFT_A; - pte_root |=3D tmp; - - tmp =3D DTE_GCR3_VAL_B(gcr3) << DTE_GCR3_SHIFT_B; - flags |=3D tmp; - - tmp =3D DTE_GCR3_VAL_C(gcr3) << DTE_GCR3_SHIFT_C; - flags |=3D tmp; - - if (amd_iommu_gpt_level =3D=3D PAGE_MODE_5_LEVEL) { - dev_table[devid].data[2] |=3D - ((u64)GUEST_PGTABLE_5_LEVEL << DTE_GPT_LEVEL_SHIFT); - } + if (dev_data->ats_enabled) + new.data[1] |=3D DTE_FLAG_IOTLB; + else + new.data[1] &=3D ~DTE_FLAG_IOTLB; =20 - /* GIOV is supported with V2 page table mode only */ - if (pdom_is_v2_pgtbl_mode(domain)) - pte_root |=3D DTE_FLAG_GIOV; - } + old_domid =3D dte->data[1] & DEV_DOMID_MASK; + new.data[1] &=3D ~DEV_DOMID_MASK; + new.data[1] |=3D domid; =20 - flags &=3D ~DEV_DOMID_MASK; - flags |=3D domid; + set_dte_gcr3_table(iommu, dev_data, &new); =20 - old_domid =3D dev_table[devid].data[1] & DEV_DOMID_MASK; - dev_table[devid].data[1] =3D flags; - dev_table[devid].data[0] =3D pte_root; + update_dte256(iommu, dev_data, &new); =20 /* * A kdump kernel might be replacing a domain ID that was copied from --=20 2.34.1 From nobody Thu Nov 28 00:48:39 2024 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2071.outbound.protection.outlook.com [40.107.96.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1FC8F71750 for ; Mon, 7 Oct 2024 04:15:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.96.71 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728274511; cv=fail; b=ncCtgsDUZnkDxQ2CfKMWwIBwiTKNdXfe20M26QU9vwoYiWCHW/GKceRUKM/kYDd+O430d/9Q2HiXG+hrlQbamRp7qU+a7BWC7qY1A9Yk00fZ+E7ilruOGrQWSSCHSJ+jxIChabXxNVCapZ3Ss5E9xER2f3LVl0WpZDFwqZ5S6xk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728274511; c=relaxed/simple; bh=G37E1vGUauxMG8WBxbH4KDe16JKZ7dpupVh9Ga9sLHA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=tM8fJ+Ka+3+HFTVxkD8lavsf+4g2G+uCxfEPpk+MeoU6cFr/CcQcwGbUpotTBJsy593DokgHfU6FQkV8An8a2YVDOi0pBUKVdFJEcIZa2toDa+LM+uoYkBA4TqM5so6ltA64oAOjE2GPtlt91SYZty+5GZdcI4obqDbiVwq00Eg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=Gcg0REYz; arc=fail smtp.client-ip=40.107.96.71 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="Gcg0REYz" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=PL6vyhde835ePU+kb9lzyjagV2xgN7QhBOwfikcnDPgfKd3hMlTofk3DqIoVa1HYUfbtJYAH8x7MSW8AJTlOSV8/6rd6IkJzL1ictC7Fj540uy5kq9zoC0sO5gaBKjOB5HnEg7N/vTkWivrtVHMPp+QnN4KajVKh3bEQGrI+lcIHcuCB3bhw4YwBzSlWsKPCmsSE3VYHSc8/jrMillHT59MA8TX1bk8Ukd0XkUL4mkcg5dSuLpWKEOp/qI85AAAGrX29UIGfD0hQ6M9sO0ePYAmCqf7G47kIfHMVYHuauEod1MECNamjvCa6scItw7375vGTEguYzTVmAK9BY4bFBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wcRMiMEDiDOE951lUW1BZrwFdDPIyAbifOs/sZWClU4=; b=h3jWnvHCox7Zs5pWn9VRNNiIgulpUKhjhCSaegb0U3Q9V5Y3hX6Am1rBEYsohNaNGsaQMrBDxlbODnh8fGEvU0AdDfvH71CJiajKyg/ceXfLdczVxyZnRyqYmErJJcs3MftGwtLQQv2WAAEefD46hLhlA2yZytpkR7Job/BO7lQbJMV1hN91Wmzfw70IwTWjRrxTEBEGu8TvJ8vE5Mx/IPVKwdOhXdUy273UOD6gU+XB2yfKN9lV1CjLYwHmW3OQJ95Bj5Wwxr/CFfI9i5eCnFlK3zq/R2fVfx4QY6W02TriqIOSPV73gkAEtyfEA/VxpPOsKPFwPl63TbX+Ef30BA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wcRMiMEDiDOE951lUW1BZrwFdDPIyAbifOs/sZWClU4=; b=Gcg0REYzOzyxk9Dc61OL4H8UOki0JdBIQ0FpBb90vCsLaduQbvd9UgTf3Y7Cn3Zeu2mr2bh6gsOto4ueFDVolndHwSs1SuGySL9rdOyga11G5hzeuTNiei2kpASUonQiAqcsRHhA5W8hxezse/yjJYA7+Q4ns4EC6kdApLz12jM= Received: from BYAPR02CA0043.namprd02.prod.outlook.com (2603:10b6:a03:54::20) by PH7PR12MB6561.namprd12.prod.outlook.com (2603:10b6:510:213::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8026.22; Mon, 7 Oct 2024 04:15:05 +0000 Received: from SJ1PEPF00002313.namprd03.prod.outlook.com (2603:10b6:a03:54:cafe::4) by BYAPR02CA0043.outlook.office365.com (2603:10b6:a03:54::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8026.20 via Frontend Transport; Mon, 7 Oct 2024 04:15:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by SJ1PEPF00002313.mail.protection.outlook.com (10.167.242.167) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8048.13 via Frontend Transport; Mon, 7 Oct 2024 04:15:05 +0000 Received: from purico-ed03host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Sun, 6 Oct 2024 23:15:00 -0500 From: Suravee Suthikulpanit To: , CC: , , , , , , , , , Suravee Suthikulpanit Subject: [PATCH v5 4/6] iommu/amd: Introduce helper function get_dte256() Date: Mon, 7 Oct 2024 04:13:51 +0000 Message-ID: <20241007041353.4756-5-suravee.suthikulpanit@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241007041353.4756-1-suravee.suthikulpanit@amd.com> References: <20241007041353.4756-1-suravee.suthikulpanit@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF00002313:EE_|PH7PR12MB6561:EE_ X-MS-Office365-Filtering-Correlation-Id: d20ef215-440a-4946-bc0f-08dce6869f40 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700013|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?+ZkB+d+917tj607KcB9a3M1Ta0MBrjl+7BUFbPjiqv4PXURUjvA5eJQQOdKV?= =?us-ascii?Q?+dHjzIUV6w2Nlnb/DY72E9R+Ww+/AMNyaeJlOtafquH1WjPZKQ6awszVOfAF?= =?us-ascii?Q?+AJhx9l3iGK7zy9oHWIlm14zPSejY6vbwPwKIHJztzNL4IHThCONrYB29lo8?= =?us-ascii?Q?B72Dgnr/+rPxIa1PWiWxLNpMBypn52GdoERzoTsfnQJQCcssY7YkifW+lQMH?= =?us-ascii?Q?ckf+t/ImNaeYFYepKRUlDDh2D4EAY/CJWv2X2Pjy4FA2mOBw1DnbmK7DL9VU?= =?us-ascii?Q?VvzGQcdktuoAYdaUUJqwNbIhjbLXurIJHkwFbiwVj2xQEE8uLPPlJ+pdOwcG?= =?us-ascii?Q?dFcgoMI0gZAw8WPvAyB40wUhZZWtcnIqOzDgxaxub7pOB5WshMNb0GOMtmhe?= =?us-ascii?Q?iwh4LNNBLTOR4g7Ss6GucjitFi8jseYo6V2MrJewjFudEpME3RTVGa9eu2jB?= =?us-ascii?Q?JhF/Zpf4Fm3fZaZBj1bYV+UFBtPNE6cE2i5qm17SgxASqEfdohjN5VAxPRmt?= =?us-ascii?Q?XWiYd4dsFK+MZb17lls/0LPlnGhNgt+yVxpERejSdgZY4QaZD4R773YiRyTN?= =?us-ascii?Q?gKTKDKf0VxHTo+zvHhf8B8b8ubl7GgsJtjuXvlYomeqtGkC0repRwLQEh0i/?= =?us-ascii?Q?1lSjDY9yrrIxNIgyCrUAx6jqi385VROBQXSTT0k4vrR57Y6/nYN0/ogcfgXp?= =?us-ascii?Q?XSRYNZLdFN9y9YSeg3WMRkRVK0Sg2J2nIo0a2TdarjbKerkjt1p7VBvQuT8r?= =?us-ascii?Q?2RUCmHKi1O7sKeAyWWnAExhAleOcsNIpJqNBCxIdn4VRsKeM7CrXHGuWbCGs?= =?us-ascii?Q?QgTYukPCSxIyA4ynSUg6SpIu7Q4OR4wPKHqbhT2tRJEy61yW9G2JF8kyHbB7?= =?us-ascii?Q?Sy249SFRLBlUddAQdErQYrXAIvEuV1/5MDruzmSui3UYiMoXZuNYRTbeU94C?= =?us-ascii?Q?Q44RV8s+CorlB7+9hpoTIb6VwcZWY1cER4JfKugvgLRviS00GFcx+KGJZd1D?= =?us-ascii?Q?4HF8l/V7E8srR5h/QcdYZihzNDK11W9S89lcCqpiTemp5fqM/7uQF4DvkDkJ?= =?us-ascii?Q?wlRvEGhlOtHD1zQzIH5TJ01lo5Pq49L5TvvnFDSMD0LHmzn+Hq/s8vYIXcFl?= =?us-ascii?Q?FFgE1iOsFo83OJH3N/rVhndymv0wmQOI/i6PUO6K5ip2qp8tvG03ZzUR/8bq?= =?us-ascii?Q?aoj+zkhSBEQWRunmbD5LDCBE3Tf1+ZwsK3a46r3oY3rjaDt9Wm0XX0ZDZ3QY?= =?us-ascii?Q?8esVVSIM60jhrbtjaDnca78N8y/MRzE6MeVsB/8Bvj7FEfaiFQtHRAnlWsHe?= =?us-ascii?Q?DnL0/DQv34x3UfmcThNCVvHvXGhPUHcl5NNGCpMz/6WVxrDCmN9UNDw4MHlN?= =?us-ascii?Q?aS8B71QGHRqJzOHOmg+gPj8lNbGl?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(36860700013)(376014)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Oct 2024 04:15:05.0031 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d20ef215-440a-4946-bc0f-08dce6869f40 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF00002313.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6561 Content-Type: text/plain; charset="utf-8" And use it in clone_alias() along with update_dte256(). Also use get_dte256() in dump_dte_entry(). Reviewed-by: Jason Gunthorpe Signed-off-by: Suravee Suthikulpanit --- drivers/iommu/amd/iommu.c | 49 ++++++++++++++++++++++++++++++++------- 1 file changed, 40 insertions(+), 9 deletions(-) diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index 6fa2f5bb5156..87a7c9dd86d9 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -187,6 +187,20 @@ static void update_dte256(struct amd_iommu *iommu, str= uct iommu_dev_data *dev_da spin_unlock(&dev_data->dte_lock); } =20 +static void get_dte256(struct amd_iommu *iommu, struct iommu_dev_data *dev= _data, + struct dev_table_entry *dte) +{ + struct dev_table_entry *ptr; + struct dev_table_entry *dev_table =3D get_dev_table(iommu); + + ptr =3D &dev_table[dev_data->devid]; + + spin_lock(&dev_data->dte_lock); + dte->data128[0] =3D ptr->data128[0]; + dte->data128[1] =3D ptr->data128[1]; + spin_unlock(&dev_data->dte_lock); +} + static inline bool pdom_is_v2_pgtbl_mode(struct protection_domain *pdom) { return (pdom && (pdom->pd_mode =3D=3D PD_MODE_V2)); @@ -335,9 +349,11 @@ static struct iommu_dev_data *search_dev_data(struct a= md_iommu *iommu, u16 devid =20 static int clone_alias(struct pci_dev *pdev, u16 alias, void *data) { + struct dev_table_entry new; struct amd_iommu *iommu; - struct dev_table_entry *dev_table; + struct iommu_dev_data *dev_data, *alias_data; u16 devid =3D pci_dev_id(pdev); + int ret =3D 0; =20 if (devid =3D=3D alias) return 0; @@ -346,13 +362,25 @@ static int clone_alias(struct pci_dev *pdev, u16 alia= s, void *data) if (!iommu) return 0; =20 - amd_iommu_set_rlookup_table(iommu, alias); - dev_table =3D get_dev_table(iommu); - memcpy(dev_table[alias].data, - dev_table[devid].data, - sizeof(dev_table[alias].data)); + /* Copy the data from pdev */ + dev_data =3D dev_iommu_priv_get(&pdev->dev); + if (!dev_data) { + ret =3D -EINVAL; + goto out; + } + get_dte256(iommu, dev_data, &new); =20 - return 0; + /* Setup alias */ + alias_data =3D search_dev_data(iommu, alias); + if (!alias_data) { + ret =3D -EINVAL; + goto out; + } + update_dte256(iommu, alias_data, &new); + + amd_iommu_set_rlookup_table(iommu, alias); +out: + return ret; } =20 static void clone_aliases(struct amd_iommu *iommu, struct device *dev) @@ -686,10 +714,13 @@ static void amd_iommu_uninit_device(struct device *de= v) static void dump_dte_entry(struct amd_iommu *iommu, u16 devid) { int i; - struct dev_table_entry *dev_table =3D get_dev_table(iommu); + struct dev_table_entry dte; + struct iommu_dev_data *dev_data =3D find_dev_data(iommu, devid); + + get_dte256(iommu, dev_data, &dte); =20 for (i =3D 0; i < 4; ++i) - pr_err("DTE[%d]: %016llx\n", i, dev_table[devid].data[i]); + pr_err("DTE[%d]: %016llx\n", i, dte.data[i]); } =20 static void dump_command(unsigned long phys_addr) --=20 2.34.1 From nobody Thu Nov 28 00:48:39 2024 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2045.outbound.protection.outlook.com [40.107.92.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 42D9242AA4 for ; Mon, 7 Oct 2024 04:15:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.45 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728274553; cv=fail; b=W1CgeUg3rdwHmUO482e0m6aOKWVQmEIGCCKkNB+NkZx/7DCvfznE1O68F7T5vp8Q1/2aV46b2OPhVUHOy2FCgw/bWjnwALcBqfZXkCft8XoroibhuJF37WMQb9u2pctcmttQB2RW+jteh3XLrVMNJpMBmgz8LqYFC00v3plTjMI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728274553; c=relaxed/simple; bh=LMi2aVeZxgbh28yNgLl/hj2uQZH4XguvxcspQF964Lw=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=DbqSJHxoAQqQjVmerVtADXmU/CBFSdlD3eNOJArvZeyeXfDA1/cwVN/Ufbk6XjhxwGeIHA0oLpaWQn95cbziNwM8j7m4zVkc8ExdcKy6kCEtm6CB/52TAB6Gi5Va8VnGOynPRnK4XQPfZp4lRX3r/WkBHPXNp77B3CTWcRUD1HA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=wBVHywBY; arc=fail smtp.client-ip=40.107.92.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="wBVHywBY" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=rNzFt5+pOa3mxCHFW9yDvCVbtxeu1DAh5JNrLludLkF8TEor6YaG09AUgZSWF1Xwa5Vzf2WopTYdJoQCSJ/d5tdRQ63M98Q9wi5A5xv4pVS0O2VM0/DxtB83KjCiABOU1r9RP4qKsot14bnSzdT55z2jFhrDMhCLzHh4RY9oV8Ec6Ab5G5ZD+MU2b50eIGy1/pG7ixmq5gVbezVCmc2I1/U/IiK0RglzRKdXHNWj2iAU1rAmVQD+7Mc0zYFIyFgF11W7ED6vPXsO4K/2Tq6j7Bpz5HJRELAkrNSoJW8vGrxx7Sz3R7jTxIazCc8Uz7nVcalSfWtWH4K/LyaanmpD/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=k6X5bBwbssLJ0ZwvNPA+Ww4UcGDIG3t1FLy05nP7/3Q=; b=JZbJz/alfwMqDzQi8QF5yeNSHErmS4cTIi0V3BvfZ8wS/nBWWj1KcT2Ob74MiKwRcJ//PwgSO7GR8nGVI5XPH/KTgYvB0Q9LAfAtJ7yi8RC71Z6ztBwqoYy8XUPC9CKRauMPiQKAy+bKOudeIC/HsIdisZu8EAse+Ddy73OaHalXsYnM/3fyx/BqAiWXM+j2p4YS1fLDzCy4LGtMFoxJ2Jczw3zKDjgRWjO4uq3TZXe8vfbnZbnagM1pMTGIDBLynIxPOIFzlfbW26929/csYuV+OcPkWrIj174c72jMYC5NBimcYdAFSFYbsyxGSHVIFM3eZofhsh4KvhZ7uz+NvA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=k6X5bBwbssLJ0ZwvNPA+Ww4UcGDIG3t1FLy05nP7/3Q=; b=wBVHywBYbJJUEoB718yveHsuyZqtSXgYZe/EtDE9VuTZfLVMCTtam47+dXGubl7s1ENA4f/S1cYn9zr7vu1XcQuq3UkyE4AjsmTO88nMwaWp9IhEyJEqLQs1sKu2boKm7n6XreA8vgpu1ribnjG1vxtowWVX7mfKPOEZmo/uYjM= Received: from SJ0PR03CA0299.namprd03.prod.outlook.com (2603:10b6:a03:39e::34) by PH7PR12MB9202.namprd12.prod.outlook.com (2603:10b6:510:2ef::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8026.20; Mon, 7 Oct 2024 04:15:48 +0000 Received: from SJ1PEPF00002312.namprd03.prod.outlook.com (2603:10b6:a03:39e:cafe::db) by SJ0PR03CA0299.outlook.office365.com (2603:10b6:a03:39e::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8026.20 via Frontend Transport; Mon, 7 Oct 2024 04:15:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by SJ1PEPF00002312.mail.protection.outlook.com (10.167.242.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8048.13 via Frontend Transport; Mon, 7 Oct 2024 04:15:47 +0000 Received: from purico-ed03host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Sun, 6 Oct 2024 23:15:04 -0500 From: Suravee Suthikulpanit To: , CC: , , , , , , , , , Suravee Suthikulpanit Subject: [PATCH v5 5/6] iommu/amd: Modify clear_dte_entry() to avoid in-place update Date: Mon, 7 Oct 2024 04:13:52 +0000 Message-ID: <20241007041353.4756-6-suravee.suthikulpanit@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241007041353.4756-1-suravee.suthikulpanit@amd.com> References: <20241007041353.4756-1-suravee.suthikulpanit@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF00002312:EE_|PH7PR12MB9202:EE_ X-MS-Office365-Filtering-Correlation-Id: 022aa93c-950b-46d1-3bcb-08dce686b8b6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700013|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?yfJQdOCOfFvK/+nRys810BvGdnra+RqRiUM6Sv6joxnAJsWcjU32iY5iY2lm?= =?us-ascii?Q?AxH73b/QMp6PHVbnL2V2xd2xK+0nczKN9KE7kN1S68inZodmLQluUvNwaJYt?= =?us-ascii?Q?eA4Nd2Ihe6588Im7I+pgJSqr0f4/QCglshR0EFhyjcpALFsFrenP2E5Lhj91?= =?us-ascii?Q?RHln4WMeQEancOL1Pkq9Y8MXb0X7V/OUe59jePdqjFstvJe3P8rqzbW5VvQ4?= =?us-ascii?Q?U6A2bdr9ySqI8qMK69HnWUIcKTjomL7gFciIeuU+JRZlNnFTPNgoOy6whr2p?= =?us-ascii?Q?BIh2hFQ1V1G7+45xncoJZXDtAneVOrwKWyPQqpxVyDwGiJOtTvqLaepCLROd?= =?us-ascii?Q?1V2Q9GrSa6q81CCVOWv4HgcWXeCendERvKrgi5b8jLJfd1G3hD5ZzCyVzd80?= =?us-ascii?Q?1lXY7ZHhN4fji1jqTHJo/WAMnAgJLaj/epjOIPnlVAdGLXHhg06TBzaVTsOW?= =?us-ascii?Q?WkTIeV06SBRGrcY4vTEcF/af9tMwMLtRO3AshB6XQaIaypS/AE46wYglFbVx?= =?us-ascii?Q?M/jVdkvqALMpbQZx+cSNLnkKELuTTkJxd0k90vTMVz1CYlWK5DKTzYq7ncle?= =?us-ascii?Q?n23vYhMzPL0AjBuonBeQV+7fte/m5jG5t47UCT8P4qcdkaXqhtu5CUU9HgsC?= =?us-ascii?Q?839PUQa50rX0+fOaLzXbcW4wb5jEj1FyOsvPfm9WNNJWuSOxi5JgS0JfYvGt?= =?us-ascii?Q?gCNV5rnP+HK8gW76CA/GR8hsFdo0ZrhCXHuyv2DIsjs9b2EuiUA+E6zC0/nw?= =?us-ascii?Q?xJOjJ8xzxVXkVmV5m6K+s1NoHaDgvOHlKG4P32HlnS4R+wP4ybDGffBs/eA/?= =?us-ascii?Q?x6/OGUnF8GjUQHy+DQPONHz2S5xtt73rd4ACW8gZBKefed9pp9CRQA/hdCiD?= =?us-ascii?Q?0XEbL5OarPwFcqM/DBAbmJzhYRLuREvGUsVCWdGLdwh1R7bYjlih5GYDLEaU?= =?us-ascii?Q?BVqOGmN3jH2bS4y0bkN8Lo6r1AeAUeg+dngPQHLi3eTCsS1aqUicbDNPXzhV?= =?us-ascii?Q?VXDUxU7ZGscrKh3NZ9WxRo67DKkimG/Bv5tzdvmfhnM1nSuunyFXqF8/+iLG?= =?us-ascii?Q?Tv+LLIrGlNDKcmV698R1eLcMfJRlWkx0j97AqTHdcMfoy8Fi6+NqCY7kdSDE?= =?us-ascii?Q?pMEECnsTMZ506B2BanfAAiSDX6EED6AmCXuZ4H1/uq2wbjtAjM6z9N7ZysSS?= =?us-ascii?Q?NlyVYrX+wQv98FuTZh86T26ZI6paSk0PYbFOjW7PvfWbP2qi4x1vL0qO0n2u?= =?us-ascii?Q?XHgKRf78BVZjRKY0JJsGqnlzQSa4Ed/rdbxR/ShzxqISNoxZVcZ0K8rkogi5?= =?us-ascii?Q?VwQAHV1lgOboDK5liexDRRMN28joJyWR8DbC1BzMn2N0QoDmRzdpGHDwKcUX?= =?us-ascii?Q?HRcDN6A=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(36860700013)(376014)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Oct 2024 04:15:47.6693 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 022aa93c-950b-46d1-3bcb-08dce686b8b6 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF00002312.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB9202 Content-Type: text/plain; charset="utf-8" Lock DTE and copy value to a temporary storage before update using cmpxchg128. Also, refactor the function to simplify logic for applying erratum 63. Signed-off-by: Suravee Suthikulpanit Reviewed-by: Jason Gunthorpe --- drivers/iommu/amd/iommu.c | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index 87a7c9dd86d9..dd7b02b00be6 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -2106,19 +2106,13 @@ static void set_dte_entry(struct amd_iommu *iommu, } } =20 -static void clear_dte_entry(struct amd_iommu *iommu, u16 devid) +static void clear_dte_entry(struct amd_iommu *iommu, struct iommu_dev_data= *dev_data) { - struct dev_table_entry *dev_table =3D get_dev_table(iommu); - - /* remove entry from the device table seen by the hardware */ - dev_table[devid].data[0] =3D DTE_FLAG_V; - - if (!amd_iommu_snp_en) - dev_table[devid].data[0] |=3D DTE_FLAG_TV; - - dev_table[devid].data[1] &=3D DTE_FLAG_MASK; + struct dev_table_entry new =3D {}; + struct dev_table_entry *dte =3D &get_dev_table(iommu)[dev_data->devid]; =20 - amd_iommu_apply_erratum_63(iommu, devid); + make_clear_dte(iommu, dte, &new); + update_dte256(iommu, dev_data, &new); } =20 /* Update and flush DTE for the given device */ @@ -2129,7 +2123,7 @@ static void dev_update_dte(struct iommu_dev_data *dev= _data, bool set) if (set) set_dte_entry(iommu, dev_data); else - clear_dte_entry(iommu, dev_data->devid); + clear_dte_entry(iommu, dev_data); =20 clone_aliases(iommu, dev_data->dev); device_flush_dte(dev_data); --=20 2.34.1 From nobody Thu Nov 28 00:48:39 2024 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2047.outbound.protection.outlook.com [40.107.223.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CBE67433CA for ; Mon, 7 Oct 2024 04:15:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.47 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728274555; cv=fail; b=nBQlVANhew5/0qKDtu9Oi89CXjHrVZ5//WQOJXWtgW+/AAkPsoIuSsuvYNXwNfpIOQ6W02FkKnnbMKdVqcTIiXRqkcT+t/gGm8/FvNkPOAH7Bgj9mQLC5nsYBcZA7tJE5kOnW3vMb6o+inZP6P4vE4hT/Cu0ShBfIq3xAKiQLB0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728274555; c=relaxed/simple; bh=GfZN3SwkoxfG4k+/7HvtZ+FdCi4I95DN1XaEyyxgarY=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Z/sOi2sp3quqU3IqeN4bongodw3Bmov104aaJbWl7/vdgGA+V54xV9gFhz7FNFPATIhrb4wqp7PHBJPdC3dFR3Gbob4rQzwvmN4l7AP+GrTRTLZnTwvnQ7jO7sEpfeH7TgXXqEF1z32YG2xvdxriC02ASet1Af8dXonhATqWwJw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=eCCxiRsQ; arc=fail smtp.client-ip=40.107.223.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="eCCxiRsQ" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=D9Ml+yFVIIOQUmpFQwHQc5h94gLlvQKQfSsnSsfpWqWXwVNRhvbCoYDlfm7YcTGTgpkYVehw5IK2fnA5kc857T9jGhYyOSSTE8wD8RAfVTQ31cMYQyiMurDqGdLMPYXg7lJvl7K2mmcvPpSn4RdO32iA/BQDBzGAWCQcoasJYCcp1+w/pzQy7BZbohdkIJjlZt7k8gdhLX63g5YvwuI06bZtBEkmBxbUdlkvDU0+e47NXQ3cA0m130pZdU/CkIgSvzNG/1SLYOciiiAWdKznNC2GsCni+Q5TGJNx7Dy6BVqk+vcssLY81v82fgB0Q77vTUPrE0Q7gqwthjOtwlIKww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xd/P6Jm0/LjtEB+j4JWFkGyaSdhmTMGgeAReCIF67rg=; b=wmemp8thkZxKI4cE2v9OoOaERENS/wnnNXVaB9c4zK1zuv+82thTbJ5/HiTZesNjEm8qwdnQST0LRg05I+qO7/frCwdqNERaQBCpDrDh/W0FRxVVmiPfziZG7ZdoTME0MtN7j7aQ+eNt6xFQE3WBaJJmxOAvA6FnKUqZqwEgtivgzN4T2EgHHAa4NzsUiMnjp65K8dvVeZpDXeiBJNXiHUSAcNOAvq/whGUapnphFKBoFa/DSjXX0lWYOephNwAhrrkEjmX//lsPeMe4o2Pbe4cfg1Q93vqwKt+Kog10xZ9NXSbPdGFB+6vhBm3Wn4MQNx5WVAZMYnzwpa5TJIe9Mw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xd/P6Jm0/LjtEB+j4JWFkGyaSdhmTMGgeAReCIF67rg=; b=eCCxiRsQtb+BF6qVZWoaaGGp8Lg+GYNfxeyFEeqPuT31UqNM6/WYaFl1Fy+eIkblVJ+SY/ufn+hPaimnDjht2ZFD3ijWUw34orc95XaTjgUkNS+vxsUqcO8Mp4O/O9WNimFs3NkmhdG/q6gjGDygjm90d0HIUcetfKTdFxYv4hk= Received: from MW4PR03CA0144.namprd03.prod.outlook.com (2603:10b6:303:8c::29) by SA0PR12MB4432.namprd12.prod.outlook.com (2603:10b6:806:98::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7982.27; Mon, 7 Oct 2024 04:15:50 +0000 Received: from SJ1PEPF00002316.namprd03.prod.outlook.com (2603:10b6:303:8c:cafe::eb) by MW4PR03CA0144.outlook.office365.com (2603:10b6:303:8c::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8026.20 via Frontend Transport; Mon, 7 Oct 2024 04:15:49 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by SJ1PEPF00002316.mail.protection.outlook.com (10.167.242.170) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8048.13 via Frontend Transport; Mon, 7 Oct 2024 04:15:49 +0000 Received: from purico-ed03host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Sun, 6 Oct 2024 23:15:45 -0500 From: Suravee Suthikulpanit To: , CC: , , , , , , , , , Suravee Suthikulpanit Subject: [PATCH v5 6/6] iommu/amd: Lock DTE before updating the entry with WRITE_ONCE() Date: Mon, 7 Oct 2024 04:13:53 +0000 Message-ID: <20241007041353.4756-7-suravee.suthikulpanit@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241007041353.4756-1-suravee.suthikulpanit@amd.com> References: <20241007041353.4756-1-suravee.suthikulpanit@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF00002316:EE_|SA0PR12MB4432:EE_ X-MS-Office365-Filtering-Correlation-Id: cec7904e-ebd9-4c6f-e348-08dce686b9d9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|36860700013|82310400026|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?dG5Z9Q4ftErB/RpCONZBgojbGS2YZVM0grtIXIsYygAbYxU6Opq3Urg4WRos?= =?us-ascii?Q?5TiWi1ozlfcZ4Q2TV6tXUfdNAmceLEf5EpUHbjszynOaCo7R+CfooUmB3wZN?= =?us-ascii?Q?DcXDsaxxW2WjFjlwYPGPpqDaXPWXnsbOeCK/6aag1a1UvpOELX8dBEyRaqct?= =?us-ascii?Q?WM+xjRyJbiN+wkKfujVVPIx36aNMDaTu1mUekp27PUnQYNsDmVrEUh0bw0Ur?= =?us-ascii?Q?Y/LoZ2p3YQEdYhDHqqCnjzq5y5l+w9wSB5StWq1P8ug2r+hQRaZGdIIlHsab?= =?us-ascii?Q?hP1/SCV6ZXdU+4u8MlsOanXxfXExXbDctI+MkIoDgiKSM39dnIY1sUBmXTsP?= =?us-ascii?Q?XYq824CWrhTs0hrA7TycBYZOAx1Hv2N7EA9vtJXEKgeTxuG5aXsnewlaQ4AO?= =?us-ascii?Q?LtK/D70Rhlsde++5zrS0d0XQjN6PUzz5x3AEciS8butucv7RDZvYrKRnLw1Y?= =?us-ascii?Q?cqXs48Pk+4YhdIz/GcOcrD10k0y7X59zvhlU4CDT0ZUWHHMKi3kZBUho6Kwj?= =?us-ascii?Q?0msPsgIkgdgF5hVz17//nZw5D7osVCLdY7oo1tyJ29hFnZAkL8drl+hsAAK5?= =?us-ascii?Q?TdI4ou2B43uHc2Ofe5tKWHXwMhiMsppyJUOvrjvapwg/YOenw4twvj3MyfVG?= =?us-ascii?Q?5jXGv3E8dGjDKUpGn4WmeWhZvUAXyRHmzMQXdFs/P0UkgKZHADisf1i8PRwM?= =?us-ascii?Q?rAL7hDJ9GPB5YnGV/P80q/MsF7p7y5FVRR0E3ajvw/2FMyzojzz5bR+XMksa?= =?us-ascii?Q?Tw324Kd+1Ainom8MkYslMcV7JRdxlgASxvXkwmu0A1F+JKJgwxo6uaSCM7Jz?= =?us-ascii?Q?FPAHrSFf+VN3fwsjbNEYB7wF0KtpPisi5i1pC5R9d7cJrWyZ6onm0Fidxpc6?= =?us-ascii?Q?oPPylb2mHs6sLKuUG6v+22UyPyTCqyCpCsrIJYjXbsjXi/tcfnhjeNnXxs7Y?= =?us-ascii?Q?GdsCYPA1XmOlvWGEwCmx24gIC/fyub1lV0I21NqGkaXtF6QXOJ2bPMyynT72?= =?us-ascii?Q?/KXblLAgT59qSI4arqlRHwoh8YI0zly8W8NMLzJM71Sazt1s+ILHlUqxevx7?= =?us-ascii?Q?WMnws/ep4L48/KrxtM24vAl+oX9OK9sBtgSFDwtZp9QZC+uI0nbloppvcSXh?= =?us-ascii?Q?9N/1he5l3DHSWxnTNCSuk6nxWZJDPDSXuZa8GxGw1TNyVWDBI/OSakbYeTW+?= =?us-ascii?Q?+UaS2RcNVjKpKvgSf3gOItiYSAwdI2XN9FSxLXok+gtVdmYaQ7Rwz0JmzYEr?= =?us-ascii?Q?hRD+Ippf66oLXaCG6Y38cBGEZl7KufmK6/l4918RGfjm4UDWRpwQ+nsHHqvn?= =?us-ascii?Q?REvuuSHxbvOi/Whw8BtEBr8ppY6ub03lxtO/42C0h1PWu/d2SknYKfx1HDlV?= =?us-ascii?Q?vw8M5jsAGLRfiCgsGMTswdJCisaV?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(36860700013)(82310400026)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Oct 2024 04:15:49.6119 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cec7904e-ebd9-4c6f-e348-08dce686b9d9 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF00002316.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4432 Content-Type: text/plain; charset="utf-8" When updating only within a 64-bit tuple of a DTE, just lock the DTE and use WRITE_ONCE() because it is writing to memory read back by HW. Suggested-by: Jason Gunthorpe Reviewed-by: Jason Gunthorpe Signed-off-by: Suravee Suthikulpanit --- drivers/iommu/amd/iommu.c | 42 ++++++++++++++++++++++----------------- 1 file changed, 24 insertions(+), 18 deletions(-) diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index dd7b02b00be6..18a60b4345f1 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -2783,12 +2783,12 @@ static int amd_iommu_set_dirty_tracking(struct iomm= u_domain *domain, bool enable) { struct protection_domain *pdomain =3D to_pdomain(domain); - struct dev_table_entry *dev_table; + struct dev_table_entry *dte; struct iommu_dev_data *dev_data; bool domain_flush =3D false; struct amd_iommu *iommu; unsigned long flags; - u64 pte_root; + u64 new; =20 spin_lock_irqsave(&pdomain->lock, flags); if (!(pdomain->dirty_tracking ^ enable)) { @@ -2797,16 +2797,15 @@ static int amd_iommu_set_dirty_tracking(struct iomm= u_domain *domain, } =20 list_for_each_entry(dev_data, &pdomain->dev_list, list) { + spin_lock(&dev_data->dte_lock); iommu =3D get_amd_iommu_from_dev_data(dev_data); - - dev_table =3D get_dev_table(iommu); - pte_root =3D dev_table[dev_data->devid].data[0]; - - pte_root =3D (enable ? pte_root | DTE_FLAG_HAD : - pte_root & ~DTE_FLAG_HAD); + dte =3D &get_dev_table(iommu)[dev_data->devid]; + new =3D dte->data[0]; + new =3D (enable ? new | DTE_FLAG_HAD : new & ~DTE_FLAG_HAD); + WRITE_ONCE(dte->data[0], new); + spin_unlock(&dev_data->dte_lock); =20 /* Flush device DTE */ - dev_table[dev_data->devid].data[0] =3D pte_root; device_flush_dte(dev_data); domain_flush =3D true; } @@ -3071,17 +3070,24 @@ static void iommu_flush_irt_and_complete(struct amd= _iommu *iommu, u16 devid) static void set_dte_irq_entry(struct amd_iommu *iommu, u16 devid, struct irq_remap_table *table) { - u64 dte; - struct dev_table_entry *dev_table =3D get_dev_table(iommu); + u64 new; + struct dev_table_entry *dte =3D &get_dev_table(iommu)[devid]; + struct iommu_dev_data *dev_data =3D search_dev_data(iommu, devid); + + if (dev_data) + spin_lock(&dev_data->dte_lock); + + new =3D dte->data[2]; + new &=3D ~DTE_IRQ_PHYS_ADDR_MASK; + new |=3D iommu_virt_to_phys(table->table); + new |=3D DTE_IRQ_REMAP_INTCTL; + new |=3D DTE_INTTABLEN; + new |=3D DTE_IRQ_REMAP_ENABLE; =20 - dte =3D dev_table[devid].data[2]; - dte &=3D ~DTE_IRQ_PHYS_ADDR_MASK; - dte |=3D iommu_virt_to_phys(table->table); - dte |=3D DTE_IRQ_REMAP_INTCTL; - dte |=3D DTE_INTTABLEN; - dte |=3D DTE_IRQ_REMAP_ENABLE; + WRITE_ONCE(dte->data[2], new); =20 - dev_table[devid].data[2] =3D dte; + if (dev_data) + spin_unlock(&dev_data->dte_lock); } =20 static struct irq_remap_table *get_irq_table(struct amd_iommu *iommu, u16 = devid) --=20 2.34.1