From nobody Sun Feb 8 06:56:06 2026 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2047.outbound.protection.outlook.com [40.107.93.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 835141B86C7 for ; Thu, 29 Aug 2024 18:08:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.47 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724954882; cv=fail; b=HmbBZl7qZint6zEw1s9DCpYXZ/M0HePAHKczB7U0YzUEpP/z4RmsJW6tqLLHyfHh8uiT6hp9LUq3dvwWSjcw52qmEM3Ka5Cqr7hUpc7Dv401aDxgpbrO+baTvRD5NwK9D3+meCSUhsGPNHEROgwcK552+3GTZb9RE19HGr/9tuw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724954882; c=relaxed/simple; bh=S2LUwt6UGv1a6AlQ3kRHKYdK1o7NAATuCp+6UNEOS/w=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=OKy8rAyyhOO62zm9ZaWJNcFQ8Gx5LccIc6HvaUpOr9x2DUV/dye4T0B66pY9kITSRgqJiqwhUN2YbAcH+evTSJ0fW3zS7SmiQTqJQ2GZIJ6AJZrEhQLV9vQTwyYFP2FHDDkaEh/kyTyvnHZpjLQpOMB+ZltF+20+B4Ds6ZulgYU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=A9kLsIF7; arc=fail smtp.client-ip=40.107.93.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="A9kLsIF7" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=UohUBrsjSUisLEnBWzWtxPpui4LoVIL8b7tC51a+uQZ7W4q2FMXW4HvvrPCq/Bj1Meal2KzAPbUasf3LGsXN9ztB1y2QlTzySDMPjDtaMfEgTCeuVoG9UUcJSZfQ6fRwb7fLh5EsKqTTT/MoG4N17rljxdcjUe9bfDaERW9cSR6uG3jgNCfUwv2TYqhFyLFJv7odJjUoLyR0dY10WLi4vgfB7XvTP/qPgH/+KMJ/t0jN3t5h6cbem7nsowe9rOHs8ZoNHynDqDKWGgDFpvw6GZMAiYrmuxls1kuU2HvxyxPP08jtgMfPkW/xSjzBczgog4TyLnaffxX+L7UwG2FR6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CPfkLVtLTG1Rf4ODxQr9nOI8kueJ45RiMXm2Z4g0fVY=; b=MKYOINWSoUdvZt7DA0hV0vTIF+Mho3hykHTbJci93wGaepFALROqBncKVT/diF5VlRwnV80iMf8RuQei/13K0ow46ef7XjdrRaoI4zfmJKIXAX+GtxDAoDf41GIVwD4cPyyg9ieM/uzChRVps7L40zH93dMDW7vDGxXMa1/nzT4RUSqqrv59sZuGIo3Pjkvnpyk8+E5xgSPh6n1S/DM6D1W9yhkTNfciGVkwEPk4rZji9A8D1W4UR9MXLszEZgIqfH/Jsduo03JezUkUJjDJdvnI0DiKqKuIO6yxx3Z2Vahx7A0vdOzI7D4eEWnlSxuLXU9j3YfUGP50jIeR+dJisg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CPfkLVtLTG1Rf4ODxQr9nOI8kueJ45RiMXm2Z4g0fVY=; b=A9kLsIF7aEfvyaoo403jPQG3hc2Us4feOGsq8QAeiHFjtInv5RHNXPKVh/Y6OL4bcB3vfQUO47efj7JmLiNsg7l9Dxrth4ziRjL8SJR+GXyjQtK33X2hNe/vapTfDy+7Dj0SKBHn2IIHTdXfmHZrS+vwbw+fcGKI0iOBCqfNWvQ= Received: from CH0P220CA0006.NAMP220.PROD.OUTLOOK.COM (2603:10b6:610:ef::27) by DM4PR12MB7503.namprd12.prod.outlook.com (2603:10b6:8:111::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.20; Thu, 29 Aug 2024 18:07:57 +0000 Received: from CH1PEPF0000AD7B.namprd04.prod.outlook.com (2603:10b6:610:ef:cafe::c7) by CH0P220CA0006.outlook.office365.com (2603:10b6:610:ef::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.29 via Frontend Transport; Thu, 29 Aug 2024 18:07:57 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CH1PEPF0000AD7B.mail.protection.outlook.com (10.167.244.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7918.13 via Frontend Transport; Thu, 29 Aug 2024 18:07:57 +0000 Received: from purico-ed03host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Thu, 29 Aug 2024 13:07:52 -0500 From: Suravee Suthikulpanit To: , CC: , , , , , , , , , Suravee Suthikulpanit Subject: [PATCH v2 1/5] iommu/amd: Disable AMD IOMMU if CMPXCHG16B feature is not supported Date: Thu, 29 Aug 2024 18:07:22 +0000 Message-ID: <20240829180726.5022-2-suravee.suthikulpanit@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240829180726.5022-1-suravee.suthikulpanit@amd.com> References: <20240829180726.5022-1-suravee.suthikulpanit@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD7B:EE_|DM4PR12MB7503:EE_ X-MS-Office365-Filtering-Correlation-Id: 0e1b8f37-83ce-4b67-574e-08dcc85582de X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?Bb1JQW6lVRgjQDvozzP2VT8i4Rq/BSeJHkxfjqTZwXja2XeCeaIplCE9sZ+M?= =?us-ascii?Q?/3ksiwD6D57cmeQnJWoUowKx2/2mgkFSRX1/TMtl+L1m2RxFMHUhQvBI5qw6?= =?us-ascii?Q?vmXxRHTWeR7rDiFtdZURLnzPhARlXNaj2h8gY3/N4D1QKlfKtXTuUC9AX85Y?= =?us-ascii?Q?wBwMKusljqnaJNlfBNydNm1DgNrentGQD65ZwEvb9//AOLKgtc/EjyxaJi1r?= =?us-ascii?Q?kynmEw+gMtneTj904h4ShOQh/aj9G9kYtVOZkPYQLrojDU0oUquTZ1wqYAxT?= =?us-ascii?Q?ThywZuvzRAIyZ4wMQ/EjDsMdl3imY7W91mPAYAkuRGVz+eceMaYu/Qy1PjTC?= =?us-ascii?Q?30HKzuarg4NXW9qBPvTU9fXk2reGnJekI/6WnfPyu6w/v3ivOPaIOMm+xGOs?= =?us-ascii?Q?OSx4DdV6ww+hUfFSMwtMQQ75kppWzSfxMK+ck1TYjjNfwgbaqc7AT2nt4AOR?= =?us-ascii?Q?WFE/1F+hSAaUJvNv9Xx/XxY8Sh3xb1DejQPTbpb76HkYIjXUcemkjHz0R4oi?= =?us-ascii?Q?2b+t8FetF0AC3C07vFZ70WvcU+TNTC23i/u9IjFMSveMvOGjbsBJqYLuOHxq?= =?us-ascii?Q?PLmCCTYq43H4VIz3KrUHOPHVaRLuPQgNWJC6w8aUBs5M1uHpMPQ6+tpC2trO?= =?us-ascii?Q?++jGY47ax+yNzccuSpALX+GPwoCyBHP2pKoGG11HaswFFBRKFp4Jjs2kGEpo?= =?us-ascii?Q?e+zlzJ7dBfZNZ5P1hRmPdWSM9gXu6wTfEFW7C9U4TAfRIDVFUQfYwkEPJRiy?= =?us-ascii?Q?CGUVv8LwRAYktiMvjmYBR9Qo0dafhPJky3L50PdPE57kHDNXNLNKCWuWf7mz?= =?us-ascii?Q?KoqhAiKpFzbXN5HdglTa/XUSKpBmwu7wIu0qcUmc3+5BoH0AgqnCLWomWHpC?= =?us-ascii?Q?cl857ij4PescSHnjBH+y1H36xjnJW/4MBLobDmzgbvj3WGuQ0OnjleAMEv8p?= =?us-ascii?Q?g2mYbfny7f+DSBjNgN/bFf3pADGbr16FKoZehW/L60NbvK9FmEVcQVA/C+EB?= =?us-ascii?Q?7eMNfGsAZBTpgBVtlx6Dlj+A84mvLZl4mBrBr3HV4qD4D4ww196EfriibMq5?= =?us-ascii?Q?MZpMIHvqhTJeQm1hX/TYNaORcR4VaNcpsqGiiSwmg1/wRjfKbROnrYwMm/aW?= =?us-ascii?Q?7PrHHzXHXHW9nkKMmB6d5kPB2gga4VZ4Wr05+bczf1izVa0Mi4wJbR6muUMG?= =?us-ascii?Q?WSV2vfGvZIwXvMOcb5a73OJr9nZ0lUeY3skR1gRo7HHH7Z3OaiEHkY33Y1Qh?= =?us-ascii?Q?1xWHe7w6kvRe07rXgvDA620P5Ou5WNHhxq/L1dn1x3r2nY4XIZEwkAw224ME?= =?us-ascii?Q?UVDaJ8K3KTUcRNPbtcZshDkZd/aQ9W+4F6/Gr19v/pt1N2L0AGbZf9yOzzbW?= =?us-ascii?Q?HzXS7TTA/tzebYhEuPM4j4Di7zUK?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(376014)(36860700013)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Aug 2024 18:07:57.0340 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0e1b8f37-83ce-4b67-574e-08dcc85582de X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CH1PEPF0000AD7B.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB7503 Content-Type: text/plain; charset="utf-8" According to the AMD IOMMU spec, the IOMMU reads the entire DTE either in two 128-bit transactions or a single 256-bit transaction. It is recommended to update DTE using 128-bit operation followed by an INVALIDATE_DEVTAB_ENTYRY command when the IV=3D1b or V=3D1b. According to the AMD BIOS and Kernel Developer's Guide (BDKG) dated back to family 10h Processor [1], which is the first introduction of AMD IOMMU, AMD processor always has CPUID Fn0000_0001_ECX[CMPXCHG16B]=3D1. Therefore, it is safe to assume cmpxchg128 is available with all AMD processor w/ IOMMU. In addition, the CMPXCHG16B feature has already been checked separately before enabling the GA, XT, and GAM modes. Consolidate the detection logic, and fail the IOMMU initialization if the feature is not supported. [1] https://www.amd.com/content/dam/amd/en/documents/archived-tech-docs/pro= grammer-references/31116.pdf Suggested-by: Jason Gunthorpe Signed-off-by: Suravee Suthikulpanit --- drivers/iommu/amd/init.c | 23 +++++++++-------------- 1 file changed, 9 insertions(+), 14 deletions(-) diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c index 6b15ce09e78d..983c09898a10 100644 --- a/drivers/iommu/amd/init.c +++ b/drivers/iommu/amd/init.c @@ -1762,13 +1762,8 @@ static int __init init_iommu_one(struct amd_iommu *i= ommu, struct ivhd_header *h, else iommu->mmio_phys_end =3D MMIO_CNTR_CONF_OFFSET; =20 - /* - * Note: GA (128-bit IRTE) mode requires cmpxchg16b supports. - * GAM also requires GA mode. Therefore, we need to - * check cmpxchg16b support before enabling it. - */ - if (!boot_cpu_has(X86_FEATURE_CX16) || - ((h->efr_attr & (0x1 << IOMMU_FEAT_GASUP_SHIFT)) =3D=3D 0)) + /* GAM requires GA mode. */ + if ((h->efr_attr & (0x1 << IOMMU_FEAT_GASUP_SHIFT)) =3D=3D 0) amd_iommu_guest_ir =3D AMD_IOMMU_GUEST_IR_LEGACY; break; case 0x11: @@ -1778,13 +1773,8 @@ static int __init init_iommu_one(struct amd_iommu *i= ommu, struct ivhd_header *h, else iommu->mmio_phys_end =3D MMIO_CNTR_CONF_OFFSET; =20 - /* - * Note: GA (128-bit IRTE) mode requires cmpxchg16b supports. - * XT, GAM also requires GA mode. Therefore, we need to - * check cmpxchg16b support before enabling them. - */ - if (!boot_cpu_has(X86_FEATURE_CX16) || - ((h->efr_reg & (0x1 << IOMMU_EFR_GASUP_SHIFT)) =3D=3D 0)) { + /* XT and GAM require GA mode. */ + if ((h->efr_reg & (0x1 << IOMMU_EFR_GASUP_SHIFT)) =3D=3D 0) { amd_iommu_guest_ir =3D AMD_IOMMU_GUEST_IR_LEGACY; break; } @@ -3049,6 +3039,11 @@ static int __init early_amd_iommu_init(void) return -EINVAL; } =20 + if (!boot_cpu_has(X86_FEATURE_CX16)) { + pr_err("Failed to initialize. The CMPXCHG16B feature is required.\n"); + return -EINVAL; + } + /* * Validate checksum here so we don't need to do it when * we actually parse the table --=20 2.34.1 From nobody Sun Feb 8 06:56:06 2026 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2056.outbound.protection.outlook.com [40.107.236.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A94741B86E0 for ; Thu, 29 Aug 2024 18:08:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.56 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724954886; cv=fail; b=TYHLuev93rb/nSzHrX4Rm9luAkNTjoz9u4ajHJz/a25sT+5rLmF1qoafuQjCpBGu7xiLN0WBwuCKkUOm7JPPG0/aiPZtF1aRyInE85BcPcIBppWNypv+0bn/CuDFgDmdRwAdHfKUIIibEQvh3qWHfE7tEcq2DXkIlVu4eYcGThQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724954886; c=relaxed/simple; bh=fWq3gGmcx1BX+5IqA98W0Nq/OgGik+GG971UkTZDkKU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=DESowuVDxUajZD36Cxkp/DWRRIHIDStqErWB7L8KF1Vf3qKwAiuKtaSuDxP8AfSO08l/NA8VHsDHXGnaD2dH0MqVGLM6hHVwQnKUjM6OUpRXeG2q9daEfxqNOsXrSA2Mdp1GlBp8C++TmP4OKQysSwCQKYsYS2kiU16JyPdJa7s= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=DvXtVEru; arc=fail smtp.client-ip=40.107.236.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="DvXtVEru" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Bu/5ESs3iY40JKyyn0AlqWPZaHAenrOIsjGsBVV3/fDOiOKULmV+wSKvTXthcVxyGDJ42QVlY+qYVKyPFhGdd1Z2Jdf70ibcy3AXyCc+j3ECUQsjp0CWifsBU3YU9IflV938omBdNM19BC15DZc4KM3joXCZh4z0Ate5wjLFsuJFgEWoRc+WubjQ09po9Z44zK1urN4kf8syDCR6zfeFlpw1JmWkZcaEHMzy+2RgkptXvgeslYq4dYf7e/qa8Oimo9G1+TyBa4nJ1nlsebTUceXmmg4RVW10Lg7mWGN+Hfq6/Gu++vq2bRamRevCbLI7ZkDaM686h/PaBDwTbl17Dw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nhHDd1yq1VCf8e6g7J3I4Zx8YmU7VedfgD+exBwfNco=; b=gWhxQMt3PdGudvPMRpgu1C60Ctq7XucXCZa+D12e2MwsjN0rjRmhtWBf0RV9eh8hu5fCQX3ebz5LVhlf4LZHIWWNieHVna588giS54oDAovibJ6mwevCuaZvIY3mCCD+3ZvE1NlKuxSWDFZ6jVaiCu+z7C4MJm3HdzphUdPlfSrZBHtBP/BFm5YR/4RFg2GidzL+YZMuov8vGyQYXq0c1oI9grfS4tcuVShzrvw0fuA7bUzz4PdW2XrDDt1lFiPBzvHDNylYOuL+TAdCPM9GWB5F831IdD5aUHLSwwMpefTiEUQIXtS8FDyTm02SEeDHWkcn7ZUdaze1h5viDqBdoA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nhHDd1yq1VCf8e6g7J3I4Zx8YmU7VedfgD+exBwfNco=; b=DvXtVEruemqTvkRxvjNs2R9H0rOkdxYXWdNlXdX0kdz/yoAZ0q8wb3ckY71/HxTrcJ/ArmXsUberFHZG8pthqs1Qt71WmiBekHgh20PhXkakkzyI0ZOeMWqwrHUL9gbr3KQomtEe6szyc0eZBuPKslXlgFgC0cVuY+v08Baivxo= Received: from SA9PR13CA0080.namprd13.prod.outlook.com (2603:10b6:806:23::25) by PH7PR12MB7257.namprd12.prod.outlook.com (2603:10b6:510:205::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.28; Thu, 29 Aug 2024 18:08:01 +0000 Received: from SA2PEPF00003F66.namprd04.prod.outlook.com (2603:10b6:806:23:cafe::a7) by SA9PR13CA0080.outlook.office365.com (2603:10b6:806:23::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.20 via Frontend Transport; Thu, 29 Aug 2024 18:08:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by SA2PEPF00003F66.mail.protection.outlook.com (10.167.248.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7918.13 via Frontend Transport; Thu, 29 Aug 2024 18:08:00 +0000 Received: from purico-ed03host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Thu, 29 Aug 2024 13:07:56 -0500 From: Suravee Suthikulpanit To: , CC: , , , , , , , , , Suravee Suthikulpanit Subject: [PATCH v2 2/5] iommu/amd: Introduce rw_semaphore for Device Table Entry (DTE) Date: Thu, 29 Aug 2024 18:07:23 +0000 Message-ID: <20240829180726.5022-3-suravee.suthikulpanit@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240829180726.5022-1-suravee.suthikulpanit@amd.com> References: <20240829180726.5022-1-suravee.suthikulpanit@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF00003F66:EE_|PH7PR12MB7257:EE_ X-MS-Office365-Filtering-Correlation-Id: 64789a6d-379a-412f-a0ff-08dcc8558517 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|376014|36860700013; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?oGbYW4LIfYQbpx574U0Uio9Qsn5I0cyOTT9bB1iUDf+aJpvOS1oh3EX32wVB?= =?us-ascii?Q?r+/I8Q0B7MVKc0TjwkEomrDuPeM2t3EXwV36sibtIYnfyfKmTK+gmbdx6W78?= =?us-ascii?Q?+Mcr9/etcaPazv9wb7nebeTFrsJRgjShAdSAljHMbrB4G1BWq7+3NJsDUCyW?= =?us-ascii?Q?70RGdoTbL4Mu3ULl4YGYKHTHqIqAioB3dbhGsBESWs/OBtB5H+FvMjuYshnv?= =?us-ascii?Q?mvtqZXQ9MAPa54385VsyZE0I0yM9nSIGEOA8wsePfmrAkuUR0ysjEVSYkPmj?= =?us-ascii?Q?Ee9XsNYdrXfdI1zD4K5QSx74tX1pvBzqR3e1EAG5QJAEab8Nnnnomaj1EeKE?= =?us-ascii?Q?nOcgHxYNtZL0Sb3Lxnq99rbSjYLgMHLJdvc+1At8rtdZSCP/hcciU9S4ZfYa?= =?us-ascii?Q?ZVyKFTDwNU+Ku3yzTibpph7Sv19SiWtHBXCnTZ4oN76Q/BCl0+eZuf5is+Wb?= =?us-ascii?Q?++JWdpRzZe/RDlU2Ih/LBVA5qliBU9+onKTaRPYJQM4cFfqG7lwH0x2SBud4?= =?us-ascii?Q?Ez5nOBvTDtU9gmbKLEIejI/Su23WimqoysmiT3UEnQxFZ3HtzHOu+JwPAxQd?= =?us-ascii?Q?/R6712t2t04d559sfBEIHwFxjqsz6CZqFiBkPMXVsaLu6M/KpQ188WaDvD+q?= =?us-ascii?Q?KmnSlhaiSZrrBsOluZcSTBmzR5OhDcN1k03GtZ8fTs7OCqq43Nuh7KjwwLCQ?= =?us-ascii?Q?QuZ/f/BuBc5SFycDQa6LvhsZAP5HRSTBWv1beb6S7beEqQdLaP/eghFVPePz?= =?us-ascii?Q?ArDCuVfj0SvfYYQ9bI+ldKNK3QsTby70jHqn+3CYpO4bziUZebGrkHyuObcZ?= =?us-ascii?Q?MKsPcX7byGMebYVCvQywWLkYw7wPVJgSSv3xwNbcW9+M5Ws81RvPGJ3XDF/s?= =?us-ascii?Q?R0c3zO+9Ll4sESdjKygXgAKnOdWtQfSoi2gI8nMOuMbgksoBz513vsCBBRT9?= =?us-ascii?Q?jer644HNdIPoCq/4DzeyUCXmNPPx3fRaVenpQdZilCLhyD26UpC4VactMeBC?= =?us-ascii?Q?x+PYKogtyQYygCqCGKpt3tKJ+ZGGHF/6PHWiGfWL5GiXxvnWd+VGFxGMV+aa?= =?us-ascii?Q?FIv3nle72Kq6/Lrws2Fnsh7jCTy5J2Jvvca2szEmJCz9iyfko0L7UoMfNkXM?= =?us-ascii?Q?oH9F7Oepsx9CAnrThJNaGB+hO+9erNfpikh2FIAZ3HVkiw50H8DQfH0J1S5n?= =?us-ascii?Q?1RilSYJdRGJQvV7WiLozJTnfhkoTzRl4nwdgdUL1l/Y44I4l6UlOLZnxtsSi?= =?us-ascii?Q?uDJv1PwNqlzNrOYLMJzmC5H0VR54mm6RguVJDwoSUINk7VbzgcSq1jFmgVX6?= =?us-ascii?Q?kPIhn+y+9gNopiHuAGZS820BEO9ypQEc4HjkugMzI8m4iWqXR9RFmUqvShP4?= =?us-ascii?Q?Bn0BIQdkU2bJZZ3lZ6A0+BQns4dUnJm73VqwSAuUFEvqqy7enETqVyl23s/P?= =?us-ascii?Q?GLftMBxmKAAabYce2q0VtXRPmVTc9pCr?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(376014)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Aug 2024 18:08:00.9055 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 64789a6d-379a-412f-a0ff-08dcc8558517 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF00003F66.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7257 Content-Type: text/plain; charset="utf-8" In preparation for 256-bit DTE update, each DTE access/update needs to be protected using synchronication mechanism to prevent conflict. Introduce a new rw-semaphore struct dev_data.dte_sem, which is per device. Also update certain helper functions to use the new dte_sem. Signed-off-by: Suravee Suthikulpanit --- drivers/iommu/amd/amd_iommu_types.h | 1 + drivers/iommu/amd/iommu.c | 10 ++++++++-- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/amd/amd_iommu_types.h b/drivers/iommu/amd/amd_io= mmu_types.h index c9f9a598eb82..65f3a073999d 100644 --- a/drivers/iommu/amd/amd_iommu_types.h +++ b/drivers/iommu/amd/amd_iommu_types.h @@ -833,6 +833,7 @@ struct devid_map { struct iommu_dev_data { /*Protect against attach/detach races */ spinlock_t lock; + struct rw_semaphore dte_sem; =20 struct list_head list; /* For domain->dev_list */ struct llist_node dev_data_list; /* For global dev_data_list */ diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index 87c5385ce3f2..994ed02842b9 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -205,6 +205,7 @@ static struct iommu_dev_data *alloc_dev_data(struct amd= _iommu *iommu, u16 devid) return NULL; =20 spin_lock_init(&dev_data->lock); + init_rwsem(&dev_data->dte_sem); dev_data->devid =3D devid; ratelimit_default_init(&dev_data->rs); =20 @@ -1946,10 +1947,13 @@ static void set_dte_entry(struct amd_iommu *iommu, } } =20 -static void clear_dte_entry(struct amd_iommu *iommu, u16 devid) +static void clear_dte_entry(struct amd_iommu *iommu, struct iommu_dev_data= *dev_data) { + u16 devid =3D dev_data->devid; struct dev_table_entry *dev_table =3D get_dev_table(iommu); =20 + down_write(&dev_data->dte_sem); + /* remove entry from the device table seen by the hardware */ dev_table[devid].data[0] =3D DTE_FLAG_V; =20 @@ -1959,6 +1963,8 @@ static void clear_dte_entry(struct amd_iommu *iommu, = u16 devid) dev_table[devid].data[1] &=3D DTE_FLAG_MASK; =20 amd_iommu_apply_erratum_63(iommu, devid); + + up_write(&dev_data->dte_sem); } =20 /* Update and flush DTE for the given device */ @@ -1969,7 +1975,7 @@ void amd_iommu_dev_update_dte(struct iommu_dev_data *= dev_data, bool set) if (set) set_dte_entry(iommu, dev_data); else - clear_dte_entry(iommu, dev_data->devid); + clear_dte_entry(iommu, dev_data); =20 clone_aliases(iommu, dev_data->dev); device_flush_dte(dev_data); --=20 2.34.1 From nobody Sun Feb 8 06:56:06 2026 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2050.outbound.protection.outlook.com [40.107.92.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 27E191B86FA for ; Thu, 29 Aug 2024 18:08:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.50 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724954890; cv=fail; b=Txp2wB5mPjLZS0GKe6G9HLEk/xz4vF5X7LjQRkKTuM0ZIiaOoQlE1re1L/FtKhcEG1S9i2CyEJXxjeuZFN8OxYU3LYju3ARVQukRDNSiDywp32BDmJ4A828oIUV5Q1BoO/t6Ob71+AP0rxnPaxOy3tMBFLBgW0Jtm8GCqdaG/iM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724954890; c=relaxed/simple; bh=1kcUNLCXGPQG7lvcwwKEhfoqaAj1wvWVOtNFeb0wKGQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=KiLFeDl1pSWI7WzEDsrY6Wluq4bbifibcFQJfwT+4UfnwlMg0r8E8BDLymGPg3f4NO1hMOuRHTo2/w7N7R43fiIeF9fRmueMEBjqSNEFSy6USzUYZSfJ5m9UR8aPkC59nB0wKWUHtEejRxWyLNKmI9pJGJ/Kp0SGKXUpRDuDDqc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=gvwbUoYd; arc=fail smtp.client-ip=40.107.92.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="gvwbUoYd" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ciWUxk6/I44Xbg+mxhRtk18PCIc6a+mzBCd+mVaG/hDiTAKU+0ikxMgf2D1HFz+7Qmy6Psj5fyQPOrnGx3o6/q1LcqC3ut/f9F5MKd3OybhDKKjZW8IvC+NmWB2aW7kk8WdxT2b6sYQlNF75d9Fg8Vt14i4CW1LH9mFt+3L+yacDon/24oy5YQ/aP4Fw4afPES6KKWevvgUPQpcKS8l/3avJSHf9jg00XZlVd/rXDpzHL98kg5a03o6/lKnwS99wGPv367uxITLa0YZ8H96UvFMk+6UeUjHmFBtmn7lRxDGGVGMK3ScLyOEsb5EAzQiu2DzeWfN0X4IAF/dodf+zJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vk/hNVwjkH6//sivZjA6qsmeD26KGCQ/cMBFldYA6gQ=; b=voCwkrm4RnLb0l/lV/HQZJM9kS7xz5T4eKyNnFx/z2k5ACIKaDgUXS9SDMDXK2Bz1fTN3Agqps9YssRNEy1oDGaJxYtC1CE0iRly3kfHqqq1hLGH0sIgoKcRhmLwP+kDPba+YOpTRUFvhaTaDnX8JOhE2JGSjLnGJVdH03y9jdejHvhp/UFerTYPeG/w3/wS41ftmi55iJKSUGbaCH5mFjk+BTlQ9G+A1/QawRf29ESEhLDQQSTLGrkcz0K5wgF4/Rf8Ps7awR/u3Cw9Ph5ytM2RRg7dY+38gs69S2jCHvlZkX9BjXqALT8em+tzwg/kIB1YPVfLe3QgVZLBmGnXyg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vk/hNVwjkH6//sivZjA6qsmeD26KGCQ/cMBFldYA6gQ=; b=gvwbUoYdoM8pGNQPQIThZWphiK86gk/g09+xtJpIv5n68a200dJ9LNQdV++0Aal6lHmNjxcigNBN1jaIpipCBlLAJkFuPEMIrux50DRw9qsoV0rxpKu4+NzqdvXKt8DwKtFEM1L6mqe0WW/n6DNd3XJPfmK3tmwG92G4LcLQYP4= Received: from SA1PR03CA0024.namprd03.prod.outlook.com (2603:10b6:806:2d3::25) by CH3PR12MB7690.namprd12.prod.outlook.com (2603:10b6:610:14e::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.28; Thu, 29 Aug 2024 18:08:05 +0000 Received: from SA2PEPF00003F62.namprd04.prod.outlook.com (2603:10b6:806:2d3:cafe::ac) by SA1PR03CA0024.outlook.office365.com (2603:10b6:806:2d3::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.28 via Frontend Transport; Thu, 29 Aug 2024 18:08:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by SA2PEPF00003F62.mail.protection.outlook.com (10.167.248.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7918.13 via Frontend Transport; Thu, 29 Aug 2024 18:08:05 +0000 Received: from purico-ed03host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Thu, 29 Aug 2024 13:07:59 -0500 From: Suravee Suthikulpanit To: , CC: , , , , , , , , , Suravee Suthikulpanit Subject: [PATCH v2 3/5] iommu/amd: Introduce helper functions to access and update 256-bit DTE Date: Thu, 29 Aug 2024 18:07:24 +0000 Message-ID: <20240829180726.5022-4-suravee.suthikulpanit@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240829180726.5022-1-suravee.suthikulpanit@amd.com> References: <20240829180726.5022-1-suravee.suthikulpanit@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF00003F62:EE_|CH3PR12MB7690:EE_ X-MS-Office365-Filtering-Correlation-Id: bfd81663-9008-4122-cdbc-08dcc85587e4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|376014|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?kuuFyklNwyBKLjpwQnbEFMcFHRhUuE7Re25iTDC4KZugOAGCZcr6S9eF3UVY?= =?us-ascii?Q?+oBxihqkuYyqwk09SvFOt2XEJNYKoVgF4ae5bDkfxkorCETnXpXXZVD5Q76w?= =?us-ascii?Q?3TW5Qp1AcN3vF6WiHIXp1m6A0+nq/ze61FqnGJQxHBukuNEYwHOgraAV0ugn?= =?us-ascii?Q?XXxttkTWLzcYXwR1HiPVFpKFs7Y77t4c03mo/xGs0drhEp5wMZinO1DPenYH?= =?us-ascii?Q?F4fiMuTrr/fu/YtokjFgqPlzU3K/po4HSQA0IhdEKAOpC7sBssNwuhBFkvll?= =?us-ascii?Q?769bAdX32h5BAXBSyyxgTpR0K9Xc1bkHfZz1tdg1ULeo164CbzQQyuPoU1LV?= =?us-ascii?Q?iXoKnKh2WC8s7QO+gUc05vGF9ozcOZcIEEZyQppUqpyaoe545E1rO9nR99uY?= =?us-ascii?Q?mK0fMG8OcrTh3DHmDa47mX0WVAiAOxw4NoV/cSoLGSjwAXdR6/In0OWrzdtV?= =?us-ascii?Q?go8hM+t7GFl3W332zE52SkNFAJ2SJP6zaeE0O/tp9dDoArj4qtVmfWJ1bicu?= =?us-ascii?Q?h18Wzuht+6g7bGGHJHlH0CDsV8HoJytVkbWMLM7x/AuDbDVUE3l3wLV9Gp97?= =?us-ascii?Q?yQhEgTVsOszhY6GQnnMx6cL5NU8blH4BAv3NHmGvJRTjerzzU2tMk+EwA89a?= =?us-ascii?Q?b9NpLSyG59bzinQ2U108OGVcuwJopX7YxR8KByfthmuBt6JrgYBf6x0shPtv?= =?us-ascii?Q?K1n6gjnah0zYMuOftMuWkkv8FdsdA3XVZoUMFUqHcwUZoAavaeEinhiv0Wgu?= =?us-ascii?Q?pUNBTsiM+Pz4JvIQmiLfCMEol87SlAUCRjAAYNv8pm5NpN6qi1s0PHDEsBan?= =?us-ascii?Q?+DWsTbalf7WnFCRupsxjxx5+fedC6pyjwpbwyahdQ6HDfO0fBMCKaQ9bXqqL?= =?us-ascii?Q?12Ad/0FejH4Z+JLP9wqnOXSpN1QEdogRYLoU5GUBo4t7AoTzeNvUdnmInx2+?= =?us-ascii?Q?0hwuSzsdxHLJ+V/88L11RHULixx8ru2xr3OHA85dZv1Hy8WyUEs1ihgNthL7?= =?us-ascii?Q?TUXO8Xa7w6DL7bECkrABZGkwZix7/6F3aXF3vSkleEVK2hk70P1KugsdjPYT?= =?us-ascii?Q?I9eFNHoEBDE+5RZ9DlvFp+zZ+p9Fj2uoO435Uj6S39rQWyONbUPH5gwcxp/u?= =?us-ascii?Q?W46PR4MaWoQilKmoBCWFLL57WQnT99yQrGH2PxemCgRTaUjlTm40iwxOmNrf?= =?us-ascii?Q?SvQgK9m7axiIo91cnbqqOP8+7cYuwEBb52uqkzW/uh6/ZSvm267J22oDhzQG?= =?us-ascii?Q?M5Jd2MNx4ZWv/Z+2MAwymJfajl2PLI66DjXOTsVDnvx0DHC22g8G8K+mL1aD?= =?us-ascii?Q?LI9SHk92fU1ZUUsWeUXCVCO6y4nNwkeJ/wCGDADVXH5/qVy+CYwUV5jYHqDE?= =?us-ascii?Q?bwVNIUk4m/4y/ZGA8+WMKFEfvBSptDu4p1DlHQ5myNYWfNfw+uGCMJwS0YHy?= =?us-ascii?Q?edrYfRcx4rUirRQc8dlhkmMCbp1SDqOy?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(376014)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Aug 2024 18:08:05.5274 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bfd81663-9008-4122-cdbc-08dcc85587e4 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF00003F62.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB7690 Content-Type: text/plain; charset="utf-8" The current implementation does not follow 128-bit write requirement to update DTE as specified in the AMD I/O Virtualization Techonology (IOMMU) Specification. Therefore, modify the struct dev_table_entry to contain union of u128 data array, and introduce two helper functions: * update_dte256() to update DTE using two 128-bit cmpxchg operations to update 256-bit DTE with the modified structure. Also use the struct iommu_dev_data.dte_sem to synchronize 256-bit data update. * get_dte256() to copy 256-bit DTE to the provided structrue. Also use the struct iommu_dev_data.dte_sem to synchronize 256-bit data access. Also, update existing code to use the new helper functions in this and subsequent patches. Suggested-by: Jason Gunthorpe Signed-off-by: Suravee Suthikulpanit --- drivers/iommu/amd/amd_iommu_types.h | 5 +- drivers/iommu/amd/iommu.c | 81 +++++++++++++++++++++++------ 2 files changed, 70 insertions(+), 16 deletions(-) diff --git a/drivers/iommu/amd/amd_iommu_types.h b/drivers/iommu/amd/amd_io= mmu_types.h index 65f3a073999d..2787d6af5a59 100644 --- a/drivers/iommu/amd/amd_iommu_types.h +++ b/drivers/iommu/amd/amd_iommu_types.h @@ -884,7 +884,10 @@ extern struct amd_iommu *amd_iommus[MAX_IOMMUS]; * Structure defining one entry in the device table */ struct dev_table_entry { - u64 data[4]; + union { + u64 data[4]; + u128 data128[2]; + }; }; =20 /* diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index 994ed02842b9..93bca5c68bca 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -85,6 +85,47 @@ static void set_dte_entry(struct amd_iommu *iommu, * *************************************************************************= ***/ =20 +static void update_dte256(struct amd_iommu *iommu, struct iommu_dev_data *= dev_data, + struct dev_table_entry *new) +{ + struct dev_table_entry *dev_table =3D get_dev_table(iommu); + struct dev_table_entry *ptr =3D &dev_table[dev_data->devid]; + struct dev_table_entry old; + u128 tmp; + + down_write(&dev_data->dte_sem); + + old.data128[0] =3D ptr->data128[0]; + old.data128[1] =3D ptr->data128[1]; + + tmp =3D cmpxchg128(&ptr->data128[1], old.data128[1], new->data128[1]); + if (tmp =3D=3D old.data128[1]) { + if (!try_cmpxchg128(&ptr->data128[0], &old.data128[0], new->data128[0]))= { + /* Restore hi 128-bit */ + cmpxchg128(&ptr->data128[1], new->data128[1], tmp); + pr_err("%s: Failed. devid=3D%#x, dte=3D%016llx:%016llx:%016llx:%016llx\= n", + __func__, dev_data->devid, new->data[0], new->data[1], + new->data[2], new->data[3]); + } + } + + up_write(&dev_data->dte_sem); +} + +static void get_dte256(struct amd_iommu *iommu, struct iommu_dev_data *dev= _data, + struct dev_table_entry *dte) +{ + struct dev_table_entry *ptr; + struct dev_table_entry *dev_table =3D get_dev_table(iommu); + + ptr =3D &dev_table[dev_data->devid]; + + down_read(&dev_data->dte_sem); + dte->data128[0] =3D ptr->data128[0]; + dte->data128[1] =3D ptr->data128[1]; + up_read(&dev_data->dte_sem); +} + static inline bool pdom_is_v2_pgtbl_mode(struct protection_domain *pdom) { return (pdom && (pdom->pd_mode =3D=3D PD_MODE_V2)); @@ -233,8 +274,9 @@ static struct iommu_dev_data *search_dev_data(struct am= d_iommu *iommu, u16 devid =20 static int clone_alias(struct pci_dev *pdev, u16 alias, void *data) { + struct dev_table_entry dte; struct amd_iommu *iommu; - struct dev_table_entry *dev_table; + struct iommu_dev_data *dev_data; u16 devid =3D pci_dev_id(pdev); =20 if (devid =3D=3D alias) @@ -244,11 +286,19 @@ static int clone_alias(struct pci_dev *pdev, u16 alia= s, void *data) if (!iommu) return 0; =20 + dev_data =3D dev_iommu_priv_get(&pdev->dev); + if (!dev_data) + return -EINVAL; + + get_dte256(iommu, dev_data, &dte); + + /* Setup for alias */ + dev_data =3D search_dev_data(iommu, alias); + if (!dev_data) + return -EINVAL; + + update_dte256(iommu, dev_data, &dte); amd_iommu_set_rlookup_table(iommu, alias); - dev_table =3D get_dev_table(iommu); - memcpy(dev_table[alias].data, - dev_table[devid].data, - sizeof(dev_table[alias].data)); =20 return 0; } @@ -584,10 +634,13 @@ static void amd_iommu_uninit_device(struct device *de= v) static void dump_dte_entry(struct amd_iommu *iommu, u16 devid) { int i; - struct dev_table_entry *dev_table =3D get_dev_table(iommu); + struct dev_table_entry dte; + struct iommu_dev_data *dev_data =3D find_dev_data(iommu, devid); + + get_dte256(iommu, dev_data, &dte); =20 for (i =3D 0; i < 4; ++i) - pr_err("DTE[%d]: %016llx\n", i, dev_table[devid].data[i]); + pr_err("DTE[%d]: %016llx\n", i, dte.data[i]); } =20 static void dump_command(unsigned long phys_addr) @@ -2667,12 +2720,10 @@ static int amd_iommu_set_dirty_tracking(struct iomm= u_domain *domain, bool enable) { struct protection_domain *pdomain =3D to_pdomain(domain); - struct dev_table_entry *dev_table; struct iommu_dev_data *dev_data; bool domain_flush =3D false; struct amd_iommu *iommu; unsigned long flags; - u64 pte_root; =20 spin_lock_irqsave(&pdomain->lock, flags); if (!(pdomain->dirty_tracking ^ enable)) { @@ -2681,16 +2732,16 @@ static int amd_iommu_set_dirty_tracking(struct iomm= u_domain *domain, } =20 list_for_each_entry(dev_data, &pdomain->dev_list, list) { - iommu =3D get_amd_iommu_from_dev_data(dev_data); + struct dev_table_entry dte; =20 - dev_table =3D get_dev_table(iommu); - pte_root =3D dev_table[dev_data->devid].data[0]; + iommu =3D get_amd_iommu_from_dev_data(dev_data); + get_dte256(iommu, dev_data, &dte); =20 - pte_root =3D (enable ? pte_root | DTE_FLAG_HAD : - pte_root & ~DTE_FLAG_HAD); + dte.data[0] =3D (enable ? dte.data[0] | DTE_FLAG_HAD : + dte.data[0] & ~DTE_FLAG_HAD); =20 /* Flush device DTE */ - dev_table[dev_data->devid].data[0] =3D pte_root; + update_dte256(iommu, dev_data, &dte); device_flush_dte(dev_data); domain_flush =3D true; } --=20 2.34.1 From nobody Sun Feb 8 06:56:06 2026 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2073.outbound.protection.outlook.com [40.107.236.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B89861B8E8F for ; Thu, 29 Aug 2024 18:08:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.73 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724954895; cv=fail; b=cix5aMsvnvEIvrJzxmeOe25P+S9E0KopOSDgrPxgq7CoSrAPjuliIgAUZEsZdjqnrn2nCCiatvjalYkKlOiUJAiufrIoVnPDuZL7MGXa1e0IQjlTIOoDMAIe9qTvOS5wCO7HAWgthvRBPd4IPAozru+rrtWjMmPBYKXgW/3FIuU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724954895; c=relaxed/simple; bh=YOeYVX3big/iEv6NXA2LCQ8cqp1h3KfdwCDf2Ub3qDc=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Go0ek9QuEgVdeW4cHTZaNoTtO73HULuGe8uxkZuHjgXd/Ry9ZyYC1shiwmVARqsbmkER5ktRrC8rh9GD1NMMPd2v2Yze/RnilQHBvjoL8Dtb9fkMwUX/f+AvQvxgC+gAHEEgjCigysEhGxkEJcSNIJJE5Q9OX+jyiuQkMC6AIDk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=M4GlIvMH; arc=fail smtp.client-ip=40.107.236.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="M4GlIvMH" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=yef6d4xxUhXHGRfUTvSzkozAf0Ue5Oa2eqNxzyleZsN8mOznFBnv65ndIuLmZc87FNeIxto47CSPpR+BpL110OZY3nwpdJbTXsMG1crsm3aDT+h3rvv+E90YRk0gNIEnUFtX5qnrH2wGVJLnZVGTVr8KWKxBMS3GSdD0QRfhKKiK521e+Hqfugf7PR0eZXZ1b/4DEv/i6CzxJgIF9yLphCRH+OLZM09oCCJ0PNY2j3cZLn8Z/Ed8sU9rhBW40tewLBYP+2nt2wf98sMuXvpSdLIkqbMBOz0f3ghB/aGvKx2Y6G76RhhFlMyu/sgA/Xa/z94WHdXk7orJVgxPJPiyzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JOaIJ3hd5hQsLtfAk18DJ9r4mbIKWJpFMkiHV4clRCk=; b=oq8kzQPvh/k5QefTtKUkgD86PqjXVD1+ADuwvEPyOJ4csRLdNllFfl922qJAGRG5IKr6OAR61AjkEFuh7IxZE1vDeCdJ2LT+s8aEJxFSKyQJFpbY1LcTCdMwb2p3nrJbhs5/4kauasb/3YAFuXeLcE2Rcf3is3x4loLSieic8mqPBIBm2EZidTw6/QqP8df7MQgaz3+TNR/kM0Eudt2t0cBwGuye14ZMFrZAF5YG4H9oDW3eV6ZrUJLoqfeul/Kc/5dK1lmPBStcwgeasEpzjEYdv+ZJ+Ev2GWpBq3InWzv+s+h2JKkR7FJyxKG87ijSBzAceisxOur0iQldZwydqQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JOaIJ3hd5hQsLtfAk18DJ9r4mbIKWJpFMkiHV4clRCk=; b=M4GlIvMHDrIKMN/VYN6LHfq0OmnSx+R62yrvtKpu4brVKeChPuulmtyeosPlSlY2lE/76vyJLdhOpoFZR2i9O/TSq4AGU/+6/lFi1BP9xBnQAHnhUjcN+6HyXJFZJEljlXmopBwVX7IZNl2HQQEl7Daz6qJdZRG9CVyItXfqNbA= Received: from SA9PR10CA0027.namprd10.prod.outlook.com (2603:10b6:806:a7::32) by MW4PR12MB7482.namprd12.prod.outlook.com (2603:10b6:303:212::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.25; Thu, 29 Aug 2024 18:08:10 +0000 Received: from SA2PEPF00003F67.namprd04.prod.outlook.com (2603:10b6:806:a7:cafe::e8) by SA9PR10CA0027.outlook.office365.com (2603:10b6:806:a7::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.27 via Frontend Transport; Thu, 29 Aug 2024 18:08:09 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by SA2PEPF00003F67.mail.protection.outlook.com (10.167.248.42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7918.13 via Frontend Transport; Thu, 29 Aug 2024 18:08:08 +0000 Received: from purico-ed03host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Thu, 29 Aug 2024 13:08:03 -0500 From: Suravee Suthikulpanit To: , CC: , , , , , , , , , Suravee Suthikulpanit Subject: [PATCH v2 4/5] iommu/amd: Modify set_dte_entry() to use 256-bit DTE helpers Date: Thu, 29 Aug 2024 18:07:25 +0000 Message-ID: <20240829180726.5022-5-suravee.suthikulpanit@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240829180726.5022-1-suravee.suthikulpanit@amd.com> References: <20240829180726.5022-1-suravee.suthikulpanit@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF00003F67:EE_|MW4PR12MB7482:EE_ X-MS-Office365-Filtering-Correlation-Id: b1a5a978-8ac3-4920-21ef-08dcc85589db X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|376014|82310400026|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?A/f7fwUPBCP7vcfiBJf3an+z/FeBZyLVXek6QZ+QEfqzgaPPVYgEYUEf2+x0?= =?us-ascii?Q?wZiSPlTcksWQ0y0SaBkAHNzYkUyFJgx2UwSfs8gVJ9YQKwJA6pbs/yWoVPfv?= =?us-ascii?Q?HDefB5QUoBlGvRWOi2ar5wtyKd1KEvovjldbiINwYNZ48/zRUysvOERND/3t?= =?us-ascii?Q?QMddLi8VcWAaRLE7TjpD/QQojkRk1lCJINM+XSbns4ulHei5Zc3ZJfW07tR3?= =?us-ascii?Q?8ZGD0BzEmA1TxHOkoqqBoYMB6OVBtvdPt0Ib6AjsnCTcS21aiOHid20NHogF?= =?us-ascii?Q?ll3wCYOgkqVl4x42jSNmq1xSdNDW52X+RgO/cApSoUEhv2YdlGyx3JpX11gG?= =?us-ascii?Q?Za9wLcGj8Nk8KpnFjfUjq3Ii+BTNA6oDJWSK5JkoICjkuBAXGnvBs3CWzw0i?= =?us-ascii?Q?Ql1aLdyv0REwqWQmfCCG34ZdIfsbZhddJVg4cgYdCZMk4tEMKnsX15xH0ROr?= =?us-ascii?Q?xtGw2QDjGz5CA2gZurd/j/eiA6c3WpkACeA5Lz3DFDatMbXyokG89mamnmpS?= =?us-ascii?Q?GNsMPbnw68ONVW41J5oJH5yySEFKb7YNnGaWRSF6Tap0LTTPMGNSA1wysdbh?= =?us-ascii?Q?XNay1tpdX0wwO2NPAgSOllxAMhOP7MH8u8b4wadm6r6Tcar9rKMszMDVQzDf?= =?us-ascii?Q?b0dfxXtC/mf61/45Aeoh8/qX5r5D05z2zExdeMpClksGfRXuax6ynhQC1iAa?= =?us-ascii?Q?MrE3HzGU84qfBGMKhUgddnEzZ4S9VvTlyZmD6fVd+RuSsKCzJwNsinaS1oWJ?= =?us-ascii?Q?IZPN00pfQmQZnMQxM2LVg0jTWEF1nIIRfqTjR82G2ckZhvLe+HTOD/R8Eniq?= =?us-ascii?Q?PmxpX05hQ1GwTCnv8D1SR4AmgLaeNwRsx4ej0Fpscn+RepMRlpD+pM3RDDEh?= =?us-ascii?Q?lCIo8zxIwUkCWmJDbMqiqAjXESZuB7yTILSNtOe87+ON7aHAnXG0GWJ5ZlCf?= =?us-ascii?Q?j2cLX9wwKdNBUjqm4dG7trn6/a2y0HDn1y2jx+/dkafDa3bdgw8yyXTCg5pE?= =?us-ascii?Q?u+KLxyD1ui4WAtXwV8AOtuFDinOoruiMiMwxmHM7EBLNrnt6pa0hgLupGFyF?= =?us-ascii?Q?hrUyFMQ07NHMXksa8Hnk//rLETeguAsUVZMNJslKauLwJkypdAxxNk7+Rah8?= =?us-ascii?Q?KhD0csizgOEX8/S+TpxkVkPL6yeASN0uftWOEakXsTqvaIySzr4VAr74ZnQ/?= =?us-ascii?Q?3YTA2MmxzEhTSgIznksT7RwNAIUk1mx4HiIg8xEGmGmQt6PyZy+C5+C+Hl7f?= =?us-ascii?Q?BoFiHH1E76bqhXVbVQ5E4XULsEy6QGz/d7k85DXa+Jj7cKEaZiyHuxQB6jEk?= =?us-ascii?Q?FLvFvXJMSqvJ0WE8pSUtlZ0uRCQGwnMLNZj9ieECyk5fGQfIAKOjhyrTZ/i2?= =?us-ascii?Q?y2oIb2eR+aNTpjazjvxbAoaitkelk7Wzuz+93xXRCdv2c1o1Aq+4Nb0MkJhu?= =?us-ascii?Q?6JPQIpGuw1OZ1kuCCwikRrFuX6XEcOIx?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(376014)(82310400026)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Aug 2024 18:08:08.7726 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b1a5a978-8ac3-4920-21ef-08dcc85589db X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF00003F67.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7482 Content-Type: text/plain; charset="utf-8" Also, the set_dte_entry() is used to program several DTE fields (e.g. stage1 table, stage2 table, domain id, and etc.), which is difficult to keep track with current implementation. Therefore, separate logic for setting up the GCR3 Table Root Pointer, GIOV, GV, GLX, and GuestPagingMode into another helper function set_dte_gcr3_table(). Signed-off-by: Suravee Suthikulpanit --- drivers/iommu/amd/iommu.c | 121 +++++++++++++++++++++----------------- 1 file changed, 67 insertions(+), 54 deletions(-) diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index 93bca5c68bca..a24986c2478b 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -1905,16 +1905,56 @@ int amd_iommu_clear_gcr3(struct iommu_dev_data *dev= _data, ioasid_t pasid) return ret; } =20 +static void set_dte_gcr3_table(struct amd_iommu *iommu, + struct iommu_dev_data *dev_data, + struct dev_table_entry *target) +{ + struct gcr3_tbl_info *gcr3_info =3D &dev_data->gcr3_info; + u64 tmp, gcr3; + + if (!gcr3_info->gcr3_tbl) + return; + + pr_debug("%s: devid=3D%#x, glx=3D%#x, gcr3_tbl=3D%#llx\n", + __func__, dev_data->devid, gcr3_info->glx, + (unsigned long long)gcr3_info->gcr3_tbl); + + tmp =3D gcr3_info->glx; + target->data[0] |=3D (tmp & DTE_GLX_MASK) << DTE_GLX_SHIFT; + if (pdom_is_v2_pgtbl_mode(dev_data->domain)) + target->data[0] |=3D DTE_FLAG_GIOV; + target->data[0] |=3D DTE_FLAG_GV; + + /* First mask out possible old values for GCR3 table */ + tmp =3D DTE_GCR3_VAL_A(~0ULL) << DTE_GCR3_SHIFT_A; + target->data[0] &=3D ~tmp; + tmp =3D DTE_GCR3_VAL_B(~0ULL) << DTE_GCR3_SHIFT_B; + tmp |=3D DTE_GCR3_VAL_C(~0ULL) << DTE_GCR3_SHIFT_C; + target->data[1] &=3D ~tmp; + + gcr3 =3D iommu_virt_to_phys(gcr3_info->gcr3_tbl); + + /* Encode GCR3 table into DTE */ + tmp =3D DTE_GCR3_VAL_A(gcr3) << DTE_GCR3_SHIFT_A; + target->data[0] |=3D tmp; + tmp =3D DTE_GCR3_VAL_B(gcr3) << DTE_GCR3_SHIFT_B; + tmp |=3D DTE_GCR3_VAL_C(gcr3) << DTE_GCR3_SHIFT_C; + target->data[1] |=3D tmp; + + /* Mask out old values for GuestPagingMode */ + target->data[2] &=3D ~(0x3ULL << DTE_GPT_LEVEL_SHIFT); + /* Guest page table can only support 4 and 5 levels */ + if (amd_iommu_gpt_level =3D=3D PAGE_MODE_5_LEVEL) + target->data[2] |=3D ((u64)GUEST_PGTABLE_5_LEVEL << DTE_GPT_LEVEL_SHIFT); +} + static void set_dte_entry(struct amd_iommu *iommu, struct iommu_dev_data *dev_data) { - u64 pte_root =3D 0; - u64 flags =3D 0; - u32 old_domid; - u16 devid =3D dev_data->devid; u16 domid; + u32 old_domid; + struct dev_table_entry new; struct protection_domain *domain =3D dev_data->domain; - struct dev_table_entry *dev_table =3D get_dev_table(iommu); struct gcr3_tbl_info *gcr3_info =3D &dev_data->gcr3_info; =20 if (gcr3_info && gcr3_info->gcr3_tbl) @@ -1922,73 +1962,46 @@ static void set_dte_entry(struct amd_iommu *iommu, else domid =3D domain->id; =20 + /* + * Need to preserve the certain fields in DTE because it contains + * interrupt-remapping and other settings, which might be + * programmed earlier by other code. + */ + get_dte256(iommu, dev_data, &new); + if (domain->iop.mode !=3D PAGE_MODE_NONE) - pte_root =3D iommu_virt_to_phys(domain->iop.root); + new.data[0] =3D iommu_virt_to_phys(domain->iop.root); =20 - pte_root |=3D (domain->iop.mode & DEV_ENTRY_MODE_MASK) + new.data[0] |=3D (domain->iop.mode & DEV_ENTRY_MODE_MASK) << DEV_ENTRY_MODE_SHIFT; =20 - pte_root |=3D DTE_FLAG_IR | DTE_FLAG_IW | DTE_FLAG_V; + new.data[0] |=3D DTE_FLAG_IR | DTE_FLAG_IW | DTE_FLAG_V; =20 /* * When SNP is enabled, Only set TV bit when IOMMU * page translation is in use. */ if (!amd_iommu_snp_en || (domid !=3D 0)) - pte_root |=3D DTE_FLAG_TV; - - flags =3D dev_table[devid].data[1]; - - if (dev_data->ats_enabled) - flags |=3D DTE_FLAG_IOTLB; + new.data[0] |=3D DTE_FLAG_TV; =20 if (dev_data->ppr) - pte_root |=3D 1ULL << DEV_ENTRY_PPR; + new.data[0] |=3D 1ULL << DEV_ENTRY_PPR; =20 if (domain->dirty_tracking) - pte_root |=3D DTE_FLAG_HAD; - - if (gcr3_info && gcr3_info->gcr3_tbl) { - u64 gcr3 =3D iommu_virt_to_phys(gcr3_info->gcr3_tbl); - u64 glx =3D gcr3_info->glx; - u64 tmp; - - pte_root |=3D DTE_FLAG_GV; - pte_root |=3D (glx & DTE_GLX_MASK) << DTE_GLX_SHIFT; - - /* First mask out possible old values for GCR3 table */ - tmp =3D DTE_GCR3_VAL_B(~0ULL) << DTE_GCR3_SHIFT_B; - flags &=3D ~tmp; + new.data[0] |=3D DTE_FLAG_HAD; =20 - tmp =3D DTE_GCR3_VAL_C(~0ULL) << DTE_GCR3_SHIFT_C; - flags &=3D ~tmp; - - /* Encode GCR3 table into DTE */ - tmp =3D DTE_GCR3_VAL_A(gcr3) << DTE_GCR3_SHIFT_A; - pte_root |=3D tmp; - - tmp =3D DTE_GCR3_VAL_B(gcr3) << DTE_GCR3_SHIFT_B; - flags |=3D tmp; - - tmp =3D DTE_GCR3_VAL_C(gcr3) << DTE_GCR3_SHIFT_C; - flags |=3D tmp; - - if (amd_iommu_gpt_level =3D=3D PAGE_MODE_5_LEVEL) { - dev_table[devid].data[2] |=3D - ((u64)GUEST_PGTABLE_5_LEVEL << DTE_GPT_LEVEL_SHIFT); - } + if (dev_data->ats_enabled) + new.data[1] |=3D DTE_FLAG_IOTLB; + else + new.data[1] &=3D ~DTE_FLAG_IOTLB; =20 - /* GIOV is supported with V2 page table mode only */ - if (pdom_is_v2_pgtbl_mode(domain)) - pte_root |=3D DTE_FLAG_GIOV; - } + old_domid =3D new.data[1] & DEV_DOMID_MASK; + new.data[1] &=3D ~DEV_DOMID_MASK; + new.data[1] |=3D domid; =20 - flags &=3D ~DEV_DOMID_MASK; - flags |=3D domid; + set_dte_gcr3_table(iommu, dev_data, &new); =20 - old_domid =3D dev_table[devid].data[1] & DEV_DOMID_MASK; - dev_table[devid].data[1] =3D flags; - dev_table[devid].data[0] =3D pte_root; + update_dte256(iommu, dev_data, &new); =20 /* * A kdump kernel might be replacing a domain ID that was copied from --=20 2.34.1 From nobody Sun Feb 8 06:56:06 2026 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2088.outbound.protection.outlook.com [40.107.212.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D622D1B81AE for ; Thu, 29 Aug 2024 18:08:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.212.88 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724954902; cv=fail; b=ZGO5bHJobzTqZFyisSHdhYy/7+qLWdZkrnbfyWvYR7mwYCB5Bwhr5Sv1dGNAfOZ2YLDjuBp4Kz76yLIejjDCrz/sLS16dTnfcjeSgPhKRvl2qf373aXqNjAB7g09UMSiQ+5cyW7z5s5HzWnf40FM3R+LSPziHHvZLt1lIrPiKko= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724954902; c=relaxed/simple; bh=QQeUaxo9BcJQdBNyLUwTRqO0jBCk7Gtjx3YfD/RChoc=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=vApNTt0/U4vdTsFmNWB3i7JVrWDvNgI/sinHKeG0mYZaNn5YyEHCHgeP1i+eOossKNgaylttOEK+x+gusDdNzLEQsA+DliM6KTMZBd1BmnBbXkhNKbv9s/25xPviF38FuwV4ewuzLlV9xz9sS+sOqrt5wZJzePRF/Xkgop6qA2I= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=jrG/HZNE; arc=fail smtp.client-ip=40.107.212.88 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="jrG/HZNE" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=fRgJ1MIWCmow6xUxmycjFcWO2mqP56fSRwkN3hbY9udjZVZ0U7uFAok9TPzfoD1bUJjoK5/LsNbh6QY+eC/Mpepj+co0TKdC7c/vOmJ0UqiTvUZz6CX5lmSWGIDShm/NKCgrJ1h39VDUMUqqAeOiLCo6gZGVqjZ1lr1MmL2SkjMWDuEOVNxCnaEHQColEjkSuBMjv5PC2JJvnI3SNYh5rNTqNT6yJVGK3OI8Lgxfz9ZkESfQFuX1b0Rtk1Br6zdl9nah+8DoHuh54rJcDEsIDPutmEaqzhD7S1Zb2AESuAOi3gcvMFA/SkeVTVjxzbCUlHvxRf8/9mNlLvEsaQvtsw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vek6uzcrlkQrVm1VPeOyDMoTvLA+7kc9SnzCJZUUGNw=; b=YY9hipx5BOMBarjs/g/bZMbyJz0nNot1SjcGURqg/f51ajoNILpWnpdWu++25hr+3tAJfPR/6aZjTsy+Vf/shaCm9znSJc0OT1nCqqCrejaTjZ32tBjcQyttCpKNs2ms/LkSdQuWgQCES7jGPTk7AXxhLiFvnMSLPI7SUt6HiXATEiKjKlVA0ccUS1YTopUDZ/bOMFFHziC/oUZagerdrNN5mPvdXnXYAKBHIyrq9Zzx7l2+/rgLBlB4v8G5dZMBXUwDuW8crXCm6aSpFqdeFy8+V2jAp9M0WDVe96m85kulfpIihHogBdsFoP8z+uprvbX9dhF+Jgt2W3GjSX9HqQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vek6uzcrlkQrVm1VPeOyDMoTvLA+7kc9SnzCJZUUGNw=; b=jrG/HZNEWkqzqSdrl3lZ77bXmtM5zu0QPhFSOTIwJ/KtCd3YVeROC13NPGsjw76z/9CJLMlQpdKd82ts40V4vvGK9DzS/LzzUoanm6Ozzj5A8sDhOEErypa2nbQwfwA1FxXgsZMqSfF5enkki0mW8XnmrPtBVz6TRUsQpxjpgUo= Received: from SA9PR10CA0026.namprd10.prod.outlook.com (2603:10b6:806:a7::31) by IA0PR12MB8694.namprd12.prod.outlook.com (2603:10b6:208:488::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.20; Thu, 29 Aug 2024 18:08:17 +0000 Received: from SA2PEPF00003F67.namprd04.prod.outlook.com (2603:10b6:806:a7:cafe::b5) by SA9PR10CA0026.outlook.office365.com (2603:10b6:806:a7::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.28 via Frontend Transport; Thu, 29 Aug 2024 18:08:17 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by SA2PEPF00003F67.mail.protection.outlook.com (10.167.248.42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7918.13 via Frontend Transport; Thu, 29 Aug 2024 18:08:17 +0000 Received: from purico-ed03host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Thu, 29 Aug 2024 13:08:07 -0500 From: Suravee Suthikulpanit To: , CC: , , , , , , , , , Suravee Suthikulpanit Subject: [PATCH v2 5/5] iommu/amd: Use 128-bit cmpxchg in set_dte_irq_entry() Date: Thu, 29 Aug 2024 18:07:26 +0000 Message-ID: <20240829180726.5022-6-suravee.suthikulpanit@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240829180726.5022-1-suravee.suthikulpanit@amd.com> References: <20240829180726.5022-1-suravee.suthikulpanit@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF00003F67:EE_|IA0PR12MB8694:EE_ X-MS-Office365-Filtering-Correlation-Id: 49aaa959-2f01-4d47-d3c8-08dcc8558efc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|82310400026|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?EbKql+TREVSBuYT0gTjvzQaWTd96OqVegWf7U3JejAtuEEDDNXaqf3+vwgWV?= =?us-ascii?Q?2t2U7xLJcREHJvV/XUoKqNQV14wmVc26tNVYBMkcgy4nLZiAjiV9E2tMUu48?= =?us-ascii?Q?klX8LDSqt7oKeqKgdEEkei9AXyABRNPmU+++Pyy/FLs/h/sVT50gJ8eqNfgp?= =?us-ascii?Q?5K5DGtMXPvZlqCBMV9dyOmrVMybVg0vyq2u5DdEAT1XYBlSzJKaS6ctbZXHx?= =?us-ascii?Q?DS5GlNmhAKsXUgoIXdPzeuEh8ayIE9O3A578EDTYlOaclPrxY6PBS8KINhgx?= =?us-ascii?Q?owclNEQoPFuXp+oUddT41noBj0TEWuIoq+g4TO/CPtNtBMscXrELNoOlcVHS?= =?us-ascii?Q?iELzbomjsr/u0ABgFGKh0LqEMcgQRhTKfcMOF8KHCdsPCl7mwenVx1MKO1dL?= =?us-ascii?Q?J+AI1s8pht2FJ4w9HMsfrNSWxbuG07l1IKFzhoa2G4leZzoyN7eIsC0abpIn?= =?us-ascii?Q?oSFg/ufoU2oHY8COJ2PMJD+g0cXL7dBusQJI+dBg4ma6mTBpcg1egRX/xB3l?= =?us-ascii?Q?3ixY5yH2WyYAj3TE6guXTIOTqfGmqnD33tUUom0akZst5EAJqcBPpOxMF2+z?= =?us-ascii?Q?GVn7O2c3GJtfagQ4pHMy/F8ZGf03M4LAjHG60WgsaIQN7btov7VnaaJnJY+V?= =?us-ascii?Q?hRl+E5MPyCa2oUl1S8LC+xIIicvk/1kbnApbiiBfTyvp0ohvHeecTjVdwJ7p?= =?us-ascii?Q?3pSteJKFhmmCRfMvEbcGjySOZRzvlv2s4RaH+UzKWYXNvQYljSUnwtcbO0Nf?= =?us-ascii?Q?30USomckXrkHtYbxqS7QPKGfLVlSvtjHBtIM46d5I7/Ho0Y+wXBc7sMF3blx?= =?us-ascii?Q?02Buz2aS3klM23ODkZ+EvHbvrl2iaq5GcOFelh6rIfd6JVgKy/88wj9PpZdR?= =?us-ascii?Q?y6rCzj/sqD10iO7UHF0x1KEw20T3hg7+cp40zu8AhCN2G/SLpPqj7QniCUP9?= =?us-ascii?Q?aYCjJPswI/SWPDpCGsXBOqQed+phmf6j/7/Rp9Ky9xB9d5o5I8hMbCkgZIdf?= =?us-ascii?Q?SerLz9p6mGBiLXysHH6Mbp+6FIZSfYJhkfbQ95svhnod1sIyKWEiEvtCq6jG?= =?us-ascii?Q?X044vgTWP7a29EKhmb8Kwj8+4G1EqW+iiAqawSZjfYPzkJGChm77tgtO2YyI?= =?us-ascii?Q?ELOZorjKSlwlpVYm2WA6rQkMyPKMURD2BS5rpuQa9qxiMi1vdei0uL0zIu3X?= =?us-ascii?Q?OT84JbFLA4au2YxXas3LNBJf52HiuSVqAaNKrjZYBdpRm9yyGDGnGVp9ygy3?= =?us-ascii?Q?O6buWSXV1J9uVk1/LusksIWjq2YZNgonWyfB0A3rxXBVAiU6+rdriR8yYKMP?= =?us-ascii?Q?b8Mx8FGgJH3RgRHEo0fP+rp7Mjg0O4d0USXnCP+MMbBtu/RIG3uGGnd1d2Lg?= =?us-ascii?Q?hwHNR+SQQGrnDkHWMlNlL+IapagqLwu2jjTF1sMTjYiAYYX60HJNPOgTD4BM?= =?us-ascii?Q?5bjd5t2MKB1LsAMCKL02K+V91R/quhgD?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(82310400026)(376014)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Aug 2024 18:08:17.3820 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 49aaa959-2f01-4d47-d3c8-08dcc8558efc X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF00003F67.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8694 Content-Type: text/plain; charset="utf-8" Interrupt-remapping-related fields are in the top 128-bit of the Device Table Entry (DTE), which should be updated using 128-bit write based on the AMD I/O Virtualization Techonology (IOMMU) Specification. Therefore, modify set_dte_irq_entry() to use 128-bit cmpxchg. Also, use struct dev_data->dte_sem to synchronize DTE access. Suggested-by: Jason Gunthorpe Signed-off-by: Suravee Suthikulpanit --- drivers/iommu/amd/iommu.c | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index a24986c2478b..4eb53bd40487 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -3020,17 +3020,24 @@ static void iommu_flush_irt_and_complete(struct amd= _iommu *iommu, u16 devid) static void set_dte_irq_entry(struct amd_iommu *iommu, u16 devid, struct irq_remap_table *table) { - u64 dte; - struct dev_table_entry *dev_table =3D get_dev_table(iommu); + u128 new, old; + struct dev_table_entry *dte =3D &get_dev_table(iommu)[devid]; + struct iommu_dev_data *dev_data =3D search_dev_data(iommu, devid); + + if (dev_data) + down_write(&dev_data->dte_sem); + + old =3D new =3D dte->data128[1]; + new &=3D ~DTE_IRQ_PHYS_ADDR_MASK; + new |=3D iommu_virt_to_phys(table->table); + new |=3D DTE_IRQ_REMAP_INTCTL; + new |=3D DTE_INTTABLEN; + new |=3D DTE_IRQ_REMAP_ENABLE; =20 - dte =3D dev_table[devid].data[2]; - dte &=3D ~DTE_IRQ_PHYS_ADDR_MASK; - dte |=3D iommu_virt_to_phys(table->table); - dte |=3D DTE_IRQ_REMAP_INTCTL; - dte |=3D DTE_INTTABLEN; - dte |=3D DTE_IRQ_REMAP_ENABLE; + WARN_ON(!try_cmpxchg128(&dte->data128[1], &old, new)); =20 - dev_table[devid].data[2] =3D dte; + if (dev_data) + up_write(&dev_data->dte_sem); } =20 static struct irq_remap_table *get_irq_table(struct amd_iommu *iommu, u16 = devid) --=20 2.34.1