From nobody Wed Feb 5 16:05:58 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+101009+1787277+3901457@groups.io; arc=fail (BodyHash is different from the expected one) Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 1678467870287967.6210931361815; Fri, 10 Mar 2023 09:04:30 -0800 (PST) Return-Path: X-Received: by 127.0.0.2 with SMTP id N8UDYY1788612xjcC1l0bVv2; Fri, 10 Mar 2023 09:04:29 -0800 X-Received: from NAM11-DM6-obe.outbound.protection.outlook.com (NAM11-DM6-obe.outbound.protection.outlook.com [40.107.223.67]) by mx.groups.io with SMTP id smtpd.web10.24918.1678467868966275614 for ; Fri, 10 Mar 2023 09:04:29 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LoZC0SWEkNbDZE2AxHnxS0Dyj0cLZtCUtUVK9AqVBnqxL5dd6QZvJQpORvb+QCY+xelPJztEBiShVKTT8zfxO86h6BWKxoLkSX8BxK/9ZEhZXNElUU0iEQQExhC6xvwZ34q9EShlhrdwCN4gkTVS/CYZICK1bCQksJ33nFkCrdLaW8M8lcNUGqRrrCVN4crmIxeR6KuS+x/bN2Jn/bAM8RxtLA7z3qvLgcCETX6d7qd45A+6ukDH8ZTN3USxjtW9lwyNsaUy40Ct1L0s72j2DLAucgaCj9NIzFIC6IXA7TqzCxdEVnpOe49t7a9AMWikazIivDx882otCGrjtakZbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yehpRBij1qDg/pv7is033EUsFJP23rLydLPYq4YKc/4=; b=YejsCnOLaO5ohXCHZgeTEA5uyaKeDmHnITgjBdobBATrvVLDVcgixUZU1iklE4lkrB5NaLOCQgRTTCCnmpvkKV2Pih1D7uAjtZClAqVTm8IgLLJfIdRENerafc4diV1BbuSSNPAVkY0lzAY0cyszLuYxIsEImrm0zzRFQCXd9wx9noaW20U0ohP99RyaBHKqqf3jFbleuiWONyWqpIp6QbonJMeMU6pdN21pO0gGkNxjQ5etdFJMcBTaaNQK7jGyJeo0YZ01eLZYouB9QhA+nwF/OIBQtjU35xUE04l8xHiNDje8ZVPPWJg7iVuAnaQn6mGCrA6QlrWsSvDW0Qki2w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=edk2.groups.io smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none X-Received: from BN9PR03CA0569.namprd03.prod.outlook.com (2603:10b6:408:138::34) by CH0PR12MB8550.namprd12.prod.outlook.com (2603:10b6:610:192::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.19; Fri, 10 Mar 2023 17:04:25 +0000 X-Received: from BL02EPF00010209.namprd05.prod.outlook.com (2603:10b6:408:138:cafe::67) by BN9PR03CA0569.outlook.office365.com (2603:10b6:408:138::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.20 via Frontend Transport; Fri, 10 Mar 2023 17:04:25 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+101009+1787277+3901457@groups.io; helo=mail02.groups.io; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C X-Received: from SATLEXMB04.amd.com (165.204.84.17) by BL02EPF00010209.mail.protection.outlook.com (10.167.241.198) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6178.13 via Frontend Transport; Fri, 10 Mar 2023 17:04:25 +0000 X-Received: from tlendack-t1.amdoffice.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 10 Mar 2023 11:04:24 -0600 From: "Lendacky, Thomas via groups.io" To: CC: Eric Dong , Ray Ni , Rahul Kumar , Gerd Hoffmann , Michael Roth , Ashish Kalra Subject: [edk2-devel] [PATCH 2/2] UefiCpuPkg/MpInitLib: Reuse VMSA allocation to avoid unreserved allocation Date: Fri, 10 Mar 2023 11:04:00 -0600 Message-ID: <7054ab9c8fb279819b7837e7958d2bc5b78dff5d.1678467840.git.thomas.lendacky@amd.com> In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF00010209:EE_|CH0PR12MB8550:EE_ X-MS-Office365-Filtering-Correlation-Id: 36732c86-dca3-460c-6051-08db21898062 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Message-Info: SLdPBDUc+ECKmRuBz5lK3qqpJpansrZIxtm7qr7yclecl20K7Gj8JVrr7x6T/r64ROPS7+NDm3DpGY4SxkO+lfpPP8RHeeo1U6sQJRGVWQXUlns3uZyIfa+xsMiJzZverUm61RcedKGrq1r5L2XW1/aKg+tvZbY1Uq2/7hX/48LvtsMyzIOSTdL6R8JaJwZd2v5ywWbpiiLsRHkLWof85vr6nrRcqTmUksbP2UNm7/hIYVbb2PzIFJI6OP0Ih2Wvr42AxCu4K+VyJ2ooaf+iQasw/A53q7zLNLkaroEP4tr8rBbTE1pgZPV60Iu44Q9bLQzkrj958ETTRa8+19BFywXbUT93r3WUpTqhYk4tDQMFAUb76nw3Wpztdrm83D7hurbyhd892dqYJcZ91nW1k5nhIYRk19k352D5kbwLdnmxe+tdyy4a6JdN7iHXqAPucQz0qLau/vx+KKtBjLmMnfkW+xV7pmjs1TfQKeIp5LIf9RtS3iIDZM/ryMoXRUOITPnL150d8unvAfCW9P03n933TeebVCEw2lwc6C22r3fUIoaGrPtF4Y3ghJGydeQetW2f1Xb5j5n5q4z5g+cNqZfmOXMHumfwBlBV/zAwRRSdmm/QK2bzyn9nDb9UdTR7fj00KXIhc+YA2uE/lw9cnRozNzMiDegUYBemSWGNXK3AQ/MzgGQc7u7Vg8riOeXSsDcIIpb86KuZJ10ykNMLSH+KbwVe3r1e6rDZ+ZCliUI= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Mar 2023 17:04:25.0755 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 36732c86-dca3-460c-6051-08db21898062 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF00010209.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR12MB8550 Precedence: Bulk List-Unsubscribe: List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,thomas.lendacky@amd.com X-Gm-Message-State: aUvmg4HbR9kGLc4aVFTfhHMIx1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1678467869; bh=Ezz+DU9qqHjYNONGduDohMnioUQpKr3HEGhzytebcgI=; h=CC:Content-Type:Date:From:Reply-To:Subject:To; b=alDL3rPaKvRxNTwKiJSyB2hYpz5FtSxz39vl5FpL9NVJLS/c18Mj0TS01qghrnTdx53 DLIAcZZLMmVwOlxbIGOi9SxfYq+j0oXXm6CnMoVaNTWhhzQixrHb//EzedJMNLB/Qty78 M79hkNbBqgdJqoQoUKu6dCNaVGQN1agPA7c= X-ZohoMail-DKIM: pass (identity @groups.io) X-ZM-MESSAGEID: 1678467872152100002 Content-Type: text/plain; charset="utf-8" https://bugzilla.tianocore.org/show_bug.cgi?id=3D4353 When parking the APs on exiting from UEFI, a new page allocation is made. This allocation, however, does not end up being marked reserved in the memory map supplied to the OS. To avoid this, re-use the VMSA by clearing the VMSA RMP flag, updating the page contents and re-setting the VMSA RMP flag. Fixes: 06544455d0d4 ("UefiCpuPkg/MpInitLib: Use SEV-SNP AP Creation ...") Signed-off-by: Tom Lendacky --- UefiCpuPkg/Library/MpInitLib/X64/AmdSev.c | 234 +++++++++++++--------- 1 file changed, 139 insertions(+), 95 deletions(-) diff --git a/UefiCpuPkg/Library/MpInitLib/X64/AmdSev.c b/UefiCpuPkg/Library= /MpInitLib/X64/AmdSev.c index 7abdda3e1c7e..ae88bbbfd828 100644 --- a/UefiCpuPkg/Library/MpInitLib/X64/AmdSev.c +++ b/UefiCpuPkg/Library/MpInitLib/X64/AmdSev.c @@ -16,58 +16,158 @@ #define IS_ALIGNED(x, y) ((((UINTN)(x) & (y - 1)) =3D=3D 0)) =20 /** - Create an SEV-SNP AP save area (VMSA) for use in running the vCPU. + Perform the requested AP Creation action. =20 - @param[in] CpuMpData Pointer to CPU MP Data - @param[in] CpuData Pointer to CPU AP Data + @param[in] SaveArea Pointer to VM save area (VMSA) @param[in] ApicId APIC ID of the vCPU + @param[in] Action AP action to perform + + @retval TRUE Action completed successfully + @retval FALSE Action did not complete successfully **/ -VOID -SevSnpCreateSaveArea ( - IN CPU_MP_DATA *CpuMpData, - IN CPU_AP_DATA *CpuData, - UINT32 ApicId +STATIC +BOOLEAN +SevSnpPerformApAction ( + IN SEV_ES_SAVE_AREA *SaveArea, + IN UINT32 ApicId, + IN UINTN Action ) { - UINT8 *Pages; - SEV_ES_SAVE_AREA *SaveArea; - IA32_CR0 ApCr0; - IA32_CR0 ResetCr0; - IA32_CR4 ApCr4; - IA32_CR4 ResetCr4; - UINTN StartIp; - UINT8 SipiVector; - UINT32 RmpAdjustStatus; - UINT64 VmgExitStatus; MSR_SEV_ES_GHCB_REGISTER Msr; GHCB *Ghcb; BOOLEAN InterruptState; UINT64 ExitInfo1; UINT64 ExitInfo2; + UINT64 VmgExitStatus; + UINT32 RmpAdjustStatus; =20 - // - // Allocate a single page for the SEV-ES Save Area and initialize it. - // Due to an erratum that prevents a VMSA being on a 2MB boundary, - // allocate an extra page to work around the issue. - // - Pages =3D AllocateReservedPages (2); - if (!Pages) { - return; + if (Action =3D=3D SVM_VMGEXIT_SNP_AP_CREATE) { + // + // To turn the page into a recognized VMSA page, issue RMPADJUST: + // Target VMPL but numerically higher than current VMPL + // Target PermissionMask is not used + // + RmpAdjustStatus =3D SevSnpRmpAdjust ( + (EFI_PHYSICAL_ADDRESS)(UINTN)SaveArea, + TRUE + ); + if (RmpAdjustStatus !=3D 0) { + DEBUG ((DEBUG_INFO, "SEV-SNP: RMPADJUST failed for VMSA creation\n")= ); + ASSERT (FALSE); + + return FALSE; + } + } + + ExitInfo1 =3D (UINT64)ApicId << 32; + ExitInfo1 |=3D Action; + ExitInfo2 =3D (UINT64)(UINTN)SaveArea; + + Msr.GhcbPhysicalAddress =3D AsmReadMsr64 (MSR_SEV_ES_GHCB); + Ghcb =3D Msr.Ghcb; + + CcExitVmgInit (Ghcb, &InterruptState); + + if (Action =3D=3D SVM_VMGEXIT_SNP_AP_CREATE) { + Ghcb->SaveArea.Rax =3D SaveArea->SevFeatures; + CcExitVmgSetOffsetValid (Ghcb, GhcbRax); } =20 - // - // Since page allocation works by allocating downward in the address spa= ce, - // try to always free the first (lower address) page to limit possible h= oles - // in the memory map. So, if the address of the second page is 2MB align= ed, - // then use the first page and free the second page. Otherwise, free the - // first page and use the second page. - // - if (IS_ALIGNED (Pages + EFI_PAGE_SIZE, SIZE_2MB)) { - SaveArea =3D (SEV_ES_SAVE_AREA *)Pages; - FreePages (Pages + EFI_PAGE_SIZE, 1); + VmgExitStatus =3D CcExitVmgExit ( + Ghcb, + SVM_EXIT_SNP_AP_CREATION, + ExitInfo1, + ExitInfo2 + ); + + CcExitVmgDone (Ghcb, InterruptState); + + if (VmgExitStatus !=3D 0) { + DEBUG ((DEBUG_INFO, "SEV-SNP: AP Destroy failed\n")); + ASSERT (FALSE); + + return FALSE; + } + + if (Action =3D=3D SVM_VMGEXIT_SNP_AP_DESTROY) { + // + // Make the current VMSA not runnable and accessible to be + // reprogrammed. + // + RmpAdjustStatus =3D SevSnpRmpAdjust ( + (EFI_PHYSICAL_ADDRESS)(UINTN)SaveArea, + FALSE + ); + if (RmpAdjustStatus !=3D 0) { + DEBUG ((DEBUG_INFO, "SEV-SNP: RMPADJUST failed for VMSA reset\n")); + ASSERT (FALSE); + + return FALSE; + } + } + + return TRUE; +} + +/** + Create an SEV-SNP AP save area (VMSA) for use in running the vCPU. + + @param[in] CpuMpData Pointer to CPU MP Data + @param[in] CpuData Pointer to CPU AP Data + @param[in] ApicId APIC ID of the vCPU +**/ +VOID +SevSnpCreateSaveArea ( + IN CPU_MP_DATA *CpuMpData, + IN CPU_AP_DATA *CpuData, + UINT32 ApicId + ) +{ + UINT8 *Pages; + SEV_ES_SAVE_AREA *SaveArea; + IA32_CR0 ApCr0; + IA32_CR0 ResetCr0; + IA32_CR4 ApCr4; + IA32_CR4 ResetCr4; + UINTN StartIp; + UINT8 SipiVector; + + if (CpuData->SevEsSaveArea =3D=3D NULL) { + // + // Allocate a single page for the SEV-ES Save Area and initialize it. + // Due to an erratum that prevents a VMSA being on a 2MB boundary, + // allocate an extra page to work around the issue. + // + Pages =3D AllocateReservedPages (2); + if (!Pages) { + return; + } + + // + // Since page allocation works by allocating downward in the address s= pace, + // try to always free the first (lower address) page to limit possible= holes + // in the memory map. So, if the address of the second page is 2MB ali= gned, + // then use the first page and free the second page. Otherwise, free t= he + // first page and use the second page. + // + if (IS_ALIGNED (Pages + EFI_PAGE_SIZE, SIZE_2MB)) { + SaveArea =3D (SEV_ES_SAVE_AREA *)Pages; + FreePages (Pages + EFI_PAGE_SIZE, 1); + } else { + SaveArea =3D (SEV_ES_SAVE_AREA *)(Pages + EFI_PAGE_SIZE); + FreePages (Pages, 1); + } + + CpuData->SevEsSaveArea =3D SaveArea; } else { - SaveArea =3D (SEV_ES_SAVE_AREA *)(Pages + EFI_PAGE_SIZE); - FreePages (Pages, 1); + SaveArea =3D CpuData->SevEsSaveArea; + + // + // Tell the hypervisor to not use the current VMSA + // + if (!SevSnpPerformApAction (SaveArea, ApicId, SVM_VMGEXIT_SNP_AP_DESTR= OY)) { + return; + } } =20 ZeroMem (SaveArea, EFI_PAGE_SIZE); @@ -152,63 +252,7 @@ SevSnpCreateSaveArea ( SaveArea->Vmpl =3D 0; SaveArea->SevFeatures =3D AsmReadMsr64 (MSR_SEV_STATUS) >> 2; =20 - // - // To turn the page into a recognized VMSA page, issue RMPADJUST: - // Target VMPL but numerically higher than current VMPL - // Target PermissionMask is not used - // - RmpAdjustStatus =3D SevSnpRmpAdjust ( - (EFI_PHYSICAL_ADDRESS)(UINTN)SaveArea, - TRUE - ); - ASSERT (RmpAdjustStatus =3D=3D 0); - - ExitInfo1 =3D (UINT64)ApicId << 32; - ExitInfo1 |=3D SVM_VMGEXIT_SNP_AP_CREATE; - ExitInfo2 =3D (UINT64)(UINTN)SaveArea; - - Msr.GhcbPhysicalAddress =3D AsmReadMsr64 (MSR_SEV_ES_GHCB); - Ghcb =3D Msr.Ghcb; - - CcExitVmgInit (Ghcb, &InterruptState); - Ghcb->SaveArea.Rax =3D SaveArea->SevFeatures; - CcExitVmgSetOffsetValid (Ghcb, GhcbRax); - VmgExitStatus =3D CcExitVmgExit ( - Ghcb, - SVM_EXIT_SNP_AP_CREATION, - ExitInfo1, - ExitInfo2 - ); - CcExitVmgDone (Ghcb, InterruptState); - - ASSERT (VmgExitStatus =3D=3D 0); - if (VmgExitStatus !=3D 0) { - RmpAdjustStatus =3D SevSnpRmpAdjust ( - (EFI_PHYSICAL_ADDRESS)(UINTN)SaveArea, - FALSE - ); - if (RmpAdjustStatus =3D=3D 0) { - FreePages (SaveArea, 1); - } else { - DEBUG ((DEBUG_INFO, "SEV-SNP: RMPADJUST failed, leaking VMSA page\n"= )); - } - - SaveArea =3D NULL; - } - - if (CpuData->SevEsSaveArea) { - RmpAdjustStatus =3D SevSnpRmpAdjust ( - (EFI_PHYSICAL_ADDRESS)(UINTN)CpuData->SevEsSaveAre= a, - FALSE - ); - if (RmpAdjustStatus =3D=3D 0) { - FreePages (CpuData->SevEsSaveArea, 1); - } else { - DEBUG ((DEBUG_INFO, "SEV-SNP: RMPADJUST failed, leaking VMSA page\n"= )); - } - } - - CpuData->SevEsSaveArea =3D SaveArea; + SevSnpPerformApAction (SaveArea, ApicId, SVM_VMGEXIT_SNP_AP_CREATE); } =20 /** --=20 2.39.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#101009): https://edk2.groups.io/g/devel/message/101009 Mute This Topic: https://groups.io/mt/97524223/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-