From nobody Wed May 15 00:48:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+110780+1787277+3901457@groups.io; helo=mail02.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+110780+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1699320292; cv=none; d=zohomail.com; s=zohoarc; b=HTpAZjINwn0bgwAh92hFxh5AmxVIzXxcxQS9hKf1X5jBZY87eX1UXd5+QjerGVk2HyPHGvTZDIECyX3xK39sZr1SJaAAgs0O1jAp7+u/ehFfEy9WLwqZFGsz1xQQv7BN3suZJ7Zz/NZJCztdfmhkCM19q5Q17ZhW2r7xcOfRhvw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1699320292; h=Cc:Cc:Date:Date:From:From:List-Subscribe:List-Id:List-Help:List-Unsubscribe:Message-ID:Reply-To:Reply-To:Sender:Subject:Subject:To:To:Message-Id; bh=AVpz6Up5rTPVA9iGNlmxC2YFOt72REtSwXz4OxHABdY=; b=CUeWYsDTUXoOBZIhHEbwxoeV4VMXcLT6uqh4R+eQNg6ZZlm0ntZsEe+1mPp9LZ4k3bx0vg1TGUcMcLznh3yLshE4VEJRQftWyefdYBR5BDUcXT8h+C/Ks4W3UFGRHIqMf7/6sE3021oY0jjqvBQbIf/y/vUy34VejXEaqY7ae54= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+110780+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 169932029250536.24414859400349; Mon, 6 Nov 2023 17:24:52 -0800 (PST) Return-Path: DKIM-Signature: a=rsa-sha256; bh=lFLsKn5N6Cwy5J1ARYiuPlqdZgz3hvLi5gCHQ6XDnzQ=; c=relaxed/simple; d=groups.io; h=From:To:Cc:Subject:Date:Message-Id:Precedence:List-Subscribe:List-Help:Sender:List-Id:Mailing-List:Delivered-To:Reply-To:List-Unsubscribe-Post:List-Unsubscribe; s=20140610; t=1699320292; v=1; b=W76t+ON8aA2moGfMFLguH0s4cRlIFMdi/e4Hx6Wegw6YHI50rMcP9qrOVZzDSKF1lCcPeySQ sslNeYNjAx/6BA3ALWpO9G+w0ZMyZob3YJ0gyMlaCrsubvVm3zrycImqJp49Xm6IKvTXeORJaDs tvkec1ha8cWgbP4rEpoVOFjg= X-Received: by 127.0.0.2 with SMTP id m3ShYY1788612x8kPDjlp5z6; Mon, 06 Nov 2023 17:24:52 -0800 X-Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) by mx.groups.io with SMTP id smtpd.web11.651.1699320290983799215 for ; Mon, 06 Nov 2023 17:24:51 -0800 X-IronPort-AV: E=McAfee;i="6600,9927,10886"; a="10946430" X-IronPort-AV: E=Sophos;i="6.03,282,1694761200"; d="scan'208";a="10946430" X-Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Nov 2023 17:24:50 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10886"; a="906241544" X-IronPort-AV: E=Sophos;i="6.03,282,1694761200"; d="scan'208";a="906241544" X-Received: from sh1gapp1009.ccr.corp.intel.com ([10.239.189.219]) by fmsmga001.fm.intel.com with ESMTP; 06 Nov 2023 17:24:48 -0800 From: "Wu, Jiaxin" To: devel@edk2.groups.io Cc: Eric Dong , Ray Ni , Zeng Star , Gerd Hoffmann , Rahul Kumar , Laszlo Ersek Subject: [edk2-devel] [PATCH v4] UefiCpuPkg/PiSmmCpuDxeSmm: Fix CP Exception when CET enable Date: Tue, 7 Nov 2023 09:24:45 +0800 Message-Id: <20231107012445.7808-1-jiaxin.wu@intel.com> Precedence: Bulk List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,jiaxin.wu@intel.com List-Unsubscribe-Post: List-Unsubscribe=One-Click List-Unsubscribe: X-Gm-Message-State: VfIQ3mwxo6lxo4tFZU5QNAjBx1787277AA= X-ZohoMail-DKIM: pass (identity @groups.io) X-ZM-MESSAGEID: 1699320292988100001 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Root cause: 1. Before DisableReadonlyPageWriteProtect() is called, the return address (#1) is pushed in shadow stack. 2. CET is disabled. 3. DisableReadonlyPageWriteProtect() returns to #1. 4. Page table is modified. 5. EnableReadonlyPageWriteProtect() is called, but the return address (#2) is not pushed in shadow stack. 6. CET is enabled. 7. EnableReadonlyPageWriteProtect() returns to #2. #CP exception happens because the actual return address (#2) doesn't match the return address stored in shadow stack (#1). Analysis: Shadow stack will stop update after CET disable (DisableCet() in DisableReadOnlyPageWriteProtect), but normal smi stack will be continue updated with the function called and return (DisableReadOnlyPageWriteProtect & EnableReadOnlyPageWriteProtect), thus leading stack mismatch after CET re-enabled (EnableCet() in EnableReadOnlyPageWriteProtect). According SDM Vol 3, 6.15-Control Protection Exception: Normal smi stack and shadow stack must be matched when CET enable, otherwise CP Exception will happen, which is caused by a near RET instruction. CET is disabled in DisableCet(), while can be enabled in EnableCet(). This way won't cause the problem because they are implemented in a way that return address of DisableCet() is poped out from shadow stack (Incsspq performs a pop to increases the shadow stack) and EnableCet() doesn't use "RET" but "JMP" to return to caller. So calling EnableCet() and DisableCet() doesn't have the same issue as calling DisableReadonlyPageWriteProtect() and EnableReadonlyPageWriteProtect(). With above root cause & analysis, define below 2 macros instead of functions for WP & CET operation: WRITE_UNPROTECT_RO_PAGES (Wp, Cet) WRITE_PROTECT_RO_PAGES (Wp, Cet) Because DisableCet() & EnableCet() must be in the same function to avoid shadow stack and normal SMI stack mismatch. Note: WRITE_UNPROTECT_RO_PAGES () must be called pair with WRITE_PROTECT_RO_PAGES () in same function. Cc: Eric Dong Cc: Ray Ni Cc: Zeng Star Cc: Gerd Hoffmann Cc: Rahul Kumar Cc: Laszlo Ersek Signed-off-by: Jiaxin Wu Reviewed-by: Eric Dong Reviewed-by: Laszlo Ersek Reviewed-by: Ray Ni --- UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.h | 59 +++++++++++++---- UefiCpuPkg/PiSmmCpuDxeSmm/SmmCpuMemoryManagement.c | 73 +++++++++---------= ---- UefiCpuPkg/PiSmmCpuDxeSmm/SmmProfile.c | 7 ++- 3 files changed, 81 insertions(+), 58 deletions(-) diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.h b/UefiCpuPkg/PiSmmC= puDxeSmm/PiSmmCpuDxeSmm.h index 654935dc76..20ada465c2 100644 --- a/UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.h +++ b/UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.h @@ -1551,29 +1551,64 @@ VOID SmmWaitForApArrival ( VOID ); =20 /** - Disable Write Protect on pages marked as read-only if Cr0.Bits.WP is 1. + Write unprotect read-only pages if Cr0.Bits.WP is 1. + + @param[out] WriteProtect If Cr0.Bits.WP is enabled. =20 - @param[out] WpEnabled If Cr0.WP is enabled. - @param[out] CetEnabled If CET is enabled. **/ VOID -DisableReadOnlyPageWriteProtect ( - OUT BOOLEAN *WpEnabled, - OUT BOOLEAN *CetEnabled +SmmWriteUnprotectReadOnlyPage ( + OUT BOOLEAN *WriteProtect ); =20 /** - Enable Write Protect on pages marked as read-only. + Write protect read-only pages. + + @param[in] WriteProtect If Cr0.Bits.WP should be enabled. =20 - @param[out] WpEnabled If Cr0.WP should be enabled. - @param[out] CetEnabled If CET should be enabled. **/ VOID -EnableReadOnlyPageWriteProtect ( - BOOLEAN WpEnabled, - BOOLEAN CetEnabled +SmmWriteProtectReadOnlyPage ( + IN BOOLEAN WriteProtect ); =20 +/// +/// Define macros to encapsulate the write unprotect/protect +/// read-only pages. +/// Below pieces of logic are defined as macros and not functions +/// because "CET" feature disable & enable must be in the same +/// function to avoid shadow stack and normal SMI stack mismatch, +/// thus WRITE_UNPROTECT_RO_PAGES () must be called pair with +/// WRITE_PROTECT_RO_PAGES () in same function. +/// +/// @param[in,out] Wp A BOOLEAN variable local to the containing +/// function, carrying write protection status from +/// WRITE_UNPROTECT_RO_PAGES() to +/// WRITE_PROTECT_RO_PAGES(). +/// +/// @param[in,out] Cet A BOOLEAN variable local to the containing +/// function, carrying control flow integrity +/// enforcement status from +/// WRITE_UNPROTECT_RO_PAGES() to +/// WRITE_PROTECT_RO_PAGES(). +/// +#define WRITE_UNPROTECT_RO_PAGES(Wp, Cet) \ + do { \ + Cet =3D ((AsmReadCr4 () & CR4_CET_ENABLE) !=3D 0); \ + if (Cet) { \ + DisableCet (); \ + } \ + SmmWriteUnprotectReadOnlyPage (&Wp); \ + } while (FALSE) + +#define WRITE_PROTECT_RO_PAGES(Wp, Cet) \ + do { \ + SmmWriteProtectReadOnlyPage (Wp); \ + if (Cet) { \ + EnableCet (); \ + } \ + } while (FALSE) + #endif diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/SmmCpuMemoryManagement.c b/UefiCpuPk= g/PiSmmCpuDxeSmm/SmmCpuMemoryManagement.c index 6f49866615..3d445df213 100644 --- a/UefiCpuPkg/PiSmmCpuDxeSmm/SmmCpuMemoryManagement.c +++ b/UefiCpuPkg/PiSmmCpuDxeSmm/SmmCpuMemoryManagement.c @@ -39,64 +39,47 @@ PAGE_TABLE_POOL *mPageTablePool =3D NULL; // If memory used by SMM page table has been mareked as ReadOnly. // BOOLEAN mIsReadOnlyPageTable =3D FALSE; =20 /** - Disable Write Protect on pages marked as read-only if Cr0.Bits.WP is 1. + Write unprotect read-only pages if Cr0.Bits.WP is 1. + + @param[out] WriteProtect If Cr0.Bits.WP is enabled. =20 - @param[out] WpEnabled If Cr0.WP is enabled. - @param[out] CetEnabled If CET is enabled. **/ VOID -DisableReadOnlyPageWriteProtect ( - OUT BOOLEAN *WpEnabled, - OUT BOOLEAN *CetEnabled +SmmWriteUnprotectReadOnlyPage ( + OUT BOOLEAN *WriteProtect ) { IA32_CR0 Cr0; =20 - *CetEnabled =3D ((AsmReadCr4 () & CR4_CET_ENABLE) !=3D 0) ? TRUE : FALSE; - Cr0.UintN =3D AsmReadCr0 (); - *WpEnabled =3D (Cr0.Bits.WP !=3D 0) ? TRUE : FALSE; - if (*WpEnabled) { - if (*CetEnabled) { - // - // CET must be disabled if WP is disabled. Disable CET before cleari= ng CR0.WP. - // - DisableCet (); - } - + Cr0.UintN =3D AsmReadCr0 (); + *WriteProtect =3D (Cr0.Bits.WP !=3D 0); + if (*WriteProtect) { Cr0.Bits.WP =3D 0; AsmWriteCr0 (Cr0.UintN); } } =20 /** - Enable Write Protect on pages marked as read-only. + Write protect read-only pages. + + @param[in] WriteProtect If Cr0.Bits.WP should be enabled. =20 - @param[out] WpEnabled If Cr0.WP should be enabled. - @param[out] CetEnabled If CET should be enabled. **/ VOID -EnableReadOnlyPageWriteProtect ( - BOOLEAN WpEnabled, - BOOLEAN CetEnabled +SmmWriteProtectReadOnlyPage ( + IN BOOLEAN WriteProtect ) { IA32_CR0 Cr0; =20 - if (WpEnabled) { + if (WriteProtect) { Cr0.UintN =3D AsmReadCr0 (); Cr0.Bits.WP =3D 1; AsmWriteCr0 (Cr0.UintN); - - if (CetEnabled) { - // - // re-enable CET. - // - EnableCet (); - } } } =20 /** Initialize a buffer pool for page table use only. @@ -119,11 +102,11 @@ BOOLEAN InitializePageTablePool ( IN UINTN PoolPages ) { VOID *Buffer; - BOOLEAN WpEnabled; + BOOLEAN WriteProtect; BOOLEAN CetEnabled; =20 // // Always reserve at least PAGE_TABLE_POOL_UNIT_PAGES, including one pag= e for // header. @@ -157,13 +140,15 @@ InitializePageTablePool ( =20 // // If page table memory has been marked as RO, mark the new pool pages a= s read-only. // if (mIsReadOnlyPageTable) { - DisableReadOnlyPageWriteProtect (&WpEnabled, &CetEnabled); + WRITE_UNPROTECT_RO_PAGES (WriteProtect, CetEnabled); + SmmSetMemoryAttributes ((EFI_PHYSICAL_ADDRESS)(UINTN)Buffer, EFI_PAGES= _TO_SIZE (PoolPages), EFI_MEMORY_RO); - EnableReadOnlyPageWriteProtect (WpEnabled, CetEnabled); + + WRITE_PROTECT_RO_PAGES (WriteProtect, CetEnabled); } =20 return TRUE; } =20 @@ -1009,11 +994,11 @@ SetMemMapAttributes ( UINTN PageTable; EFI_STATUS Status; IA32_MAP_ENTRY *Map; UINTN Count; UINT64 MemoryAttribute; - BOOLEAN WpEnabled; + BOOLEAN WriteProtect; BOOLEAN CetEnabled; =20 SmmGetSystemConfigurationTable (&gEdkiiPiSmmMemoryAttributesTableGuid, (= VOID **)&MemoryAttributesTable); if (MemoryAttributesTable =3D=3D NULL) { DEBUG ((DEBUG_INFO, "MemoryAttributesTable - NULL\n")); @@ -1055,11 +1040,11 @@ SetMemMapAttributes ( Status =3D PageTableParse (PageTable, mPagingMode, Map, &Count); } =20 ASSERT_RETURN_ERROR (Status); =20 - DisableReadOnlyPageWriteProtect (&WpEnabled, &CetEnabled); + WRITE_UNPROTECT_RO_PAGES (WriteProtect, CetEnabled); =20 MemoryMap =3D MemoryMapStart; for (Index =3D 0; Index < MemoryMapEntryCount; Index++) { DEBUG ((DEBUG_VERBOSE, "SetAttribute: Memory Entry - 0x%lx, 0x%x\n", M= emoryMap->PhysicalStart, MemoryMap->NumberOfPages)); if (MemoryMap->Type =3D=3D EfiRuntimeServicesCode) { @@ -1085,11 +1070,12 @@ SetMemMapAttributes ( ); =20 MemoryMap =3D NEXT_MEMORY_DESCRIPTOR (MemoryMap, DescriptorSize); } =20 - EnableReadOnlyPageWriteProtect (WpEnabled, CetEnabled); + WRITE_PROTECT_RO_PAGES (WriteProtect, CetEnabled); + FreePool (Map); =20 PatchSmmSaveStateMap (); PatchGdtIdtMap (); =20 @@ -1392,18 +1378,18 @@ SetUefiMemMapAttributes ( EFI_STATUS Status; EFI_MEMORY_DESCRIPTOR *MemoryMap; UINTN MemoryMapEntryCount; UINTN Index; EFI_MEMORY_DESCRIPTOR *Entry; - BOOLEAN WpEnabled; + BOOLEAN WriteProtect; BOOLEAN CetEnabled; =20 PERF_FUNCTION_BEGIN (); =20 DEBUG ((DEBUG_INFO, "SetUefiMemMapAttributes\n")); =20 - DisableReadOnlyPageWriteProtect (&WpEnabled, &CetEnabled); + WRITE_UNPROTECT_RO_PAGES (WriteProtect, CetEnabled); =20 if (mUefiMemoryMap !=3D NULL) { MemoryMapEntryCount =3D mUefiMemoryMapSize/mUefiDescriptorSize; MemoryMap =3D mUefiMemoryMap; for (Index =3D 0; Index < MemoryMapEntryCount; Index++) { @@ -1479,11 +1465,11 @@ SetUefiMemMapAttributes ( =20 Entry =3D NEXT_MEMORY_DESCRIPTOR (Entry, mUefiMemoryAttributesTable-= >DescriptorSize); } } =20 - EnableReadOnlyPageWriteProtect (WpEnabled, CetEnabled); + WRITE_PROTECT_RO_PAGES (WriteProtect, CetEnabled); =20 // // Do not free mUefiMemoryAttributesTable, it will be checked in IsSmmCo= mmBufferForbiddenAddress(). // =20 @@ -1870,11 +1856,11 @@ IfReadOnlyPageTableNeeded ( VOID SetPageTableAttributes ( VOID ) { - BOOLEAN WpEnabled; + BOOLEAN WriteProtect; BOOLEAN CetEnabled; =20 if (!IfReadOnlyPageTableNeeded ()) { return; } @@ -1884,20 +1870,21 @@ SetPageTableAttributes ( =20 // // Disable write protection, because we need mark page table to be write= protected. // We need *write* page table memory, to mark itself to be *read only*. // - DisableReadOnlyPageWriteProtect (&WpEnabled, &CetEnabled); + WRITE_UNPROTECT_RO_PAGES (WriteProtect, CetEnabled); =20 // Set memory used by page table as Read Only. DEBUG ((DEBUG_INFO, "Start...\n")); EnablePageTableProtection (); =20 // // Enable write protection, after page table attribute updated. // - EnableReadOnlyPageWriteProtect (TRUE, CetEnabled); + WRITE_PROTECT_RO_PAGES (TRUE, CetEnabled); + mIsReadOnlyPageTable =3D TRUE; =20 // // Flush TLB after mark all page table pool as read only. // diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/SmmProfile.c b/UefiCpuPkg/PiSmmCpuDx= eSmm/SmmProfile.c index 7ac3c66f91..8142d3ceac 100644 --- a/UefiCpuPkg/PiSmmCpuDxeSmm/SmmProfile.c +++ b/UefiCpuPkg/PiSmmCpuDxeSmm/SmmProfile.c @@ -592,11 +592,11 @@ InitPaging ( UINT64 Base; UINT64 Length; UINT64 Limit; UINT64 PreviousAddress; UINT64 MemoryAttrMask; - BOOLEAN WpEnabled; + BOOLEAN WriteProtect; BOOLEAN CetEnabled; =20 PERF_FUNCTION_BEGIN (); =20 PageTable =3D AsmReadCr3 (); @@ -604,11 +604,12 @@ InitPaging ( Limit =3D BASE_4GB; } else { Limit =3D (IsRestrictedMemoryAccess ()) ? LShiftU64 (1, mPhysicalAddre= ssBits) : BASE_4GB; } =20 - DisableReadOnlyPageWriteProtect (&WpEnabled, &CetEnabled); + WRITE_UNPROTECT_RO_PAGES (WriteProtect, CetEnabled); + // // [0, 4k] may be non-present. // PreviousAddress =3D ((PcdGet8 (PcdNullPointerDetectionPropertyMask) & BI= T1) !=3D 0) ? BASE_4KB : 0; =20 @@ -670,11 +671,11 @@ InitPaging ( // Status =3D ConvertMemoryPageAttributes (PageTable, mPagingMode, Previo= usAddress, Limit - PreviousAddress, MemoryAttrMask, TRUE, NULL); ASSERT_RETURN_ERROR (Status); } =20 - EnableReadOnlyPageWriteProtect (WpEnabled, CetEnabled); + WRITE_PROTECT_RO_PAGES (WriteProtect, CetEnabled); =20 // // Flush TLB // CpuFlushTlb (); --=20 2.16.2.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#110780): https://edk2.groups.io/g/devel/message/110780 Mute This Topic: https://groups.io/mt/102434876/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-