From nobody Fri May 3 03:22:15 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+96724+1787277+3901457@groups.io; helo=mail02.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+96724+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1669785243; cv=none; d=zohomail.com; s=zohoarc; b=JMDRbmAwt07djgJJoavDPmh8KScfPZlo6Qe7MPc7giQuNnU+kGggG0DJJj7V3d4dp9GSyUO+mVkEczVUXQAHaqyv6SJAvioZolNA3fI2v8YvAvBraM/kQRumIrAWV3CwSho75ClxnslGycY9/pEWZhPrbFhX1n3v/PNVsPO5rRE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1669785243; h=Cc:Date:From:List-Subscribe:List-Id:List-Help:List-Unsubscribe:Message-ID:Reply-To:Sender:Subject:To; bh=9gJ7Bj34DCpvYlcZYGAX8PpRMiBKWlkPEAlvyJCAU7o=; b=dovDtW16262sOUHJs5SVc8p+JeJlm+iLbJUuYX6ygZVzKxAgLXgL/tvOxSDSRJFaQ19X3Q2D8+biF+TB/3Fq+4rhpy83ozlPWEU2tgoNGygOLaMuodduu0IfmaVUFHpyjolM1ghlsipAWsYOTWrpcaPCiYs0Rn6GGSIMrjoCNWQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+96724+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 1669785243286950.3017953553798; Tue, 29 Nov 2022 21:14:03 -0800 (PST) Return-Path: X-Received: by 127.0.0.2 with SMTP id OoI2YY1788612xaqsLsEJWsC; Tue, 29 Nov 2022 21:14:02 -0800 X-Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mx.groups.io with SMTP id smtpd.web11.3560.1669785241449806836 for ; Tue, 29 Nov 2022 21:14:01 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10546"; a="316458458" X-IronPort-AV: E=Sophos;i="5.96,205,1665471600"; d="scan'208";a="316458458" X-Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Nov 2022 21:14:00 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10546"; a="674896520" X-IronPort-AV: E=Sophos;i="5.96,205,1665471600"; d="scan'208";a="674896520" X-Received: from sh1gapp1009.ccr.corp.intel.com ([10.239.189.79]) by orsmga008.jf.intel.com with ESMTP; 29 Nov 2022 21:13:59 -0800 From: "Wu, Jiaxin" To: devel@edk2.groups.io Cc: Eric Dong , Ray Ni , Zeng Star Subject: [edk2-devel] [PATCH v3] UefiCpuPkg: Check SMM Delayed/Blocked AP Count Date: Wed, 30 Nov 2022 13:13:56 +0800 Message-Id: <20221130051356.13652-1-jiaxin.wu@intel.com> Precedence: Bulk List-Unsubscribe: List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,jiaxin.wu@intel.com X-Gm-Message-State: VtOQL5bTo3z3UMwWQJnJ9X1ix1787277AA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1669785242; bh=HrF+UjgSMUuogNbtAjL0nTN848SfQjwSPmOcVleQ/kQ=; h=Cc:Date:From:Reply-To:Subject:To; b=JZbJfL5XNsdCJN8XWYnCk9XPTzjuuFimJGOD9JrcGsBLR/AV+wd7ITJaqU4RO3VpJ3l lqWtKjZ7Xo4uKnz3lwSXiHEya2doaI+JRVWh+PEpO9IyufJ7NdJAaafA2C70BBjFeGxeK h+k8KAmurScBeAY1q5swZGC9qB6FxjgL7mY= X-ZohoMail-DKIM: pass (identity @groups.io) X-ZM-MESSAGEID: 1669785243930100002 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3D4173 Due to more core count increasement, it's hard to reflect all APs state via AP bitvector support in the register. Actually, SMM CPU driver doesn't need to check each AP state to know all CPUs in SMI or not, one alternative method is to check the SMM Delayed & Blocked AP Count number: APs in SMI + Blocked Count + Disabled Count >=3D All supported Aps (code comments explained why can be > All supported Aps) With above change, the returned value of "SmmRegSmmEnable" & "SmmRegSmmDelayed" & "SmmRegSmmBlocked" from SmmCpuFeaturesLib should be the AP count number within the existing CPU package. For register that return the bitvector state, require SmmCpuFeaturesGetSmmRegister() returns count number of all bit per logical processor within the same package. For register that return the AP count, require SmmCpuFeaturesGetSmmRegister() returns the register value directly. v3: - Refine the coding style v2: - Rename "mPackageBspInfo" to "mPackageFirstThreadIndex" - Clarify the expected value of "SmmRegSmmEnable" & "SmmRegSmmDelayed" & "SmmRegSmmBlocked" returned from SmmCpuFeaturesLib. - Thread: https://edk2.groups.io/g/devel/message/96722 v1: - Thread: https://edk2.groups.io/g/devel/message/96671 Cc: Eric Dong Cc: Ray Ni Cc: Zeng Star Signed-off-by: Jiaxin Wu Reviewed-by: Ray Ni --- UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c | 195 +++++++++++++++++++++++++= ---- UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.c | 5 + UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.h | 16 ++- 3 files changed, 184 insertions(+), 32 deletions(-) diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c b/UefiCpuPkg/PiSmmCpuDxe= Smm/MpService.c index c79da418e3..a0967eb69c 100644 --- a/UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c +++ b/UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c @@ -22,10 +22,15 @@ UINTN mSemaphoreSize; SPIN_LOCK *mPFLock =3D NULL; SMM_CPU_SYNC_MODE mCpuSmmSyncMode; BOOLEAN mMachineCheckSupported =3D FALSE; MM_COMPLETION mSmmStartupThisApToken; =20 +// +// Processor specified by mPackageFirstThreadIndex[PackageIndex] will do t= he package-scope register check. +// +UINT32 *mPackageFirstThreadIndex =3D NULL; + extern UINTN mSmmShadowStackSize; =20 /** Performs an atomic compare exchange operation to get semaphore. The compare exchange operation must be performed using @@ -155,54 +160,129 @@ ReleaseAllAPs ( } } } =20 /** - Checks if all CPUs (with certain exceptions) have checked in for this SM= I run + Check whether the index of CPU perform the package level register + programming during System Management Mode initialization. =20 - @param Exceptions CPU Arrival exception flags. + The index of Processor specified by mPackageFirstThreadIndex[PackageInde= x] + will do the package-scope register programming. =20 - @retval TRUE if all CPUs the have checked in. - @retval FALSE if at least one Normal AP hasn't checked in. + @param[in] CpuIndex Processor Index. + + @retval TRUE Perform the package level register programming. + @retval FALSE Don't perform the package level register programming. =20 **/ BOOLEAN -AllCpusInSmmWithExceptions ( - SMM_CPU_ARRIVAL_EXCEPTIONS Exceptions +IsPackageFirstThread ( + IN UINTN CpuIndex ) { - UINTN Index; - SMM_CPU_DATA_BLOCK *CpuData; - EFI_PROCESSOR_INFORMATION *ProcessorInfo; + UINT32 PackageIndex; =20 - ASSERT (*mSmmMpSyncData->Counter <=3D mNumberOfCpus); + PackageIndex =3D gSmmCpuPrivate->ProcessorInfo[CpuIndex].Location.Packa= ge; =20 - if (*mSmmMpSyncData->Counter =3D=3D mNumberOfCpus) { - return TRUE; + ASSERT (mPackageFirstThreadIndex !=3D NULL); + + // + // Set the value of mPackageFirstThreadIndex[PackageIndex]. + // The package-scope register are checked by the first processor (CpuInd= ex) in Package. + // + // If mPackageFirstThreadIndex[PackageIndex] equals to (UINT32)-1, then = update + // to current CpuIndex. If it doesn't equal to (UINT32)-1, don't change = it. + // + if (mPackageFirstThreadIndex[PackageIndex] =3D=3D (UINT32)-1) { + mPackageFirstThreadIndex[PackageIndex] =3D (UINT32)CpuIndex; } =20 - CpuData =3D mSmmMpSyncData->CpuData; - ProcessorInfo =3D gSmmCpuPrivate->ProcessorInfo; - for (Index =3D 0; Index < mMaxNumberOfCpus; Index++) { - if (!(*(CpuData[Index].Present)) && (ProcessorInfo[Index].ProcessorId = !=3D INVALID_APIC_ID)) { - if (((Exceptions & ARRIVAL_EXCEPTION_DELAYED) !=3D 0) && (SmmCpuFeat= uresGetSmmRegister (Index, SmmRegSmmDelayed) !=3D 0)) { - continue; - } + return (BOOLEAN)(mPackageFirstThreadIndex[PackageIndex] =3D=3D CpuIndex); +} + +/** + Returns the Number of SMM Delayed & Blocked & Disabled Thread Count. + + @param[in,out] DelayedCount The Number of SMM Delayed Thread Count. + @param[in,out] BlockedCount The Number of SMM Blocked Thread Count. + @param[in,out] DisabledCount The Number of SMM Disabled Thread Count. + +**/ +VOID +GetSmmDelayedBlockedDisabledCount ( + IN OUT UINT32 *DelayedCount, + IN OUT UINT32 *BlockedCount, + IN OUT UINT32 *DisabledCount + ) +{ + UINTN Index; =20 - if (((Exceptions & ARRIVAL_EXCEPTION_BLOCKED) !=3D 0) && (SmmCpuFeat= uresGetSmmRegister (Index, SmmRegSmmBlocked) !=3D 0)) { - continue; + for (Index =3D 0; Index < mNumberOfCpus; Index++) { + if (IsPackageFirstThread (Index)) { + if (DelayedCount !=3D NULL) { + *DelayedCount +=3D (UINT32)SmmCpuFeaturesGetSmmRegister (Index, Sm= mRegSmmDelayed); } =20 - if (((Exceptions & ARRIVAL_EXCEPTION_SMI_DISABLED) !=3D 0) && (SmmCp= uFeaturesGetSmmRegister (Index, SmmRegSmmEnable) !=3D 0)) { - continue; + if (BlockedCount !=3D NULL) { + *BlockedCount +=3D (UINT32)SmmCpuFeaturesGetSmmRegister (Index, Sm= mRegSmmBlocked); } =20 - return FALSE; + if (DisabledCount !=3D NULL) { + *DisabledCount +=3D (UINT32)SmmCpuFeaturesGetSmmRegister (Index, S= mmRegSmmEnable); + } } } +} =20 - return TRUE; +/** + Checks if all CPUs (except Blocked & Disabled) have checked in for this = SMI run + + @retval TRUE if all CPUs the have checked in. + @retval FALSE if at least one Normal AP hasn't checked in. + +**/ +BOOLEAN +AllCpusInSmmExceptBlockedDisabled ( + VOID + ) +{ + UINT32 BlockedCount; + UINT32 DisabledCount; + + BlockedCount =3D 0; + DisabledCount =3D 0; + + // + // Check to make sure mSmmMpSyncData->Counter is valid and not locked. + // + ASSERT (*mSmmMpSyncData->Counter <=3D mNumberOfCpus); + + // + // Check whether all CPUs in SMM. + // + if (*mSmmMpSyncData->Counter =3D=3D mNumberOfCpus) { + return TRUE; + } + + // + // Check for the Blocked & Disabled Exceptions Case. + // + GetSmmDelayedBlockedDisabledCount (NULL, &BlockedCount, &DisabledCount); + + // + // *mSmmMpSyncData->Counter might be updated by all APs concurrently. Th= e value + // can be dynamic changed. If some Aps enter the SMI after the BlockedCo= unt & + // DisabledCount check, then the *mSmmMpSyncData->Counter will be increa= sed, thus + // leading the *mSmmMpSyncData->Counter + BlockedCount + DisabledCount >= mNumberOfCpus. + // since the BlockedCount & DisabledCount are local variable, it's ok he= re only for + // the checking of all CPUs In Smm. + // + if (*mSmmMpSyncData->Counter + BlockedCount + DisabledCount >=3D mNumber= OfCpus) { + return TRUE; + } + + return FALSE; } =20 /** Has OS enabled Lmce in the MSR_IA32_MCG_EXT_CTL =20 @@ -266,10 +346,15 @@ SmmWaitForApArrival ( { UINT64 Timer; UINTN Index; BOOLEAN LmceEn; BOOLEAN LmceSignal; + UINT32 DelayedCount; + UINT32 BlockedCount; + + DelayedCount =3D 0; + BlockedCount =3D 0; =20 ASSERT (*mSmmMpSyncData->Counter <=3D mNumberOfCpus); =20 LmceEn =3D FALSE; LmceSignal =3D FALSE; @@ -294,11 +379,11 @@ SmmWaitForApArrival ( // for (Timer =3D StartSyncTimer (); !IsSyncTimerTimeout (Timer) && !(LmceEn && LmceSignal); ) { - mSmmMpSyncData->AllApArrivedWithException =3D AllCpusInSmmWithExceptio= ns (ARRIVAL_EXCEPTION_BLOCKED | ARRIVAL_EXCEPTION_SMI_DISABLED); + mSmmMpSyncData->AllApArrivedWithException =3D AllCpusInSmmExceptBlocke= dDisabled (); if (mSmmMpSyncData->AllApArrivedWithException) { break; } =20 CpuPause (); @@ -335,19 +420,27 @@ SmmWaitForApArrival ( // for (Timer =3D StartSyncTimer (); !IsSyncTimerTimeout (Timer); ) { - mSmmMpSyncData->AllApArrivedWithException =3D AllCpusInSmmWithExcept= ions (ARRIVAL_EXCEPTION_BLOCKED | ARRIVAL_EXCEPTION_SMI_DISABLED); + mSmmMpSyncData->AllApArrivedWithException =3D AllCpusInSmmExceptBloc= kedDisabled (); if (mSmmMpSyncData->AllApArrivedWithException) { break; } =20 CpuPause (); } } =20 + if (!mSmmMpSyncData->AllApArrivedWithException) { + // + // Check for the Blocked & Delayed Case. + // + GetSmmDelayedBlockedDisabledCount (&DelayedCount, &BlockedCount, NULL); + DEBUG ((DEBUG_INFO, "SmmWaitForApArrival: Delayed AP Count =3D %d, Blo= cked AP Count =3D %d\n", DelayedCount, BlockedCount)); + } + return; } =20 /** Replace OS MTRR's with SMI MTRR's. @@ -737,10 +830,11 @@ APHandler ( // BSP timeout in the first round // if (mSmmMpSyncData->BspIndex !=3D -1) { // // BSP Index is known + // Existing AP is in SMI now but BSP not in, so, try bring BSP in SM= M. // BspIndex =3D mSmmMpSyncData->BspIndex; ASSERT (CpuIndex !=3D BspIndex); =20 // @@ -761,16 +855,19 @@ APHandler ( =20 if (!(*mSmmMpSyncData->InsideSmm)) { // // Give up since BSP is unable to enter SMM // and signal the completion of this AP + // Reduce the mSmmMpSyncData->Counter! + // WaitForSemaphore (mSmmMpSyncData->Counter); return; } } else { // // Don't know BSP index. Give up without sending IPI to BSP. + // Reduce the mSmmMpSyncData->Counter! // WaitForSemaphore (mSmmMpSyncData->Counter); return; } } @@ -1666,14 +1763,17 @@ SmiRendezvous ( // goto Exit; } else { // // Signal presence of this processor + // mSmmMpSyncData->Counter is increased here! + // "ReleaseSemaphore (mSmmMpSyncData->Counter) =3D=3D 0" means BSP has= already ended the synchronization. // if (ReleaseSemaphore (mSmmMpSyncData->Counter) =3D=3D 0) { // // BSP has already ended the synchronization, so QUIT!!! + // Existing AP is too late now to enter SMI since BSP has already en= ded the synchronization!!! // =20 // // Wait for BSP's signal to finish SMI // @@ -1781,10 +1881,51 @@ Exit: // Restore Cr2 // RestoreCr2 (Cr2); } =20 +/** + Initialize PackageBsp Info. Processor specified by mPackageFirstThreadIn= dex[PackageIndex] + will do the package-scope register programming. Set default CpuIndex to = (UINT32)-1, which + means not specified yet. + +**/ +VOID +InitPackageFirstThreadIndexInfo ( + VOID + ) +{ + UINT32 Index; + UINT32 PackageId; + UINT32 PackageCount; + + PackageId =3D 0; + PackageCount =3D 0; + + // + // Count the number of package, set to max PackageId + 1 + // + for (Index =3D 0; Index < mNumberOfCpus; Index++) { + if (PackageId < gSmmCpuPrivate->ProcessorInfo[Index].Location.Package)= { + PackageId =3D gSmmCpuPrivate->ProcessorInfo[Index].Location.Package; + } + } + + PackageCount =3D PackageId + 1; + + mPackageFirstThreadIndex =3D (UINT32 *)AllocatePool (sizeof (UINT32) * P= ackageCount); + ASSERT (mPackageFirstThreadIndex !=3D NULL); + if (mPackageFirstThreadIndex =3D=3D NULL) { + return; + } + + // + // Set default CpuIndex to (UINT32)-1, which means not specified yet. + // + SetMem32 (mPackageFirstThreadIndex, sizeof (UINT32) * PackageCount, (UIN= T32)-1); +} + /** Allocate buffer for SpinLock and Wrapper function buffer. =20 **/ VOID diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.c b/UefiCpuPkg/PiSmmC= puDxeSmm/PiSmmCpuDxeSmm.c index 40aabeda72..37e3cfc449 100644 --- a/UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.c +++ b/UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.c @@ -1043,10 +1043,15 @@ PiCpuSmmEntry ( // // Initialize global buffer for MM MP. // InitializeDataForMmMp (); =20 + // + // Initialize Package First Thread Index Info. + // + InitPackageFirstThreadIndexInfo (); + // // Install the SMM Mp Protocol into SMM protocol database // Status =3D gSmst->SmmInstallProtocolInterface ( &mSmmCpuHandle, diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.h b/UefiCpuPkg/PiSmmC= puDxeSmm/PiSmmCpuDxeSmm.h index ef8bf5947d..0bfba7e359 100644 --- a/UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.h +++ b/UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.h @@ -192,15 +192,10 @@ typedef struct { =20 #define EXCEPTION_VECTOR_NUMBER 0x20 =20 #define INVALID_APIC_ID 0xFFFFFFFFFFFFFFFFULL =20 -typedef UINT32 SMM_CPU_ARRIVAL_EXCEPTIONS; -#define ARRIVAL_EXCEPTION_BLOCKED 0x1 -#define ARRIVAL_EXCEPTION_DELAYED 0x2 -#define ARRIVAL_EXCEPTION_SMI_DISABLED 0x4 - // // Wrapper used to convert EFI_AP_PROCEDURE2 and EFI_AP_PROCEDURE. // typedef struct { EFI_AP_PROCEDURE Procedure; @@ -1460,10 +1455,21 @@ EFI_STATUS RegisterStartupProcedure ( IN EFI_AP_PROCEDURE Procedure, IN OUT VOID *ProcedureArguments OPTIONAL ); =20 +/** + Initialize PackageBsp Info. Processor specified by mPackageFirstThreadIn= dex[PackageIndex] + will do the package-scope register programming. Set default CpuIndex to = (UINT32)-1, which + means not specified yet. + +**/ +VOID +InitPackageFirstThreadIndexInfo ( + VOID + ); + /** Allocate buffer for SpinLock and Wrapper function buffer. =20 **/ VOID --=20 2.16.2.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#96724): https://edk2.groups.io/g/devel/message/96724 Mute This Topic: https://groups.io/mt/95352666/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-