From nobody Thu Nov 14 16:59:45 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+69975+1787277+3901457@groups.io; helo=mail02.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+69975+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1610049341; cv=none; d=zohomail.com; s=zohoarc; b=JEv+c7OStFbu8yUJS9YMUSPwFzMEi7lA3IgMD4rXJLWNgvyN5NkV577wi3t4kCiqNsXskcD9H9ZszQzUn8Oz67REOIV5uGnIaM4z6/0swsAVT7rhakqhHrTAl04+C4Jv80vwlK1h7vfrFalFCd2eqQMhXdc8acRgx/kBHgUJfxU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1610049341; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Id:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=xf9ag9FnPy6J074YyDEJJFwk4cRgGpAL/9DIb/V35YE=; b=Z+bkmeHMi0r3IbXkOa0qB4T58DnfaBxuuPJZ6NWTAL6CPfBvMO4gBWJ8u5bxMEMPOPxMGRua8bzOhLxx0NbtlrGntYhtjj++7PuAiOW6DzhYv8E7uUST19BDbm6FU11ArHjVVPG1G9KYKpdklMvMnzj0uuiDMmpSBO10YS8pn8I= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+69975+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 1610049341307532.2437772434225; Thu, 7 Jan 2021 11:55:41 -0800 (PST) Return-Path: X-Received: by 127.0.0.2 with SMTP id kSQMYY1788612xpcagnKPvkn; Thu, 07 Jan 2021 11:55:40 -0800 X-Received: from aserp2120.oracle.com (aserp2120.oracle.com [141.146.126.78]) by mx.groups.io with SMTP id smtpd.web11.2146.1610049335374482640 for ; Thu, 07 Jan 2021 11:55:35 -0800 X-Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 107JtVks013664; Thu, 7 Jan 2021 19:55:32 GMT X-Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2120.oracle.com with ESMTP id 35wepme2jw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Thu, 07 Jan 2021 19:55:32 +0000 X-Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 107JtSh4109071; Thu, 7 Jan 2021 19:55:31 GMT X-Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userp3030.oracle.com with ESMTP id 35w3g3835t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 07 Jan 2021 19:55:31 +0000 X-Received: from abhmp0002.oracle.com (abhmp0002.oracle.com [141.146.116.8]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 107JtU30021843; Thu, 7 Jan 2021 19:55:30 GMT X-Received: from localhost.localdomain (/70.36.60.91) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 07 Jan 2021 11:55:30 -0800 From: "Ankur Arora" To: devel@edk2.groups.io Cc: lersek@redhat.com, imammedo@redhat.com, boris.ostrovsky@oracle.com, Ankur Arora , Jordan Justen , Ard Biesheuvel , Eric Dong , Ray Ni , Rahul Kumar , Aaron Young Subject: [edk2-devel] [PATCH v2 08/10] OvmfCpuPkg/CpuHotplug: add a hot-unplug handler called at SMI exit Date: Thu, 7 Jan 2021 11:55:13 -0800 Message-Id: <20210107195515.106158-9-ankur.a.arora@oracle.com> In-Reply-To: <20210107195515.106158-1-ankur.a.arora@oracle.com> References: <20210107195515.106158-1-ankur.a.arora@oracle.com> MIME-Version: 1.0 Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,ankur.a.arora@oracle.com X-Gm-Message-State: KKbelP8WicfYreFQlSJVU6mNx1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1610049340; bh=f+j/6y27w3u7qgHkLvtNoj+bUQEQrU/ua0TlP7O9sps=; h=Cc:Date:From:Reply-To:Subject:To; b=DMzZWaAfPWNvtUC6XIn78W+eyxp3/d8Rjm/tAJ60BteTWfYO9yoKnDYqkmaHmtmNCWe 24cJqGmb4E35ItlQeg6M5jKEkopfw7u6vNtbLsOGGyMuuWM4OAFOLF4dXpE0J8+DKWxOl uCwFK/rfKpcCaTUQG4Ac/WtoZcXP0YZ9Sf4= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Type: text/plain; charset="utf-8" Add CpuUnplugExitWork(), to be called from SmmCpuFeaturesRendezvousExit() to do the final ejection as part of CPU hot-unplug. On the BSP, CpuUnplugExitWork() calls QEMU to do the ejection for each CPU that is unplugged. QEMU handles this by signalling the remote VCPU thread which forces the SMI AP to context switch out of the SMI and with its QEMU state destroyed. On the AP, CpuUnplugExitWork() provides a holding area where the CPU spins until context switched out by QEMU via the BSP. Given that the context switch would end up with the AP state being cleaned up, this means that the AP CPU will never return to finish the SMI handling, and thus would not restore some of the CPU state that it ordinarily would (in SmiRendezvous() and in SmiEntry.nasm::CommonHandler). This unrestored state includes FPU state, CET enable, stuffing of RSB and the final RSM. Given that the CPU state is destroyed by QEMU on unplug, this should be okay. Cc: Laszlo Ersek Cc: Jordan Justen Cc: Ard Biesheuvel Cc: Eric Dong Cc: Ray Ni Cc: Rahul Kumar Cc: Igor Mammedov Cc: Boris Ostrovsky Cc: Aaron Young Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3D3132 Signed-off-by: Ankur Arora --- OvmfPkg/CpuHotplugSmm/CpuHotplug.c | 68 +++++++++++++++++++++++++++++++= ++++ UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c | 6 ++++ 2 files changed, 74 insertions(+) diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplug.c b/OvmfPkg/CpuHotplugSmm/Cpu= Hotplug.c index 20d92a35da39..379c9a66f261 100644 --- a/OvmfPkg/CpuHotplugSmm/CpuHotplug.c +++ b/OvmfPkg/CpuHotplugSmm/CpuHotplug.c @@ -421,6 +421,69 @@ Fatal: return EFI_INTERRUPT_PENDING; } =20 +EFI_STATUS +EFIAPI +CpuUnplugExitWork( + IN UINTN CpuIndex, + IN BOOLEAN IsBSP + ) +{ + APIC_ID RemoveApicId; + + RemoveApicId =3D mHotUnplugWork[CpuIndex]; + + if (!IsBSP && RemoveApicId =3D=3D MAX_UINT32) { + return EFI_SUCCESS; + } + + if (IsBSP) { + UINT32 Idx; + for (Idx =3D 0; Idx < mCpuHotPlugData->ArrayLength; Idx++) { + RemoveApicId =3D mHotUnplugWork[Idx]; + + if (RemoveApicId !=3D MAX_UINT32) { + // + // The CPU(s) to be unplugged have received the BSP's signal to exit the + // SMI and either will execute SmmCpuFeaturesSmiRendezvousExit() and this + // callback or are waiting here. + // + // Tell HW to put it out of its misery. + // + QemuCpuhpWriteCpuSelector (mMmCpuIo, RemoveApicId); + QemuCpuhpWriteCpuStatus (mMmCpuIo, QEMU_CPUHP_STAT_EJECTED); + + // + // Barrier to ensure that the compiler doesn't reorder the next store + // + MemoryFence(); + + // + // Clear the unplug status to make sure that an invalid SMI later + // does not try to do an unplug or go to the dead loop. + // + mHotUnplugWork[Idx] =3D MAX_UINT32; + + DEBUG ((DEBUG_INFO, "%a: Unplugged CPU " FMT_APIC_ID "\n", + __FUNCTION__, RemoveApicId)); + } + } + return EFI_SUCCESS; + } + =20 + // + // CPU(s) being unplugged get here from SmmCpuFeaturesSmiRendezvousExit() + // after having been cleared to exit the SMI by the monarch and thus have + // no SMM processing remaining. + // + // Given that we cannot allow them to escape to the guest, we pen them + // here until the SMM monarch tells the HW to unplug them. + // + + CpuDeadLoop (); + + return EFI_ABORTED; +} + =20 // // Entry point function of this driver. @@ -573,6 +636,11 @@ CpuHotplugEntry ( } =20 // + // Register handler for hot-unplugging an AP. + // + MmRegisterShutdownInterface(CpuUnplugExitWork); + + // // Register the handler for the CPU Hotplug MMI. // Status =3D gMmst->MmiHandlerRegister ( diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c b/UefiCpuPkg/PiSmmCpuDxe= Smm/MpService.c index fb6aab17de37..f246d730d1e2 100644 --- a/UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c +++ b/UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c @@ -1727,6 +1727,12 @@ SmiRendezvous ( } } =20 + // + // Note that the BSP will unplug any CPUs that have been marked for + // hot-unplug at any point after it sets AllCpusInSync =3D FALSE + // so it cannot depend on an AP executing code post that point. + // + Exit: SmmCpuFeaturesRendezvousExit (CpuIndex, IsBsp); =20 --=20 2.9.3 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#69975): https://edk2.groups.io/g/devel/message/69975 Mute This Topic: https://groups.io/mt/79507544/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-