OvmfPkg/OvmfPkg.dec | 4 + OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf | 2 + .../SmmCpuFeaturesLib/SmmCpuFeaturesLib.inf | 4 + OvmfPkg/CpuHotplugSmm/QemuCpuhp.h | 7 + OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h | 2 + OvmfPkg/Include/Pcd/CpuHotEjectData.h | 60 +++ OvmfPkg/CpuHotplugSmm/CpuHotplug.c | 576 +++++++++++++++++---- OvmfPkg/CpuHotplugSmm/QemuCpuhp.c | 123 +++-- .../Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.c | 111 ++++ OvmfPkg/SmmControl2Dxe/SmiFeatures.c | 18 +- 10 files changed, 775 insertions(+), 132 deletions(-) create mode 100644 OvmfPkg/Include/Pcd/CpuHotEjectData.h
Hi,
This series adds OVMF support for CPU hot-unplug.
QEMU secureboot hot-unplug logic corresponding to this is in upstream.
Also posted here:
https://lore.kernel.org/qemu-devel/20201207140739.3829993-1-imammedo@redhat.com/
Testing (with QEMU 5.2.50):
- Stable with randomized CPU plug/unplug (guest maxcpus=33,128)
- Synthetic tests with simultaneous multi CPU hot-unplug
Also at:
github.com/terminus/edk2/ hot-unplug-v9
Changelog:
v9:
- Rebased on top of edd46cd407ea
- Clarify comments around memory-barriers in patches 7, 8, 9
- Address other review comments from v8
v8:
- Fixes a couple of ECC issues in the code (in patches 7, 9)
URL: https://patchew.org/EDK2/20210222071928.1401820-1-ankur.a.arora@oracle.com/
v7:
- Address review comments from v6.
- Fix ejection bug where we were using APIC ID to do the ejection
rather than the Qemu Selector.
- Describes safety properties and ordering needed for concurrent
accesses to CPU_HOT_EJECT_DATA->QemuSelectorMap, and
CPU_HOT_EJECT_DATA->Handler.
URL: https://patchew.org/EDK2/20210219090444.1332380-1-ankur.a.arora@oracle.com/
v6:
- addresses v5 review comments.
URL: https://patchew.org/EDK2/20210129005950.467638-1-ankur.a.arora@oracle.com/
v5:
- fixes ECC errors (all but one in "OvmfPkg/CpuHotplugSmm: add
add Qemu Cpu Status helper").
URL: https://patchew.org/EDK2/20210126064440.299596-1-ankur.a.arora@oracle.com/
v4:
- Gets rid of unnecessary UefiCpuPkg changes
URL: https://patchew.org/EDK2/20210118063457.358581-1-ankur.a.arora@oracle.com/
v3:
- Use a saner PCD based interface to share state between PiSmmCpuDxeSmm
and OvmfPkg/CpuHotplugSmm
- Cleaner split of the hot-unplug code
URL: https://patchew.org/EDK2/20210115074533.277448-1-ankur.a.arora@oracle.com/
v2:
- Do the ejection via SmmCpuFeaturesRendezvousExit()
URL: https://patchew.org/EDK2/20210107195515.106158-1-ankur.a.arora@oracle.com/
RFC:
URL: https://patchew.org/EDK2/20201208053432.2690694-1-ankur.a.arora@oracle.com/
Please review.
Thanks
Ankur
Ankur Arora (10):
OvmfPkg/CpuHotplugSmm: refactor hotplug logic
OvmfPkg/CpuHotplugSmm: collect hot-unplug events
OvmfPkg/CpuHotplugSmm: add Qemu Cpu Status helper
OvmfPkg/CpuHotplugSmm: introduce UnplugCpus()
OvmfPkg: define CPU_HOT_EJECT_DATA
OvmfPkg/SmmCpuFeaturesLib: init CPU ejection state
OvmfPkg/SmmCpuFeaturesLib: call CPU hot-eject handler
OvmfPkg/CpuHotplugSmm: add EjectCpu()
OvmfPkg/CpuHotplugSmm: do actual CPU hot-eject
OvmfPkg/SmmControl2Dxe: negotiate CPU hot-unplug
OvmfPkg/OvmfPkg.dec | 4 +
OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf | 2 +
.../SmmCpuFeaturesLib/SmmCpuFeaturesLib.inf | 4 +
OvmfPkg/CpuHotplugSmm/QemuCpuhp.h | 7 +
OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h | 2 +
OvmfPkg/Include/Pcd/CpuHotEjectData.h | 60 +++
OvmfPkg/CpuHotplugSmm/CpuHotplug.c | 576 +++++++++++++++++----
OvmfPkg/CpuHotplugSmm/QemuCpuhp.c | 123 +++--
.../Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.c | 111 ++++
OvmfPkg/SmmControl2Dxe/SmiFeatures.c | 18 +-
10 files changed, 775 insertions(+), 132 deletions(-)
create mode 100644 OvmfPkg/Include/Pcd/CpuHotEjectData.h
--
2.9.3
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#72702): https://edk2.groups.io/g/devel/message/72702
Mute This Topic: https://groups.io/mt/81273600/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-
On 03/12/21 07:26, Ankur Arora wrote:
> Hi,
>
> This series adds OVMF support for CPU hot-unplug.
>
> QEMU secureboot hot-unplug logic corresponding to this is in upstream.
> Also posted here:
> https://lore.kernel.org/qemu-devel/20201207140739.3829993-1-imammedo@redhat.com/
>
> Testing (with QEMU 5.2.50):
> - Stable with randomized CPU plug/unplug (guest maxcpus=33,128)
> - Synthetic tests with simultaneous multi CPU hot-unplug
>
> Also at:
> github.com/terminus/edk2/ hot-unplug-v9
>
> Changelog:
>
> v9:
> - Rebased on top of edd46cd407ea
> - Clarify comments around memory-barriers in patches 7, 8, 9
> - Address other review comments from v8
Merged as commit range 4751a48aeb2a..f3bdfc41866e, via
<https://github.com/tianocore/edk2/pull/1494>, with the following R-b's
/ comments / light tweaks added:
> 1: 4b4466ed7c44 = 1: 0cb242e33693 OvmfPkg/CpuHotplugSmm: refactor hotplug logic
> 2: 719828efdebe ! 2: a752dd07466c OvmfPkg/CpuHotplugSmm: collect hot-unplug events
> @@ -20,6 +20,7 @@
> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
> Message-Id: <20210312062656.2477515-3-ankur.a.arora@oracle.com>
> + Reviewed-by: Laszlo Ersek <lersek@redhat.com>
>
> diff --git a/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h b/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h
> --- a/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h
> 3: 7d732efcb7af = 3: 2d92e052c3af OvmfPkg/CpuHotplugSmm: add Qemu Cpu Status helper
> 4: 64649eeeee42 = 4: 15e6ae8ea400 OvmfPkg/CpuHotplugSmm: introduce UnplugCpus()
> 5: 7190a5d134a7 ! 5: 8ade9d425a6e OvmfPkg: define CPU_HOT_EJECT_DATA
> @@ -15,6 +15,7 @@
> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
> Message-Id: <20210312062656.2477515-6-ankur.a.arora@oracle.com>
> + Reviewed-by: Laszlo Ersek <lersek@redhat.com>
>
> diff --git a/OvmfPkg/OvmfPkg.dec b/OvmfPkg/OvmfPkg.dec
> --- a/OvmfPkg/OvmfPkg.dec
> 6: 1fd019dac476 ! 6: b6d5996706dd OvmfPkg/SmmCpuFeaturesLib: init CPU ejection state
> @@ -22,6 +22,7 @@
> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
> Message-Id: <20210312062656.2477515-7-ankur.a.arora@oracle.com>
> + Reviewed-by: Laszlo Ersek <lersek@redhat.com>
>
> diff --git a/OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.inf b/OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.inf
> --- a/OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.inf
> 7: c8758d9b9764 ! 7: af9c77e151fa OvmfPkg/SmmCpuFeaturesLib: call CPU hot-eject handler
> @@ -19,6 +19,7 @@
> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
> Message-Id: <20210312062656.2477515-8-ankur.a.arora@oracle.com>
> + Reviewed-by: Laszlo Ersek <lersek@redhat.com>
>
> diff --git a/OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.c b/OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.c
> --- a/OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.c
> 8: 5ff8e8700678 ! 8: 30c69d2cfa63 OvmfPkg/CpuHotplugSmm: add EjectCpu()
> @@ -19,6 +19,7 @@
> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
> Message-Id: <20210312062656.2477515-9-ankur.a.arora@oracle.com>
> + Reviewed-by: Laszlo Ersek <lersek@redhat.com>
>
> diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf b/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
> --- a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
> 9: d6660f8f2f14 ! 9: f05328886302 OvmfPkg/CpuHotplugSmm: do actual CPU hot-eject
> @@ -27,6 +27,9 @@
> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
> Message-Id: <20210312062656.2477515-10-ankur.a.arora@oracle.com>
> + Reviewed-by: Laszlo Ersek <lersek@redhat.com>
> + [lersek@redhat.com: unneeded inner QemuSelector declaration in EjectCpu()
> + triggers VS warning #4456 (local variable shadowed); remove it]
>
> diff --git a/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h b/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h
> --- a/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h
> @@ -102,8 +105,6 @@
> + UINT32 Idx;
> +
> + for (Idx = 0; Idx < mCpuHotEjectData->ArrayLength; Idx++) {
> -+ UINT64 QemuSelector;
> -+
> + QemuSelector = mCpuHotEjectData->QemuSelectorMap[Idx];
> +
> + if (QemuSelector != CPU_EJECT_QEMU_SELECTOR_INVALID) {
> 10: 0a9a1cd40b98 ! 10: f3bdfc41866e OvmfPkg/SmmControl2Dxe: negotiate CPU hot-unplug
> @@ -14,6 +14,9 @@
> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
> Message-Id: <20210312062656.2477515-11-ankur.a.arora@oracle.com>
> + [lersek@redhat.com: preserve the empty line between the ICH9_LPC_SMI_F_*
> + group of macro definitions and the SCRATCH_BUFFER type definition]
> + Reviewed-by: Laszlo Ersek <lersek@redhat.com>
>
> diff --git a/OvmfPkg/SmmControl2Dxe/SmiFeatures.c b/OvmfPkg/SmmControl2Dxe/SmiFeatures.c
> --- a/OvmfPkg/SmmControl2Dxe/SmiFeatures.c
> @@ -22,16 +25,15 @@
> // "etc/smi/supported-features" and "etc/smi/requested-features" fw_cfg files.
> //
> #define ICH9_LPC_SMI_F_CPU_HOTPLUG BIT1
> --
> +//
> +// The following bit value stands for "enable CPU hot-unplug, and inject an SMI
> +// with control value ICH9_APM_CNT_CPU_HOTPLUG upon hot-unplug", in the
> +// "etc/smi/supported-features" and "etc/smi/requested-features" fw_cfg files.
> +//
> +#define ICH9_LPC_SMI_F_CPU_HOT_UNPLUG BIT2
> +
> //
> // Provides a scratch buffer (allocated in EfiReservedMemoryType type memory)
> - // for the S3 boot script fragment to write to and read from.
> @@
> QemuFwCfgReadBytes (sizeof mSmiFeatures, &mSmiFeatures);
>
Thanks for the contribution,
Laszlo
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#72875): https://edk2.groups.io/g/devel/message/72875
Mute This Topic: https://groups.io/mt/81273600/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-
On 2021-03-16 7:07 a.m., Laszlo Ersek wrote:
> On 03/12/21 07:26, Ankur Arora wrote:
>> Hi,
>>
>> This series adds OVMF support for CPU hot-unplug.
>>
>> QEMU secureboot hot-unplug logic corresponding to this is in upstream.
>> Also posted here:
>> https://lore.kernel.org/qemu-devel/20201207140739.3829993-1-imammedo@redhat.com/
>>
>> Testing (with QEMU 5.2.50):
>> - Stable with randomized CPU plug/unplug (guest maxcpus=33,128)
>> - Synthetic tests with simultaneous multi CPU hot-unplug
>>
>> Also at:
>> github.com/terminus/edk2/ hot-unplug-v9
>>
>> Changelog:
>>
>> v9:
>> - Rebased on top of edd46cd407ea
>> - Clarify comments around memory-barriers in patches 7, 8, 9
>> - Address other review comments from v8
>
> Merged as commit range 4751a48aeb2a..f3bdfc41866e, via
> <https://github.com/tianocore/edk2/pull/1494>, with the following R-b's
> / comments / light tweaks added:
Awesome! Thanks Laszlo.
And, thanks especially for the thorough reviews.
Ankur
>
>> 1: 4b4466ed7c44 = 1: 0cb242e33693 OvmfPkg/CpuHotplugSmm: refactor hotplug logic
>> 2: 719828efdebe ! 2: a752dd07466c OvmfPkg/CpuHotplugSmm: collect hot-unplug events
>> @@ -20,6 +20,7 @@
>> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
>> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
>> Message-Id: <20210312062656.2477515-3-ankur.a.arora@oracle.com>
>> + Reviewed-by: Laszlo Ersek <lersek@redhat.com>
>>
>> diff --git a/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h b/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h
>> --- a/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h
>> 3: 7d732efcb7af = 3: 2d92e052c3af OvmfPkg/CpuHotplugSmm: add Qemu Cpu Status helper
>> 4: 64649eeeee42 = 4: 15e6ae8ea400 OvmfPkg/CpuHotplugSmm: introduce UnplugCpus()
>> 5: 7190a5d134a7 ! 5: 8ade9d425a6e OvmfPkg: define CPU_HOT_EJECT_DATA
>> @@ -15,6 +15,7 @@
>> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
>> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
>> Message-Id: <20210312062656.2477515-6-ankur.a.arora@oracle.com>
>> + Reviewed-by: Laszlo Ersek <lersek@redhat.com>
>>
>> diff --git a/OvmfPkg/OvmfPkg.dec b/OvmfPkg/OvmfPkg.dec
>> --- a/OvmfPkg/OvmfPkg.dec
>> 6: 1fd019dac476 ! 6: b6d5996706dd OvmfPkg/SmmCpuFeaturesLib: init CPU ejection state
>> @@ -22,6 +22,7 @@
>> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
>> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
>> Message-Id: <20210312062656.2477515-7-ankur.a.arora@oracle.com>
>> + Reviewed-by: Laszlo Ersek <lersek@redhat.com>
>>
>> diff --git a/OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.inf b/OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.inf
>> --- a/OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.inf
>> 7: c8758d9b9764 ! 7: af9c77e151fa OvmfPkg/SmmCpuFeaturesLib: call CPU hot-eject handler
>> @@ -19,6 +19,7 @@
>> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
>> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
>> Message-Id: <20210312062656.2477515-8-ankur.a.arora@oracle.com>
>> + Reviewed-by: Laszlo Ersek <lersek@redhat.com>
>>
>> diff --git a/OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.c b/OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.c
>> --- a/OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.c
>> 8: 5ff8e8700678 ! 8: 30c69d2cfa63 OvmfPkg/CpuHotplugSmm: add EjectCpu()
>> @@ -19,6 +19,7 @@
>> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
>> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
>> Message-Id: <20210312062656.2477515-9-ankur.a.arora@oracle.com>
>> + Reviewed-by: Laszlo Ersek <lersek@redhat.com>
>>
>> diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf b/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
>> --- a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
>> 9: d6660f8f2f14 ! 9: f05328886302 OvmfPkg/CpuHotplugSmm: do actual CPU hot-eject
>> @@ -27,6 +27,9 @@
>> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
>> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
>> Message-Id: <20210312062656.2477515-10-ankur.a.arora@oracle.com>
>> + Reviewed-by: Laszlo Ersek <lersek@redhat.com>
>> + [lersek@redhat.com: unneeded inner QemuSelector declaration in EjectCpu()
>> + triggers VS warning #4456 (local variable shadowed); remove it]
>>
>> diff --git a/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h b/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h
>> --- a/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h
>> @@ -102,8 +105,6 @@
>> + UINT32 Idx;
>> +
>> + for (Idx = 0; Idx < mCpuHotEjectData->ArrayLength; Idx++) {
>> -+ UINT64 QemuSelector;
>> -+
>> + QemuSelector = mCpuHotEjectData->QemuSelectorMap[Idx];
>> +
>> + if (QemuSelector != CPU_EJECT_QEMU_SELECTOR_INVALID) {
>> 10: 0a9a1cd40b98 ! 10: f3bdfc41866e OvmfPkg/SmmControl2Dxe: negotiate CPU hot-unplug
>> @@ -14,6 +14,9 @@
>> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
>> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
>> Message-Id: <20210312062656.2477515-11-ankur.a.arora@oracle.com>
>> + [lersek@redhat.com: preserve the empty line between the ICH9_LPC_SMI_F_*
>> + group of macro definitions and the SCRATCH_BUFFER type definition]
>> + Reviewed-by: Laszlo Ersek <lersek@redhat.com>
>>
>> diff --git a/OvmfPkg/SmmControl2Dxe/SmiFeatures.c b/OvmfPkg/SmmControl2Dxe/SmiFeatures.c
>> --- a/OvmfPkg/SmmControl2Dxe/SmiFeatures.c
>> @@ -22,16 +25,15 @@
>> // "etc/smi/supported-features" and "etc/smi/requested-features" fw_cfg files.
>> //
>> #define ICH9_LPC_SMI_F_CPU_HOTPLUG BIT1
>> --
>> +//
>> +// The following bit value stands for "enable CPU hot-unplug, and inject an SMI
>> +// with control value ICH9_APM_CNT_CPU_HOTPLUG upon hot-unplug", in the
>> +// "etc/smi/supported-features" and "etc/smi/requested-features" fw_cfg files.
>> +//
>> +#define ICH9_LPC_SMI_F_CPU_HOT_UNPLUG BIT2
>> +
>> //
>> // Provides a scratch buffer (allocated in EfiReservedMemoryType type memory)
>> - // for the S3 boot script fragment to write to and read from.
>> @@
>> QemuFwCfgReadBytes (sizeof mSmiFeatures, &mSmiFeatures);
>>
>
> Thanks for the contribution,
> Laszlo
>
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#72926): https://edk2.groups.io/g/devel/message/72926
Mute This Topic: https://groups.io/mt/81273600/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-
© 2016 - 2026 Red Hat, Inc.