[PATCH V1 0/4] Arch agnostic ACPI changes to support vCPU Hotplug (on Archs like ARM)

Salil Mehta via posted 4 patches 1 week, 1 day ago
hw/acpi/cpu.c                  | 70 +++++++++++++++++++++++++++++++---
hw/acpi/generic_event_device.c | 11 ++++++
include/hw/acpi/cpu.h          | 21 ++++++++++
include/hw/core/cpu.h          | 21 ++++++++++
5 files changed, 119 insertions(+), 5 deletions(-)
[PATCH V1 0/4] Arch agnostic ACPI changes to support vCPU Hotplug (on Archs like ARM)
Posted by Salil Mehta via 1 week, 1 day ago
Certain CPU architecture specifications [1][2][3] prohibit changes to the CPUs
*presence* after the kernel has booted. This is because many system
initializations depend on the exact CPU count at boot time and do not expect it
to change afterward. For example, components like interrupt controllers that are
closely coupled with CPUs, or various per-CPU features, may not support
configuration changes once the kernel has been initialized.

This requirement poses a challenge for virtualization features like vCPU
hotplug. To address this, changes to the ACPI AML are necessary to update the
`_STA.PRES` (presence) and `_STA.ENA` (enabled) bits accordingly during guest
initialization, as well as when vCPUs are hot-plugged or hot-unplugged. The
presence of unplugged vCPUs may need to be deliberately *simulated* at the ACPI
level to maintain a *persistent* view of vCPUs for the guest kernel.

This patch set introduces the following features:

1. ACPI Interface with Explicit PRESENT and ENABLED CPU States: It allows the
   guest kernel to evaluate these states using the `_STA` ACPI method.
   
2. Initialization of ACPI CPU States: These states are initialized during
   `machvirt_init` and when vCPUs are hot-(un)plugged. This enables hotpluggable
   vCPUs to be exposed to the guest kernel via ACPI.

3. Support for Migrating ACPI CPU States: The patch set ensures the migration of
   the newly introduced `is_{present,enabled}` ACPI CPU states to the
   destination VM.

The approach is flexible enough to accommodate ARM-like architectures that
intend to implement vCPU hotplug functionality. It is suitable for architectures
facing similar constraints to ARM or those that plan to implement vCPU
hotplugging independently of hardware support (if available).

This patch set is derived from the ARM-specific vCPU hotplug implementation [4]
and includes migration components adaptable to other architectures, following
suggestions [5] made by Igor Mammedov <imammedo@redhat.com>.

It can be applied independently, ensuring compatibility with existing hotplug
support in other architectures. I have tested this patch set in conjunction with
the ARM-specific vCPU hotplug changes (included in the upcoming RFC V5 [6]), and
everything worked as expected. I kindly request maintainers of other
architectures to provide a "Tested-by" after running their respective regression
tests.

Many thanks!


References:
[1] KVMForum 2023 Presentation: Challenges Revisited in Supporting Virt CPU Hotplug on
    architectures that don’t Support CPU Hotplug (like ARM64)
    a. Kernel Link: https://kvm-forum.qemu.org/2023/KVM-forum-cpu-hotplug_7OJ1YyJ.pdf
    b. Qemu Link:  https://kvm-forum.qemu.org/2023/Challenges_Revisited_in_Supporting_Virt_CPU_Hotplug_-__ii0iNb3.pdf
[2] KVMForum 2020 Presentation: Challenges in Supporting Virtual CPU Hotplug on
    SoC Based Systems (like ARM64)
    Link: https://kvmforum2020.sched.com/event/eE4m
[3] Check comment 5 in the bugzilla entry
    Link: https://bugzilla.tianocore.org/show_bug.cgi?id=4481#c5
[4] [PATCH RFC V4 00/33] Support of Virtual CPU Hotplug for ARMv8 Arch
    Link: https://lore.kernel.org/qemu-devel/20241009031815.250096-1-salil.mehta@huawei.com/T/#mf32be203baa568a871dc625b732f666a4c4f1e68
[5] Architecture agnostic ACPI VMSD state migration (Discussion)
    Link: https://lore.kernel.org/qemu-devel/20240715155436.577d34c5@imammedo.users.ipa.redhat.com/
[6] Upcoming RFC V5, Support of Virtual CPU Hotplug for ARMv8 Arch
    Link: https://github.com/salil-mehta/qemu/commits/virt-cpuhp-armv8/rfc-v5

Salil Mehta (4):
  hw/acpi: Initialize ACPI Hotplug CPU Status with Support for vCPU
    `Persistence`
  hw/acpi: Update ACPI CPU Status `is_{present, enabled}` during vCPU
    hot(un)plug
  hw/acpi: Reflect ACPI vCPU {present,enabled} states in ACPI
    _STA.{PRES,ENA} Bits
  hw/acpi: Populate vCPU Hotplug VMSD to migrate `is_{present,enabled}`
    states

 cpu-target.c         patches.vcpuhp.rfc-v5.arch.agnostic.acpi          |  1 +
 hw/acpi/cpu.c                  | 70 +++++++++++++++++++++++++++++++---
 hw/acpi/generic_event_device.c | 11 ++++++
 include/hw/acpi/cpu.h          | 21 ++++++++++
 include/hw/core/cpu.h          | 21 ++++++++++
 5 files changed, 119 insertions(+), 5 deletions(-)

-- 
2.34.1


Re: [PATCH V1 0/4] Arch agnostic ACPI changes to support vCPU Hotplug (on Archs like ARM)
Posted by Miguel Luis 1 week ago
Hi Salil,

I’ve ran the usual tests successfully of hotplug/unplug from the number of cold-booted cpus up to maxcpus and migration on ARM. Please feel free to add:

Tested-by: Miguel Luis <miguel.luis@oracle.com>

Thanks
Miguel

> On 14 Oct 2024, at 19:22, Salil Mehta <salil.mehta@huawei.com> wrote:
> 
> Certain CPU architecture specifications [1][2][3] prohibit changes to the CPUs
> *presence* after the kernel has booted. This is because many system
> initializations depend on the exact CPU count at boot time and do not expect it
> to change afterward. For example, components like interrupt controllers that are
> closely coupled with CPUs, or various per-CPU features, may not support
> configuration changes once the kernel has been initialized.
> 
> This requirement poses a challenge for virtualization features like vCPU
> hotplug. To address this, changes to the ACPI AML are necessary to update the
> `_STA.PRES` (presence) and `_STA.ENA` (enabled) bits accordingly during guest
> initialization, as well as when vCPUs are hot-plugged or hot-unplugged. The
> presence of unplugged vCPUs may need to be deliberately *simulated* at the ACPI
> level to maintain a *persistent* view of vCPUs for the guest kernel.
> 
> This patch set introduces the following features:
> 
> 1. ACPI Interface with Explicit PRESENT and ENABLED CPU States: It allows the
>   guest kernel to evaluate these states using the `_STA` ACPI method.
> 
> 2. Initialization of ACPI CPU States: These states are initialized during
>   `machvirt_init` and when vCPUs are hot-(un)plugged. This enables hotpluggable
>   vCPUs to be exposed to the guest kernel via ACPI.
> 
> 3. Support for Migrating ACPI CPU States: The patch set ensures the migration of
>   the newly introduced `is_{present,enabled}` ACPI CPU states to the
>   destination VM.
> 
> The approach is flexible enough to accommodate ARM-like architectures that
> intend to implement vCPU hotplug functionality. It is suitable for architectures
> facing similar constraints to ARM or those that plan to implement vCPU
> hotplugging independently of hardware support (if available).
> 
> This patch set is derived from the ARM-specific vCPU hotplug implementation [4]
> and includes migration components adaptable to other architectures, following
> suggestions [5] made by Igor Mammedov <imammedo@redhat.com>.
> 
> It can be applied independently, ensuring compatibility with existing hotplug
> support in other architectures. I have tested this patch set in conjunction with
> the ARM-specific vCPU hotplug changes (included in the upcoming RFC V5 [6]), and
> everything worked as expected. I kindly request maintainers of other
> architectures to provide a "Tested-by" after running their respective regression
> tests.
> 
> Many thanks!
> 
> 
> References:
> [1] KVMForum 2023 Presentation: Challenges Revisited in Supporting Virt CPU Hotplug on
>    architectures that don’t Support CPU Hotplug (like ARM64)
>    a. Kernel Link: https://kvm-forum.qemu.org/2023/KVM-forum-cpu-hotplug_7OJ1YyJ.pdf
>    b. Qemu Link:  https://kvm-forum.qemu.org/2023/Challenges_Revisited_in_Supporting_Virt_CPU_Hotplug_-__ii0iNb3.pdf
> [2] KVMForum 2020 Presentation: Challenges in Supporting Virtual CPU Hotplug on
>    SoC Based Systems (like ARM64)
>    Link: https://kvmforum2020.sched.com/event/eE4m
> [3] Check comment 5 in the bugzilla entry
>    Link: https://bugzilla.tianocore.org/show_bug.cgi?id=4481#c5
> [4] [PATCH RFC V4 00/33] Support of Virtual CPU Hotplug for ARMv8 Arch
>    Link: https://lore.kernel.org/qemu-devel/20241009031815.250096-1-salil.mehta@huawei.com/T/#mf32be203baa568a871dc625b732f666a4c4f1e68
> [5] Architecture agnostic ACPI VMSD state migration (Discussion)
>    Link: https://lore.kernel.org/qemu-devel/20240715155436.577d34c5@imammedo.users.ipa.redhat.com/
> [6] Upcoming RFC V5, Support of Virtual CPU Hotplug for ARMv8 Arch
>    Link: https://github.com/salil-mehta/qemu/commits/virt-cpuhp-armv8/rfc-v5
> 
> Salil Mehta (4):
>  hw/acpi: Initialize ACPI Hotplug CPU Status with Support for vCPU
>    `Persistence`
>  hw/acpi: Update ACPI CPU Status `is_{present, enabled}` during vCPU
>    hot(un)plug
>  hw/acpi: Reflect ACPI vCPU {present,enabled} states in ACPI
>    _STA.{PRES,ENA} Bits
>  hw/acpi: Populate vCPU Hotplug VMSD to migrate `is_{present,enabled}`
>    states
> 
> cpu-target.c         patches.vcpuhp.rfc-v5.arch.agnostic.acpi          |  1 +
> hw/acpi/cpu.c                  | 70 +++++++++++++++++++++++++++++++---
> hw/acpi/generic_event_device.c | 11 ++++++
> include/hw/acpi/cpu.h          | 21 ++++++++++
> include/hw/core/cpu.h          | 21 ++++++++++
> 5 files changed, 119 insertions(+), 5 deletions(-)
> 
> -- 
> 2.34.1
> 

Re: [PATCH V1 0/4] Arch agnostic ACPI changes to support vCPU Hotplug (on Archs like ARM)
Posted by Gustavo Romero 4 days, 2 hours ago
Hi Miguel,

On 10/15/24 15:41, Miguel Luis wrote:
> Hi Salil,
> 
> I’ve ran the usual tests successfully of hotplug/unplug from the number of cold-booted cpus up to maxcpus and migration on ARM. Please feel free to add:

Do you mind sharing what cpus you used for your tests?

Did you use tcg or kvm for the tests?

Thanks!


Cheers,
Gustavo
  
> Tested-by: Miguel Luis <miguel.luis@oracle.com>
> 
> Thanks
> Miguel
> 
>> On 14 Oct 2024, at 19:22, Salil Mehta <salil.mehta@huawei.com> wrote:
>>
>> Certain CPU architecture specifications [1][2][3] prohibit changes to the CPUs
>> *presence* after the kernel has booted. This is because many system
>> initializations depend on the exact CPU count at boot time and do not expect it
>> to change afterward. For example, components like interrupt controllers that are
>> closely coupled with CPUs, or various per-CPU features, may not support
>> configuration changes once the kernel has been initialized.
>>
>> This requirement poses a challenge for virtualization features like vCPU
>> hotplug. To address this, changes to the ACPI AML are necessary to update the
>> `_STA.PRES` (presence) and `_STA.ENA` (enabled) bits accordingly during guest
>> initialization, as well as when vCPUs are hot-plugged or hot-unplugged. The
>> presence of unplugged vCPUs may need to be deliberately *simulated* at the ACPI
>> level to maintain a *persistent* view of vCPUs for the guest kernel.
>>
>> This patch set introduces the following features:
>>
>> 1. ACPI Interface with Explicit PRESENT and ENABLED CPU States: It allows the
>>    guest kernel to evaluate these states using the `_STA` ACPI method.
>>
>> 2. Initialization of ACPI CPU States: These states are initialized during
>>    `machvirt_init` and when vCPUs are hot-(un)plugged. This enables hotpluggable
>>    vCPUs to be exposed to the guest kernel via ACPI.
>>
>> 3. Support for Migrating ACPI CPU States: The patch set ensures the migration of
>>    the newly introduced `is_{present,enabled}` ACPI CPU states to the
>>    destination VM.
>>
>> The approach is flexible enough to accommodate ARM-like architectures that
>> intend to implement vCPU hotplug functionality. It is suitable for architectures
>> facing similar constraints to ARM or those that plan to implement vCPU
>> hotplugging independently of hardware support (if available).
>>
>> This patch set is derived from the ARM-specific vCPU hotplug implementation [4]
>> and includes migration components adaptable to other architectures, following
>> suggestions [5] made by Igor Mammedov <imammedo@redhat.com>.
>>
>> It can be applied independently, ensuring compatibility with existing hotplug
>> support in other architectures. I have tested this patch set in conjunction with
>> the ARM-specific vCPU hotplug changes (included in the upcoming RFC V5 [6]), and
>> everything worked as expected. I kindly request maintainers of other
>> architectures to provide a "Tested-by" after running their respective regression
>> tests.
>>
>> Many thanks!
>>
>>
>> References:
>> [1] KVMForum 2023 Presentation: Challenges Revisited in Supporting Virt CPU Hotplug on
>>     architectures that don’t Support CPU Hotplug (like ARM64)
>>     a. Kernel Link: https://kvm-forum.qemu.org/2023/KVM-forum-cpu-hotplug_7OJ1YyJ.pdf
>>     b. Qemu Link:  https://kvm-forum.qemu.org/2023/Challenges_Revisited_in_Supporting_Virt_CPU_Hotplug_-__ii0iNb3.pdf
>> [2] KVMForum 2020 Presentation: Challenges in Supporting Virtual CPU Hotplug on
>>     SoC Based Systems (like ARM64)
>>     Link: https://kvmforum2020.sched.com/event/eE4m
>> [3] Check comment 5 in the bugzilla entry
>>     Link: https://bugzilla.tianocore.org/show_bug.cgi?id=4481#c5
>> [4] [PATCH RFC V4 00/33] Support of Virtual CPU Hotplug for ARMv8 Arch
>>     Link: https://lore.kernel.org/qemu-devel/20241009031815.250096-1-salil.mehta@huawei.com/T/#mf32be203baa568a871dc625b732f666a4c4f1e68
>> [5] Architecture agnostic ACPI VMSD state migration (Discussion)
>>     Link: https://lore.kernel.org/qemu-devel/20240715155436.577d34c5@imammedo.users.ipa.redhat.com/
>> [6] Upcoming RFC V5, Support of Virtual CPU Hotplug for ARMv8 Arch
>>     Link: https://github.com/salil-mehta/qemu/commits/virt-cpuhp-armv8/rfc-v5
>>
>> Salil Mehta (4):
>>   hw/acpi: Initialize ACPI Hotplug CPU Status with Support for vCPU
>>     `Persistence`
>>   hw/acpi: Update ACPI CPU Status `is_{present, enabled}` during vCPU
>>     hot(un)plug
>>   hw/acpi: Reflect ACPI vCPU {present,enabled} states in ACPI
>>     _STA.{PRES,ENA} Bits
>>   hw/acpi: Populate vCPU Hotplug VMSD to migrate `is_{present,enabled}`
>>     states
>>
>> cpu-target.c         patches.vcpuhp.rfc-v5.arch.agnostic.acpi          |  1 +
>> hw/acpi/cpu.c                  | 70 +++++++++++++++++++++++++++++++---
>> hw/acpi/generic_event_device.c | 11 ++++++
>> include/hw/acpi/cpu.h          | 21 ++++++++++
>> include/hw/core/cpu.h          | 21 ++++++++++
>> 5 files changed, 119 insertions(+), 5 deletions(-)
>>
>> -- 
>> 2.34.1
>>
> 


Re: [PATCH V1 0/4] Arch agnostic ACPI changes to support vCPU Hotplug (on Archs like ARM)
Posted by Miguel Luis 1 day, 12 hours ago
Hi Gustavo,

> On 18 Oct 2024, at 17:57, Gustavo Romero <gustavo.romero@linaro.org> wrote:
> 
> Hi Miguel,
> 
> On 10/15/24 15:41, Miguel Luis wrote:
>> Hi Salil,
>> I’ve ran the usual tests successfully of hotplug/unplug from the number of cold-booted cpus up to maxcpus and migration on ARM. Please feel free to add:
> 
> Do you mind sharing what cpus you used for your tests?
> 

Not at all. I’ve used -cpu host and -cpu max. I had used RFC-v5 from [1] in order to test
this patchset as they are at the base of it.

> Did you use tcg or kvm for the tests?

I’ve used both in the following configurations:

-M virt -accel kvm -cpu host
-M virt,gic_version=3 -accel kvm -cpu host
-M virt,gic_version=3 -accel tcg -cpu max

And 

-M virt -accel tcg -cpu max

This last one presented defects in QEMU on behalf of RFC-V5 but, if you don’t
mind, instead of me transcribing everything here, could you please take a look
at [2] ?

I could have given more detail from start, apologies for any confusion on my part.

Thanks,
Miguel

[1]: https://github.com/salil-mehta/qemu/commits/virt-cpuhp-armv8/rfc-v5/
[2]: https://lore.kernel.org/qemu-devel/30478B18-932D-413C-BFF0-5FD70510D9D5@oracle.com/

> 
> Thanks!
> 
> 
> Cheers,
> Gustavo
> 
>> Tested-by: Miguel Luis <miguel.luis@oracle.com>
>> Thanks
>> Miguel
>>> On 14 Oct 2024, at 19:22, Salil Mehta <salil.mehta@huawei.com> wrote:
>>> 
>>> Certain CPU architecture specifications [1][2][3] prohibit changes to the CPUs
>>> *presence* after the kernel has booted. This is because many system
>>> initializations depend on the exact CPU count at boot time and do not expect it
>>> to change afterward. For example, components like interrupt controllers that are
>>> closely coupled with CPUs, or various per-CPU features, may not support
>>> configuration changes once the kernel has been initialized.
>>> 
>>> This requirement poses a challenge for virtualization features like vCPU
>>> hotplug. To address this, changes to the ACPI AML are necessary to update the
>>> `_STA.PRES` (presence) and `_STA.ENA` (enabled) bits accordingly during guest
>>> initialization, as well as when vCPUs are hot-plugged or hot-unplugged. The
>>> presence of unplugged vCPUs may need to be deliberately *simulated* at the ACPI
>>> level to maintain a *persistent* view of vCPUs for the guest kernel.
>>> 
>>> This patch set introduces the following features:
>>> 
>>> 1. ACPI Interface with Explicit PRESENT and ENABLED CPU States: It allows the
>>>   guest kernel to evaluate these states using the `_STA` ACPI method.
>>> 
>>> 2. Initialization of ACPI CPU States: These states are initialized during
>>>   `machvirt_init` and when vCPUs are hot-(un)plugged. This enables hotpluggable
>>>   vCPUs to be exposed to the guest kernel via ACPI.
>>> 
>>> 3. Support for Migrating ACPI CPU States: The patch set ensures the migration of
>>>   the newly introduced `is_{present,enabled}` ACPI CPU states to the
>>>   destination VM.
>>> 
>>> The approach is flexible enough to accommodate ARM-like architectures that
>>> intend to implement vCPU hotplug functionality. It is suitable for architectures
>>> facing similar constraints to ARM or those that plan to implement vCPU
>>> hotplugging independently of hardware support (if available).
>>> 
>>> This patch set is derived from the ARM-specific vCPU hotplug implementation [4]
>>> and includes migration components adaptable to other architectures, following
>>> suggestions [5] made by Igor Mammedov <imammedo@redhat.com>.
>>> 
>>> It can be applied independently, ensuring compatibility with existing hotplug
>>> support in other architectures. I have tested this patch set in conjunction with
>>> the ARM-specific vCPU hotplug changes (included in the upcoming RFC V5 [6]), and
>>> everything worked as expected. I kindly request maintainers of other
>>> architectures to provide a "Tested-by" after running their respective regression
>>> tests.
>>> 
>>> Many thanks!
>>> 
>>> 
>>> References:
>>> [1] KVMForum 2023 Presentation: Challenges Revisited in Supporting Virt CPU Hotplug on
>>>    architectures that don’t Support CPU Hotplug (like ARM64)
>>>    a. Kernel Link: https://kvm-forum.qemu.org/2023/KVM-forum-cpu-hotplug_7OJ1YyJ.pdf
>>>    b. Qemu Link:  https://kvm-forum.qemu.org/2023/Challenges_Revisited_in_Supporting_Virt_CPU_Hotplug_-__ii0iNb3.pdf
>>> [2] KVMForum 2020 Presentation: Challenges in Supporting Virtual CPU Hotplug on
>>>    SoC Based Systems (like ARM64)
>>>    Link: https://kvmforum2020.sched.com/event/eE4m
>>> [3] Check comment 5 in the bugzilla entry
>>>    Link: https://bugzilla.tianocore.org/show_bug.cgi?id=4481#c5
>>> [4] [PATCH RFC V4 00/33] Support of Virtual CPU Hotplug for ARMv8 Arch
>>>    Link: https://lore.kernel.org/qemu-devel/20241009031815.250096-1-salil.mehta@huawei.com/T/#mf32be203baa568a871dc625b732f666a4c4f1e68
>>> [5] Architecture agnostic ACPI VMSD state migration (Discussion)
>>>    Link: https://lore.kernel.org/qemu-devel/20240715155436.577d34c5@imammedo.users.ipa.redhat.com/
>>> [6] Upcoming RFC V5, Support of Virtual CPU Hotplug for ARMv8 Arch
>>>    Link: https://github.com/salil-mehta/qemu/commits/virt-cpuhp-armv8/rfc-v5
>>> 
>>> Salil Mehta (4):
>>>  hw/acpi: Initialize ACPI Hotplug CPU Status with Support for vCPU
>>>    `Persistence`
>>>  hw/acpi: Update ACPI CPU Status `is_{present, enabled}` during vCPU
>>>    hot(un)plug
>>>  hw/acpi: Reflect ACPI vCPU {present,enabled} states in ACPI
>>>    _STA.{PRES,ENA} Bits
>>>  hw/acpi: Populate vCPU Hotplug VMSD to migrate `is_{present,enabled}`
>>>    states
>>> 
>>> cpu-target.c         patches.vcpuhp.rfc-v5.arch.agnostic.acpi          |  1 +
>>> hw/acpi/cpu.c                  | 70 +++++++++++++++++++++++++++++++---
>>> hw/acpi/generic_event_device.c | 11 ++++++
>>> include/hw/acpi/cpu.h          | 21 ++++++++++
>>> include/hw/core/cpu.h          | 21 ++++++++++
>>> 5 files changed, 119 insertions(+), 5 deletions(-)
>>> 
>>> -- 
>>> 2.34.1
>>> 
> 

Re: [PATCH V1 0/4] Arch agnostic ACPI changes to support vCPU Hotplug (on Archs like ARM)
Posted by Gustavo Romero 8 hours ago
Hi Miguel!

On 10/21/24 05:04, Miguel Luis wrote:
> Hi Gustavo,
> 
>> On 18 Oct 2024, at 17:57, Gustavo Romero <gustavo.romero@linaro.org> wrote:
>>
>> Hi Miguel,
>>
>> On 10/15/24 15:41, Miguel Luis wrote:
>>> Hi Salil,
>>> I’ve ran the usual tests successfully of hotplug/unplug from the number of cold-booted cpus up to maxcpus and migration on ARM. Please feel free to add:
>>
>> Do you mind sharing what cpus you used for your tests?
>>
> 
> Not at all. I’ve used -cpu host and -cpu max. I had used RFC-v5 from [1] in order to test
> this patchset as they are at the base of it.

ah, got it. Thanks for the clarification, that's what
I was wondering at first :)


>> Did you use tcg or kvm for the tests?
> 
> I’ve used both in the following configurations:
> 
> -M virt -accel kvm -cpu host
> -M virt,gic_version=3 -accel kvm -cpu host
> -M virt,gic_version=3 -accel tcg -cpu max
> 
> And
> 
> -M virt -accel tcg -cpu max
> 
> This last one presented defects in QEMU on behalf of RFC-V5 but, if you don’t
> mind, instead of me transcribing everything here, could you please take a look
> at [2] ?
> 
> I could have given more detail from start, apologies for any confusion on my part.

Sure, thanks for the pointers ;)


Cheers,
Gustavo

Re: [PATCH V1 0/4] Arch agnostic ACPI changes to support vCPU Hotplug (on Archs like ARM)
Posted by maobibo 1 week ago
With cpu-add/cpu-del command tested on LoongArch system, no migration 
tested. There is no negative influence with LoongArch cpu hotplug.

Regards
Bibo Mao

On 2024/10/15 上午3:22, Salil Mehta via wrote:
> Certain CPU architecture specifications [1][2][3] prohibit changes to the CPUs
> *presence* after the kernel has booted. This is because many system
> initializations depend on the exact CPU count at boot time and do not expect it
> to change afterward. For example, components like interrupt controllers that are
> closely coupled with CPUs, or various per-CPU features, may not support
> configuration changes once the kernel has been initialized.
> 
> This requirement poses a challenge for virtualization features like vCPU
> hotplug. To address this, changes to the ACPI AML are necessary to update the
> `_STA.PRES` (presence) and `_STA.ENA` (enabled) bits accordingly during guest
> initialization, as well as when vCPUs are hot-plugged or hot-unplugged. The
> presence of unplugged vCPUs may need to be deliberately *simulated* at the ACPI
> level to maintain a *persistent* view of vCPUs for the guest kernel.
> 
> This patch set introduces the following features:
> 
> 1. ACPI Interface with Explicit PRESENT and ENABLED CPU States: It allows the
>     guest kernel to evaluate these states using the `_STA` ACPI method.
>     
> 2. Initialization of ACPI CPU States: These states are initialized during
>     `machvirt_init` and when vCPUs are hot-(un)plugged. This enables hotpluggable
>     vCPUs to be exposed to the guest kernel via ACPI.
> 
> 3. Support for Migrating ACPI CPU States: The patch set ensures the migration of
>     the newly introduced `is_{present,enabled}` ACPI CPU states to the
>     destination VM.
> 
> The approach is flexible enough to accommodate ARM-like architectures that
> intend to implement vCPU hotplug functionality. It is suitable for architectures
> facing similar constraints to ARM or those that plan to implement vCPU
> hotplugging independently of hardware support (if available).
> 
> This patch set is derived from the ARM-specific vCPU hotplug implementation [4]
> and includes migration components adaptable to other architectures, following
> suggestions [5] made by Igor Mammedov <imammedo@redhat.com>.
> 
> It can be applied independently, ensuring compatibility with existing hotplug
> support in other architectures. I have tested this patch set in conjunction with
> the ARM-specific vCPU hotplug changes (included in the upcoming RFC V5 [6]), and
> everything worked as expected. I kindly request maintainers of other
> architectures to provide a "Tested-by" after running their respective regression
> tests.
> 
> Many thanks!
> 
> 
> References:
> [1] KVMForum 2023 Presentation: Challenges Revisited in Supporting Virt CPU Hotplug on
>      architectures that don’t Support CPU Hotplug (like ARM64)
>      a. Kernel Link: https://kvm-forum.qemu.org/2023/KVM-forum-cpu-hotplug_7OJ1YyJ.pdf
>      b. Qemu Link:  https://kvm-forum.qemu.org/2023/Challenges_Revisited_in_Supporting_Virt_CPU_Hotplug_-__ii0iNb3.pdf
> [2] KVMForum 2020 Presentation: Challenges in Supporting Virtual CPU Hotplug on
>      SoC Based Systems (like ARM64)
>      Link: https://kvmforum2020.sched.com/event/eE4m
> [3] Check comment 5 in the bugzilla entry
>      Link: https://bugzilla.tianocore.org/show_bug.cgi?id=4481#c5
> [4] [PATCH RFC V4 00/33] Support of Virtual CPU Hotplug for ARMv8 Arch
>      Link: https://lore.kernel.org/qemu-devel/20241009031815.250096-1-salil.mehta@huawei.com/T/#mf32be203baa568a871dc625b732f666a4c4f1e68
> [5] Architecture agnostic ACPI VMSD state migration (Discussion)
>      Link: https://lore.kernel.org/qemu-devel/20240715155436.577d34c5@imammedo.users.ipa.redhat.com/
> [6] Upcoming RFC V5, Support of Virtual CPU Hotplug for ARMv8 Arch
>      Link: https://github.com/salil-mehta/qemu/commits/virt-cpuhp-armv8/rfc-v5
> 
> Salil Mehta (4):
>    hw/acpi: Initialize ACPI Hotplug CPU Status with Support for vCPU
>      `Persistence`
>    hw/acpi: Update ACPI CPU Status `is_{present, enabled}` during vCPU
>      hot(un)plug
>    hw/acpi: Reflect ACPI vCPU {present,enabled} states in ACPI
>      _STA.{PRES,ENA} Bits
>    hw/acpi: Populate vCPU Hotplug VMSD to migrate `is_{present,enabled}`
>      states
> 
>   cpu-target.c         patches.vcpuhp.rfc-v5.arch.agnostic.acpi          |  1 +
>   hw/acpi/cpu.c                  | 70 +++++++++++++++++++++++++++++++---
>   hw/acpi/generic_event_device.c | 11 ++++++
>   include/hw/acpi/cpu.h          | 21 ++++++++++
>   include/hw/core/cpu.h          | 21 ++++++++++
>   5 files changed, 119 insertions(+), 5 deletions(-)
> 


RE: [PATCH V1 0/4] Arch agnostic ACPI changes to support vCPU Hotplug (on Archs like ARM)
Posted by Salil Mehta via 1 week ago
HI Bibo,

>  From: maobibo <maobibo@loongson.cn>
>  Sent: Tuesday, October 15, 2024 4:31 AM
>  To: Salil Mehta <salil.mehta@huawei.com>; qemu-devel@nongnu.org;
>  qemu-arm@nongnu.org; mst@redhat.com
>  
>  With cpu-add/cpu-del command tested on LoongArch system, no migration
>  tested. There is no negative influence with LoongArch cpu hotplug.

Thanks for the confirmation. 

Just curious are guys still using cpu-{add,del} interface for vCPU hotplug?
I thought it was deprecated for device_{add,del}?

https://wiki.qemu.org/Features/CPUHotplug
https://www.qemu.org/docs/master/system/cpu-hotplug.html


Thanks
Salil.

>  
>  Regards
>  Bibo Mao
>  
>  On 2024/10/15 上午3:22, Salil Mehta via wrote:
>  > Certain CPU architecture specifications [1][2][3] prohibit changes to
>  > the CPUs
>  > *presence* after the kernel has booted. This is because many system
>  > initializations depend on the exact CPU count at boot time and do not
>  > expect it to change afterward. For example, components like interrupt
>  > controllers that are closely coupled with CPUs, or various per-CPU
>  > features, may not support configuration changes once the kernel has
>  been initialized.
>  >
>  > This requirement poses a challenge for virtualization features like
>  > vCPU hotplug. To address this, changes to the ACPI AML are necessary
>  > to update the `_STA.PRES` (presence) and `_STA.ENA` (enabled) bits
>  > accordingly during guest initialization, as well as when vCPUs are
>  > hot-plugged or hot-unplugged. The presence of unplugged vCPUs may
>  need
>  > to be deliberately *simulated* at the ACPI level to maintain a *persistent*
>  view of vCPUs for the guest kernel.
>  >
>  > This patch set introduces the following features:
>  >
>  > 1. ACPI Interface with Explicit PRESENT and ENABLED CPU States: It allows
>  the
>  >     guest kernel to evaluate these states using the `_STA` ACPI method.
>  >
>  > 2. Initialization of ACPI CPU States: These states are initialized during
>  >     `machvirt_init` and when vCPUs are hot-(un)plugged. This enables
>  hotpluggable
>  >     vCPUs to be exposed to the guest kernel via ACPI.
>  >
>  > 3. Support for Migrating ACPI CPU States: The patch set ensures the
>  migration of
>  >     the newly introduced `is_{present,enabled}` ACPI CPU states to the
>  >     destination VM.
>  >
>  > The approach is flexible enough to accommodate ARM-like architectures
>  > that intend to implement vCPU hotplug functionality. It is suitable
>  > for architectures facing similar constraints to ARM or those that plan
>  > to implement vCPU hotplugging independently of hardware support (if
>  available).
>  >
>  > This patch set is derived from the ARM-specific vCPU hotplug
>  > implementation [4] and includes migration components adaptable to
>  > other architectures, following suggestions [5] made by Igor Mammedov
>  <imammedo@redhat.com>.
>  >
>  > It can be applied independently, ensuring compatibility with existing
>  > hotplug support in other architectures. I have tested this patch set
>  > in conjunction with the ARM-specific vCPU hotplug changes (included in
>  > the upcoming RFC V5 [6]), and everything worked as expected. I kindly
>  > request maintainers of other architectures to provide a "Tested-by"
>  > after running their respective regression tests.
>  >
>  > Many thanks!
>  >
>  >
>  > References:
>  > [1] KVMForum 2023 Presentation: Challenges Revisited in Supporting Virt
>  CPU Hotplug on
>  >      architectures that don’t Support CPU Hotplug (like ARM64)
>  >      a. Kernel Link: https://kvm-forum.qemu.org/2023/KVM-forum-cpu-
>  hotplug_7OJ1YyJ.pdf
>  >      b. Qemu Link:
>  > https://kvm-
>  forum.qemu.org/2023/Challenges_Revisited_in_Supporting_Vir
>  > t_CPU_Hotplug_-__ii0iNb3.pdf [2] KVMForum 2020 Presentation:
>  > Challenges in Supporting Virtual CPU Hotplug on
>  >      SoC Based Systems (like ARM64)
>  >      Link: https://kvmforum2020.sched.com/event/eE4m
>  > [3] Check comment 5 in the bugzilla entry
>  >      Link: https://bugzilla.tianocore.org/show_bug.cgi?id=4481#c5
>  > [4] [PATCH RFC V4 00/33] Support of Virtual CPU Hotplug for ARMv8 Arch
>  >      Link:
>  > https://lore.kernel.org/qemu-devel/20241009031815.250096-1-salil.mehta
>  > @huawei.com/T/#mf32be203baa568a871dc625b732f666a4c4f1e68
>  > [5] Architecture agnostic ACPI VMSD state migration (Discussion)
>  >      Link:
>  > https://lore.kernel.org/qemu-
>  devel/20240715155436.577d34c5@imammedo.us
>  > ers.ipa.redhat.com/ [6] Upcoming RFC V5, Support of Virtual CPU
>  > Hotplug for ARMv8 Arch
>  >      Link:
>  > https://github.com/salil-mehta/qemu/commits/virt-cpuhp-armv8/rfc-v5
>  >
>  > Salil Mehta (4):
>  >    hw/acpi: Initialize ACPI Hotplug CPU Status with Support for vCPU
>  >      `Persistence`
>  >    hw/acpi: Update ACPI CPU Status `is_{present, enabled}` during vCPU
>  >      hot(un)plug
>  >    hw/acpi: Reflect ACPI vCPU {present,enabled} states in ACPI
>  >      _STA.{PRES,ENA} Bits
>  >    hw/acpi: Populate vCPU Hotplug VMSD to migrate
>  `is_{present,enabled}`
>  >      states
>  >
>  >   cpu-target.c         patches.vcpuhp.rfc-v5.arch.agnostic.acpi          |  1 +
>  >   hw/acpi/cpu.c                  | 70 +++++++++++++++++++++++++++++++---
>  >   hw/acpi/generic_event_device.c | 11 ++++++
>  >   include/hw/acpi/cpu.h          | 21 ++++++++++
>  >   include/hw/core/cpu.h          | 21 ++++++++++
>  >   5 files changed, 119 insertions(+), 5 deletions(-)
>  >

Re: [PATCH V1 0/4] Arch agnostic ACPI changes to support vCPU Hotplug (on Archs like ARM)
Posted by maobibo 6 days, 14 hours ago

On 2024/10/15 下午10:31, Salil Mehta wrote:
> HI Bibo,
> 
>>   From: maobibo <maobibo@loongson.cn>
>>   Sent: Tuesday, October 15, 2024 4:31 AM
>>   To: Salil Mehta <salil.mehta@huawei.com>; qemu-devel@nongnu.org;
>>   qemu-arm@nongnu.org; mst@redhat.com
>>   
>>   With cpu-add/cpu-del command tested on LoongArch system, no migration
>>   tested. There is no negative influence with LoongArch cpu hotplug.
> 
> Thanks for the confirmation.
> 
> Just curious are guys still using cpu-{add,del} interface for vCPU hotplug?
> I thought it was deprecated for device_{add,del}?
I use the device_add/del hmp command, such as
   # device_add 
la464-loongarch-cpu,socket-id=0,core-id=1,thread-id=0,id=cpu-1
   # device_del cpu-1

Tested-by: Bibo Mao <maobibo@loongson.cn>

Regards
Bibo Mao
> 
> https://wiki.qemu.org/Features/CPUHotplug
> https://www.qemu.org/docs/master/system/cpu-hotplug.html
> 
> 
> Thanks
> Salil.
> 
>>   
>>   Regards
>>   Bibo Mao
>>   
>>   On 2024/10/15 上午3:22, Salil Mehta via wrote:
>>   > Certain CPU architecture specifications [1][2][3] prohibit changes to
>>   > the CPUs
>>   > *presence* after the kernel has booted. This is because many system
>>   > initializations depend on the exact CPU count at boot time and do not
>>   > expect it to change afterward. For example, components like interrupt
>>   > controllers that are closely coupled with CPUs, or various per-CPU
>>   > features, may not support configuration changes once the kernel has
>>   been initialized.
>>   >
>>   > This requirement poses a challenge for virtualization features like
>>   > vCPU hotplug. To address this, changes to the ACPI AML are necessary
>>   > to update the `_STA.PRES` (presence) and `_STA.ENA` (enabled) bits
>>   > accordingly during guest initialization, as well as when vCPUs are
>>   > hot-plugged or hot-unplugged. The presence of unplugged vCPUs may
>>   need
>>   > to be deliberately *simulated* at the ACPI level to maintain a *persistent*
>>   view of vCPUs for the guest kernel.
>>   >
>>   > This patch set introduces the following features:
>>   >
>>   > 1. ACPI Interface with Explicit PRESENT and ENABLED CPU States: It allows
>>   the
>>   >     guest kernel to evaluate these states using the `_STA` ACPI method.
>>   >
>>   > 2. Initialization of ACPI CPU States: These states are initialized during
>>   >     `machvirt_init` and when vCPUs are hot-(un)plugged. This enables
>>   hotpluggable
>>   >     vCPUs to be exposed to the guest kernel via ACPI.
>>   >
>>   > 3. Support for Migrating ACPI CPU States: The patch set ensures the
>>   migration of
>>   >     the newly introduced `is_{present,enabled}` ACPI CPU states to the
>>   >     destination VM.
>>   >
>>   > The approach is flexible enough to accommodate ARM-like architectures
>>   > that intend to implement vCPU hotplug functionality. It is suitable
>>   > for architectures facing similar constraints to ARM or those that plan
>>   > to implement vCPU hotplugging independently of hardware support (if
>>   available).
>>   >
>>   > This patch set is derived from the ARM-specific vCPU hotplug
>>   > implementation [4] and includes migration components adaptable to
>>   > other architectures, following suggestions [5] made by Igor Mammedov
>>   <imammedo@redhat.com>.
>>   >
>>   > It can be applied independently, ensuring compatibility with existing
>>   > hotplug support in other architectures. I have tested this patch set
>>   > in conjunction with the ARM-specific vCPU hotplug changes (included in
>>   > the upcoming RFC V5 [6]), and everything worked as expected. I kindly
>>   > request maintainers of other architectures to provide a "Tested-by"
>>   > after running their respective regression tests.
>>   >
>>   > Many thanks!
>>   >
>>   >
>>   > References:
>>   > [1] KVMForum 2023 Presentation: Challenges Revisited in Supporting Virt
>>   CPU Hotplug on
>>   >      architectures that don’t Support CPU Hotplug (like ARM64)
>>   >      a. Kernel Link: https://kvm-forum.qemu.org/2023/KVM-forum-cpu-
>>   hotplug_7OJ1YyJ.pdf
>>   >      b. Qemu Link:
>>   > https://kvm-
>>   forum.qemu.org/2023/Challenges_Revisited_in_Supporting_Vir
>>   > t_CPU_Hotplug_-__ii0iNb3.pdf [2] KVMForum 2020 Presentation:
>>   > Challenges in Supporting Virtual CPU Hotplug on
>>   >      SoC Based Systems (like ARM64)
>>   >      Link: https://kvmforum2020.sched.com/event/eE4m
>>   > [3] Check comment 5 in the bugzilla entry
>>   >      Link: https://bugzilla.tianocore.org/show_bug.cgi?id=4481#c5
>>   > [4] [PATCH RFC V4 00/33] Support of Virtual CPU Hotplug for ARMv8 Arch
>>   >      Link:
>>   > https://lore.kernel.org/qemu-devel/20241009031815.250096-1-salil.mehta
>>   > @huawei.com/T/#mf32be203baa568a871dc625b732f666a4c4f1e68
>>   > [5] Architecture agnostic ACPI VMSD state migration (Discussion)
>>   >      Link:
>>   > https://lore.kernel.org/qemu-
>>   devel/20240715155436.577d34c5@imammedo.us
>>   > ers.ipa.redhat.com/ [6] Upcoming RFC V5, Support of Virtual CPU
>>   > Hotplug for ARMv8 Arch
>>   >      Link:
>>   > https://github.com/salil-mehta/qemu/commits/virt-cpuhp-armv8/rfc-v5
>>   >
>>   > Salil Mehta (4):
>>   >    hw/acpi: Initialize ACPI Hotplug CPU Status with Support for vCPU
>>   >      `Persistence`
>>   >    hw/acpi: Update ACPI CPU Status `is_{present, enabled}` during vCPU
>>   >      hot(un)plug
>>   >    hw/acpi: Reflect ACPI vCPU {present,enabled} states in ACPI
>>   >      _STA.{PRES,ENA} Bits
>>   >    hw/acpi: Populate vCPU Hotplug VMSD to migrate
>>   `is_{present,enabled}`
>>   >      states
>>   >
>>   >   cpu-target.c         patches.vcpuhp.rfc-v5.arch.agnostic.acpi          |  1 +
>>   >   hw/acpi/cpu.c                  | 70 +++++++++++++++++++++++++++++++---
>>   >   hw/acpi/generic_event_device.c | 11 ++++++
>>   >   include/hw/acpi/cpu.h          | 21 ++++++++++
>>   >   include/hw/core/cpu.h          | 21 ++++++++++
>>   >   5 files changed, 119 insertions(+), 5 deletions(-)
>>   >
> 


Re: [PATCH V1 0/4] Arch agnostic ACPI changes to support vCPU Hotplug (on Archs like ARM)
Posted by Igor Mammedov 4 days, 5 hours ago
On Mon, 14 Oct 2024 20:22:01 +0100
Salil Mehta <salil.mehta@huawei.com> wrote:

> Certain CPU architecture specifications [1][2][3] prohibit changes to the CPUs
> *presence* after the kernel has booted. This is because many system
> initializations depend on the exact CPU count at boot time and do not expect it
> to change afterward. For example, components like interrupt controllers that are
> closely coupled with CPUs, or various per-CPU features, may not support
> configuration changes once the kernel has been initialized.
> 
> This requirement poses a challenge for virtualization features like vCPU
> hotplug. To address this, changes to the ACPI AML are necessary to update the
> `_STA.PRES` (presence) and `_STA.ENA` (enabled) bits accordingly during guest
> initialization, as well as when vCPUs are hot-plugged or hot-unplugged. The
> presence of unplugged vCPUs may need to be deliberately *simulated* at the ACPI
> level to maintain a *persistent* view of vCPUs for the guest kernel.

the series is peppered with *simulated* idea, which after looking at code
I read as 'fake'. While it's obvious to author why things need to be faked
at this time, it will be forgotten later on. And cause a lot swearing from
whoever will have to deal with this code down the road.

Salil, I'm sorry that review comes out as mostly negative but for me having to
repeat 'simulated' some many times, hints that the there is
something wrong with design and that we should re-evaluate the approach.

ps:
see comments on 1/4 for suggestions


> This patch set introduces the following features:
> 
> 1. ACPI Interface with Explicit PRESENT and ENABLED CPU States: It allows the
>    guest kernel to evaluate these states using the `_STA` ACPI method.
>    
> 2. Initialization of ACPI CPU States: These states are initialized during
>    `machvirt_init` and when vCPUs are hot-(un)plugged. This enables hotpluggable
>    vCPUs to be exposed to the guest kernel via ACPI.
> 
> 3. Support for Migrating ACPI CPU States: The patch set ensures the migration of
>    the newly introduced `is_{present,enabled}` ACPI CPU states to the
>    destination VM.
> 
> The approach is flexible enough to accommodate ARM-like architectures that
> intend to implement vCPU hotplug functionality. It is suitable for architectures
> facing similar constraints to ARM or those that plan to implement vCPU
> hotplugging independently of hardware support (if available).
> 
> This patch set is derived from the ARM-specific vCPU hotplug implementation [4]
> and includes migration components adaptable to other architectures, following
> suggestions [5] made by Igor Mammedov <imammedo@redhat.com>.
> 
> It can be applied independently, ensuring compatibility with existing hotplug
> support in other architectures. I have tested this patch set in conjunction with
> the ARM-specific vCPU hotplug changes (included in the upcoming RFC V5 [6]), and
> everything worked as expected. I kindly request maintainers of other
> architectures to provide a "Tested-by" after running their respective regression
> tests.
> 
> Many thanks!
> 
> 
> References:
> [1] KVMForum 2023 Presentation: Challenges Revisited in Supporting Virt CPU Hotplug on
>     architectures that don’t Support CPU Hotplug (like ARM64)
>     a. Kernel Link: https://kvm-forum.qemu.org/2023/KVM-forum-cpu-hotplug_7OJ1YyJ.pdf
>     b. Qemu Link:  https://kvm-forum.qemu.org/2023/Challenges_Revisited_in_Supporting_Virt_CPU_Hotplug_-__ii0iNb3.pdf
> [2] KVMForum 2020 Presentation: Challenges in Supporting Virtual CPU Hotplug on
>     SoC Based Systems (like ARM64)
>     Link: https://kvmforum2020.sched.com/event/eE4m
> [3] Check comment 5 in the bugzilla entry
>     Link: https://bugzilla.tianocore.org/show_bug.cgi?id=4481#c5
> [4] [PATCH RFC V4 00/33] Support of Virtual CPU Hotplug for ARMv8 Arch
>     Link: https://lore.kernel.org/qemu-devel/20241009031815.250096-1-salil.mehta@huawei.com/T/#mf32be203baa568a871dc625b732f666a4c4f1e68
> [5] Architecture agnostic ACPI VMSD state migration (Discussion)
>     Link: https://lore.kernel.org/qemu-devel/20240715155436.577d34c5@imammedo.users.ipa.redhat.com/
> [6] Upcoming RFC V5, Support of Virtual CPU Hotplug for ARMv8 Arch
>     Link: https://github.com/salil-mehta/qemu/commits/virt-cpuhp-armv8/rfc-v5
> 
> Salil Mehta (4):
>   hw/acpi: Initialize ACPI Hotplug CPU Status with Support for vCPU
>     `Persistence`
>   hw/acpi: Update ACPI CPU Status `is_{present, enabled}` during vCPU
>     hot(un)plug
>   hw/acpi: Reflect ACPI vCPU {present,enabled} states in ACPI
>     _STA.{PRES,ENA} Bits
>   hw/acpi: Populate vCPU Hotplug VMSD to migrate `is_{present,enabled}`
>     states
> 
>  cpu-target.c         patches.vcpuhp.rfc-v5.arch.agnostic.acpi          |  1 +
>  hw/acpi/cpu.c                  | 70 +++++++++++++++++++++++++++++++---
>  hw/acpi/generic_event_device.c | 11 ++++++
>  include/hw/acpi/cpu.h          | 21 ++++++++++
>  include/hw/core/cpu.h          | 21 ++++++++++
>  5 files changed, 119 insertions(+), 5 deletions(-)
> 
Re: [PATCH V1 0/4] Arch agnostic ACPI changes to support vCPU Hotplug (on Archs like ARM)
Posted by Gustavo Romero 1 day, 18 hours ago
Hi Igor,

On 10/18/24 11:46, Igor Mammedov wrote:
> On Mon, 14 Oct 2024 20:22:01 +0100
> Salil Mehta <salil.mehta@huawei.com> wrote:
> 
>> Certain CPU architecture specifications [1][2][3] prohibit changes to the CPUs
>> *presence* after the kernel has booted. This is because many system
>> initializations depend on the exact CPU count at boot time and do not expect it
>> to change afterward. For example, components like interrupt controllers that are
>> closely coupled with CPUs, or various per-CPU features, may not support
>> configuration changes once the kernel has been initialized.
>>
>> This requirement poses a challenge for virtualization features like vCPU
>> hotplug. To address this, changes to the ACPI AML are necessary to update the
>> `_STA.PRES` (presence) and `_STA.ENA` (enabled) bits accordingly during guest
>> initialization, as well as when vCPUs are hot-plugged or hot-unplugged. The
>> presence of unplugged vCPUs may need to be deliberately *simulated* at the ACPI
>> level to maintain a *persistent* view of vCPUs for the guest kernel.
> 
> the series is peppered with *simulated* idea, which after looking at code
> I read as 'fake'. While it's obvious to author why things need to be faked
> at this time, it will be forgotten later on. And cause a lot swearing from
> whoever will have to deal with this code down the road.
> 
> Salil, I'm sorry that review comes out as mostly negative but for me having to
> repeat 'simulated' some many times, hints that the there is
> something wrong with design and that we should re-evaluate the approach.

I think 'simulated' was indeed a badly chosen name because the series,
as read it, is merely saving the hotplugged/unplugged vCPU states to the
ACPI for persistence and later restoration, so I think s/simulated/saved/ is
reasonable here, i.e., ACPI now reflects the vCPU states after hotplug/unplug.
QEMU manipulates the ACPI tables and methods to reflect the machine config.
and this series is essentially doing the same. Why that's not ok?


Cheers,
Gustavo

> ps:
> see comments on 1/4 for suggestions
> 
> 
>> This patch set introduces the following features:
>>
>> 1. ACPI Interface with Explicit PRESENT and ENABLED CPU States: It allows the
>>     guest kernel to evaluate these states using the `_STA` ACPI method.
>>     
>> 2. Initialization of ACPI CPU States: These states are initialized during
>>     `machvirt_init` and when vCPUs are hot-(un)plugged. This enables hotpluggable
>>     vCPUs to be exposed to the guest kernel via ACPI.
>>
>> 3. Support for Migrating ACPI CPU States: The patch set ensures the migration of
>>     the newly introduced `is_{present,enabled}` ACPI CPU states to the
>>     destination VM.
>>
>> The approach is flexible enough to accommodate ARM-like architectures that
>> intend to implement vCPU hotplug functionality. It is suitable for architectures
>> facing similar constraints to ARM or those that plan to implement vCPU
>> hotplugging independently of hardware support (if available).
>>
>> This patch set is derived from the ARM-specific vCPU hotplug implementation [4]
>> and includes migration components adaptable to other architectures, following
>> suggestions [5] made by Igor Mammedov <imammedo@redhat.com>.
>>
>> It can be applied independently, ensuring compatibility with existing hotplug
>> support in other architectures. I have tested this patch set in conjunction with
>> the ARM-specific vCPU hotplug changes (included in the upcoming RFC V5 [6]), and
>> everything worked as expected. I kindly request maintainers of other
>> architectures to provide a "Tested-by" after running their respective regression
>> tests.
>>
>> Many thanks!
>>
>>
>> References:
>> [1] KVMForum 2023 Presentation: Challenges Revisited in Supporting Virt CPU Hotplug on
>>      architectures that don’t Support CPU Hotplug (like ARM64)
>>      a. Kernel Link: https://kvm-forum.qemu.org/2023/KVM-forum-cpu-hotplug_7OJ1YyJ.pdf
>>      b. Qemu Link:  https://kvm-forum.qemu.org/2023/Challenges_Revisited_in_Supporting_Virt_CPU_Hotplug_-__ii0iNb3.pdf
>> [2] KVMForum 2020 Presentation: Challenges in Supporting Virtual CPU Hotplug on
>>      SoC Based Systems (like ARM64)
>>      Link: https://kvmforum2020.sched.com/event/eE4m
>> [3] Check comment 5 in the bugzilla entry
>>      Link: https://bugzilla.tianocore.org/show_bug.cgi?id=4481#c5
>> [4] [PATCH RFC V4 00/33] Support of Virtual CPU Hotplug for ARMv8 Arch
>>      Link: https://lore.kernel.org/qemu-devel/20241009031815.250096-1-salil.mehta@huawei.com/T/#mf32be203baa568a871dc625b732f666a4c4f1e68
>> [5] Architecture agnostic ACPI VMSD state migration (Discussion)
>>      Link: https://lore.kernel.org/qemu-devel/20240715155436.577d34c5@imammedo.users.ipa.redhat.com/
>> [6] Upcoming RFC V5, Support of Virtual CPU Hotplug for ARMv8 Arch
>>      Link: https://github.com/salil-mehta/qemu/commits/virt-cpuhp-armv8/rfc-v5
>>
>> Salil Mehta (4):
>>    hw/acpi: Initialize ACPI Hotplug CPU Status with Support for vCPU
>>      `Persistence`
>>    hw/acpi: Update ACPI CPU Status `is_{present, enabled}` during vCPU
>>      hot(un)plug
>>    hw/acpi: Reflect ACPI vCPU {present,enabled} states in ACPI
>>      _STA.{PRES,ENA} Bits
>>    hw/acpi: Populate vCPU Hotplug VMSD to migrate `is_{present,enabled}`
>>      states
>>
>>   cpu-target.c         patches.vcpuhp.rfc-v5.arch.agnostic.acpi          |  1 +
>>   hw/acpi/cpu.c                  | 70 +++++++++++++++++++++++++++++++---
>>   hw/acpi/generic_event_device.c | 11 ++++++
>>   include/hw/acpi/cpu.h          | 21 ++++++++++
>>   include/hw/core/cpu.h          | 21 ++++++++++
>>   5 files changed, 119 insertions(+), 5 deletions(-)
>>
>