[Qemu-devel] [PATCH 0/4] spapr: disable hotplugging without OS

Laurent Vivier posted 4 patches 6 years, 10 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20170523111812.13469-1-lvivier@redhat.com
Test checkpatch passed
Test docker passed
Test s390x passed
There is a newer version of this series
hw/ppc/spapr.c              | 103 ++++++++++++++++++++++++++++++++++++--------
hw/ppc/spapr_drc.c          |  20 ++-------
hw/ppc/spapr_hcall.c        |   5 ++-
hw/ppc/spapr_ovec.c         |   8 ++++
include/hw/ppc/spapr.h      |   2 +
include/hw/ppc/spapr_drc.h  |   1 -
include/hw/ppc/spapr_ovec.h |   7 +++
7 files changed, 109 insertions(+), 37 deletions(-)
[Qemu-devel] [PATCH 0/4] spapr: disable hotplugging without OS
Posted by Laurent Vivier 6 years, 10 months ago
If the OS is not started, QEMU sends an event to the OS
that is lost and cannot be recovered. An unplug is not
able to restore QEMU in a coherent state.
So, while the OS is not started, disable CPU and memory hotplug.
We use option vector 6 to know if the OS is started

This series moves error checking for memory hotplug
in a pre_plug function, and introduces the option
vector 6 management. It also revert previous
fix which was not really fixing the hotplug problem
when the OS is not running.

Laurent Vivier (4):
  spapr: add pre_plug function for memory
  spapr: add option vector 6
  spapr: disable hotplugging without OS
  Revert "spapr: fix memory hot-unplugging"

 hw/ppc/spapr.c              | 103 ++++++++++++++++++++++++++++++++++++--------
 hw/ppc/spapr_drc.c          |  20 ++-------
 hw/ppc/spapr_hcall.c        |   5 ++-
 hw/ppc/spapr_ovec.c         |   8 ++++
 include/hw/ppc/spapr.h      |   2 +
 include/hw/ppc/spapr_drc.h  |   1 -
 include/hw/ppc/spapr_ovec.h |   7 +++
 7 files changed, 109 insertions(+), 37 deletions(-)

-- 
2.9.4


Re: [Qemu-devel] [Qemu-ppc] [PATCH 0/4] spapr: disable hotplugging without OS
Posted by Daniel Henrique Barboza 6 years, 10 months ago
Hi Laurent,

This is an interesting patch series. I've been working in the last weeks 
in the DRC
migration, mainly to solve the problem in which a hot CPU unplug will 
not succeed after
a migration if the CPU was hotplugged in the source. The problem 
happened when
migrating with virsh because Libvirt hotplugs the CPU in both source and 
target, and
the DRC state of the hotplugged after the migration is inconsistent. 
This series solves the
issue by preventing it from happening in the first place. Of course that 
migrating DRC states
has other uses (pending unplug operations during a migration, for 
example) so both patch
series can coexist.

One possible issue I see with this series is that it breaks Libvirt 
migration entirely if
a CPU/mem hotplug happens in the target. With your series applied the 
migration
fails before start with:


# ./virsh -c 'qemu:///system' migrate --live --domain dhb_ub1704_nfs 
--desturi qemu+ssh://target_ip/system --timeout 60 --verbose
error: internal error: unable to execute QEMU command 'device_add': CPU 
hotplug not supported without OS

Note that I say "possible issue" because, although I believe we do not 
want to break Libvirt
if possible, I also believe that we need to think about what makes sense 
in QEMU first.



Thanks,


Daniel


On 05/23/2017 08:18 AM, Laurent Vivier wrote:
> If the OS is not started, QEMU sends an event to the OS
> that is lost and cannot be recovered. An unplug is not
> able to restore QEMU in a coherent state.
> So, while the OS is not started, disable CPU and memory hotplug.
> We use option vector 6 to know if the OS is started
>
> This series moves error checking for memory hotplug
> in a pre_plug function, and introduces the option
> vector 6 management. It also revert previous
> fix which was not really fixing the hotplug problem
> when the OS is not running.
>
> Laurent Vivier (4):
>    spapr: add pre_plug function for memory
>    spapr: add option vector 6
>    spapr: disable hotplugging without OS
>    Revert "spapr: fix memory hot-unplugging"
>
>   hw/ppc/spapr.c              | 103 ++++++++++++++++++++++++++++++++++++--------
>   hw/ppc/spapr_drc.c          |  20 ++-------
>   hw/ppc/spapr_hcall.c        |   5 ++-
>   hw/ppc/spapr_ovec.c         |   8 ++++
>   include/hw/ppc/spapr.h      |   2 +
>   include/hw/ppc/spapr_drc.h  |   1 -
>   include/hw/ppc/spapr_ovec.h |   7 +++
>   7 files changed, 109 insertions(+), 37 deletions(-)
>


Re: [Qemu-devel] [Qemu-ppc] [PATCH 0/4] spapr: disable hotplugging without OS
Posted by Daniel Henrique Barboza 6 years, 10 months ago

On 05/23/2017 02:52 PM, Daniel Henrique Barboza wrote:
> Hi Laurent,
>
> This is an interesting patch series. I've been working in the last 
> weeks in the DRC
> migration, mainly to solve the problem in which a hot CPU unplug will 
> not succeed after
> a migration if the CPU was hotplugged in the source. The problem 
> happened when
> migrating with virsh because Libvirt hotplugs the CPU in both source 
> and target, and
> the DRC state of the hotplugged after the migration is inconsistent. 
> This series solves the
> issue by preventing it from happening in the first place. Of course 
> that migrating DRC states
> has other uses (pending unplug operations during a migration, for 
> example) so both patch
> series can coexist.
>
> One possible issue I see with this series is that it breaks Libvirt 
> migration entirely if
> a CPU/mem hotplug happens in the target. With your series applied the 
> migration
> fails before start with:

Sorry: if a migration happens in the *source*, before the migration.

>
>
> # ./virsh -c 'qemu:///system' migrate --live --domain dhb_ub1704_nfs 
> --desturi qemu+ssh://target_ip/system --timeout 60 --verbose
> error: internal error: unable to execute QEMU command 'device_add': 
> CPU hotplug not supported without OS
>
> Note that I say "possible issue" because, although I believe we do not 
> want to break Libvirt
> if possible, I also believe that we need to think about what makes 
> sense in QEMU first.
>
>
>
> Thanks,
>
>
> Daniel
>
>
> On 05/23/2017 08:18 AM, Laurent Vivier wrote:
>> If the OS is not started, QEMU sends an event to the OS
>> that is lost and cannot be recovered. An unplug is not
>> able to restore QEMU in a coherent state.
>> So, while the OS is not started, disable CPU and memory hotplug.
>> We use option vector 6 to know if the OS is started
>>
>> This series moves error checking for memory hotplug
>> in a pre_plug function, and introduces the option
>> vector 6 management. It also revert previous
>> fix which was not really fixing the hotplug problem
>> when the OS is not running.
>>
>> Laurent Vivier (4):
>>    spapr: add pre_plug function for memory
>>    spapr: add option vector 6
>>    spapr: disable hotplugging without OS
>>    Revert "spapr: fix memory hot-unplugging"
>>
>>   hw/ppc/spapr.c              | 103 
>> ++++++++++++++++++++++++++++++++++++--------
>>   hw/ppc/spapr_drc.c          |  20 ++-------
>>   hw/ppc/spapr_hcall.c        |   5 ++-
>>   hw/ppc/spapr_ovec.c         |   8 ++++
>>   include/hw/ppc/spapr.h      |   2 +
>>   include/hw/ppc/spapr_drc.h  |   1 -
>>   include/hw/ppc/spapr_ovec.h |   7 +++
>>   7 files changed, 109 insertions(+), 37 deletions(-)
>>
>
>


Re: [Qemu-devel] [Qemu-ppc] [PATCH 0/4] spapr: disable hotplugging without OS
Posted by Daniel Henrique Barboza 6 years, 10 months ago

On 05/23/2017 03:07 PM, Daniel Henrique Barboza wrote:
>
>
> On 05/23/2017 02:52 PM, Daniel Henrique Barboza wrote:
>> Hi Laurent,
>>
>> This is an interesting patch series. I've been working in the last 
>> weeks in the DRC
>> migration, mainly to solve the problem in which a hot CPU unplug will 
>> not succeed after
>> a migration if the CPU was hotplugged in the source. The problem 
>> happened when
>> migrating with virsh because Libvirt hotplugs the CPU in both source 
>> and target, and
>> the DRC state of the hotplugged after the migration is inconsistent. 
>> This series solves the
>> issue by preventing it from happening in the first place. Of course 
>> that migrating DRC states
>> has other uses (pending unplug operations during a migration, for 
>> example) so both patch
>> series can coexist.
>>
>> One possible issue I see with this series is that it breaks Libvirt 
>> migration entirely if
>> a CPU/mem hotplug happens in the target. With your series applied the 
>> migration
>> fails before start with:
>
> Sorry: if a migration happens in the *source*, before the migration.

Hehe nothing like fixing a typo with another one ...

This is the Libvirt use case that fails with this patch set applied in 
QEMU master, Libvirt 3.4.0 compiled from
source:

# ./virsh start dhb_ub1704_nfs 2
#
# ./virsh setvcpus dhb_ub1704_nfs 2 --live
#
# ./virsh -c 'qemu:///system' migrate --live --domain dhb_ub1704_nfs 
--desturi qemu+ssh://target_ip/system --timeout 60 --verbose
error: internal error: unable to execute QEMU command 'device_add': CPU 
hotplug not supported without OS

This is the error msg that appears in Libvirt daemon:

2017-05-23 18:17:17.844+0000: 159678: error : 
qemuMonitorJSONCheckError:389 : internal error: unable to execute QEMU 
command 'device_add': CPU hotplug not supported without OS



Daniel

>
>>
>>
>> # ./virsh -c 'qemu:///system' migrate --live --domain dhb_ub1704_nfs 
>> --desturi qemu+ssh://target_ip/system --timeout 60 --verbose
>> error: internal error: unable to execute QEMU command 'device_add': 
>> CPU hotplug not supported without OS
>>
>> Note that I say "possible issue" because, although I believe we do 
>> not want to break Libvirt
>> if possible, I also believe that we need to think about what makes 
>> sense in QEMU first.
>>
>>
>>
>> Thanks,
>>
>>
>> Daniel
>>
>>
>> On 05/23/2017 08:18 AM, Laurent Vivier wrote:
>>> If the OS is not started, QEMU sends an event to the OS
>>> that is lost and cannot be recovered. An unplug is not
>>> able to restore QEMU in a coherent state.
>>> So, while the OS is not started, disable CPU and memory hotplug.
>>> We use option vector 6 to know if the OS is started
>>>
>>> This series moves error checking for memory hotplug
>>> in a pre_plug function, and introduces the option
>>> vector 6 management. It also revert previous
>>> fix which was not really fixing the hotplug problem
>>> when the OS is not running.
>>>
>>> Laurent Vivier (4):
>>>    spapr: add pre_plug function for memory
>>>    spapr: add option vector 6
>>>    spapr: disable hotplugging without OS
>>>    Revert "spapr: fix memory hot-unplugging"
>>>
>>>   hw/ppc/spapr.c              | 103 
>>> ++++++++++++++++++++++++++++++++++++--------
>>>   hw/ppc/spapr_drc.c          |  20 ++-------
>>>   hw/ppc/spapr_hcall.c        |   5 ++-
>>>   hw/ppc/spapr_ovec.c         |   8 ++++
>>>   include/hw/ppc/spapr.h      |   2 +
>>>   include/hw/ppc/spapr_drc.h  |   1 -
>>>   include/hw/ppc/spapr_ovec.h |   7 +++
>>>   7 files changed, 109 insertions(+), 37 deletions(-)
>>>
>>
>>
>
>


Re: [Qemu-devel] [Qemu-ppc] [PATCH 0/4] spapr: disable hotplugging without OS
Posted by Laurent Vivier 6 years, 10 months ago
On 23/05/2017 20:22, Daniel Henrique Barboza wrote:
> 
> 
> This is the Libvirt use case that fails with this patch set applied in
> QEMU master, Libvirt 3.4.0 compiled from
> source:
> 
> # ./virsh start dhb_ub1704_nfs 2
> #
> # ./virsh setvcpus dhb_ub1704_nfs 2 --live
> #
> # ./virsh -c 'qemu:///system' migrate --live --domain dhb_ub1704_nfs
> --desturi qemu+ssh://target_ip/system --timeout 60 --verbose
> error: internal error: unable to execute QEMU command 'device_add': CPU
> hotplug not supported without OS

Good point.

I think I should refine my series to allow hotplug if the machine is not
started.

Thanks,
Laurent