[Qemu-devel] [PATCHv2 0/8] spapr: DRC cleanups (part VI)

David Gibson posted 8 patches 6 years, 9 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20170712055317.26225-1-david@gibson.dropbear.id.au
Test FreeBSD passed
Test checkpatch passed
Test docker passed
Test s390x passed
hw/ppc/spapr.c             |  89 +++-------
hw/ppc/spapr_drc.c         | 399 ++++++++++++++++++++++++---------------------
hw/ppc/spapr_pci.c         |  17 +-
hw/ppc/trace-events        |   3 +-
include/hw/ppc/spapr_drc.h |  74 ++++++---
5 files changed, 301 insertions(+), 281 deletions(-)
[Qemu-devel] [PATCHv2 0/8] spapr: DRC cleanups (part VI)
Posted by David Gibson 6 years, 9 months ago
This sixth set of DRC cleanup patches is a complete rework of DRC
state management.  We stop tracking some unnecessary things, and
change the basic state representation to a simpler and more robust
model.

Many of the patches in this set "break" migration from earlier git
snapshots, but not from any released qemu version.  The previous
migration stream format had multiple problems, so better to fix them
now, before 2.10 is out.

Although there are certainly more things that can be improved in the
DRC system, with this series we should have a solid foundation for
migrating DRCs - the state trasferred is about as minimal and well
defined as it's possible to be.

Changes since v1:
  * Rebased onto current tree
  * Added cleanup to unplug path
  * Added restriction of DR-indicator to physical DRCs
  * Included revised version of Laurent's patch to correctly handle
    things "hot" plugged before incoming migration

David Gibson (7):
  spapr: Remove 'awaiting_allocation' DRC flag
  spapr: Simplify unplug path
  spapr: Refactor spapr_drc_detach()
  spapr: Cleanups relating to DRC awaiting_release field
  spapr: Consolidate DRC state variables
  spapr: Remove sPAPRConfigureConnectorState sub-structure
  spapr: Implement DR-indicator for physical DRCs only

Laurent Vivier (1):
  spapr: Treat devices added before inbound migration as coldplugged

 hw/ppc/spapr.c             |  89 +++-------
 hw/ppc/spapr_drc.c         | 399 ++++++++++++++++++++++++---------------------
 hw/ppc/spapr_pci.c         |  17 +-
 hw/ppc/trace-events        |   3 +-
 include/hw/ppc/spapr_drc.h |  74 ++++++---
 5 files changed, 301 insertions(+), 281 deletions(-)

-- 
2.9.4


Re: [Qemu-devel] [Qemu-ppc] [PATCHv2 0/8] spapr: DRC cleanups (part VI)
Posted by Daniel Henrique Barboza 6 years, 9 months ago
The dreaded Libvirt hotplug-migrate-hotunplug scenario is working nicely.

device_add when the machine is in RUN_STATE_PRELAUNCH (-S) still doesn't
work but it is expected - as discussed in "[RFC drcVI PATCH] spapr: 
reset DRCs
on migration pre_load​", this scenario can't be fixed solely by this DRC 
cleanup.

Given that we'll review the DT code sometime in the future I guess we can
postpone the fix for device_adding in pre-launch for that time.


Tested-by: Daniel Barboza <danielhb@linux.vnet.ibm.com>

On 07/12/2017 02:53 AM, David Gibson wrote:
> This sixth set of DRC cleanup patches is a complete rework of DRC
> state management.  We stop tracking some unnecessary things, and
> change the basic state representation to a simpler and more robust
> model.
>
> Many of the patches in this set "break" migration from earlier git
> snapshots, but not from any released qemu version.  The previous
> migration stream format had multiple problems, so better to fix them
> now, before 2.10 is out.
>
> Although there are certainly more things that can be improved in the
> DRC system, with this series we should have a solid foundation for
> migrating DRCs - the state trasferred is about as minimal and well
> defined as it's possible to be.
>
> Changes since v1:
>    * Rebased onto current tree
>    * Added cleanup to unplug path
>    * Added restriction of DR-indicator to physical DRCs
>    * Included revised version of Laurent's patch to correctly handle
>      things "hot" plugged before incoming migration
>
> David Gibson (7):
>    spapr: Remove 'awaiting_allocation' DRC flag
>    spapr: Simplify unplug path
>    spapr: Refactor spapr_drc_detach()
>    spapr: Cleanups relating to DRC awaiting_release field
>    spapr: Consolidate DRC state variables
>    spapr: Remove sPAPRConfigureConnectorState sub-structure
>    spapr: Implement DR-indicator for physical DRCs only
>
> Laurent Vivier (1):
>    spapr: Treat devices added before inbound migration as coldplugged
>
>   hw/ppc/spapr.c             |  89 +++-------
>   hw/ppc/spapr_drc.c         | 399 ++++++++++++++++++++++++---------------------
>   hw/ppc/spapr_pci.c         |  17 +-
>   hw/ppc/trace-events        |   3 +-
>   include/hw/ppc/spapr_drc.h |  74 ++++++---
>   5 files changed, 301 insertions(+), 281 deletions(-)
>


Re: [Qemu-devel] [Qemu-ppc] [PATCHv2 0/8] spapr: DRC cleanups (part VI)
Posted by David Gibson 6 years, 9 months ago
On Wed, Jul 12, 2017 at 10:48:38AM -0300, Daniel Henrique Barboza wrote:
> The dreaded Libvirt hotplug-migrate-hotunplug scenario is working nicely.

Good to hear.

> device_add when the machine is in RUN_STATE_PRELAUNCH (-S) still doesn't
> work but it is expected - as discussed in "[RFC drcVI PATCH] spapr: reset
> DRCs
> on migration pre_load​", this scenario can't be fixed solely by this DRC
> cleanup.

Hmm.. what's the exact test case you're using here?  The prelaunch
case I tried _did_ work (queueing the event during prelaunch, then
completing the hotplug sequence once the guest had booted).

> Given that we'll review the DT code sometime in the future I guess we can
> postpone the fix for device_adding in pre-launch for that time.
> 
> 
> Tested-by: Daniel Barboza <danielhb@linux.vnet.ibm.com>
> 
> On 07/12/2017 02:53 AM, David Gibson wrote:
> > This sixth set of DRC cleanup patches is a complete rework of DRC
> > state management.  We stop tracking some unnecessary things, and
> > change the basic state representation to a simpler and more robust
> > model.
> > 
> > Many of the patches in this set "break" migration from earlier git
> > snapshots, but not from any released qemu version.  The previous
> > migration stream format had multiple problems, so better to fix them
> > now, before 2.10 is out.
> > 
> > Although there are certainly more things that can be improved in the
> > DRC system, with this series we should have a solid foundation for
> > migrating DRCs - the state trasferred is about as minimal and well
> > defined as it's possible to be.
> > 
> > Changes since v1:
> >    * Rebased onto current tree
> >    * Added cleanup to unplug path
> >    * Added restriction of DR-indicator to physical DRCs
> >    * Included revised version of Laurent's patch to correctly handle
> >      things "hot" plugged before incoming migration
> > 
> > David Gibson (7):
> >    spapr: Remove 'awaiting_allocation' DRC flag
> >    spapr: Simplify unplug path
> >    spapr: Refactor spapr_drc_detach()
> >    spapr: Cleanups relating to DRC awaiting_release field
> >    spapr: Consolidate DRC state variables
> >    spapr: Remove sPAPRConfigureConnectorState sub-structure
> >    spapr: Implement DR-indicator for physical DRCs only
> > 
> > Laurent Vivier (1):
> >    spapr: Treat devices added before inbound migration as coldplugged
> > 
> >   hw/ppc/spapr.c             |  89 +++-------
> >   hw/ppc/spapr_drc.c         | 399 ++++++++++++++++++++++++---------------------
> >   hw/ppc/spapr_pci.c         |  17 +-
> >   hw/ppc/trace-events        |   3 +-
> >   include/hw/ppc/spapr_drc.h |  74 ++++++---
> >   5 files changed, 301 insertions(+), 281 deletions(-)
> > 
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
Re: [Qemu-devel] [Qemu-ppc] [PATCHv2 0/8] spapr: DRC cleanups (part VI)
Posted by Daniel Henrique Barboza 6 years, 9 months ago

On 07/12/2017 09:57 PM, David Gibson wrote:
> On Wed, Jul 12, 2017 at 10:48:38AM -0300, Daniel Henrique Barboza wrote:
>> The dreaded Libvirt hotplug-migrate-hotunplug scenario is working nicely.
> Good to hear.
>
>> device_add when the machine is in RUN_STATE_PRELAUNCH (-S) still doesn't
>> work but it is expected - as discussed in "[RFC drcVI PATCH] spapr: reset
>> DRCs
>> on migration pre_load​", this scenario can't be fixed solely by this DRC
>> cleanup.
> Hmm.. what's the exact test case you're using here?  The prelaunch
> case I tried _did_ work (queueing the event during prelaunch, then
> completing the hotplug sequence once the guest had booted).

This is the test case:

sudo ./qemu-system-ppc64 -name migrate_qemu -boot strict=on --enable-kvm 
-device nec-usb-xhci,id=usb,bus=pci.0,addr=0xf -device 
spapr-vscsi,id=scsi0,reg=0x2000 -smp 
1,maxcpus=4,sockets=4,cores=1,threads=1 --machine 
pseries,accel=kvm,usb=off,dump-guest-core=off -m 4G,slots=32,maxmem=32G 
-drive 
file=/home/danielhb/vm_imgs/ubuntu1704.qcow2,format=qcow2,if=none,id=drive-virtio-disk0,cache=none 
-device 
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x2,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 
-nographic -S
QEMU 2.9.50 monitor - type 'help' for more information
(qemu)
(qemu) device_add host-spapr-cpu-core,id=core1,core-id=1
(qemu) cont

(...)

After OS boots:

danielhb@ubuntu1704:~$ lscpu
Architecture:          ppc64le
Byte Order:            Little Endian
CPU(s):                1
On-line CPU(s) list:   0
Thread(s) per core:    1
Core(s) per socket:    1
Socket(s):             1
NUMA node(s):          1
Model:                 2.1 (pvr 004b 0201)
Model name:            POWER8E (raw), altivec supported
Hypervisor vendor:     horizontal
Virtualization type:   full
L1d cache:             64K
L1i cache:             32K
NUMA node0 CPU(s):     0
danielhb@ubuntu1704:~$ (qemu)
(qemu) info cpus
* CPU #0: nip=0xc0000000000a3e0c thread_id=6134
   CPU #1: nip=0x0000000000000000 (halted) thread_id=6163
(qemu) info hotpluggable-cpus
Hotpluggable CPUs:
   type: "host-spapr-cpu-core"
   vcpus_count: "1"
   CPUInstance Properties:
     core-id: "3"
   type: "host-spapr-cpu-core"
   vcpus_count: "1"
   CPUInstance Properties:
     core-id: "2"
   type: "host-spapr-cpu-core"
   vcpus_count: "1"
   qom_path: "/machine/peripheral/core1"
   CPUInstance Properties:
     core-id: "1"
   type: "host-spapr-cpu-core"
   vcpus_count: "1"
   qom_path: "/machine/unattached/device[0]"
   CPUInstance Properties:
     core-id: "0"
(qemu)


I am not aware of anyone (e.g. Libvirt) using device_add at that point 
so it's not
urgent to make this work AFAIC. I was just curious of why it doesn't.


Daniel



>
>> Given that we'll review the DT code sometime in the future I guess we can
>> postpone the fix for device_adding in pre-launch for that time.
>>
>>
>> Tested-by: Daniel Barboza <danielhb@linux.vnet.ibm.com>
>>
>> On 07/12/2017 02:53 AM, David Gibson wrote:
>>> This sixth set of DRC cleanup patches is a complete rework of DRC
>>> state management.  We stop tracking some unnecessary things, and
>>> change the basic state representation to a simpler and more robust
>>> model.
>>>
>>> Many of the patches in this set "break" migration from earlier git
>>> snapshots, but not from any released qemu version.  The previous
>>> migration stream format had multiple problems, so better to fix them
>>> now, before 2.10 is out.
>>>
>>> Although there are certainly more things that can be improved in the
>>> DRC system, with this series we should have a solid foundation for
>>> migrating DRCs - the state trasferred is about as minimal and well
>>> defined as it's possible to be.
>>>
>>> Changes since v1:
>>>     * Rebased onto current tree
>>>     * Added cleanup to unplug path
>>>     * Added restriction of DR-indicator to physical DRCs
>>>     * Included revised version of Laurent's patch to correctly handle
>>>       things "hot" plugged before incoming migration
>>>
>>> David Gibson (7):
>>>     spapr: Remove 'awaiting_allocation' DRC flag
>>>     spapr: Simplify unplug path
>>>     spapr: Refactor spapr_drc_detach()
>>>     spapr: Cleanups relating to DRC awaiting_release field
>>>     spapr: Consolidate DRC state variables
>>>     spapr: Remove sPAPRConfigureConnectorState sub-structure
>>>     spapr: Implement DR-indicator for physical DRCs only
>>>
>>> Laurent Vivier (1):
>>>     spapr: Treat devices added before inbound migration as coldplugged
>>>
>>>    hw/ppc/spapr.c             |  89 +++-------
>>>    hw/ppc/spapr_drc.c         | 399 ++++++++++++++++++++++++---------------------
>>>    hw/ppc/spapr_pci.c         |  17 +-
>>>    hw/ppc/trace-events        |   3 +-
>>>    include/hw/ppc/spapr_drc.h |  74 ++++++---
>>>    5 files changed, 301 insertions(+), 281 deletions(-)
>>>


Re: [Qemu-devel] [Qemu-ppc] [PATCHv2 0/8] spapr: DRC cleanups (part VI)
Posted by David Gibson 6 years, 9 months ago
On Thu, Jul 13, 2017 at 07:13:23AM -0300, Daniel Henrique Barboza wrote:
> 
> 
> On 07/12/2017 09:57 PM, David Gibson wrote:
> > On Wed, Jul 12, 2017 at 10:48:38AM -0300, Daniel Henrique Barboza wrote:
> > > The dreaded Libvirt hotplug-migrate-hotunplug scenario is working nicely.
> > Good to hear.
> > 
> > > device_add when the machine is in RUN_STATE_PRELAUNCH (-S) still doesn't
> > > work but it is expected - as discussed in "[RFC drcVI PATCH] spapr: reset
> > > DRCs
> > > on migration pre_load​", this scenario can't be fixed solely by this DRC
> > > cleanup.
> > Hmm.. what's the exact test case you're using here?  The prelaunch
> > case I tried _did_ work (queueing the event during prelaunch, then
> > completing the hotplug sequence once the guest had booted).
> 
> This is the test case:
> 
> sudo ./qemu-system-ppc64 -name migrate_qemu -boot strict=on --enable-kvm
> -device nec-usb-xhci,id=usb,bus=pci.0,addr=0xf -device
> spapr-vscsi,id=scsi0,reg=0x2000 -smp 1,maxcpus=4,sockets=4,cores=1,threads=1
> --machine pseries,accel=kvm,usb=off,dump-guest-core=off -m
> 4G,slots=32,maxmem=32G -drive file=/home/danielhb/vm_imgs/ubuntu1704.qcow2,format=qcow2,if=none,id=drive-virtio-disk0,cache=none
> -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x2,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
> -nographic -S
> QEMU 2.9.50 monitor - type 'help' for more information
> (qemu)
> (qemu) device_add host-spapr-cpu-core,id=core1,core-id=1
> (qemu) cont
> 
> (...)
> 
> After OS boots:
> 
> danielhb@ubuntu1704:~$ lscpu
> Architecture:          ppc64le
> Byte Order:            Little Endian
> CPU(s):                1
> On-line CPU(s) list:   0
> Thread(s) per core:    1
> Core(s) per socket:    1
> Socket(s):             1
> NUMA node(s):          1
> Model:                 2.1 (pvr 004b 0201)
> Model name:            POWER8E (raw), altivec supported
> Hypervisor vendor:     horizontal
> Virtualization type:   full
> L1d cache:             64K
> L1i cache:             32K
> NUMA node0 CPU(s):     0
> danielhb@ubuntu1704:~$ (qemu)
> (qemu) info cpus
> * CPU #0: nip=0xc0000000000a3e0c thread_id=6134
>   CPU #1: nip=0x0000000000000000 (halted) thread_id=6163
> (qemu) info hotpluggable-cpus
> Hotpluggable CPUs:
>   type: "host-spapr-cpu-core"
>   vcpus_count: "1"
>   CPUInstance Properties:
>     core-id: "3"
>   type: "host-spapr-cpu-core"
>   vcpus_count: "1"
>   CPUInstance Properties:
>     core-id: "2"
>   type: "host-spapr-cpu-core"
>   vcpus_count: "1"
>   qom_path: "/machine/peripheral/core1"
>   CPUInstance Properties:
>     core-id: "1"
>   type: "host-spapr-cpu-core"
>   vcpus_count: "1"
>   qom_path: "/machine/unattached/device[0]"
>   CPUInstance Properties:
>     core-id: "0"
> (qemu)

Huh.  I tried basically the same thing, and I get the second cpu once
the OS is booted.  My first guess would be that the difference is in
the guest (mine is RHEL 7.3).  Have you double checked that rtas_errd
and drmgr are present in your guest?

> I am not aware of anyone (e.g. Libvirt) using device_add at that point so
> it's not
> urgent to make this work AFAIC. I was just curious of why it
> doesn't.

Hm, I thought libvirt did use device_add here in at least some
circumstances.

I've gone ahead and sent a pull req with the changes, since AFAICT his
is not a regression, so the DRC cleanups still improve matters
overall.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
Re: [Qemu-devel] [Qemu-ppc] [PATCHv2 0/8] spapr: DRC cleanups (part VI)
Posted by Daniel Henrique Barboza 6 years, 9 months ago

On 07/14/2017 03:53 AM, David Gibson wrote:
> On Thu, Jul 13, 2017 at 07:13:23AM -0300, Daniel Henrique Barboza wrote:
>>
>> On 07/12/2017 09:57 PM, David Gibson wrote:
>>> On Wed, Jul 12, 2017 at 10:48:38AM -0300, Daniel Henrique Barboza wrote:
>>>> The dreaded Libvirt hotplug-migrate-hotunplug scenario is working nicely.
>>> Good to hear.
>>>
>>>> device_add when the machine is in RUN_STATE_PRELAUNCH (-S) still doesn't
>>>> work but it is expected - as discussed in "[RFC drcVI PATCH] spapr: reset
>>>> DRCs
>>>> on migration pre_load​", this scenario can't be fixed solely by this DRC
>>>> cleanup.
>>> Hmm.. what's the exact test case you're using here?  The prelaunch
>>> case I tried _did_ work (queueing the event during prelaunch, then
>>> completing the hotplug sequence once the guest had booted).
>> This is the test case:
>>
>> sudo ./qemu-system-ppc64 -name migrate_qemu -boot strict=on --enable-kvm
>> -device nec-usb-xhci,id=usb,bus=pci.0,addr=0xf -device
>> spapr-vscsi,id=scsi0,reg=0x2000 -smp 1,maxcpus=4,sockets=4,cores=1,threads=1
>> --machine pseries,accel=kvm,usb=off,dump-guest-core=off -m
>> 4G,slots=32,maxmem=32G -drive file=/home/danielhb/vm_imgs/ubuntu1704.qcow2,format=qcow2,if=none,id=drive-virtio-disk0,cache=none
>> -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x2,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
>> -nographic -S
>> QEMU 2.9.50 monitor - type 'help' for more information
>> (qemu)
>> (qemu) device_add host-spapr-cpu-core,id=core1,core-id=1
>> (qemu) cont
>>
>> (...)
>>
>> After OS boots:
>>
>> danielhb@ubuntu1704:~$ lscpu
>> Architecture:          ppc64le
>> Byte Order:            Little Endian
>> CPU(s):                1
>> On-line CPU(s) list:   0
>> Thread(s) per core:    1
>> Core(s) per socket:    1
>> Socket(s):             1
>> NUMA node(s):          1
>> Model:                 2.1 (pvr 004b 0201)
>> Model name:            POWER8E (raw), altivec supported
>> Hypervisor vendor:     horizontal
>> Virtualization type:   full
>> L1d cache:             64K
>> L1i cache:             32K
>> NUMA node0 CPU(s):     0
>> danielhb@ubuntu1704:~$ (qemu)
>> (qemu) info cpus
>> * CPU #0: nip=0xc0000000000a3e0c thread_id=6134
>>    CPU #1: nip=0x0000000000000000 (halted) thread_id=6163
>> (qemu) info hotpluggable-cpus
>> Hotpluggable CPUs:
>>    type: "host-spapr-cpu-core"
>>    vcpus_count: "1"
>>    CPUInstance Properties:
>>      core-id: "3"
>>    type: "host-spapr-cpu-core"
>>    vcpus_count: "1"
>>    CPUInstance Properties:
>>      core-id: "2"
>>    type: "host-spapr-cpu-core"
>>    vcpus_count: "1"
>>    qom_path: "/machine/peripheral/core1"
>>    CPUInstance Properties:
>>      core-id: "1"
>>    type: "host-spapr-cpu-core"
>>    vcpus_count: "1"
>>    qom_path: "/machine/unattached/device[0]"
>>    CPUInstance Properties:
>>      core-id: "0"
>> (qemu)
> Huh.  I tried basically the same thing, and I get the second cpu once
> the OS is booted.  My first guess would be that the difference is in
> the guest (mine is RHEL 7.3).  Have you double checked that rtas_errd
> and drmgr are present in your guest?

Yeah the guest has drmgr and rtas_errd running. As you said, there might be
something different in the guests that explains why yours work and mine 
doesn't.
Coupling that with the fact that this is not a common usage, I believe we
can leave it as a FYI/reminder if we need to revisit this issue in the
future.


Daniel

>
>> I am not aware of anyone (e.g. Libvirt) using device_add at that point so
>> it's not
>> urgent to make this work AFAIC. I was just curious of why it
>> doesn't.
> Hm, I thought libvirt did use device_add here in at least some
> circumstances.
>
> I've gone ahead and sent a pull req with the changes, since AFAICT his
> is not a regression, so the DRC cleanups still improve matters
> overall.
>


Re: [Qemu-devel] [Qemu-ppc] [PATCHv2 0/8] spapr: DRC cleanups (part VI)
Posted by David Gibson 6 years, 9 months ago
On Fri, Jul 14, 2017 at 10:50:06AM -0300, Daniel Henrique Barboza wrote:
> 
> 
> On 07/14/2017 03:53 AM, David Gibson wrote:
> > On Thu, Jul 13, 2017 at 07:13:23AM -0300, Daniel Henrique Barboza wrote:
> > > 
> > > On 07/12/2017 09:57 PM, David Gibson wrote:
> > > > On Wed, Jul 12, 2017 at 10:48:38AM -0300, Daniel Henrique Barboza wrote:
> > > > > The dreaded Libvirt hotplug-migrate-hotunplug scenario is working nicely.
> > > > Good to hear.
> > > > 
> > > > > device_add when the machine is in RUN_STATE_PRELAUNCH (-S) still doesn't
> > > > > work but it is expected - as discussed in "[RFC drcVI PATCH] spapr: reset
> > > > > DRCs
> > > > > on migration pre_load​", this scenario can't be fixed solely by this DRC
> > > > > cleanup.
> > > > Hmm.. what's the exact test case you're using here?  The prelaunch
> > > > case I tried _did_ work (queueing the event during prelaunch, then
> > > > completing the hotplug sequence once the guest had booted).
> > > This is the test case:
> > > 
> > > sudo ./qemu-system-ppc64 -name migrate_qemu -boot strict=on --enable-kvm
> > > -device nec-usb-xhci,id=usb,bus=pci.0,addr=0xf -device
> > > spapr-vscsi,id=scsi0,reg=0x2000 -smp 1,maxcpus=4,sockets=4,cores=1,threads=1
> > > --machine pseries,accel=kvm,usb=off,dump-guest-core=off -m
> > > 4G,slots=32,maxmem=32G -drive file=/home/danielhb/vm_imgs/ubuntu1704.qcow2,format=qcow2,if=none,id=drive-virtio-disk0,cache=none
> > > -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x2,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
> > > -nographic -S
> > > QEMU 2.9.50 monitor - type 'help' for more information
> > > (qemu)
> > > (qemu) device_add host-spapr-cpu-core,id=core1,core-id=1
> > > (qemu) cont
> > > 
> > > (...)
> > > 
> > > After OS boots:
> > > 
> > > danielhb@ubuntu1704:~$ lscpu
> > > Architecture:          ppc64le
> > > Byte Order:            Little Endian
> > > CPU(s):                1
> > > On-line CPU(s) list:   0
> > > Thread(s) per core:    1
> > > Core(s) per socket:    1
> > > Socket(s):             1
> > > NUMA node(s):          1
> > > Model:                 2.1 (pvr 004b 0201)
> > > Model name:            POWER8E (raw), altivec supported
> > > Hypervisor vendor:     horizontal
> > > Virtualization type:   full
> > > L1d cache:             64K
> > > L1i cache:             32K
> > > NUMA node0 CPU(s):     0
> > > danielhb@ubuntu1704:~$ (qemu)
> > > (qemu) info cpus
> > > * CPU #0: nip=0xc0000000000a3e0c thread_id=6134
> > >    CPU #1: nip=0x0000000000000000 (halted) thread_id=6163
> > > (qemu) info hotpluggable-cpus
> > > Hotpluggable CPUs:
> > >    type: "host-spapr-cpu-core"
> > >    vcpus_count: "1"
> > >    CPUInstance Properties:
> > >      core-id: "3"
> > >    type: "host-spapr-cpu-core"
> > >    vcpus_count: "1"
> > >    CPUInstance Properties:
> > >      core-id: "2"
> > >    type: "host-spapr-cpu-core"
> > >    vcpus_count: "1"
> > >    qom_path: "/machine/peripheral/core1"
> > >    CPUInstance Properties:
> > >      core-id: "1"
> > >    type: "host-spapr-cpu-core"
> > >    vcpus_count: "1"
> > >    qom_path: "/machine/unattached/device[0]"
> > >    CPUInstance Properties:
> > >      core-id: "0"
> > > (qemu)
> > Huh.  I tried basically the same thing, and I get the second cpu once
> > the OS is booted.  My first guess would be that the difference is in
> > the guest (mine is RHEL 7.3).  Have you double checked that rtas_errd
> > and drmgr are present in your guest?
> 
> Yeah the guest has drmgr and rtas_errd running. As you said, there might be
> something different in the guests that explains why yours work and mine
> doesn't.
> Coupling that with the fact that this is not a common usage, I believe we
> can leave it as a FYI/reminder if we need to revisit this issue in the
> future.

I concur.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
Re: [Qemu-devel] [PATCHv2 0/8] spapr: DRC cleanups (part VI)
Posted by David Gibson 6 years, 9 months ago
On Wed, Jul 12, 2017 at 03:53:09PM +1000, David Gibson wrote:
> This sixth set of DRC cleanup patches is a complete rework of DRC
> state management.  We stop tracking some unnecessary things, and
> change the basic state representation to a simpler and more robust
> model.
> 
> Many of the patches in this set "break" migration from earlier git
> snapshots, but not from any released qemu version.  The previous
> migration stream format had multiple problems, so better to fix them
> now, before 2.10 is out.
> 
> Although there are certainly more things that can be improved in the
> DRC system, with this series we should have a solid foundation for
> migrating DRCs - the state trasferred is about as minimal and well
> defined as it's possible to be.

Thanks for the reviews.  I've made a few tweaks as suggested.  I'm now
sufficiently confident of the series, that I've merged it into
ppc-for-2.10.

> 
> Changes since v1:
>   * Rebased onto current tree
>   * Added cleanup to unplug path
>   * Added restriction of DR-indicator to physical DRCs
>   * Included revised version of Laurent's patch to correctly handle
>     things "hot" plugged before incoming migration
> 
> David Gibson (7):
>   spapr: Remove 'awaiting_allocation' DRC flag
>   spapr: Simplify unplug path
>   spapr: Refactor spapr_drc_detach()
>   spapr: Cleanups relating to DRC awaiting_release field
>   spapr: Consolidate DRC state variables
>   spapr: Remove sPAPRConfigureConnectorState sub-structure
>   spapr: Implement DR-indicator for physical DRCs only
> 
> Laurent Vivier (1):
>   spapr: Treat devices added before inbound migration as coldplugged
> 
>  hw/ppc/spapr.c             |  89 +++-------
>  hw/ppc/spapr_drc.c         | 399 ++++++++++++++++++++++++---------------------
>  hw/ppc/spapr_pci.c         |  17 +-
>  hw/ppc/trace-events        |   3 +-
>  include/hw/ppc/spapr_drc.h |  74 ++++++---
>  5 files changed, 301 insertions(+), 281 deletions(-)
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson