[PATCH v2 00/13] microvm: add acpi support

Gerd Hoffmann posted 13 patches 5 years, 7 months ago
Test docker-mingw@fedora failed
Test checkpatch failed
Test asan failed
Test docker-quick@centos7 failed
Test FreeBSD failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20200505134305.22666-1-kraxel@redhat.com
Maintainers: Eduardo Habkost <ehabkost@redhat.com>, Igor Mammedov <imammedo@redhat.com>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>, "Michael S. Tsirkin" <mst@redhat.com>, Richard Henderson <rth@twiddle.net>, Paolo Bonzini <pbonzini@redhat.com>, Sergio Lopez <slp@redhat.com>
There is a newer version of this series
hw/i386/acpi-common.h                  |  38 ++++
hw/i386/acpi-microvm.h                 |   6 +
include/hw/acpi/acpi-defs.h            |   2 +
include/hw/acpi/generic_event_device.h |  10 +
include/hw/i386/microvm.h              |  10 +-
hw/acpi/aml-build.c                    |   4 +-
hw/i386/acpi-build.c                   | 198 +-------------------
hw/i386/acpi-common.c                  | 206 ++++++++++++++++++++
hw/i386/acpi-microvm.c                 | 249 +++++++++++++++++++++++++
hw/i386/generic_event_device_x86.c     | 114 +++++++++++
hw/i386/microvm.c                      |  36 +++-
hw/i386/Kconfig                        |   1 +
hw/i386/Makefile.objs                  |   3 +
13 files changed, 676 insertions(+), 201 deletions(-)
create mode 100644 hw/i386/acpi-common.h
create mode 100644 hw/i386/acpi-microvm.h
create mode 100644 hw/i386/acpi-common.c
create mode 100644 hw/i386/acpi-microvm.c
create mode 100644 hw/i386/generic_event_device_x86.c
[PATCH v2 00/13] microvm: add acpi support
Posted by Gerd Hoffmann 5 years, 7 months ago
I know that not supporting ACPI in microvm is intentional.  If you still
don't want ACPI this is perfectly fine, you can use the usual -no-acpi
switch to toggle ACPI support.

These are the advantages you are going to loose then:

  (1) virtio-mmio device discovery without command line hacks (tweaking
      the command line is a problem when not using direct kernel boot).
  (2) Better IO-APIC support, we can use IRQ lines 16-23.
  (3) ACPI power button (aka powerdown request) works.
  (4) machine poweroff (aka S5 state) works.

Together with seabios patches for virtio-mmio support this allows to
boot standard fedora images (cloud, coreos, workstation live) with the
microvm machine type.

git branch for testing (including updated seabios):
	https://git.kraxel.org/cgit/qemu/log/?h=sirius/microvm

changes in v2:
  * some acpi cleanups are an separate patch series now.
  * switched to hw reduced acpi & generic event device.
  * misc fixes here and there.

cheers,
  Gerd

Gerd Hoffmann (13):
  acpi: make build_madt() more generic.
  acpi: create acpi-common.c and move madt code
  acpi: madt: skip pci override on pci-less systems (microvm)
  acpi: move acpi_build_facs to acpi-common.c
  acpi: move acpi_init_common_fadt_data to acpi-common.c
  acpi: move acpi_align_size to acpi-common.h
  acpi: fadt: add hw-reduced sleep register support
  acpi: generic event device for x86
  microvm: add minimal acpi support
  microvm: disable virtio-mmio cmdline hack
  microvm: add acpi_dsdt_add_virtio() for x86
  microvm: make virtio irq base runtime configurable
  microvm/acpi: use GSI 16-23 for virtio

 hw/i386/acpi-common.h                  |  38 ++++
 hw/i386/acpi-microvm.h                 |   6 +
 include/hw/acpi/acpi-defs.h            |   2 +
 include/hw/acpi/generic_event_device.h |  10 +
 include/hw/i386/microvm.h              |  10 +-
 hw/acpi/aml-build.c                    |   4 +-
 hw/i386/acpi-build.c                   | 198 +-------------------
 hw/i386/acpi-common.c                  | 206 ++++++++++++++++++++
 hw/i386/acpi-microvm.c                 | 249 +++++++++++++++++++++++++
 hw/i386/generic_event_device_x86.c     | 114 +++++++++++
 hw/i386/microvm.c                      |  36 +++-
 hw/i386/Kconfig                        |   1 +
 hw/i386/Makefile.objs                  |   3 +
 13 files changed, 676 insertions(+), 201 deletions(-)
 create mode 100644 hw/i386/acpi-common.h
 create mode 100644 hw/i386/acpi-microvm.h
 create mode 100644 hw/i386/acpi-common.c
 create mode 100644 hw/i386/acpi-microvm.c
 create mode 100644 hw/i386/generic_event_device_x86.c

-- 
2.18.4


Re: [PATCH v2 00/13] microvm: add acpi support
Posted by Michael S. Tsirkin 5 years, 7 months ago
On Tue, May 05, 2020 at 03:42:52PM +0200, Gerd Hoffmann wrote:
> I know that not supporting ACPI in microvm is intentional.  If you still
> don't want ACPI this is perfectly fine, you can use the usual -no-acpi
> switch to toggle ACPI support.
> 
> These are the advantages you are going to loose then:
> 
>   (1) virtio-mmio device discovery without command line hacks (tweaking
>       the command line is a problem when not using direct kernel boot).
>   (2) Better IO-APIC support, we can use IRQ lines 16-23.
>   (3) ACPI power button (aka powerdown request) works.
>   (4) machine poweroff (aka S5 state) works.

Questions

- what's the tradeoff in startup time?
- what should be the default?

Based on above I'd be inclined to say default should stay no acpi and
users should enable acpi with an option.

> Together with seabios patches for virtio-mmio support this allows to
> boot standard fedora images (cloud, coreos, workstation live) with the
> microvm machine type.
> 
> git branch for testing (including updated seabios):
> 	https://git.kraxel.org/cgit/qemu/log/?h=sirius/microvm
> 
> changes in v2:
>   * some acpi cleanups are an separate patch series now.
>   * switched to hw reduced acpi & generic event device.
>   * misc fixes here and there.
> 
> cheers,
>   Gerd
> 
> Gerd Hoffmann (13):
>   acpi: make build_madt() more generic.
>   acpi: create acpi-common.c and move madt code
>   acpi: madt: skip pci override on pci-less systems (microvm)
>   acpi: move acpi_build_facs to acpi-common.c
>   acpi: move acpi_init_common_fadt_data to acpi-common.c
>   acpi: move acpi_align_size to acpi-common.h
>   acpi: fadt: add hw-reduced sleep register support
>   acpi: generic event device for x86
>   microvm: add minimal acpi support
>   microvm: disable virtio-mmio cmdline hack
>   microvm: add acpi_dsdt_add_virtio() for x86
>   microvm: make virtio irq base runtime configurable
>   microvm/acpi: use GSI 16-23 for virtio
> 
>  hw/i386/acpi-common.h                  |  38 ++++
>  hw/i386/acpi-microvm.h                 |   6 +
>  include/hw/acpi/acpi-defs.h            |   2 +
>  include/hw/acpi/generic_event_device.h |  10 +
>  include/hw/i386/microvm.h              |  10 +-
>  hw/acpi/aml-build.c                    |   4 +-
>  hw/i386/acpi-build.c                   | 198 +-------------------
>  hw/i386/acpi-common.c                  | 206 ++++++++++++++++++++
>  hw/i386/acpi-microvm.c                 | 249 +++++++++++++++++++++++++
>  hw/i386/generic_event_device_x86.c     | 114 +++++++++++
>  hw/i386/microvm.c                      |  36 +++-
>  hw/i386/Kconfig                        |   1 +
>  hw/i386/Makefile.objs                  |   3 +
>  13 files changed, 676 insertions(+), 201 deletions(-)
>  create mode 100644 hw/i386/acpi-common.h
>  create mode 100644 hw/i386/acpi-microvm.h
>  create mode 100644 hw/i386/acpi-common.c
>  create mode 100644 hw/i386/acpi-microvm.c
>  create mode 100644 hw/i386/generic_event_device_x86.c
> 
> -- 
> 2.18.4


Re: [PATCH v2 00/13] microvm: add acpi support
Posted by Philippe Mathieu-Daudé 5 years, 7 months ago
On 5/5/20 4:04 PM, Michael S. Tsirkin wrote:
> On Tue, May 05, 2020 at 03:42:52PM +0200, Gerd Hoffmann wrote:
>> I know that not supporting ACPI in microvm is intentional.  If you still
>> don't want ACPI this is perfectly fine, you can use the usual -no-acpi
>> switch to toggle ACPI support.
>>
>> These are the advantages you are going to loose then:
>>
>>    (1) virtio-mmio device discovery without command line hacks (tweaking
>>        the command line is a problem when not using direct kernel boot).
>>    (2) Better IO-APIC support, we can use IRQ lines 16-23.
>>    (3) ACPI power button (aka powerdown request) works.
>>    (4) machine poweroff (aka S5 state) works.
> 
> Questions
> 
> - what's the tradeoff in startup time?
> - what should be the default?
> 
> Based on above I'd be inclined to say default should stay no acpi and
> users should enable acpi with an option.

As this machine was added to have the least minimum hardware required, 
I'd keep the default with no ACPI and have user requiring it to use an 
option. My 2 cents obviously.

> 
>> Together with seabios patches for virtio-mmio support this allows to
>> boot standard fedora images (cloud, coreos, workstation live) with the
>> microvm machine type.
>>
>> git branch for testing (including updated seabios):
>> 	https://git.kraxel.org/cgit/qemu/log/?h=sirius/microvm
>>
>> changes in v2:
>>    * some acpi cleanups are an separate patch series now.
>>    * switched to hw reduced acpi & generic event device.
>>    * misc fixes here and there.
>>
>> cheers,
>>    Gerd
>>
>> Gerd Hoffmann (13):
>>    acpi: make build_madt() more generic.
>>    acpi: create acpi-common.c and move madt code
>>    acpi: madt: skip pci override on pci-less systems (microvm)
>>    acpi: move acpi_build_facs to acpi-common.c
>>    acpi: move acpi_init_common_fadt_data to acpi-common.c
>>    acpi: move acpi_align_size to acpi-common.h
>>    acpi: fadt: add hw-reduced sleep register support
>>    acpi: generic event device for x86
>>    microvm: add minimal acpi support
>>    microvm: disable virtio-mmio cmdline hack
>>    microvm: add acpi_dsdt_add_virtio() for x86
>>    microvm: make virtio irq base runtime configurable
>>    microvm/acpi: use GSI 16-23 for virtio
>>
>>   hw/i386/acpi-common.h                  |  38 ++++
>>   hw/i386/acpi-microvm.h                 |   6 +
>>   include/hw/acpi/acpi-defs.h            |   2 +
>>   include/hw/acpi/generic_event_device.h |  10 +
>>   include/hw/i386/microvm.h              |  10 +-
>>   hw/acpi/aml-build.c                    |   4 +-
>>   hw/i386/acpi-build.c                   | 198 +-------------------
>>   hw/i386/acpi-common.c                  | 206 ++++++++++++++++++++
>>   hw/i386/acpi-microvm.c                 | 249 +++++++++++++++++++++++++
>>   hw/i386/generic_event_device_x86.c     | 114 +++++++++++
>>   hw/i386/microvm.c                      |  36 +++-
>>   hw/i386/Kconfig                        |   1 +
>>   hw/i386/Makefile.objs                  |   3 +
>>   13 files changed, 676 insertions(+), 201 deletions(-)
>>   create mode 100644 hw/i386/acpi-common.h
>>   create mode 100644 hw/i386/acpi-microvm.h
>>   create mode 100644 hw/i386/acpi-common.c
>>   create mode 100644 hw/i386/acpi-microvm.c
>>   create mode 100644 hw/i386/generic_event_device_x86.c
>>
>> -- 
>> 2.18.4
> 
> 


Re: [PATCH v2 00/13] microvm: add acpi support
Posted by Sergio Lopez 5 years, 7 months ago
On Tue, May 05, 2020 at 04:16:00PM +0200, Philippe Mathieu-Daudé wrote:
> On 5/5/20 4:04 PM, Michael S. Tsirkin wrote:
> > On Tue, May 05, 2020 at 03:42:52PM +0200, Gerd Hoffmann wrote:
> > > I know that not supporting ACPI in microvm is intentional.  If you still
> > > don't want ACPI this is perfectly fine, you can use the usual -no-acpi
> > > switch to toggle ACPI support.
> > > 
> > > These are the advantages you are going to loose then:
> > > 
> > >    (1) virtio-mmio device discovery without command line hacks (tweaking
> > >        the command line is a problem when not using direct kernel boot).
> > >    (2) Better IO-APIC support, we can use IRQ lines 16-23.
> > >    (3) ACPI power button (aka powerdown request) works.
> > >    (4) machine poweroff (aka S5 state) works.
> > 
> > Questions
> > 
> > - what's the tradeoff in startup time?
> > - what should be the default?
> > 
> > Based on above I'd be inclined to say default should stay no acpi and
> > users should enable acpi with an option.
> 
> As this machine was added to have the least minimum hardware required, I'd
> keep the default with no ACPI and have user requiring it to use an option.
> My 2 cents obviously.

I also share this opinion. And I would prefer it to be a machine type
option, defaulting to "off", but I guess that's a matter of taste.

Sergio.

> > 
> > > Together with seabios patches for virtio-mmio support this allows to
> > > boot standard fedora images (cloud, coreos, workstation live) with the
> > > microvm machine type.
> > > 
> > > git branch for testing (including updated seabios):
> > > 	https://git.kraxel.org/cgit/qemu/log/?h=sirius/microvm
> > > 
> > > changes in v2:
> > >    * some acpi cleanups are an separate patch series now.
> > >    * switched to hw reduced acpi & generic event device.
> > >    * misc fixes here and there.
> > > 
> > > cheers,
> > >    Gerd
> > > 
> > > Gerd Hoffmann (13):
> > >    acpi: make build_madt() more generic.
> > >    acpi: create acpi-common.c and move madt code
> > >    acpi: madt: skip pci override on pci-less systems (microvm)
> > >    acpi: move acpi_build_facs to acpi-common.c
> > >    acpi: move acpi_init_common_fadt_data to acpi-common.c
> > >    acpi: move acpi_align_size to acpi-common.h
> > >    acpi: fadt: add hw-reduced sleep register support
> > >    acpi: generic event device for x86
> > >    microvm: add minimal acpi support
> > >    microvm: disable virtio-mmio cmdline hack
> > >    microvm: add acpi_dsdt_add_virtio() for x86
> > >    microvm: make virtio irq base runtime configurable
> > >    microvm/acpi: use GSI 16-23 for virtio
> > > 
> > >   hw/i386/acpi-common.h                  |  38 ++++
> > >   hw/i386/acpi-microvm.h                 |   6 +
> > >   include/hw/acpi/acpi-defs.h            |   2 +
> > >   include/hw/acpi/generic_event_device.h |  10 +
> > >   include/hw/i386/microvm.h              |  10 +-
> > >   hw/acpi/aml-build.c                    |   4 +-
> > >   hw/i386/acpi-build.c                   | 198 +-------------------
> > >   hw/i386/acpi-common.c                  | 206 ++++++++++++++++++++
> > >   hw/i386/acpi-microvm.c                 | 249 +++++++++++++++++++++++++
> > >   hw/i386/generic_event_device_x86.c     | 114 +++++++++++
> > >   hw/i386/microvm.c                      |  36 +++-
> > >   hw/i386/Kconfig                        |   1 +
> > >   hw/i386/Makefile.objs                  |   3 +
> > >   13 files changed, 676 insertions(+), 201 deletions(-)
> > >   create mode 100644 hw/i386/acpi-common.h
> > >   create mode 100644 hw/i386/acpi-microvm.h
> > >   create mode 100644 hw/i386/acpi-common.c
> > >   create mode 100644 hw/i386/acpi-microvm.c
> > >   create mode 100644 hw/i386/generic_event_device_x86.c
> > > 
> > > -- 
> > > 2.18.4
> > 
> > 
> 
Re: [PATCH v2 00/13] microvm: add acpi support
Posted by Gerd Hoffmann 5 years, 7 months ago
On Tue, May 05, 2020 at 10:04:02AM -0400, Michael S. Tsirkin wrote:
> On Tue, May 05, 2020 at 03:42:52PM +0200, Gerd Hoffmann wrote:
> > I know that not supporting ACPI in microvm is intentional.  If you still
> > don't want ACPI this is perfectly fine, you can use the usual -no-acpi
> > switch to toggle ACPI support.
> > 
> > These are the advantages you are going to loose then:
> > 
> >   (1) virtio-mmio device discovery without command line hacks (tweaking
> >       the command line is a problem when not using direct kernel boot).
> >   (2) Better IO-APIC support, we can use IRQ lines 16-23.
> >   (3) ACPI power button (aka powerdown request) works.
> >   (4) machine poweroff (aka S5 state) works.
> 
> Questions
> 
> - what's the tradeoff in startup time?

In the noise.  0.28-0.29 seconds on my hardware to the "i8042: PNP: No
PS/2 controller found" message, no matter whenever acpi is on or off.
With "quiet" (acpi prints more and logging to the serial console is
slow).

At that point -no-acpi takes one second to figure the ps2 controller
really isn't there (as discussed before).

Another interesting difference is interrupt handling.

The -no-acpi version:

           CPU0       
  2:          0    XT-PIC      cascade
  4:        284   IO-APIC   4-edge      ttyS0
  8:          0   IO-APIC   8-edge      rtc0
 14:       5399   IO-APIC  14-edge      virtio1
 15:         58   IO-APIC  15-edge      virtio0
NMI:          0   Non-maskable interrupts
[ ... ]

The acpi version:

           CPU0       
  1:          0   IO-APIC   9-edge      ACPI:Ged
  2:        231   IO-APIC  23-fasteoi   virtio0
  3:       6291   IO-APIC  22-fasteoi   virtio1
  4:       1758   IO-APIC   4-edge      ttyS0
  5:          0   IO-APIC   8-edge      rtc0
NMI:          0   Non-maskable interrupts
[ ... ]

> - what should be the default?

IMO it makes sense to enable it by default.  You get working
power management.  You can boot stock cloud images (patched
seabios parses the dsdt to find virtio-mmio devices to boot
from virtio-mmio disks).

It's easier to leave behind legacy stuff:  The kernel trusts the
firmware and doesn't go into "trying harder to find ps2 kbd" mode.
Also what is this "cascade" thing in /proc/interrupts above? [1]

I expect dropping the rtc is easier with acpi too, the kernel probably
wouldn't try to find it then.  Right now seabios needs rtc cmos for
ram size probing, so I didn't test that yet.

On the other hand I don't really see any disadvantages.  The tables are
small ...

# find /sys/firmware/acpi/tables/ -type f | xargs ls -l
-r--------. 1 root root  70 May  6 06:48 /sys/firmware/acpi/tables/APIC
-r--------. 1 root root 472 May  6 06:48 /sys/firmware/acpi/tables/DSDT
-r--------. 1 root root 268 May  6 06:48 /sys/firmware/acpi/tables/FACP

... and simple (no methods) so you can hardly call that "bloat".

> Based on above I'd be inclined to say default should stay no acpi and
> users should enable acpi with an option.

I disagree, but I can live with off by default too.  We already have
acpi=OnOffAuto for X86MachineState, so it is just a matter of handling
microvm.acpi=auto accordingly in x86_machine_is_acpi_enabled().

take care,
  Gerd

[1] Rhetorical question, I know what it is. [2]
[2] I don't want remember though.


Re: [PATCH v2 00/13] microvm: add acpi support
Posted by Michael S. Tsirkin 5 years, 7 months ago
On Wed, May 06, 2020 at 01:46:35PM +0200, Gerd Hoffmann wrote:
> On Tue, May 05, 2020 at 10:04:02AM -0400, Michael S. Tsirkin wrote:
> > On Tue, May 05, 2020 at 03:42:52PM +0200, Gerd Hoffmann wrote:
> > > I know that not supporting ACPI in microvm is intentional.  If you still
> > > don't want ACPI this is perfectly fine, you can use the usual -no-acpi
> > > switch to toggle ACPI support.
> > > 
> > > These are the advantages you are going to loose then:
> > > 
> > >   (1) virtio-mmio device discovery without command line hacks (tweaking
> > >       the command line is a problem when not using direct kernel boot).
> > >   (2) Better IO-APIC support, we can use IRQ lines 16-23.
> > >   (3) ACPI power button (aka powerdown request) works.
> > >   (4) machine poweroff (aka S5 state) works.
> > 
> > Questions
> > 
> > - what's the tradeoff in startup time?
> 
> In the noise.  0.28-0.29 seconds on my hardware to the "i8042: PNP: No
> PS/2 controller found" message, no matter whenever acpi is on or off.
> With "quiet" (acpi prints more and logging to the serial console is
> slow).
> 
> At that point -no-acpi takes one second to figure the ps2 controller
> really isn't there (as discussed before).
> 
> Another interesting difference is interrupt handling.
> 
> The -no-acpi version:
> 
>            CPU0       
>   2:          0    XT-PIC      cascade
>   4:        284   IO-APIC   4-edge      ttyS0
>   8:          0   IO-APIC   8-edge      rtc0
>  14:       5399   IO-APIC  14-edge      virtio1
>  15:         58   IO-APIC  15-edge      virtio0
> NMI:          0   Non-maskable interrupts
> [ ... ]
> 
> The acpi version:
> 
>            CPU0       
>   1:          0   IO-APIC   9-edge      ACPI:Ged
>   2:        231   IO-APIC  23-fasteoi   virtio0
>   3:       6291   IO-APIC  22-fasteoi   virtio1
>   4:       1758   IO-APIC   4-edge      ttyS0
>   5:          0   IO-APIC   8-edge      rtc0
> NMI:          0   Non-maskable interrupts
> [ ... ]
> 
> > - what should be the default?
> 
> IMO it makes sense to enable it by default.  You get working
> power management.  You can boot stock cloud images (patched
> seabios parses the dsdt to find virtio-mmio devices to boot
> from virtio-mmio disks).
> 
> It's easier to leave behind legacy stuff:  The kernel trusts the
> firmware and doesn't go into "trying harder to find ps2 kbd" mode.
> Also what is this "cascade" thing in /proc/interrupts above? [1]
> 
> I expect dropping the rtc is easier with acpi too, the kernel probably
> wouldn't try to find it then.  Right now seabios needs rtc cmos for
> ram size probing, so I didn't test that yet.
> 
> On the other hand I don't really see any disadvantages.  The tables are
> small ...
> 
> # find /sys/firmware/acpi/tables/ -type f | xargs ls -l
> -r--------. 1 root root  70 May  6 06:48 /sys/firmware/acpi/tables/APIC
> -r--------. 1 root root 472 May  6 06:48 /sys/firmware/acpi/tables/DSDT
> -r--------. 1 root root 268 May  6 06:48 /sys/firmware/acpi/tables/FACP
> 
> ... and simple (no methods) so you can hardly call that "bloat".
> 
> > Based on above I'd be inclined to say default should stay no acpi and
> > users should enable acpi with an option.
> 
> I disagree, but I can live with off by default too.  We already have
> acpi=OnOffAuto for X86MachineState, so it is just a matter of handling
> microvm.acpi=auto accordingly in x86_machine_is_acpi_enabled().
> 
> take care,
>   Gerd
> 
> [1] Rhetorical question, I know what it is. [2]
> [2] I don't want remember though.

Let's leave flipping the default as a separate patch, to be
decided on merits after a bunch of people test with/without.

-- 
MST


Re: [PATCH v2 00/13] microvm: add acpi support
Posted by Gerd Hoffmann 5 years, 7 months ago
  Hi,

> > I expect dropping the rtc is easier with acpi too, the kernel probably
> > wouldn't try to find it then.  Right now seabios needs rtc cmos for
> > ram size probing, so I didn't test that yet.

Confirmed.  With rtc=off I get this ...

           CPU0       
  1:          0   IO-APIC   9-edge      ACPI:Ged
  2:         62   IO-APIC  23-fasteoi   virtio0
  3:       5664   IO-APIC  22-fasteoi   virtio1
  4:       1854   IO-APIC   4-edge      ttyS0
NMI:          0   Non-maskable interrupts
[ ... ]

... i.e. rtc irq is gone.

> Let's leave flipping the default as a separate patch, to be
> decided on merits after a bunch of people test with/without.

Sure.

take care,
  Gerd