qboot changes for PVH boot

[Qemu-devel] [RFC 0/3] qboot changes for PVH boot

Posted by Liam Merwick 5 years, 4 months ago

For certain applications it is desirable to rapidly boot a KVM virtual
machine. In cases where legacy hardware and software support within the
guest is not needed, QEMU should be able to boot directly into the
uncompressed Linux kernel binary with minimal firmware involvement.

There already exists an ABI to allow this for Xen PVH guests and the ABI
is supported by Linux and FreeBSD:

   https://xenbits.xen.org/docs/unstable/misc/pvh.html

Details on the Linux changes: https://lkml.org/lkml/2018/4/16/1002

This patch series provides qboot support to populate the start_info struct
needed by the direct boot ABI and to configure the guest e820 tables to
enable QEMU to use that same entry point for booting KVM guests.

Usіng the methods/scripts documented by the NEMU team at

   https://github.com/intel/nemu/wiki/Measuring-Boot-Latency
   https://lists.gnu.org/archive/html/qemu-devel/2018-12/msg00200.html

below are some timings measured (vmlinux and bzImage from the same build).
Time to get to kernel start is almost halved (95ṁs -> 48ms)

QEMU + qboot + vmlinux (PVH + 4.20-rc4)
 qemu_init_end: 41.550521
 fw_start: 41.667139 (+0.116618)
 fw_do_boot: 47.448495 (+5.781356)
 linux_startup_64: 47.720785 (+0.27229)
 linux_start_kernel: 48.399541 (+0.678756)
 linux_start_user: 296.952056 (+248.552515)

QEMU + qboot + bzImage:
 qemu_init_end: 29.209276
 fw_start: 29.317342 (+0.108066)
 linux_start_boot: 36.679362 (+7.36202)
 linux_startup_64: 94.531349 (+57.851987)
 linux_start_kernel: 94.900913 (+0.369564)
 linux_start_user: 401.060971 (+306.160058)

QEMU + bzImage:
 qemu_init_end: 30.424430
 linux_startup_64: 893.770334 (+863.345904)
 linux_start_kernel: 894.17049 (+0.400156)
 linux_start_user: 1208.679768 (+314.509278)


Liam Merwick (3):
  pvh: Add x86/HVM direct boot ABI header file
  pvh: use x86/HVM direct boot ABI
  pvh: add benchmark exit point

 benchmark.h          |   3 +-
 fw_cfg.c             |  79 +++++++++++++++++++++++++++-
 include/start_info.h | 146 +++++++++++++++++++++++++++++++++++++++++++++++++++
 linuxboot.c          |   2 +-
 main.c               |   3 ++
 tables.c             |   9 ++++
 6 files changed, 239 insertions(+), 3 deletions(-)
 create mode 100644 include/start_info.h

-- 
1.8.3.1

Re: [Qemu-devel] [RFC 0/3] qboot changes for PVH boot

Posted by Paolo Bonzini 5 years, 4 months ago

On 05/12/18 23:31, Liam Merwick wrote:
> For certain applications it is desirable to rapidly boot a KVM virtual
> machine. In cases where legacy hardware and software support within the
> guest is not needed, QEMU should be able to boot directly into the
> uncompressed Linux kernel binary with minimal firmware involvement.
> 
> There already exists an ABI to allow this for Xen PVH guests and the ABI
> is supported by Linux and FreeBSD:
> 
>    https://xenbits.xen.org/docs/unstable/misc/pvh.html
> 
> Details on the Linux changes: https://lkml.org/lkml/2018/4/16/1002
> 
> This patch series provides qboot support to populate the start_info struct
> needed by the direct boot ABI and to configure the guest e820 tables to
> enable QEMU to use that same entry point for booting KVM guests.
> 
> Usіng the methods/scripts documented by the NEMU team at
> 
>    https://github.com/intel/nemu/wiki/Measuring-Boot-Latency
>    https://lists.gnu.org/archive/html/qemu-devel/2018-12/msg00200.html
> 
> below are some timings measured (vmlinux and bzImage from the same build).
> Time to get to kernel start is almost halved (95ṁs -> 48ms)

I had a few comments on patch 2, but overall it looks fine.  Can you
send a pull request on GitHub?

Thanks,

Paolo

> QEMU + qboot + vmlinux (PVH + 4.20-rc4)
>  qemu_init_end: 41.550521
>  fw_start: 41.667139 (+0.116618)
>  fw_do_boot: 47.448495 (+5.781356)
>  linux_startup_64: 47.720785 (+0.27229)
>  linux_start_kernel: 48.399541 (+0.678756)
>  linux_start_user: 296.952056 (+248.552515)
> 
> QEMU + qboot + bzImage:
>  qemu_init_end: 29.209276
>  fw_start: 29.317342 (+0.108066)
>  linux_start_boot: 36.679362 (+7.36202)
>  linux_startup_64: 94.531349 (+57.851987)
>  linux_start_kernel: 94.900913 (+0.369564)
>  linux_start_user: 401.060971 (+306.160058)
> 
> QEMU + bzImage:
>  qemu_init_end: 30.424430
>  linux_startup_64: 893.770334 (+863.345904)
>  linux_start_kernel: 894.17049 (+0.400156)
>  linux_start_user: 1208.679768 (+314.509278)
> 
> 
> Liam Merwick (3):
>   pvh: Add x86/HVM direct boot ABI header file
>   pvh: use x86/HVM direct boot ABI
>   pvh: add benchmark exit point
> 
>  benchmark.h          |   3 +-
>  fw_cfg.c             |  79 +++++++++++++++++++++++++++-
>  include/start_info.h | 146 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  linuxboot.c          |   2 +-
>  main.c               |   3 ++
>  tables.c             |   9 ++++
>  6 files changed, 239 insertions(+), 3 deletions(-)
>  create mode 100644 include/start_info.h
>

Re: [Qemu-devel] [RFC 0/3] qboot changes for PVH boot

Posted by Liam Merwick 5 years, 4 months ago


On 06/12/2018 20:13, Paolo Bonzini wrote:
> On 05/12/18 23:31, Liam Merwick wrote:
>> For certain applications it is desirable to rapidly boot a KVM virtual
>> machine. In cases where legacy hardware and software support within the
>> guest is not needed, QEMU should be able to boot directly into the
>> uncompressed Linux kernel binary with minimal firmware involvement.
>>
>> There already exists an ABI to allow this for Xen PVH guests and the ABI
>> is supported by Linux and FreeBSD:
>>
>>     https://xenbits.xen.org/docs/unstable/misc/pvh.html
>>
>> Details on the Linux changes: https://lkml.org/lkml/2018/4/16/1002
>>
>> This patch series provides qboot support to populate the start_info struct
>> needed by the direct boot ABI and to configure the guest e820 tables to
>> enable QEMU to use that same entry point for booting KVM guests.
>>
>> Usіng the methods/scripts documented by the NEMU team at
>>
>>     https://github.com/intel/nemu/wiki/Measuring-Boot-Latency
>>     https://lists.gnu.org/archive/html/qemu-devel/2018-12/msg00200.html
>>
>> below are some timings measured (vmlinux and bzImage from the same build).
>> Time to get to kernel start is almost halved (95ṁs -> 48ms)
> 
> I had a few comments on patch 2, but overall it looks fine.  Can you
> send a pull request on GitHub?
> 

Thanks Paolo.  I have made those changes and submitted a PR:

https://github.com/bonzini/qboot/pull/17

Regards,
Liam


> 
>> QEMU + qboot + vmlinux (PVH + 4.20-rc4)
>>   qemu_init_end: 41.550521
>>   fw_start: 41.667139 (+0.116618)
>>   fw_do_boot: 47.448495 (+5.781356)
>>   linux_startup_64: 47.720785 (+0.27229)
>>   linux_start_kernel: 48.399541 (+0.678756)
>>   linux_start_user: 296.952056 (+248.552515)
>>
>> QEMU + qboot + bzImage:
>>   qemu_init_end: 29.209276
>>   fw_start: 29.317342 (+0.108066)
>>   linux_start_boot: 36.679362 (+7.36202)
>>   linux_startup_64: 94.531349 (+57.851987)
>>   linux_start_kernel: 94.900913 (+0.369564)
>>   linux_start_user: 401.060971 (+306.160058)
>>
>> QEMU + bzImage:
>>   qemu_init_end: 30.424430
>>   linux_startup_64: 893.770334 (+863.345904)
>>   linux_start_kernel: 894.17049 (+0.400156)
>>   linux_start_user: 1208.679768 (+314.509278)
>>
>>
>> Liam Merwick (3):
>>    pvh: Add x86/HVM direct boot ABI header file
>>    pvh: use x86/HVM direct boot ABI
>>    pvh: add benchmark exit point
>>
>>   benchmark.h          |   3 +-
>>   fw_cfg.c             |  79 +++++++++++++++++++++++++++-
>>   include/start_info.h | 146 +++++++++++++++++++++++++++++++++++++++++++++++++++
>>   linuxboot.c          |   2 +-
>>   main.c               |   3 ++
>>   tables.c             |   9 ++++
>>   6 files changed, 239 insertions(+), 3 deletions(-)
>>   create mode 100644 include/start_info.h
>>
>