[Qemu-devel] [PATCH v2 0/2] pc: mmap kernel (ELF image) and initrd

Stefano Garzarella posted 2 patches 4 years, 9 months ago
Test docker-clang@ubuntu failed
Test s390x passed
Test asan passed
Test docker-mingw@fedora passed
Test FreeBSD passed
Test checkpatch passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20190723140445.12748-1-sgarzare@redhat.com
Maintainers: "Michael S. Tsirkin" <mst@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>, Eduardo Habkost <ehabkost@redhat.com>, Richard Henderson <rth@twiddle.net>
There is a newer version of this series
hw/i386/pc.c         | 15 ++++++++---
include/hw/elf_ops.h | 64 ++++++++++++++++++++++++--------------------
2 files changed, 46 insertions(+), 33 deletions(-)
[Qemu-devel] [PATCH v2 0/2] pc: mmap kernel (ELF image) and initrd
Posted by Stefano Garzarella 4 years, 9 months ago
In order to reduce the memory footprint when PVH kernel and initrd
are used, we map them into memory instead of reading them.
In this way we can share them between multiple instances of QEMU.

v2:
- Patch 1: used g_mapped_file_new_from_fd() with 'writeble' set to 'true',
           since we can modify the mapped buffer. [Paolo, Peter]

These are the results using a PVH kernel and initrd (cpio):
- memory footprint (using smem) [MB]
        QEMU              before                   now
    # instances        USS      PSS            USS      PSS
         1           102.0M   105.8M         102.3M   106.2M
         2            94.6M   101.2M          72.3M    90.1M
         4            94.1M    98.0M          72.0M    81.5M
         8            94.0M    96.2M          71.8M    76.9M
        16            93.9M    95.1M          71.6M    74.3M

    Initrd size: 3.0M
    Kernel
        image size: 28M
        sections size [size -A -d vmlinux]:  18.9M

- boot time [ms]
                          before                   now
 qemu_init_end:           63.85                   55.91
 linux_start_kernel:      82.11 (+18.26)          74.51 (+18.60)
 linux_start_user:       169.94 (+87.83)         159.06 (+84.56)

QEMU command used:
./qemu-system-x86_64 -bios /path/to/seabios/out/bios.bin -no-hpet \
    -machine q35,accel=kvm,kernel_irqchip,nvdimm,sata=off,smbus=off,vmport=off \
    -cpu host -m 1G -smp 1 -vga none -display none -no-user-config -nodefaults \
    -kernel /path/to/vmlinux -initrd /path/to/rootfs.cpio \
    -append 'root=/dev/mem0 ro console=hvc0 pci=lastbus=0 nosmap'

Stefano Garzarella (2):
  elf-ops.h: Map into memory the ELF to load
  hw/i386/pc: Map into memory the initrd

 hw/i386/pc.c         | 15 ++++++++---
 include/hw/elf_ops.h | 64 ++++++++++++++++++++++++--------------------
 2 files changed, 46 insertions(+), 33 deletions(-)

-- 
2.20.1


Re: [Qemu-devel] [PATCH v2 0/2] pc: mmap kernel (ELF image) and initrd
Posted by Montes, Julio 4 years, 9 months ago
Stefano, Brilliant job!

I can confirm that with these patches the memory footprint is smaller
and the boot time is the same for kata

Here the results using kata metrics

https://pasteboard.co/Ipl06Q0.png
https://pasteboard.co/Ipl3p4d.png

Thanks

-
Julio


On Tue, 2019-07-23 at 16:04 +0200, Stefano Garzarella wrote:
> In order to reduce the memory footprint when PVH kernel and initrd
> are used, we map them into memory instead of reading them.
> In this way we can share them between multiple instances of QEMU.
> 
> v2:
> - Patch 1: used g_mapped_file_new_from_fd() with 'writeble' set to
> 'true',
>            since we can modify the mapped buffer. [Paolo, Peter]
> 
> These are the results using a PVH kernel and initrd (cpio):
> - memory footprint (using smem) [MB]
>         QEMU              before                   now
>     # instances        USS      PSS            USS      PSS
>          1           102.0M   105.8M         102.3M   106.2M
>          2            94.6M   101.2M          72.3M    90.1M
>          4            94.1M    98.0M          72.0M    81.5M
>          8            94.0M    96.2M          71.8M    76.9M
>         16            93.9M    95.1M          71.6M    74.3M
> 
>     Initrd size: 3.0M
>     Kernel
>         image size: 28M
>         sections size [size -A -d vmlinux]:  18.9M
> 
> - boot time [ms]
>                           before                   now
>  qemu_init_end:           63.85                   55.91
>  linux_start_kernel:      82.11 (+18.26)          74.51 (+18.60)
>  linux_start_user:       169.94 (+87.83)         159.06 (+84.56)
> 
> QEMU command used:
> ./qemu-system-x86_64 -bios /path/to/seabios/out/bios.bin -no-hpet \
>     -machine
> q35,accel=kvm,kernel_irqchip,nvdimm,sata=off,smbus=off,vmport=off \
>     -cpu host -m 1G -smp 1 -vga none -display none -no-user-config
> -nodefaults \
>     -kernel /path/to/vmlinux -initrd /path/to/rootfs.cpio \
>     -append 'root=/dev/mem0 ro console=hvc0 pci=lastbus=0 nosmap'
> 
> Stefano Garzarella (2):
>   elf-ops.h: Map into memory the ELF to load
>   hw/i386/pc: Map into memory the initrd
> 
>  hw/i386/pc.c         | 15 ++++++++---
>  include/hw/elf_ops.h | 64 ++++++++++++++++++++++++----------------
> ----
>  2 files changed, 46 insertions(+), 33 deletions(-)
> 
Re: [Qemu-devel] [PATCH v2 0/2] pc: mmap kernel (ELF image) and initrd
Posted by Stefano Garzarella 4 years, 9 months ago
On Tue, Jul 23, 2019 at 05:37:18PM +0000, Montes, Julio wrote:
> Stefano, Brilliant job!
> 
> I can confirm that with these patches the memory footprint is smaller
> and the boot time is the same for kata
> 
> Here the results using kata metrics
> 
> https://pasteboard.co/Ipl06Q0.png
> https://pasteboard.co/Ipl3p4d.png
> 

Hi Julio,
thank you very much for testing :)

When you measure the PPS, how many QEMU instances did you start?
Did you use also the initrd?

Thanks,
Stefano

Re: [Qemu-devel] [PATCH v2 0/2] pc: mmap kernel (ELF image) and initrd
Posted by Montes, Julio 4 years, 9 months ago
Hi Stefano

On Wed, 2019-07-24 at 09:36 +0200, Stefano Garzarella wrote:
> On Tue, Jul 23, 2019 at 05:37:18PM +0000, Montes, Julio wrote:
> > Stefano, Brilliant job!
> > 
> > I can confirm that with these patches the memory footprint is
> > smaller
> > and the boot time is the same for kata
> > 
> > Here the results using kata metrics
> > 
> > https://pasteboard.co/Ipl06Q0.png
> > https://pasteboard.co/Ipl3p4d.png
> > 
> 
> Hi Julio,
> thank you very much for testing :)
> 
> When you measure the PPS, how many QEMU instances did you start?
> Did you use also the initrd?

50 VMs with a nvdimm/pmem device as rootfs, I will test your v3 with
initrd :)

> 
> Thanks,
> Stefano
Re: [Qemu-devel] [PATCH v2 0/2] pc: mmap kernel (ELF image) and initrd
Posted by Stefano Garzarella 4 years, 9 months ago
On Wed, Jul 24, 2019 at 01:03:42PM +0000, Montes, Julio wrote:
> Hi Stefano
> 
> On Wed, 2019-07-24 at 09:36 +0200, Stefano Garzarella wrote:
> > On Tue, Jul 23, 2019 at 05:37:18PM +0000, Montes, Julio wrote:
> > > Stefano, Brilliant job!
> > > 
> > > I can confirm that with these patches the memory footprint is
> > > smaller
> > > and the boot time is the same for kata
> > > 
> > > Here the results using kata metrics
> > > 
> > > https://pasteboard.co/Ipl06Q0.png
> > > https://pasteboard.co/Ipl3p4d.png
> > > 
> > 
> > Hi Julio,
> > thank you very much for testing :)
> > 
> > When you measure the PPS, how many QEMU instances did you start?
> > Did you use also the initrd?
> 
> 50 VMs with a nvdimm/pmem device as rootfs, I will test your v3 with
> initrd :)

Cool :)

Okay, I'm sending a v4 to fix some issues...

Thanks,
Stefano