Recent linux kernels enable KASLR to randomize phys/virt memory
addresses. This series aims to provide enough information in qemu
dumps so that crash utility can work with randomized kernel too (it
hasn't been tested on other archs than x86 though, help welcome).

The previous design to provide qemu with debug details (using qemu-ga
and a dedicated vmcoreinfo ACPI device) failed to satisfy the
requirements during previous iterations.

In particular, the previous proposed vmcoreinfo ACPI device had the
following issues:
- hazardous memory handling with no explicit synchronization
- occupy 2 fw-cfg entries (for memory and pointer)
- occupy 4k of guest memory (this could have been tweaked)
- did not provide ACPI methods (this could have been added)
- may be difficult to maintain compatibility (according to Michael)

This is a new proposal, that leverage fw-cfg device instead of adding
a new device. A "etc/vmcoreinfo" entry is added, where the guest,
during boot or later, can write the addr/size location of an ELF note
to be appended in the qemu dump.

Note: only guest kernel is expected to write to a fw-cfg entry.  This
method is not meant for general qemu/user-space communication. There
are more appropriate devices for this purpose, and the guest kernel
should not expose this facility.

This is quite easier to implement, and uses less of the limited fw-cfg
slots, and guest memory. It also solves the synchronization issue, and
may be easier to discover or to maintain compatibility.

The Linux ELF note is expected to be the VMCOREINFO note, which will
have a special handling in qemu in this case helping kaslr-kernel
debugging. But it could be any valid ELF note.

Crash 7.1.9 will parse the "phys_base" value from the VMCOREINFO note,
and thus will work with KASLR-dump produced by this series.

The series implements the note addition in qemu ELF/kdump,
as well as the python scripts/dump-guest-memory.py.

To test:

Using kernel from https://github.com/elmarco/linux fw-cfg branch,
Compile and run guest kernel with CONFIG_RANDOMIZE_BASE=y & CONFIG_FW_CFG_SYSFS=y.

Run qemu with -global fw_cfg.vmcoreinfo=on

Produce an ELF dump:
{ "execute": "dump-guest-memory", "arguments": { "protocol": "file:dump", "paging": false } }

Produce a kdump:
{ "execute": "dump-guest-memory", "arguments": { "protocol": "file:dump", "paging": false, "format": "kdump-zlib" } }

Or with (gdb) dump-guest-memory, with scripts/dump-guest-memory.py script.

Analyze with crash >= 7.1.9 (or the git version for 4.13 fixes..):

$ crash vmlinux dump

v5:
- removed x-write-pointer-available patch from this series
- drop vmcoreinfo device
- add write callback to fw_cfg entries
- add a writable fw_cfg "vmcoreinfo" entry
- split phys_base update from VMCOREINFO note in a seperate patch
- most patches had non-trivial changes, dropping reviewed-by tags

v4: from Laszlo review
- switch to warn_report*()
- update test to follow vmgenid and use boot-sector infrastructure
- fix range checks in the python script
- add vmcoreinfo_get() stub

v3: from Laszlo review
- change vmcoreinfo offset to 36
- reset err to null after report
- use PRIu32
- change name_size and desc_size against MAX_VMCOREINFO_SIZE
- python code simplification
- check boundaries of blocks in phys_memory_read()
- fix some vmgi vs vmci names
- add more comments in code
- fix comment indentation
- add r-b tags

v2: from Laszlo review
- vmci: fix guest endianess handling
- vmci: fix wrong sizeof()
- vmci: add back reset logic from vmgenid
- dump: have 1MB size limit for vmcoreinfo
- dump: fix potential off-by-1 buffer manipulation
- dump: use temporary variable for qemu_strtou64
- dump: fixed VMCOREINFO duplication in kdump
- update gdb script to not call into qemu process
- update MAINTAINERS with some new files

Marc-André Lureau (8):
  fw_cfg: rename read callback
  fw_cfg: add write callback
  fw_cfg: add vmcoreinfo file
  dump: add guest ELF note
  dump: update phys_base header field based on VMCOREINFO content
  kdump: set vmcoreinfo location
  scripts/dump-guest-memory.py: add vmcoreinfo
  MAINTAINERS: add Dump maintainers

 scripts/dump-guest-memory.py |  51 ++++++++++++
 include/hw/compat.h          |   8 ++
 include/hw/loader.h          |   2 +-
 include/hw/nvram/fw_cfg.h    |  18 ++++-
 include/sysemu/dump.h        |   2 +
 dump.c                       | 179 +++++++++++++++++++++++++++++++++++++++++++
 hw/acpi/vmgenid.c            |   2 +-
 hw/core/loader.c             |   4 +-
 hw/i386/acpi-build.c         |   2 +-
 hw/isa/lpc_ich9.c            |   4 +-
 hw/nvram/fw_cfg.c            |  64 ++++++++++++----
 MAINTAINERS                  |   9 +++
 docs/specs/fw_cfg.txt        |  24 ++++++
 13 files changed, 343 insertions(+), 26 deletions(-)

-- 
2.14.0.1.geff633fa0

On Mon, Aug 07, 2017 at 08:16:10PM +0200, Marc-André Lureau wrote:
> Recent linux kernels enable KASLR to randomize phys/virt memory
> addresses. This series aims to provide enough information in qemu
> dumps so that crash utility can work with randomized kernel too (it
> hasn't been tested on other archs than x86 though, help welcome).
> 
> The previous design to provide qemu with debug details (using qemu-ga
> and a dedicated vmcoreinfo ACPI device) failed to satisfy the
> requirements during previous iterations.
> 
> In particular, the previous proposed vmcoreinfo ACPI device had the
> following issues:
> - hazardous memory handling with no explicit synchronization
> - occupy 2 fw-cfg entries (for memory and pointer)
> - occupy 4k of guest memory (this could have been tweaked)
> - did not provide ACPI methods (this could have been added)
> - may be difficult to maintain compatibility (according to Michael)
> 
> This is a new proposal, that leverage fw-cfg device instead of adding
> a new device. A "etc/vmcoreinfo" entry is added, where the guest,
> during boot or later, can write the addr/size location of an ELF note
> to be appended in the qemu dump.

Looks ok to me. Pls remember to ping after release.

> Note: only guest kernel is expected to write to a fw-cfg entry.  This
> method is not meant for general qemu/user-space communication. There
> are more appropriate devices for this purpose, and the guest kernel
> should not expose this facility.
> 
> This is quite easier to implement, and uses less of the limited fw-cfg
> slots, and guest memory. It also solves the synchronization issue, and
> may be easier to discover or to maintain compatibility.
> 
> The Linux ELF note is expected to be the VMCOREINFO note, which will
> have a special handling in qemu in this case helping kaslr-kernel
> debugging. But it could be any valid ELF note.
> 
> Crash 7.1.9 will parse the "phys_base" value from the VMCOREINFO note,
> and thus will work with KASLR-dump produced by this series.
> 
> The series implements the note addition in qemu ELF/kdump,
> as well as the python scripts/dump-guest-memory.py.
> 
> To test:
> 
> Using kernel from https://github.com/elmarco/linux fw-cfg branch,
> Compile and run guest kernel with CONFIG_RANDOMIZE_BASE=y & CONFIG_FW_CFG_SYSFS=y.
> 
> Run qemu with -global fw_cfg.vmcoreinfo=on
> 
> Produce an ELF dump:
> { "execute": "dump-guest-memory", "arguments": { "protocol": "file:dump", "paging": false } }
> 
> Produce a kdump:
> { "execute": "dump-guest-memory", "arguments": { "protocol": "file:dump", "paging": false, "format": "kdump-zlib" } }
> 
> Or with (gdb) dump-guest-memory, with scripts/dump-guest-memory.py script.
> 
> Analyze with crash >= 7.1.9 (or the git version for 4.13 fixes..):
> 
> $ crash vmlinux dump
> 
> v5:
> - removed x-write-pointer-available patch from this series
> - drop vmcoreinfo device
> - add write callback to fw_cfg entries
> - add a writable fw_cfg "vmcoreinfo" entry
> - split phys_base update from VMCOREINFO note in a seperate patch
> - most patches had non-trivial changes, dropping reviewed-by tags
> 
> v4: from Laszlo review
> - switch to warn_report*()
> - update test to follow vmgenid and use boot-sector infrastructure
> - fix range checks in the python script
> - add vmcoreinfo_get() stub
> 
> v3: from Laszlo review
> - change vmcoreinfo offset to 36
> - reset err to null after report
> - use PRIu32
> - change name_size and desc_size against MAX_VMCOREINFO_SIZE
> - python code simplification
> - check boundaries of blocks in phys_memory_read()
> - fix some vmgi vs vmci names
> - add more comments in code
> - fix comment indentation
> - add r-b tags
> 
> v2: from Laszlo review
> - vmci: fix guest endianess handling
> - vmci: fix wrong sizeof()
> - vmci: add back reset logic from vmgenid
> - dump: have 1MB size limit for vmcoreinfo
> - dump: fix potential off-by-1 buffer manipulation
> - dump: use temporary variable for qemu_strtou64
> - dump: fixed VMCOREINFO duplication in kdump
> - update gdb script to not call into qemu process
> - update MAINTAINERS with some new files
> 
> Marc-André Lureau (8):
>   fw_cfg: rename read callback
>   fw_cfg: add write callback
>   fw_cfg: add vmcoreinfo file
>   dump: add guest ELF note
>   dump: update phys_base header field based on VMCOREINFO content
>   kdump: set vmcoreinfo location
>   scripts/dump-guest-memory.py: add vmcoreinfo
>   MAINTAINERS: add Dump maintainers
> 
>  scripts/dump-guest-memory.py |  51 ++++++++++++
>  include/hw/compat.h          |   8 ++
>  include/hw/loader.h          |   2 +-
>  include/hw/nvram/fw_cfg.h    |  18 ++++-
>  include/sysemu/dump.h        |   2 +
>  dump.c                       | 179 +++++++++++++++++++++++++++++++++++++++++++
>  hw/acpi/vmgenid.c            |   2 +-
>  hw/core/loader.c             |   4 +-
>  hw/i386/acpi-build.c         |   2 +-
>  hw/isa/lpc_ich9.c            |   4 +-
>  hw/nvram/fw_cfg.c            |  64 ++++++++++++----
>  MAINTAINERS                  |   9 +++
>  docs/specs/fw_cfg.txt        |  24 ++++++
>  13 files changed, 343 insertions(+), 26 deletions(-)
> 
> -- 
> 2.14.0.1.geff633fa0

On Mon, Aug 07, 2017 at 08:16:10PM +0200, Marc-André Lureau wrote:
> Recent linux kernels enable KASLR to randomize phys/virt memory
> addresses. This series aims to provide enough information in qemu
> dumps so that crash utility can work with randomized kernel too (it
> hasn't been tested on other archs than x86 though, help welcome).
> 
> The previous design to provide qemu with debug details (using qemu-ga
> and a dedicated vmcoreinfo ACPI device) failed to satisfy the
> requirements during previous iterations.
> 
> In particular, the previous proposed vmcoreinfo ACPI device had the
> following issues:
> - hazardous memory handling with no explicit synchronization
> - occupy 2 fw-cfg entries (for memory and pointer)
> - occupy 4k of guest memory (this could have been tweaked)
> - did not provide ACPI methods (this could have been added)
> - may be difficult to maintain compatibility (according to Michael)
> 
> This is a new proposal, that leverage fw-cfg device instead of adding
> a new device. A "etc/vmcoreinfo" entry is added, where the guest,
> during boot or later, can write the addr/size location of an ELF note
> to be appended in the qemu dump.
> 
> Note: only guest kernel is expected to write to a fw-cfg entry.  This
> method is not meant for general qemu/user-space communication. There
> are more appropriate devices for this purpose, and the guest kernel
> should not expose this facility.
> 
> This is quite easier to implement, and uses less of the limited fw-cfg
> slots, and guest memory. It also solves the synchronization issue, and
> may be easier to discover or to maintain compatibility.
> 
> The Linux ELF note is expected to be the VMCOREINFO note, which will
> have a special handling in qemu in this case helping kaslr-kernel
> debugging. But it could be any valid ELF note.
> 
> Crash 7.1.9 will parse the "phys_base" value from the VMCOREINFO note,
> and thus will work with KASLR-dump produced by this series.
> 
> The series implements the note addition in qemu ELF/kdump,
> as well as the python scripts/dump-guest-memory.py.

So still some comments and I'll be on vacation -
I think if it's a separate device someone else can
merge it easily.

FW CFG API changes themselves are OK if they satisfy
needs of this device.

> To test:
> 
> Using kernel from https://github.com/elmarco/linux fw-cfg branch,
> Compile and run guest kernel with CONFIG_RANDOMIZE_BASE=y & CONFIG_FW_CFG_SYSFS=y.
> 
> Run qemu with -global fw_cfg.vmcoreinfo=on
> 
> Produce an ELF dump:
> { "execute": "dump-guest-memory", "arguments": { "protocol": "file:dump", "paging": false } }
> 
> Produce a kdump:
> { "execute": "dump-guest-memory", "arguments": { "protocol": "file:dump", "paging": false, "format": "kdump-zlib" } }
> 
> Or with (gdb) dump-guest-memory, with scripts/dump-guest-memory.py script.
> 
> Analyze with crash >= 7.1.9 (or the git version for 4.13 fixes..):
> 
> $ crash vmlinux dump
> 
> v5:
> - removed x-write-pointer-available patch from this series
> - drop vmcoreinfo device
> - add write callback to fw_cfg entries
> - add a writable fw_cfg "vmcoreinfo" entry
> - split phys_base update from VMCOREINFO note in a seperate patch
> - most patches had non-trivial changes, dropping reviewed-by tags
> 
> v4: from Laszlo review
> - switch to warn_report*()
> - update test to follow vmgenid and use boot-sector infrastructure
> - fix range checks in the python script
> - add vmcoreinfo_get() stub
> 
> v3: from Laszlo review
> - change vmcoreinfo offset to 36
> - reset err to null after report
> - use PRIu32
> - change name_size and desc_size against MAX_VMCOREINFO_SIZE
> - python code simplification
> - check boundaries of blocks in phys_memory_read()
> - fix some vmgi vs vmci names
> - add more comments in code
> - fix comment indentation
> - add r-b tags
> 
> v2: from Laszlo review
> - vmci: fix guest endianess handling
> - vmci: fix wrong sizeof()
> - vmci: add back reset logic from vmgenid
> - dump: have 1MB size limit for vmcoreinfo
> - dump: fix potential off-by-1 buffer manipulation
> - dump: use temporary variable for qemu_strtou64
> - dump: fixed VMCOREINFO duplication in kdump
> - update gdb script to not call into qemu process
> - update MAINTAINERS with some new files
> 
> Marc-André Lureau (8):
>   fw_cfg: rename read callback
>   fw_cfg: add write callback
>   fw_cfg: add vmcoreinfo file
>   dump: add guest ELF note
>   dump: update phys_base header field based on VMCOREINFO content
>   kdump: set vmcoreinfo location
>   scripts/dump-guest-memory.py: add vmcoreinfo
>   MAINTAINERS: add Dump maintainers
> 
>  scripts/dump-guest-memory.py |  51 ++++++++++++
>  include/hw/compat.h          |   8 ++
>  include/hw/loader.h          |   2 +-
>  include/hw/nvram/fw_cfg.h    |  18 ++++-
>  include/sysemu/dump.h        |   2 +
>  dump.c                       | 179 +++++++++++++++++++++++++++++++++++++++++++
>  hw/acpi/vmgenid.c            |   2 +-
>  hw/core/loader.c             |   4 +-
>  hw/i386/acpi-build.c         |   2 +-
>  hw/isa/lpc_ich9.c            |   4 +-
>  hw/nvram/fw_cfg.c            |  64 ++++++++++++----
>  MAINTAINERS                  |   9 +++
>  docs/specs/fw_cfg.txt        |  24 ++++++
>  13 files changed, 343 insertions(+), 26 deletions(-)
> 
> -- 
> 2.14.0.1.geff633fa0
>