This work being submitted in this series was made possible with a great thanks
to Star Lab Cop ration and their generous funding of this project.

The patch series is based on the existing xsm series for starting the idle
domain privileged. After that, the first four commits were previously submitted
as an RFC and expanded by an additional commit to refactor xen commandline
handling. The remaining preliminary patches are FDT refactoring and a doc
update. From there is where the series begins to  morph x86 arch to support
building multiple domains at boot.

This series has been fairly well tested using qemu with a multiboot1 bootoader
and under EFI + GRUB multiboot2 boot. While there are likely some rough spots
remaining in the series, it is at a point now where the series should be reviewed,
exercised, and tested for consideration into tree.

Information, including docuemntation, meeting minutes, presentations, and past
series postings can be found on the Xen wiki:

https://wiki.xenproject.org/wiki/Hyperlaunch

Daniel P. Smith (18):
  kconfig: allow configuration of maximum modules
  introduction of generalized boot info
  x86: adopt new boot info structures
  x86: refactor entrypoints to new boot info
  x86: refactor xen cmdline into general framework
  fdt: make fdt handling reusable across arch
  docs: update hyperlaunch device tree documentation
  kconfig: introduce domain builder config option
  x86: introduce abstractions for domain builder
  x86: introduce the domain builder
  x86: initial conversion to domain builder
  x86: convert dom0 creation to domain builder
  x86: generalize physmap logic
  x86: generalize vcpu for domain building
  x86: rework domain page allocation
  x86: add pv multidomain construction
  builder: introduce domain builder hypfs tree
  tools: introduce example late pv helper

 .gitignore                                    |   1 +
 .../designs/launch/hyperlaunch-devicetree.rst | 497 +++++++++++-------
 tools/helpers/Makefile                        |  11 +
 tools/helpers/builder-hypfs.c                 | 253 +++++++++
 tools/helpers/hypfs-helpers.h                 |   9 +
 tools/helpers/late-init-pv.c                  | 287 ++++++++++
 tools/helpers/late-init-pv.h                  |  29 +
 tools/helpers/xs-helpers.c                    | 117 +++++
 tools/helpers/xs-helpers.h                    |  27 +
 xen/arch/Kconfig                              |  12 +
 xen/arch/arm/bootfdt.c                        | 115 +---
 xen/arch/arm/include/asm/setup.h              |   5 +-
 xen/arch/x86/Makefile                         |   1 +
 xen/arch/x86/boot/boot_info32.h               |  97 ++++
 xen/arch/x86/boot/defs.h                      |  17 +-
 xen/arch/x86/boot/reloc.c                     | 187 +++++--
 xen/arch/x86/bzimage.c                        |  18 +-
 xen/arch/x86/cpu/microcode/core.c             | 133 +++--
 xen/arch/x86/dom0_build.c                     | 129 +----
 xen/arch/x86/domain_builder.c                 | 284 ++++++++++
 xen/arch/x86/efi/efi-boot.h                   |  96 ++--
 xen/arch/x86/guest/xen/pvh-boot.c             |  64 ++-
 xen/arch/x86/hvm/dom0_build.c                 |  62 +--
 xen/arch/x86/include/asm/bootdomain.h         |  30 ++
 xen/arch/x86/include/asm/bootinfo.h           |  99 ++++
 xen/arch/x86/include/asm/bzimage.h            |   5 +-
 xen/arch/x86/include/asm/dom0_build.h         |  27 +-
 xen/arch/x86/include/asm/guest/pvh-boot.h     |   6 +-
 xen/arch/x86/include/asm/setup.h              |  18 +-
 xen/arch/x86/pv/Makefile                      |   2 +-
 .../x86/pv/{dom0_build.c => domain_builder.c} | 141 ++---
 xen/arch/x86/pv/shim.c                        |   4 +-
 xen/arch/x86/setup.c                          | 392 ++++++--------
 xen/common/Kconfig                            |   5 +
 xen/common/Makefile                           |   4 +-
 xen/common/domain-builder/Kconfig             |  36 ++
 xen/common/domain-builder/Makefile            |   3 +
 xen/common/domain-builder/core.c              | 207 ++++++++
 xen/common/domain-builder/fdt.c               | 295 +++++++++++
 xen/common/domain-builder/fdt.h               |   7 +
 xen/common/domain-builder/hypfs.c             | 193 +++++++
 xen/common/efi/boot.c                         |   4 +-
 xen/common/fdt.c                              | 131 +++++
 xen/common/sched/core.c                       |  25 +-
 xen/include/xen/bootdomain.h                  |  58 ++
 xen/include/xen/bootinfo.h                    | 132 +++++
 xen/include/xen/device_tree.h                 |  50 +-
 xen/include/xen/domain_builder.h              |  88 ++++
 xen/include/xen/fdt.h                         |  79 +++
 xen/include/xen/sched.h                       |   3 +-
 xen/include/xsm/xsm.h                         |  26 +-
 xen/xsm/xsm_core.c                            |  43 +-
 xen/xsm/xsm_policy.c                          |  56 +-
 53 files changed, 3544 insertions(+), 1076 deletions(-)
 create mode 100644 tools/helpers/builder-hypfs.c
 create mode 100644 tools/helpers/hypfs-helpers.h
 create mode 100644 tools/helpers/late-init-pv.c
 create mode 100644 tools/helpers/late-init-pv.h
 create mode 100644 tools/helpers/xs-helpers.c
 create mode 100644 tools/helpers/xs-helpers.h
 create mode 100644 xen/arch/x86/boot/boot_info32.h
 create mode 100644 xen/arch/x86/domain_builder.c
 create mode 100644 xen/arch/x86/include/asm/bootdomain.h
 create mode 100644 xen/arch/x86/include/asm/bootinfo.h
 rename xen/arch/x86/pv/{dom0_build.c => domain_builder.c} (88%)
 create mode 100644 xen/common/domain-builder/Kconfig
 create mode 100644 xen/common/domain-builder/Makefile
 create mode 100644 xen/common/domain-builder/core.c
 create mode 100644 xen/common/domain-builder/fdt.c
 create mode 100644 xen/common/domain-builder/fdt.h
 create mode 100644 xen/common/domain-builder/hypfs.c
 create mode 100644 xen/common/fdt.c
 create mode 100644 xen/include/xen/bootdomain.h
 create mode 100644 xen/include/xen/bootinfo.h
 create mode 100644 xen/include/xen/domain_builder.h
 create mode 100644 xen/include/xen/fdt.h

-- 
2.20.1

Hi Daniel,

> -----Original Message-----
> Subject: [PATCH v1 00/18] Hyperlaunch

With the adjustments that I suggested in other messages, this patch builds and boots for me on x86 (including a device tree with a domU). I will continue to poke around and see if I discover any other rough edges.

One strange behavior I see is that xen fails to start the Dom0 kernel on a warm reboot. I'm using qemu_system_x86 with the KVM backend to test out the patch. After starting qemu, xen will boot correctly only once. If I attempt to reboot the virtual system (through the 'reboot' command in dom0 or the 'system_reset' qemu monitor command) without exiting/starting a new qemu process on the host machine, xen panics while booting after printing this:

(XEN) *** Building Dom0 ***
(XEN) Dom0 has maximum 856 PIRQs
(XEN) *** Constructing a PV Dom0 ***
(XEN) ELF: not an ELF binary
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Could not construct domain 0
(XEN) ****************************************

This happens with the BUILDER_FDT config option on and off, and regardless of what dtb (if any) I pass to xen. I don't see this behavior if I switch back to xen's master branch.

Hopefully that explanation made sense. Let me know if I can provide any further information about my setup.

Thanks,
Jackson

Also, I apologize that my last messages included a digital signature. Should be fixed now.

On 7/19/22 13:06, Smith, Jackson wrote:
> Hi Daniel,
> 
>> -----Original Message-----
>> Subject: [PATCH v1 00/18] Hyperlaunch
> 
> With the adjustments that I suggested in other messages, this patch builds and boots for me on x86 (including a device tree with a domU). I will continue to poke around and see if I discover any other rough edges.

Thank you so much for reviewing and testing!

> One strange behavior I see is that xen fails to start the Dom0 kernel on a warm reboot. I'm using qemu_system_x86 with the KVM backend to test out the patch. After starting qemu, xen will boot correctly only once. If I attempt to reboot the virtual system (through the 'reboot' command in dom0 or the 'system_reset' qemu monitor command) without exiting/starting a new qemu process on the host machine, xen panics while booting after printing this:
> 
> (XEN) *** Building Dom0 ***
> (XEN) Dom0 has maximum 856 PIRQs
> (XEN) *** Constructing a PV Dom0 ***
> (XEN) ELF: not an ELF binary
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 0:
> (XEN) Could not construct domain 0
> (XEN) ****************************************
> 
> This happens with the BUILDER_FDT config option on and off, and regardless of what dtb (if any) I pass to xen. I don't see this behavior if I switch back to xen's master branch.
> 
> Hopefully that explanation made sense. Let me know if I can provide any further information about my setup.

That is certainly a very strange behavior. I never tested reboot as I
assumed it should just go through the same process as could boot. I will
add this to my tests to run and see if I can track down why it is happening.

V/r,
Daniel P. Smith