[PATCH 0/2] Introducing hyperlaunch capability design (formerly: DomB mode of dom0less)

Daniel P. Smith posted 2 patches 3 years ago
Only 0 patches received!
.../designs/launch/hyperlaunch-devicetree.rst | 343 +++++++
docs/designs/launch/hyperlaunch.rst           | 900 ++++++++++++++++++
2 files changed, 1243 insertions(+)
create mode 100644 docs/designs/launch/hyperlaunch-devicetree.rst
create mode 100644 docs/designs/launch/hyperlaunch.rst
[PATCH 0/2] Introducing hyperlaunch capability design (formerly: DomB mode of dom0less)
Posted by Daniel P. Smith 3 years ago
We are submitting for inclusion in the Xen documentation:

- the hyperlaunch design document, and
- the hyperlaunch device tree design document

to describe a new method for launching the Xen hypervisor.

The hyperlaunch feature builds upon prior dom0less work and the DomB
project to bring a flexible and security-minded means to launch a
variety of VM configurations as part of the startup of Xen.

Signed-off-by: Christopher Clark <christopher.clark@starlab.io>
Signed-off by: Daniel P. Smith <dpsmith@apertussolutions.com>


Daniel P. Smith (2):
  docs/designs/launch: hyperlaunch design document
  docs/designs/launch: hyperlaunch device tree

 .../designs/launch/hyperlaunch-devicetree.rst | 343 +++++++
 docs/designs/launch/hyperlaunch.rst           | 900 ++++++++++++++++++
 2 files changed, 1243 insertions(+)
 create mode 100644 docs/designs/launch/hyperlaunch-devicetree.rst
 create mode 100644 docs/designs/launch/hyperlaunch.rst

-- 
2.20.1



Re: [PATCH 0/2] Introducing hyperlaunch capability design (formerly: DomB mode of dom0less)
Posted by Daniel P. Smith 3 years ago
On 3/15/21 11:18 PM, Daniel P. Smith wrote:
> We are submitting for inclusion in the Xen documentation:
> 
> - the hyperlaunch design document, and
> - the hyperlaunch device tree design document
> 
> to describe a new method for launching the Xen hypervisor.
> 
> The hyperlaunch feature builds upon prior dom0less work and the DomB
> project to bring a flexible and security-minded means to launch a
> variety of VM configurations as part of the startup of Xen.
> 
> Signed-off-by: Christopher Clark <christopher.clark@starlab.io>
> Signed-off by: Daniel P. Smith <dpsmith@apertussolutions.com>
> 
> 
> Daniel P. Smith (2):
>   docs/designs/launch: hyperlaunch design document
>   docs/designs/launch: hyperlaunch device tree
> 
>  .../designs/launch/hyperlaunch-devicetree.rst | 343 +++++++
>  docs/designs/launch/hyperlaunch.rst           | 900 ++++++++++++++++++
>  2 files changed, 1243 insertions(+)
>  create mode 100644 docs/designs/launch/hyperlaunch-devicetree.rst
>  create mode 100644 docs/designs/launch/hyperlaunch.rst
> 
To assist in reading, please find attached rendered copies of the design
docs. It should be noted that due to poor rendering by pandoc, we forced
the tables to stay as ASCII tables in the patches whereas the attached
docs have the tables rendered properly by rst2pdf.

V/r
Daniel P. Smith
Re: [PATCH 0/2] Introducing hyperlaunch capability design (formerly: DomB mode of dom0less)
Posted by Jan Beulich 2 years, 12 months ago
On 16.03.2021 04:56, Daniel P. Smith wrote:
> To assist in reading, please find attached rendered copies of the design
> docs. It should be noted that due to poor rendering by pandoc, we forced
> the tables to stay as ASCII tables in the patches whereas the attached
> docs have the tables rendered properly by rst2pdf.

In section 3.6 I found "As a result, on x86 the hyperlaunch capability does
not rely on nor preclude any specific BIOS boot protocol, i.e legacy BIOS
boot or UEFI boot. The only requirement is that the boot loader supports
the Multiboot2 (MB2) protocol." I'm afraid the two sentences contradict
one another, as UEFI on its own doesn't provide MB2 functionality. It is
my understanding that you don't require this anyway - what you need is a
way to load modules beyond just Dom0 kernel and an initrd.

I'm also struggling to see how you mean to associate the (MB2) modules
passed to Xen with the individual functions. I.e. I don't see how it will
be ensure that the embedded mb-index is in sync with the order or modules
actually supplied.

As to modules - iirc there are placement restrictions by at least some
boot loaders (below 4Gb). If that's correct, do you have any thoughts
towards dealing with the limited space available to hold these modules?
I've seem systems with lots of memory, but with as little as just 1Gb
below the 4Gb boundary.

Jan

Re: [PATCH 0/2] Introducing hyperlaunch capability design (formerly: DomB mode of dom0less)
Posted by Christopher Clark 2 years, 11 months ago
On Tue, Mar 30, 2021 at 7:31 AM Jan Beulich <jbeulich@suse.com> wrote:
>
> On 16.03.2021 04:56, Daniel P. Smith wrote:
> > To assist in reading, please find attached rendered copies of the design
> > docs. It should be noted that due to poor rendering by pandoc, we forced
> > the tables to stay as ASCII tables in the patches whereas the attached
> > docs have the tables rendered properly by rst2pdf.
>
> In section 3.6 I found "As a result, on x86 the hyperlaunch capability does
> not rely on nor preclude any specific BIOS boot protocol, i.e legacy BIOS
> boot or UEFI boot. The only requirement is that the boot loader supports
> the Multiboot2 (MB2) protocol." I'm afraid the two sentences contradict
> one another, as UEFI on its own doesn't provide MB2 functionality. It is
> my understanding that you don't require this anyway - what you need is a
> way to load modules beyond just Dom0 kernel and an initrd.

Thanks - we'll amend the doc. Given the already sizeable scope of the
project, our current approach for host UEFI is to recommend use of
GRUB.efi to load Xen and the initial domains via the multiboot2 method.

> I'm also struggling to see how you mean to associate the (MB2) modules
> passed to Xen with the individual functions. I.e. I don't see how it will
> be ensure that the embedded mb-index is in sync with the order or modules
> actually supplied.

To make sure I'm following the concern raised here, my understanding is:

- the multiboot module list is ordered and stable, so that the order
  that the bootloader provides the modules in is the same order in which
  Xen receives them, in the multiboot data structure, so the modules can
  be referenced by an index into that list

- for a given Launch Control Module file (expressed in a Device Tree, as
  described in our second design document), the provided multiboot
  modules will need to match, both in type (kernel, ramdisk, microcode,
  xsm policy) and order

- we think that at some point the LCM will end up being dynamically
  generated by an enlightened bootloader, assembling it from the
  bootloader config file, which will list the modules and their types

> As to modules - iirc there are placement restrictions by at least some
> boot loaders (below 4Gb). If that's correct, do you have any thoughts
> towards dealing with the limited space available to hold these modules?
> I've seem systems with lots of memory, but with as little as just 1Gb
> below the 4Gb boundary.

At this point, I don't think that we'll break anything that currently
works, and that hardware or platform limitations can always constrain
what is deployable.

Christopher

Re: [PATCH 0/2] Introducing hyperlaunch capability design (formerly: DomB mode of dom0less)
Posted by Jan Beulich 2 years, 11 months ago
On 07.04.2021 21:23, Christopher Clark wrote:
> On Tue, Mar 30, 2021 at 7:31 AM Jan Beulich <jbeulich@suse.com> wrote:
>>
>> On 16.03.2021 04:56, Daniel P. Smith wrote:
>>> To assist in reading, please find attached rendered copies of the design
>>> docs. It should be noted that due to poor rendering by pandoc, we forced
>>> the tables to stay as ASCII tables in the patches whereas the attached
>>> docs have the tables rendered properly by rst2pdf.
>>
>> In section 3.6 I found "As a result, on x86 the hyperlaunch capability does
>> not rely on nor preclude any specific BIOS boot protocol, i.e legacy BIOS
>> boot or UEFI boot. The only requirement is that the boot loader supports
>> the Multiboot2 (MB2) protocol." I'm afraid the two sentences contradict
>> one another, as UEFI on its own doesn't provide MB2 functionality. It is
>> my understanding that you don't require this anyway - what you need is a
>> way to load modules beyond just Dom0 kernel and an initrd.
> 
> Thanks - we'll amend the doc. Given the already sizeable scope of the
> project, our current approach for host UEFI is to recommend use of
> GRUB.efi to load Xen and the initial domains via the multiboot2 method.
> 
>> I'm also struggling to see how you mean to associate the (MB2) modules
>> passed to Xen with the individual functions. I.e. I don't see how it will
>> be ensure that the embedded mb-index is in sync with the order or modules
>> actually supplied.
> 
> To make sure I'm following the concern raised here, my understanding is:
> 
> - the multiboot module list is ordered and stable, so that the order
>   that the bootloader provides the modules in is the same order in which
>   Xen receives them, in the multiboot data structure, so the modules can
>   be referenced by an index into that list

In a separate context (parallel ongoing discussion under "multiboot2
and module2 boot issues via GRUB2") Andrew raised the (imo valid)
point of this getting the more fragile the more modules there are.

> - for a given Launch Control Module file (expressed in a Device Tree, as
>   described in our second design document), the provided multiboot
>   modules will need to match, both in type (kernel, ramdisk, microcode,
>   xsm policy) and order

"Need to match" meaning what? You don't clarify how boot loader
config and device tree blob are meant to be kept in sync.

> - we think that at some point the LCM will end up being dynamically
>   generated by an enlightened bootloader, assembling it from the
>   bootloader config file, which will list the modules and their types
> 
>> As to modules - iirc there are placement restrictions by at least some
>> boot loaders (below 4Gb). If that's correct, do you have any thoughts
>> towards dealing with the limited space available to hold these modules?
>> I've seem systems with lots of memory, but with as little as just 1Gb
>> below the 4Gb boundary.
> 
> At this point, I don't think that we'll break anything that currently
> works, and that hardware or platform limitations can always constrain
> what is deployable.

I'm not concerned of you breaking anything. I merely raised a possible
scalability issue of your current proposal.

Jan

Re: [PATCH 0/2] Introducing hyperlaunch capability design (formerly: DomB mode of dom0less)
Posted by Christopher Clark 2 years, 11 months ago
On Wed, Apr 7, 2021 at 10:56 PM Jan Beulich <jbeulich@suse.com> wrote:
>
> On 07.04.2021 21:23, Christopher Clark wrote:
> > On Tue, Mar 30, 2021 at 7:31 AM Jan Beulich <jbeulich@suse.com> wrote:
> >>
> >> On 16.03.2021 04:56, Daniel P. Smith wrote:
> >>> To assist in reading, please find attached rendered copies of the design
> >>> docs. It should be noted that due to poor rendering by pandoc, we forced
> >>> the tables to stay as ASCII tables in the patches whereas the attached
> >>> docs have the tables rendered properly by rst2pdf.
> >>
> >> In section 3.6 I found "As a result, on x86 the hyperlaunch capability does
> >> not rely on nor preclude any specific BIOS boot protocol, i.e legacy BIOS
> >> boot or UEFI boot. The only requirement is that the boot loader supports
> >> the Multiboot2 (MB2) protocol." I'm afraid the two sentences contradict
> >> one another, as UEFI on its own doesn't provide MB2 functionality. It is
> >> my understanding that you don't require this anyway - what you need is a
> >> way to load modules beyond just Dom0 kernel and an initrd.
> >
> > Thanks - we'll amend the doc. Given the already sizeable scope of the
> > project, our current approach for host UEFI is to recommend use of
> > GRUB.efi to load Xen and the initial domains via the multiboot2 method.
> >
> >> I'm also struggling to see how you mean to associate the (MB2) modules
> >> passed to Xen with the individual functions. I.e. I don't see how it will
> >> be ensure that the embedded mb-index is in sync with the order or modules
> >> actually supplied.
> >
> > To make sure I'm following the concern raised here, my understanding is:
> >
> > - the multiboot module list is ordered and stable, so that the order
> >   that the bootloader provides the modules in is the same order in which
> >   Xen receives them, in the multiboot data structure, so the modules can
> >   be referenced by an index into that list
>
> In a separate context (parallel ongoing discussion under "multiboot2
> and module2 boot issues via GRUB2") Andrew raised the (imo valid)
> point of this getting the more fragile the more modules there are.

It is a separate context though: ie. your (controversial!) proposal to
use the Xen command line to direct the hypervisor to concatenate
multiboot modules to form a single initrd before assembling a domain.

Boot via Multiboot2 relies on the positional correctness (ie. ordering)
of the modules supplied.

Andrew's first message in the thread describes Xen's longstanding
convention for handling the multiboot modules that it receives:

    Xen is capable of taking various things via modules, such as an
    XSM/Flask policy, or microcode, so has logic to identify these if
    present and separate them from "other stuff".  However, there is a
    hardcoded expectation that the first module is the dom0 kernel, and
    the next unrecognised module, if present, is *the* initrd.

We are proposing to make that hardcoded expectation become configurable
via the definition provided in the Launch Control Module. The modules
supplied will have to be correctly ordered, matching that definition.

Xen has behaviour today for when the boot materials are incorrect
(eg. when dom0 initrd is supplied before the dom0 kernel) and that is
reasonable, and the same can apply to misconfigured systems in future
too.

> > - for a given Launch Control Module file (expressed in a Device Tree, as
> >   described in our second design document), the provided multiboot
> >   modules will need to match, both in type (kernel, ramdisk, microcode,
> >   xsm policy) and order
>
> "Need to match" meaning what? You don't clarify how boot loader
> config and device tree blob are meant to be kept in sync.

A given Launch Control Module (or device tree blob) defines the
expectation for boot module ordering. The boot loader config needs to be
written with an awareness of that ordering.

Tooling to support that synchronization can be provided - as noted here:

> > - we think that at some point the LCM will end up being dynamically
> >   generated by an enlightened bootloader, assembling it from the
> >   bootloader config file, which will list the modules and their types

which should help address usability concerns.

> >> As to modules - iirc there are placement restrictions by at least some
> >> boot loaders (below 4Gb). If that's correct, do you have any thoughts
> >> towards dealing with the limited space available to hold these modules?
> >> I've seem systems with lots of memory, but with as little as just 1Gb
> >> below the 4Gb boundary.
> >
> > At this point, I don't think that we'll break anything that currently
> > works, and that hardware or platform limitations can always constrain
> > what is deployable.
>
> I'm not concerned of you breaking anything. I merely raised a possible
> scalability issue of your current proposal.

Fair. Thanks for the review.

Christopher