Theses are the broad changes needed for a split hardware / control
domain.

An earlier posting gave device_model privileges to hardware domain.  For
this posting, it was split out into a new capability.  This way the
operator can choose where to run the device models without making the
hardware domain have the permissions.

The first patch add XSM_HW_PRIV for the hardware hypercalls.  Unlike the
first posting, the control domain can call these hypercalls even though
it doesn't really make sense.  The idea was to keep the control domain
all powerful from an XSM perspective.

SILO is changed to allow control, hardwware or xenstore to service
domUs.  Xenstore and hardware will use grants for PV interfaces.
Control wouldn't typically provide PV interfaces to domUs, but it is
given the permision to do so.  Again, to keep control all powerful.

xsm/dummy: Allow hwdom SYSCTL_readconsole/physinfo this is not strictly
needed.  xenconsoled could read Xen's dmesg.  If it's in hwdom, then
that permission would be required.  SYSCTL_physinfo is mainly to silence
xl messages, which are mostly cosmetic.

Jason Andryuk (4):
  xen/xsm: Add XSM_HW_PRIV
  xsm/silo: Support hwdom/control domains
  xen: Add DOMAIN_CAPS_DEVICE_MODEL & XEN_DOMCTL_CDF_device_model
  xsm/dummy: Allow hwdom SYSCTL_readconsole/physinfo

 tools/ocaml/libs/xc/xenctrl.ml          |  1 +
 tools/ocaml/libs/xc/xenctrl.mli         |  1 +
 xen/arch/arm/domain.c                   |  3 ++-
 xen/arch/arm/platform_hypercall.c       |  2 +-
 xen/arch/x86/msi.c                      |  2 +-
 xen/arch/x86/physdev.c                  | 12 ++++-----
 xen/arch/x86/platform_hypercall.c       |  2 +-
 xen/common/device-tree/dom0less-build.c |  3 +++
 xen/common/domain.c                     |  3 ++-
 xen/common/sysctl.c                     |  2 +-
 xen/drivers/passthrough/pci.c           |  5 ++--
 xen/drivers/pci/physdev.c               |  2 +-
 xen/include/public/bootfdt.h            | 12 +++++++--
 xen/include/public/domctl.h             |  4 ++-
 xen/include/xen/sched.h                 |  9 +++++++
 xen/include/xsm/dummy.h                 | 36 ++++++++++++++++++-------
 xen/include/xsm/xsm.h                   |  1 +
 xen/xsm/silo.c                          | 10 +++++--
 18 files changed, 80 insertions(+), 30 deletions(-)

-- 
2.49.0

On 11.06.2025 00:57, Jason Andryuk wrote:
> Theses are the broad changes needed for a split hardware / control
> domain.
> 
> An earlier posting gave device_model privileges to hardware domain.  For
> this posting, it was split out into a new capability.  This way the
> operator can choose where to run the device models without making the
> hardware domain have the permissions.
> 
> The first patch add XSM_HW_PRIV for the hardware hypercalls.  Unlike the
> first posting, the control domain can call these hypercalls even though
> it doesn't really make sense.  The idea was to keep the control domain
> all powerful from an XSM perspective.
> 
> SILO is changed to allow control, hardwware or xenstore to service
> domUs.  Xenstore and hardware will use grants for PV interfaces.
> Control wouldn't typically provide PV interfaces to domUs, but it is
> given the permision to do so.  Again, to keep control all powerful.
> 
> xsm/dummy: Allow hwdom SYSCTL_readconsole/physinfo this is not strictly
> needed.  xenconsoled could read Xen's dmesg.  If it's in hwdom, then
> that permission would be required.  SYSCTL_physinfo is mainly to silence
> xl messages, which are mostly cosmetic.
> 
> Jason Andryuk (4):
>   xen/xsm: Add XSM_HW_PRIV
>   xsm/silo: Support hwdom/control domains
>   xen: Add DOMAIN_CAPS_DEVICE_MODEL & XEN_DOMCTL_CDF_device_model
>   xsm/dummy: Allow hwdom SYSCTL_readconsole/physinfo

Overall I can't help the impression that this level of disaggregation simply
requires the use of Flask.

Jan

On 2025-06-11 09:28, Jan Beulich wrote:
> On 11.06.2025 00:57, Jason Andryuk wrote:
>> Theses are the broad changes needed for a split hardware / control
>> domain.
>>
>> An earlier posting gave device_model privileges to hardware domain.  For
>> this posting, it was split out into a new capability.  This way the
>> operator can choose where to run the device models without making the
>> hardware domain have the permissions.
>>
>> The first patch add XSM_HW_PRIV for the hardware hypercalls.  Unlike the
>> first posting, the control domain can call these hypercalls even though
>> it doesn't really make sense.  The idea was to keep the control domain
>> all powerful from an XSM perspective.
>>
>> SILO is changed to allow control, hardwware or xenstore to service
>> domUs.  Xenstore and hardware will use grants for PV interfaces.
>> Control wouldn't typically provide PV interfaces to domUs, but it is
>> given the permision to do so.  Again, to keep control all powerful.
>>
>> xsm/dummy: Allow hwdom SYSCTL_readconsole/physinfo this is not strictly
>> needed.  xenconsoled could read Xen's dmesg.  If it's in hwdom, then
>> that permission would be required.  SYSCTL_physinfo is mainly to silence
>> xl messages, which are mostly cosmetic.
>>
>> Jason Andryuk (4):
>>    xen/xsm: Add XSM_HW_PRIV
>>    xsm/silo: Support hwdom/control domains
>>    xen: Add DOMAIN_CAPS_DEVICE_MODEL & XEN_DOMCTL_CDF_device_model
>>    xsm/dummy: Allow hwdom SYSCTL_readconsole/physinfo
> 
> Overall I can't help the impression that this level of disaggregation simply
> requires the use of Flask.

I have thought about that.  The problem with Flask is the complexity of 
the security server.  We don't want to have to deal with all that code. 
A fixed policy is easier for our coverage testing.

Exposing separate control, hardware and xenstore capabilities, it makes 
sense for the default policy to function with them.  This would be a 
coarse level of functionality, and Flask would remain for fine-grain and 
MAC enforcement.

Regards,
Jason

On 11.06.2025 07:08, Jason Andryuk wrote:
> On 2025-06-11 09:28, Jan Beulich wrote:
>> On 11.06.2025 00:57, Jason Andryuk wrote:
>>> Theses are the broad changes needed for a split hardware / control
>>> domain.
>>>
>>> An earlier posting gave device_model privileges to hardware domain.  For
>>> this posting, it was split out into a new capability.  This way the
>>> operator can choose where to run the device models without making the
>>> hardware domain have the permissions.
>>>
>>> The first patch add XSM_HW_PRIV for the hardware hypercalls.  Unlike the
>>> first posting, the control domain can call these hypercalls even though
>>> it doesn't really make sense.  The idea was to keep the control domain
>>> all powerful from an XSM perspective.
>>>
>>> SILO is changed to allow control, hardwware or xenstore to service
>>> domUs.  Xenstore and hardware will use grants for PV interfaces.
>>> Control wouldn't typically provide PV interfaces to domUs, but it is
>>> given the permision to do so.  Again, to keep control all powerful.
>>>
>>> xsm/dummy: Allow hwdom SYSCTL_readconsole/physinfo this is not strictly
>>> needed.  xenconsoled could read Xen's dmesg.  If it's in hwdom, then
>>> that permission would be required.  SYSCTL_physinfo is mainly to silence
>>> xl messages, which are mostly cosmetic.
>>>
>>> Jason Andryuk (4):
>>>    xen/xsm: Add XSM_HW_PRIV
>>>    xsm/silo: Support hwdom/control domains
>>>    xen: Add DOMAIN_CAPS_DEVICE_MODEL & XEN_DOMCTL_CDF_device_model
>>>    xsm/dummy: Allow hwdom SYSCTL_readconsole/physinfo
>>
>> Overall I can't help the impression that this level of disaggregation simply
>> requires the use of Flask.
> 
> I have thought about that.  The problem with Flask is the complexity of 
> the security server.  We don't want to have to deal with all that code. 
> A fixed policy is easier for our coverage testing.
> 
> Exposing separate control, hardware and xenstore capabilities, it makes 
> sense for the default policy to function with them.

Yet as indicated in replies to individual patches - the boundaries between
the three aren't clear, which imo goes (to some degree at least) against the
purpose of "disaggregation".

Jan

>  This would be a 
> coarse level of functionality, and Flask would remain for fine-grain and 
> MAC enforcement.
> 
> Regards,
> Jason

On Thu, 12 Jun 2025, Jan Beulich wrote:
> On 11.06.2025 07:08, Jason Andryuk wrote:
> > On 2025-06-11 09:28, Jan Beulich wrote:
> >> On 11.06.2025 00:57, Jason Andryuk wrote:
> >>> Theses are the broad changes needed for a split hardware / control
> >>> domain.
> >>>
> >>> An earlier posting gave device_model privileges to hardware domain.  For
> >>> this posting, it was split out into a new capability.  This way the
> >>> operator can choose where to run the device models without making the
> >>> hardware domain have the permissions.
> >>>
> >>> The first patch add XSM_HW_PRIV for the hardware hypercalls.  Unlike the
> >>> first posting, the control domain can call these hypercalls even though
> >>> it doesn't really make sense.  The idea was to keep the control domain
> >>> all powerful from an XSM perspective.
> >>>
> >>> SILO is changed to allow control, hardwware or xenstore to service
> >>> domUs.  Xenstore and hardware will use grants for PV interfaces.
> >>> Control wouldn't typically provide PV interfaces to domUs, but it is
> >>> given the permision to do so.  Again, to keep control all powerful.
> >>>
> >>> xsm/dummy: Allow hwdom SYSCTL_readconsole/physinfo this is not strictly
> >>> needed.  xenconsoled could read Xen's dmesg.  If it's in hwdom, then
> >>> that permission would be required.  SYSCTL_physinfo is mainly to silence
> >>> xl messages, which are mostly cosmetic.
> >>>
> >>> Jason Andryuk (4):
> >>>    xen/xsm: Add XSM_HW_PRIV
> >>>    xsm/silo: Support hwdom/control domains
> >>>    xen: Add DOMAIN_CAPS_DEVICE_MODEL & XEN_DOMCTL_CDF_device_model
> >>>    xsm/dummy: Allow hwdom SYSCTL_readconsole/physinfo
> >>
> >> Overall I can't help the impression that this level of disaggregation simply
> >> requires the use of Flask.
> > 
> > I have thought about that.  The problem with Flask is the complexity of 
> > the security server.  We don't want to have to deal with all that code. 
> > A fixed policy is easier for our coverage testing.
> > 
> > Exposing separate control, hardware and xenstore capabilities, it makes 
> > sense for the default policy to function with them.
> 
> Yet as indicated in replies to individual patches - the boundaries between
> the three aren't clear, which imo goes (to some degree at least) against the
> purpose of "disaggregation".

At a high level things are clear. We have a static number of VMs, so
with Dom0less we don't need domain creation capabilities. Dom0 can be
de-privileged and becomes the Hardware Domain. To allow monitoring and
other privileged operations an optional Control Domain might be present
(or not).

- The Control Domain has d->is_privileged == true but has no direct
  access to hardware by default. With Dom0less ability to create all the
  VMs at boot, the Control Domain is not required for domain creation,
  so it is just optional.

- The Hardware Domain has all hardware by default but d->is_privileged
  == false. The Hardware Domain should be as close to a DomU as possible
  in terms of privileges but it must be able to run PV backends and
  Virtio backends.

- Xenstored will typically be in the Hardware Domain to enable PV
  backends, especially as the Control Domain is optional. It could also
  live in its own separate domain to speed up the boot. If a user
  chooses to deploy a Control Domain, xenstored could also run there.
  It could be entirely missing if a user only deploys Virtio. It makes
  sense to provide flexibility on this.

We have other instances of XSM policies without Flask, such as SILO. I
think the industrial/automotive use-case is clear enough and shared
among many Xen community members so I think it makes sense to optimize a
policy for it without having to involving Flask because Flask's
complexity would cause trouble in terms of coverage testing and
validation.