From: Stefano Stabellini <stefano.stabellini@amd.com>

Define "Hardware Domain", "Control Domain", "DomUs", "SafeVMs" and
"UnsafeVMs".

Explain the limitations that VirtIO introduces and the way they affect
the defined roles.

Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
---
 docs/fusa/arch_specs/roles.rst | 200 +++++++++++++++++++++++++++++++++
 docs/fusa/index.rst            |   1 +
 2 files changed, 201 insertions(+)
 create mode 100644 docs/fusa/arch_specs/roles.rst

diff --git a/docs/fusa/arch_specs/roles.rst b/docs/fusa/arch_specs/roles.rst
new file mode 100644
index 0000000000..0b3fe821c3
--- /dev/null
+++ b/docs/fusa/arch_specs/roles.rst
@@ -0,0 +1,200 @@
+===============================
+Xen Domain Roles and Privileges
+===============================
+
+Roles
+=====
+
+We define the following roles that Xen domains can have:
+- Hardware Domain
+- Control Domain
+- DomU
+
+In addition, we classify domains based on the criticality of the tasks
+they perform:
+- SafeVM
+- UnSafeVM
+
+DomU
+----
+
+A DomU is a regular domain without extra privileges to perform
+operations that affect other domains or the entire platform. A DomU is
+therefore unprivileged and can only affect itself.
+
+Hardware Domain
+---------------
+
+The Hardware Domain is a domain that has direct access to all hardware
+peripherals by default, except for those controlled by Xen (CPU, MMU,
+IOMMU, Interrupt Controller, Timers, UART). Hardware resources can be
+directly assigned to other domains ("passthrough"), but the default is
+that they get assigned to the Hardware Domain. As such, there can only
+be one Hardware Domain in the system. The Hardware Domain is not
+privileged: it cannot perform operations that affect other domains or
+the entire platform.
+
+In cases where VirtIO is used to share access to physical devices among
+multiple domains, the Hardware Domain requires some limited privileges
+to run the VirtIO backends.
+
+Control Domain
+--------------
+
+The Control Domain has full privileges over the system. As such, it can
+perform privileged operations, typically implemented as hypercalls, that
+affect other domains and even the entire platform, such as platform
+reboot. There can only be one Control Domain in the system. A
+non-exhaustive list of these operations follows:
+
+- Reboot other domains
+- Reboot and shutdown the platform
+- Suspend/resume other domains
+- Pause/unpause other domains
+
+The Control Domain is optional.
+
+SafeVMs and UnSafeVMs
+---------------------
+
+A SafeVM is a domain running a safety-critical task. An UnsafeVM is a
+domain running non-safety-critical software only. No safety-critical
+tasks are running in the UnsafeVM.
+
+The Control Domain must be a SafeVM because the Control Domain has
+enough privileges to affect the entire system.
+
+Without VirtIO (see below), both the Hardware Domain and DomUs can be
+SafeVMs or UnsafeVMs. There are no restrictions. For example, the
+following configuration is compatible with safety:
+
+- Hardware Domain: UnsafeVM
+- 2x DomU: SafeVMs
+- 2x DomU: UnsafeVMs
+
+Privileges
+==========
+
+Full Privileges
+---------------
+
+A domain with full privileges has unrestricted access to all services
+provided by Xen, including the ability to perform operations that impact
+other VMs and the entire platform. Only one domain holds these full
+privileges: the Control Domain.
+
+Unprivileged
+------------
+
+An Unprivileged domain lacks special access rights, allowing it to
+perform only operations that affect itself without impacting other
+domains. For example, an Unprivileged domain cannot map the memory of
+another domain.
+
+VirtIO Privileges
+-----------------
+
+Domains running VirtIO backends, such as QEMU, require specific
+privileges over domains with VirtIO frontends to enable proper
+functionality. For instance, they need permission to map the memory of
+the domain with the VirtIO frontend. These additional privileges are
+necessary for standard VirtIO operation but can be removed when using
+the VirtIO extensions described in the VirtIO chapter. This is the list
+of VirtIO privileges:
+- mapping memory of VirtIO frontend domains (foreign map hypercall)
+- device model hypercalls targeting VirtIO frontend domains
+
+VirtIO
+======
+
+VirtIO is a specification and set of drivers to share a physical device
+among multiple domains. For example, assuming the Hardware Domain has
+access to an Ethernet device, VirtIO provides a way for the Hardware
+Domain to allow one or more DomUs to access the Ethernet device.
+
+VirtIO is based on a frontend/backend architecture: the frontend is a
+driver similar to a regular Ethernet device driver (using Ethernet as an
+example), running in the DomU. The backend is a "proxy" driver running
+in the same domain as the device driver and have direct access to the
+device. That domain is typically the Hardware Domain because it is the
+default domain for hardware assignment, but it is also possible to run
+VirtIO backends in other domains, provided they have been configured with
+passthrough access to a physical device. These other domains are called
+"Driver Domains," and the limitations discussed in the context of VirtIO
+for the Hardware Domain also apply to Driver Domains.
+
+There are multiple issues in VirtIO that affect freedom from
+interference, so deploying VirtIO introduces limits on the configuration
+regarding what domains can run safety-critical workloads.
+
+It is important to note that VirtIO is not the only way for domains to
+communicate with each other. Specifically, there are other protocols
+such as Argo and plain shared memory that do not have the same
+interference concerns.
+
+FFI: Shared Memory and Interrupts
+---------------------------------
+
+All VirtIO protocols are based on ring buffers over shared memory,
+called "VirtQueues." Although there are well-known methods to set up
+ring buffers over shared memory that are free from interference, in
+practice, the existing VirtIO specification and current frontend/backend
+driver implementations do not follow these best practices. Thus, the
+ring buffers over shared memory can be a source of interference.
+
+Given the current state of the VirtIO specification and drivers, it is
+recommended that both VirtIO frontends and backends run in UnsafeVMs
+only. This way, SafeVMs will not be affected by interference caused by
+VirtIO protocols.
+
+As ring buffers exist over memory shared between domains, it is the
+domains' responsibility to ensure they are safe. This is outside of
+Xen's responsibility, and out of scope for this document.
+
+FFI: Memory Mappings
+--------------------
+
+VirtIO expects the backend to access ("map") arbitrary memory addresses
+passed by the frontend. In other words, the backend needs to map any
+memory of the frontend, which is a privileged operation in Xen.
+
+Xen implements a Xen-specific extension to VirtIO to address this issue:
+"VirtIO with Grants."
+
+Xen has a feature called the "grant table" that guests can use to
+voluntarily share individual memory pages with other domains. This way,
+the backend does not need the privilege to map any memory page of the
+domain running the frontend, as the frontend whitelists pages for the
+backend to access.
+
+Unless "VirtIO with Grants" is used, to enable VirtIO, a domain with
+VirtIO backends (the Hardware Domain or a Driver Domain) is given the
+extra privilege of being able to map any memory of domains with VirtIO
+frontends.  As this is a source of interference, only UnsafeVMs can run
+VirtIO frontends without "VirtIO with Grants".
+
+FFI: Synchronous Emulation
+--------------------------
+
+VirtIO expects certain operations to be synchronous: the frontend
+virtual CPU requesting the operation is fully blocked until the backend
+completes the operation. If the backend does nothing, the frontend vCPU
+remains blocked. This is a source of interference.
+
+When VirtIO is enabled, the domain with the backends (Hardware Domain or
+Driver Domain) implements these synchronous operations for domains
+running VirtIO frontends.
+
+Xen implements a Xen-specific extensions to VirtIO to address this
+issue: "non-blocking VirtIO." Two extra virtio-pci registers are
+provided, allowing the frontend to perform non-blocking operations. By
+using the two Xen-specific non-blocking registers, the frontend virtual
+CPU doesn't have to block while waiting for the backend to complete the
+operations.
+
+Unless "non-blocking VirtIO" is used, to enable VirtIO, a domain with
+VirtIO backends (the Hardware Domain or a Driver Domain) is given the
+extra privilege of being able to implement these synchronous operations
+for domains running VirtIO frontends. As this is a source of
+interference, only UnsafeVMs can run VirtIO frontends without
+"non-blocking VirtIO."
diff --git a/docs/fusa/index.rst b/docs/fusa/index.rst
index 5f1e8acfc4..9d989f5105 100644
--- a/docs/fusa/index.rst
+++ b/docs/fusa/index.rst
@@ -7,3 +7,4 @@ Functional Safety documentation
    :maxdepth: 2
 
    reqs/index
+   arch_specs/roles
-- 
2.25.1