This patch series provides emulated SMMUv3 support in Xen, enabling Stage-1
translation for guest operating systems.
Stage-1 translation is required to provide isolation between different
devices within an operating system. Xen already supports Stage-2 translation,
but there is no support for Stage-1 translation. The goal of this work is to
support Stage-1 translation for Xen guests.
This patch series is a continuation of the work by Rahul Singh:
https://patchwork.kernel.org/project/xen-devel/cover/cover.1669888522.git.rahul.singh@arm.com/
The original patch series has been aligned with the current Xen code
structure, with the addition of a translation layer providing 1:N
vIOMMU-to-pIOMMU mapping. This allows passthrough of devices attached to
different physical IOMMUs.
We cannot trust the guest OS to control the SMMUv3 hardware directly, as a
compromised guest OS could corrupt the SMMUv3 configuration and make the
system vulnerable. The guest owns the Stage-1 page tables and Stage-1
configuration structures. Xen handles the root configuration structure for
security reasons, including the Stage-2 configuration.
Xen emulates the SMMUv3 hardware and exposes a virtual SMMUv3 to the guest.
The guest can use the native SMMUv3 driver to configure Stage-1 translation.
When the guest configures the SMMUv3 for Stage-1 translation, Xen traps the
access and configures the physical SMMU accordingly.
The final patch in the series provides a design document for the emulated
IOMMU (arm-viommu.rst), which was previously discussed with the maintainers.
Details regarding the implementation, future work, and security risks are
outlined in this document.
---
Changes in v2:
- Updated design and implementation with vIOMMU->pIOMMU mapping layer
- Addressed security risks in the design, provided initial performance
measurements
- Addressed comments from previous version
- Tested on Renesas R-Car platform, initial performance measurements for
stage-1 vs stage-1-less guests
---
---
Changes in v3:
- Bump domctl version, added explicit padding for the new domctl structures
- Remove unnecessary changes according to review comments
- Add "ARM" prefix for vIOMMU Kconfig options, since only ARM architecture is
supported at this point
- Re-generate go code
- Add missing commit sign-off tags
---
---
Changes in v4:
- Added details on current support, with TODOs marking unsupported features
and limitations
- Updated emulated IOMMU feature handling to advertise features supported by
the underlying SMMU, except for features explicitly disabled in the
emulation layer
- Reworked event queue and GERROR support, including GERROR
assert/acknowledge logic, IRQ injection, and event queue register access
on page 1
- Enabled Stage-1-only support when the underlying hardware does not support
Stage-2
- Removed the Xen command-line boot option ("viommu") for enabling the
vIOMMU, as it was considered redundant. The vIOMMU is now enabled through
Kconfig when the underlying SMMU hardware is discovered at runtime
- Added locking and access checks for vIOMMU register reads and writes
- Ported errata workarounds for MMU-600/MMU-700 products from Linux to
disable nesting
- Updated the vSID translation layer to map vSIDs to (pSMMU, pSID) pairs,
resolving the duplicated pSID issue
- Updated Stage-2 configuration to share the VMID between devices belonging
to the same domain and pSMMU
- Added bug fixes and code formatting changes
- Updated the design document (arm-viommu.rst) with details on current
support, limitations, and future workb
To reduce the number of patches, PCI updates for the emulated IOMMU are
excluded from this series. PCI support and long-running command queue
handling will be sent as separate patches.
Since this is a significant rework of the patch series, Reviewed-by tags
from previous iteration are included only for patches that are unchanged.
---
---
Changes in v5:
- Addressed review comments from v4:
- Handle stage-1-only hardware, allowing stage-1-only mode exclusively
for direct-mapped domains.
- dom0less domain creation: disable vIOMMU if passthrough is not
permitted for the domain.
- Align invalid STE handling and error reporting with the SMMUv3
specification.
- Map guest bypass STE configuration to stage-2-only configuration.
- Refuse vIOMMU initialization if nesting is disabled for the target
IOMMU hardware, or if the hardware does not support stage-2 translation
and the domain is non-direct-mapped.
- Support hardware layouts where pIOMMUs reside behind different buses.
- Fix event forwarding by translating pSID to vSID before forwarding
events to the guest.
- Propagate errors when updating the domain DT iommus property.
- Dynamically allocate vSID-to-pSID mapping entries instead of using
a pre-allocated array.
- tools: Improve vSID lookup logic by including both the DT path and
domain ID in the lookup key.
- Document stage-1-only mode in the design documentation.
- Apply minor MISRA fixes and fix RISC-V build.
---
Jean-Philippe Brucker (1):
xen/arm: smmuv3: Maintain a SID->device structure
Milan Djokic (4):
xen/arm: smmuv3: Extend HW features detection and apply errata
xen/arm: vIOMMU vSID->pSID mapping layer
libxl/arm: Introduce domctl command for IOMMU vSID mapping
doc/arm: vIOMMU design document
Rahul Singh (17):
xen/arm: smmuv3: Add support for stage-1 and nested stage translation
xen/arm: smmuv3: Alloc io_domain for each device
xen/arm: vIOMMU: add generic vIOMMU framework
xen/arm: vsmmuv3: Add dummy support for virtual SMMUv3 for guests
tools: Add XEN_DOMCTL_CONFIG_VIOMMU_* and viommu config param
xen/arm: vsmmuv3: Add support for registers emulation
xen/arm: vsmmuv3: Add support for cmdqueue handling
xen/arm: vsmmuv3: Add support for command CMD_CFGI_STE
xen/arm: vsmmuv3: Attach Stage-1 configuration to SMMUv3 hardware
xen/arm: vsmmuv3: Add support for event queue and global error
xen/arm: vsmmuv3: IOMMU device tree node for dom0
xen/arm: vsmmuv3: Emulated SMMUv3 device tree node for dom0less
arm/libxl: vsmmuv3: Emulated SMMUv3 device tree node in libxl
xen/arm: vsmmuv3: Insert vIRQ properties into vIOMMU device tree node
xen/arm: vsmmuv3: Add support to send stage-1 event to guest
libxl/arm: vIOMMU: Modify the partial device tree for iommus
xen/arm: vIOMMU: Modify the partial device tree for dom0less
docs/designs/arm-viommu.rst | 530 ++++++++
docs/man/xl.cfg.5.pod.in | 22 +
tools/golang/xenlight/helpers.gen.go | 2 +
tools/golang/xenlight/types.gen.go | 7 +
tools/include/libxl.h | 5 +
tools/include/xenctrl.h | 13 +
tools/libs/ctrl/xc_domain.c | 25 +
tools/libs/light/libxl_arm.c | 215 +++-
tools/libs/light/libxl_types.idl | 6 +
tools/xl/xl_parse.c | 9 +
xen/arch/arm/dom0less-build.c | 125 +-
xen/arch/arm/domain.c | 35 +-
xen/arch/arm/domain_build.c | 106 +-
xen/arch/arm/domctl.c | 38 +
xen/arch/arm/include/asm/domain.h | 5 +
xen/arch/arm/include/asm/iommu.h | 7 +
xen/arch/arm/include/asm/viommu.h | 123 ++
xen/common/device-tree/dom0less-build.c | 72 +-
xen/drivers/passthrough/Kconfig | 13 +
xen/drivers/passthrough/arm/Makefile | 2 +
xen/drivers/passthrough/arm/smmu-v3.c | 622 ++++++++-
xen/drivers/passthrough/arm/smmu-v3.h | 75 +-
xen/drivers/passthrough/arm/viommu.c | 142 +++
xen/drivers/passthrough/arm/vsmmu-v3.c | 1533 +++++++++++++++++++++++
xen/drivers/passthrough/arm/vsmmu-v3.h | 28 +
xen/include/public/arch-arm.h | 13 +-
xen/include/public/device_tree_defs.h | 1 +
xen/include/public/domctl.h | 22 +
xen/include/xen/iommu.h | 10 +
xen/xsm/flask/hooks.c | 4 +
xen/xsm/flask/policy/access_vectors | 2 +
31 files changed, 3733 insertions(+), 79 deletions(-)
create mode 100644 docs/designs/arm-viommu.rst
create mode 100644 xen/arch/arm/include/asm/viommu.h
create mode 100644 xen/drivers/passthrough/arm/viommu.c
create mode 100644 xen/drivers/passthrough/arm/vsmmu-v3.c
create mode 100644 xen/drivers/passthrough/arm/vsmmu-v3.h
--
2.43.0