Hi Penny,
Thanks for the update. I tested the series in a couple of different
configurations and it works great!
You can add my Tested-by to all patches
On Mon, 20 Dec 2021, Penny Zheng wrote:
> Cases where domU needs direct-map memory map:
> * IOMMU not present in the system.
> * IOMMU disabled if it doesn't cover a specific device and all the guests
> are trusted. Thinking a mixed scenario, where a few devices with IOMMU and
> a few without, then guest DMA security still could not be totally guaranteed.
> So users may want to disable the IOMMU, to at least gain some performance
> improvement from IOMMU disabled.
> * IOMMU disabled as a workaround when it doesn't have enough bandwidth.
> To be specific, in a few extreme situation, when multiple devices do DMA
> concurrently, these requests may exceed IOMMU's transmission capacity.
> * IOMMU disabled when it adds too much latency on DMA. For example,
> TLB may be missing in some IOMMU hardware, which may bring latency in DMA
> progress, so users may want to disable it in some realtime scenario.
> * Guest OS relies on the host memory layout
>
> "direct-map" property shall be added under the appropriate domain node,
> when users requesting direct-map memory mapping for the domain.
>
> Right now, direct-map is only supported when domain on Static Allocation,
> that is, "xen,static-mem" is also necessary in the domain configuration.
>
> Looking into related [design link](
> https://lists.xenproject.org/archives/html/xen-devel/2021-05/msg00882.html)
> for more details.
>
> The whole design is about Static Allocation and direct-map, and this
> Patch Serie only covers parts of it, which are direct-map memory map.
> Other features will be delievered through different patch series.
>
> See https://lists.xenproject.org/archives/html/xen-devel/2021-09/msg00855.html
> for Domain on Static Allocation.
>
> This patch serie is based on
> https://lists.xenproject.org/archives/html/xen-devel/2021-10/msg00822.html\
> ---
> v4 changes:
> - introduce internal const CDF_xxx flags for domain creation
> - introduce internal flag CDF_privileged
> - introduce new internal flag CDF_directmap
> - add a directmap flag under struct arch_domain and use it to
> reimplement is_domain_direct_mapped.
> - expand arch_domain_create/domain_create to include internal-only parameter
> "const unsigned int flags"
> - use mfn_eq() instead, because it is the only value used to indicate
> there is an error and this is more lightweight than mfn_valid()
> - rename function allocate_static_memory_11() to assign_static_memory_11()
> to make clear there is actually no allocation done. Instead we are only
> mapping pre-defined host regions to pre-defined guest regions.
> - remove tot_size to directly substract psize from kinfo->unassigned_mem
> - check kinfo->unassigned_mem doesn't underflow or overflow
> - remove nested if/else
> - remove ASSERT_UNREACHABLE() to avoid breaking compilation on prod build with
> CONFIG_GICV3=n
> - comment and commit message refinement
> ---
> v3 changes:
> - move flag XEN_DOMCTL_CDF_INTERNAL_directmap back to xen/include/xen/domain.h,
> to let it be only available for domain created by XEN.
> - name it with extra "INTERNAL" and add comments to warn developers not
> to accidently use its bitfield when introducing new XEN_DOMCTL_CDF_xxx flag.
> - reject this flag in x86'es arch_sanitise_domain_config()
> - add ASSERT_UNREACHABLE to catch any misuse in allocate_static_memory()
> and allocate_static_memory_11()
> - add another check of validating flag XEN_DOMCTL_CDF_INTERNAL_directmap only
> when CONFIG_STATIC_MEMORY is set.
> - simply map the CPU interface at the GPA vgic_v2_hw.cbase
> - drop 'cells += (GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS)'
> - rename 'is_domain_use_host_layout()' to 'domain_use_host_layout()'
> ---
> v2 changes:
> - remove the introduce of internal flag
> - Refine is_domain_direct_mapped to check whether the flag
> XEN_DOMCTL_CDF_directmap is set
> - reword "1:1 direct-map" to just "direct-map"
> - split the common codes into two helpers: parse_static_mem_prop and
> acquire_static_memory_bank to deduce complexity.
> - introduce a new helper allocate_static_memory_11 for allocating static
> memory for direct-map guests
> - remove panic action since it is fine to assign a non-DMA capable device when
> IOMMU and direct-map both off
> - remove redistributor accessor
> - introduce new helper "is_domain_use_host_layout()"
> - explain why vpl011 initialization before creating its device tree node
> - error out if the domain is direct-mapped and the IRQ is not found
> - harden the code and add a check/comment when the hardware UART region
> is smaller than CUEST_VPL011_SIZE.
> Penny Zheng (4):
> xen/arm: introduce new helper parse_static_mem_prop and
> acquire_static_memory_bank
> xen/arm: introduce direct-map for domUs
> xen/arm: add ASSERT_UNREACHABLE in allocate_static_memory
> xen/arm: gate make_gicv3_domU_node with CONFIG_GICV3
>
> Stefano Stabellini (7):
> xen: introduce internal CDF_xxx flags for domain creation
> xen: introduce CDF_directmap
> xen/arm: avoid setting XEN_DOMCTL_CDF_iommu when IOMMU off
> xen/arm: if direct-map domain use native addresses for GICv2
> xen/arm: if direct-map domain use native addresses for GICv3
> xen/arm: if direct-map domain use native UART address and IRQ number
> for vPL011
> xen/docs: Document how to do passthrough without IOMMU
>
> docs/misc/arm/device-tree/booting.txt | 6 +
> docs/misc/arm/passthrough-noiommu.txt | 52 +++++
> xen/arch/arm/domain.c | 5 +-
> xen/arch/arm/domain_build.c | 308 +++++++++++++++++++++-----
> xen/arch/arm/include/asm/domain.h | 19 +-
> xen/arch/arm/include/asm/new_vgic.h | 10 +
> xen/arch/arm/include/asm/vgic.h | 11 +
> xen/arch/arm/include/asm/vpl011.h | 2 +
> xen/arch/arm/vgic-v2.c | 34 ++-
> xen/arch/arm/vgic-v3.c | 26 ++-
> xen/arch/arm/vgic/vgic-v2.c | 34 ++-
> xen/arch/arm/vpl011.c | 60 ++++-
> xen/arch/x86/domain.c | 3 +-
> xen/arch/x86/setup.c | 2 +-
> xen/common/domain.c | 12 +-
> xen/common/sched/core.c | 2 +-
> xen/include/xen/domain.h | 9 +-
> xen/include/xen/sched.h | 2 +-
> 18 files changed, 490 insertions(+), 107 deletions(-)
> create mode 100644 docs/misc/arm/passthrough-noiommu.txt
>
> --
> 2.25.1
>