From nobody Sun Dec 22 02:48:51 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1727195082; cv=none; d=zohomail.com; s=zohoarc; b=D6MPfBF5OiQ3HzduuChiqNMfAd16WOAow2jBXPRBlQ0iKpGQy8xpTbVs6mmPyap0S/hLxR1bvGUTWhy9XdJNBz4JxfYnwgCBEpR2VOsf2xL/YS5UAWLsPb/YKbpf/+4hzC7xdRQvwcouSCTV+MFpVubaF7/wuh0dgOuBi4JtmLI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1727195082; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=I1Emyc3BWl2EUF/DtLpoTZ7Li7YQQAezyxPBC0AV7dU=; b=chpm4L3I0qIeHrmiXrhcl358HvVFg7R7fxLJ/hkidsFvppe4lZpdpVFojJ/tZ1n4w4G+DVwV+bu028lnIcU0sbc7GpYNibDRIJ+VDCzJG9tWi3QJcFmwNnRJ4w0aWswDBzjGq09BtNfhhtwILPP/T7spoNTanrtIAVXAVkwzgZ8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1727195082856683.3415155661494; Tue, 24 Sep 2024 09:24:42 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.803100.1213549 (Exim 4.92) (envelope-from ) id 1st8Kz-0006kT-S0; Tue, 24 Sep 2024 16:24:29 +0000 Received: by outflank-mailman (output) from mailman id 803100.1213549; Tue, 24 Sep 2024 16:24:29 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1st8Kz-0006kK-PM; Tue, 24 Sep 2024 16:24:29 +0000 Received: by outflank-mailman (input) for mailman id 803100; Tue, 24 Sep 2024 16:24:28 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1st8Ky-00063x-Ez for xen-devel@lists.xenproject.org; Tue, 24 Sep 2024 16:24:28 +0000 Received: from mail-vs1-xe32.google.com (mail-vs1-xe32.google.com [2607:f8b0:4864:20::e32]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 77c7bc05-7a91-11ef-a0ba-8be0dac302b0; Tue, 24 Sep 2024 18:24:27 +0200 (CEST) Received: by mail-vs1-xe32.google.com with SMTP id ada2fe7eead31-49bd32f6a11so1965814137.1 for ; Tue, 24 Sep 2024 09:24:27 -0700 (PDT) Received: from gmail.com ([168.243.189.171]) by smtp.gmail.com with ESMTPSA id ada2fe7eead31-4a15178e3e2sm1015805137.8.2024.09.24.09.24.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Sep 2024 09:24:24 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 77c7bc05-7a91-11ef-a0ba-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727195066; x=1727799866; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=I1Emyc3BWl2EUF/DtLpoTZ7Li7YQQAezyxPBC0AV7dU=; b=E9iDFSivwfKdX7ZqFywpqxtglFsUl2jMLa2mwnHwC4Q03SIywQkCl9qY69ymVWD8E/ z5/jlLhE5nQ42cHw7TzKNWMDMrq0f8LYSx1MYEsgnkJAb0wBR4R2L9Urc3m91gis/xgf lRL7C1ZmWjxYqj2okSfGMaLjDWxCqYTkZOFcS0FcaBJlfShN98zQZm7sa1Hs27R+y5tH BLmAR8FIBeuZCGgOsqbZOFhcQ2a0DYMo3Dc2txBiWDE95WDmJSSb77FKkyufJMXrJRXr z9MfgOSsxkrH5L9KDz7lUqmAX42QhkyHFgskJPxwfHoyscKoQ7XlrL80DYjvDx98I+42 Xvbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727195066; x=1727799866; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=I1Emyc3BWl2EUF/DtLpoTZ7Li7YQQAezyxPBC0AV7dU=; b=DIAX8ZSxBAxHjw7pI47wz/sgq4LVf0shV//MFXJowsT+ldhmcB0WHoad+UQkfglntl cZxH5SWniSQEYo5lDUPzP0oMY87b7C8uqPx5nZnw+YyhMOnNGkEzn/wqk+UvvnGm4tSD o26OsAFjVz7t5ckdFOe1ki5izT30VwFHq4Afco8FtXXGqQ3oE6lLkUTnxZowPJblp/51 vb+FumbcwulxwLWR+FD9rs3DPIAyubtlXCiuu03X3zSfnXutFbF1oMyh/HTovUakFS3E LicaxpBo1HWkiQ6us7k0/MIq7XXVLKBzJvn6fRlyBP1f1tGp0/JenmYq3lVU0Jv7e8RM 5FIQ== X-Gm-Message-State: AOJu0YxKjtfkNWjFDPsfT6tAEOBapHUuPiGCreRG3PVv9g99/TLi+eSZ RB3SecrfaHKb/s2te7/KyktdEdXxUy/h6XZUK31dSjGNfCRH7m3Mqk5f8zyo X-Google-Smtp-Source: AGHT+IF0kSdttzBy8iaGxdALf2ISdaugQF0Xv1SVfUBNBqBSaPhgtSeV1mmH2MkSc0jggbXr4XzpRw== X-Received: by 2002:a05:6102:c52:b0:493:3cde:e46 with SMTP id ada2fe7eead31-4a15dc8f173mr90496137.13.1727195065934; Tue, 24 Sep 2024 09:24:25 -0700 (PDT) From: "Edgar E. Iglesias" To: xen-devel@lists.xenproject.org Cc: sstabellini@kernel.org, julien@xen.org, bertrand.marquis@arm.com, michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, dpsmith@apertussolutions.com, edgar.iglesias@amd.com, Stewart Hildebrand Subject: [PATCH v1 3/6] xen/arm: create dom0less virtio-pci DT node Date: Tue, 24 Sep 2024 18:23:56 +0200 Message-ID: <20240924162359.1390487-4-edgar.iglesias@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240924162359.1390487-1-edgar.iglesias@gmail.com> References: <20240924162359.1390487-1-edgar.iglesias@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1727195084271116600 Content-Type: text/plain; charset="utf-8" From: Stewart Hildebrand When virtio-pci is specified in the dom0less domU properties, create a virtio-pci node in the guest's device tree. Set up an mmio handler with a register for the guest to poll when the backend has connected and virtio-pci bus is ready to be probed. Grant tables may be used by specifying virtio-pci =3D "grants";. [Edgar: Use GPEX PCI INTX interrupt swizzling (from PCI specs). Make grants iommu-map cover the entire PCI bus. Add virtio-pci-ranges to specify memory-map for direct-mapped guests. Document virtio-pci dom0less fdt bindings.] Signed-off-by: Stewart Hildebrand Signed-off-by: Edgar E. Iglesias --- docs/misc/arm/device-tree/booting.txt | 21 +++ xen/arch/arm/dom0less-build.c | 238 ++++++++++++++++++++++++++ xen/arch/arm/include/asm/kernel.h | 15 ++ 3 files changed, 274 insertions(+) diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-t= ree/booting.txt index 3a04f5c57f..82f3bd7026 100644 --- a/docs/misc/arm/device-tree/booting.txt +++ b/docs/misc/arm/device-tree/booting.txt @@ -276,6 +276,27 @@ with the following properties: passed through. This option is the default if this property is missing and the user does not provide the device partial device tree for the d= omain. =20 +- virtio-pci + + A string property specifying whether virtio-pci is enabled for the + domain and if grant table mappings should be used. If no value is set + this property is treated as a boolean and the same way as if set to + "enabled". + Possible property values are: + + - "enabled" + Virtio-pci is enabled for the domain. + + - "grants" + Virtio-pci is enabled for the domain and an grants IOMMU node will be + generated in the domains device-tree. + +- virtio-pci-ranges + + An optional array of 6 u32 values specifying the 2 cells base addresse= s of + the ECAM, Memory and Prefetchable-Memory regions for virtio-pci. This = is + useful to avoid memory-map collisions when using direct-mapped guests. + Under the "xen,domain" compatible node, one or more sub-nodes are present for the DomU kernel and ramdisk. =20 diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c index 09b65e44ae..dab24fa9e2 100644 --- a/xen/arch/arm/dom0less-build.c +++ b/xen/arch/arm/dom0less-build.c @@ -586,6 +586,189 @@ static int __init domain_handle_dtb_bootmodule(struct= domain *d, return res; } =20 +static int __init make_virtio_pci_domU_node(const struct kernel_info *kinf= o) +{ + void *fdt =3D kinfo->fdt; + /* reg is sized to be used for all the needed properties below */ + __be32 reg[(1 + (GUEST_ROOT_ADDRESS_CELLS * 2) + GUEST_ROOT_SIZE_CELLS) + * 2]; + __be32 irq_map[4 * 4 * 8]; + __be32 *cells; + char buf[22]; /* pcie@ + max 16 char address + '\0' */ + int res; + int devfn, intx_pin; + static const char compat[] =3D "pci-host-ecam-generic"; + static const char reg_names[] =3D "ecam"; + + if ( p2m_ipa_bits <=3D 40 ) { + printk("PA bits %d is too small!\nvirtio-pci is only supported " + "on platforms with PA bits > 40\n", p2m_ipa_bits); + return -EINVAL; + } + + snprintf(buf, sizeof(buf), "pcie@%lx", kinfo->virtio_pci.ecam.base); + dt_dprintk("Create virtio-pci node\n"); + res =3D fdt_begin_node(fdt, buf); + if ( res ) + return res; + + res =3D fdt_property(fdt, "compatible", compat, sizeof(compat)); + if ( res ) + return res; + + res =3D fdt_property_string(fdt, "device_type", "pci"); + if ( res ) + return res; + + /* Create reg property */ + cells =3D ®[0]; + dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_C= ELLS, + kinfo->virtio_pci.ecam.base, GUEST_VIRTIO_PCI_ECAM_= SIZE); + + res =3D fdt_property(fdt, "reg", reg, + (GUEST_ROOT_ADDRESS_CELLS + + GUEST_ROOT_SIZE_CELLS) * sizeof(*reg)); + if ( res ) + return res; + + res =3D fdt_property(fdt, "reg-names", reg_names, sizeof(reg_names)); + if ( res ) + return res; + + /* Create bus-range property */ + cells =3D ®[0]; + dt_set_cell(&cells, 1, 0); + dt_set_cell(&cells, 1, 0xff); + res =3D fdt_property(fdt, "bus-range", reg, 2 * sizeof(*reg)); + if ( res ) + return res; + + res =3D fdt_property_cell(fdt, "#address-cells", 3); + if ( res ) + return res; + + res =3D fdt_property_cell(fdt, "#size-cells", 2); + if ( res ) + return res; + + res =3D fdt_property_string(fdt, "status", "okay"); + if ( res ) + return res; + + /* + * Create ranges property as: + * <(PCI bitfield) (PCI address) (CPU address) (Size)> + */ + cells =3D ®[0]; + dt_set_cell(&cells, 1, GUEST_VIRTIO_PCI_MEM_TYPE); + dt_set_cell(&cells, GUEST_ROOT_ADDRESS_CELLS, kinfo->virtio_pci.mem.ba= se); + dt_set_cell(&cells, GUEST_ROOT_ADDRESS_CELLS, kinfo->virtio_pci.mem.ba= se); + dt_set_cell(&cells, GUEST_ROOT_SIZE_CELLS, GUEST_VIRTIO_PCI_MEM_SIZE); + dt_set_cell(&cells, 1, GUEST_VIRTIO_PCI_PREFETCH_MEM_TYPE); + dt_set_cell(&cells, GUEST_ROOT_ADDRESS_CELLS, kinfo->virtio_pci.pf_mem= .base); + dt_set_cell(&cells, GUEST_ROOT_ADDRESS_CELLS, kinfo->virtio_pci.pf_mem= .base); + dt_set_cell(&cells, GUEST_ROOT_SIZE_CELLS, GUEST_VIRTIO_PCI_PREFETCH_M= EM_SIZE); + res =3D fdt_property(fdt, "ranges", reg, 14 * sizeof(*reg)); + if ( res ) + return res; + + res =3D fdt_property(fdt, "dma-coherent", "", 0); + if ( res ) + return res; + + res =3D fdt_property_cell(fdt, "#interrupt-cells", 1); + if ( res ) + return res; + + /* + * PCI-to-PCI bridge specification + * 9.1: Interrupt routing. Table 9-1 + * + * the PCI Express Base Specification, Revision 2.1 + * 2.2.8.1: INTx interrupt signaling - Rules + * the Implementation Note + * Table 2-20 + */ + cells =3D &irq_map[0]; + for (devfn =3D 0; devfn <=3D 0x18; devfn +=3D 8) { + for (intx_pin =3D 0; intx_pin < 4; intx_pin++) { + int irq =3D GUEST_VIRTIO_PCI_SPI_FIRST - 32; + irq +=3D ((intx_pin + PCI_SLOT(devfn)) % 4); + + dt_set_cell(&cells, 1, devfn << 8); + dt_set_cell(&cells, 1, 0); + dt_set_cell(&cells, 1, 0); + dt_set_cell(&cells, 1, intx_pin + 1); + dt_set_cell(&cells, 1, kinfo->phandle_gic); + /* 3 GIC cells. */ + dt_set_cell(&cells, 1, 0); + dt_set_cell(&cells, 1, irq); + dt_set_cell(&cells, 1, DT_IRQ_TYPE_LEVEL_HIGH); + } + } + + /* Assert we've sized irq_map correctly. */ + BUG_ON(cells - &irq_map[0] !=3D ARRAY_SIZE(irq_map)); + + res =3D fdt_property(fdt, "interrupt-map", irq_map, sizeof(irq_map)); + if ( res ) + return res; + + cells =3D ®[0]; + dt_set_cell(&cells, 1, cpu_to_be16(PCI_DEVFN(3, 0))); + dt_set_cell(&cells, 1, 0x0); + dt_set_cell(&cells, 1, 0x0); + dt_set_cell(&cells, 1, 0x7); + res =3D fdt_property(fdt, "interrupt-map-mask", reg, 4 * sizeof(*reg)); + if ( res ) + return res; + + if ( kinfo->virtio_pci.mode =3D=3D VIRTIO_PCI_GRANTS ) + { + cells =3D ®[0]; + dt_set_cell(&cells, 1, 0x0); + dt_set_cell(&cells, 1, GUEST_PHANDLE_IOMMU); + dt_set_cell(&cells, 1, 0x0); + dt_set_cell(&cells, 1, 0x10000); + res =3D fdt_property(fdt, "iommu-map", reg, 4 * sizeof(*reg)); + if ( res ) + return res; + } + + res =3D fdt_property_cell(fdt, "linux,pci-domain", 1); + if ( res ) + return res; + + res =3D fdt_end_node(fdt); + if ( res ) + return res; + + if ( kinfo->virtio_pci.mode =3D=3D VIRTIO_PCI_GRANTS ) + { + snprintf(buf, sizeof(buf), "xen_iommu"); + + res =3D fdt_begin_node(fdt, buf); + if ( res ) + return res; + + res =3D fdt_property_string(fdt, "compatible", "xen,grant-dma"); + if ( res ) + return res; + + res =3D fdt_property_cell(fdt, "#iommu-cells", 1); + if ( res ) + return res; + + res =3D fdt_property_cell(fdt, "phandle", GUEST_PHANDLE_IOMMU); + if ( res ) + return res; + + res =3D fdt_end_node(fdt); + } + + return res; +} + /* * The max size for DT is 2MB. However, the generated DT is small (not inc= luding * domU passthrough DT nodes whose size we account separately), 4KB are en= ough @@ -693,6 +876,13 @@ static int __init prepare_dtb_domU(struct domain *d, s= truct kernel_info *kinfo) goto err; } =20 + if ( kinfo->virtio_pci.mode ) + { + ret =3D make_virtio_pci_domU_node(kinfo); + if ( ret ) + goto err; + } + ret =3D fdt_end_node(kinfo->fdt); if ( ret < 0 ) goto err; @@ -744,11 +934,24 @@ static int __init alloc_xenstore_evtchn(struct domain= *d) return 0; } =20 +static u64 combine_u64(u32 v[2]) +{ + u64 v64; + + v64 =3D v[0]; + v64 <<=3D 32; + v64 |=3D v[1]; + return v64; +} + static int __init construct_domU(struct domain *d, const struct dt_device_node *node) { struct kernel_info kinfo =3D KERNEL_INFO_INIT; const char *dom0less_enhanced; + const char *virtio_pci; + /* virtio-pci ECAM, MEM, PF-MEM each carrying 2 x Address cells. */ + u32 virtio_pci_ranges[3 * 2]; int rc; u64 mem; u32 p2m_mem_mb; @@ -779,6 +982,41 @@ static int __init construct_domU(struct domain *d, =20 kinfo.vpl011 =3D dt_property_read_bool(node, "vpl011"); =20 + rc =3D dt_property_read_string(node, "virtio-pci", &virtio_pci); + if ( !rc ) + { + if ( !strcmp(virtio_pci, "enabled") ) + kinfo.virtio_pci.mode =3D VIRTIO_PCI; + else if ( !strcmp(virtio_pci, "grants") ) + kinfo.virtio_pci.mode =3D VIRTIO_PCI_GRANTS; + else + { + printk("Invalid \"virtio-pci\" property value (%s)\n", virtio_= pci); + return -EINVAL; + } + } + else if ( rc =3D=3D -ENODATA ) + { + /* Handle missing property value */ + kinfo.virtio_pci.mode =3D dt_property_read_bool(node, "virtio-pci"= ); + } + + if ( kinfo.virtio_pci.mode !=3D VIRTIO_PCI_NONE ) + { + rc =3D dt_property_read_u32_array(node, "virtio-pci-ranges", + virtio_pci_ranges, + ARRAY_SIZE(virtio_pci_ranges)); + if ( rc =3D=3D 0 ) { + kinfo.virtio_pci.ecam.base =3D combine_u64(&virtio_pci_ranges[= 0]); + kinfo.virtio_pci.mem.base =3D combine_u64(&virtio_pci_ranges[2= ]); + kinfo.virtio_pci.pf_mem.base =3D combine_u64(&virtio_pci_range= s[4]); + } else { + kinfo.virtio_pci.ecam.base =3D GUEST_VIRTIO_PCI_ECAM_BASE; + kinfo.virtio_pci.mem.base =3D GUEST_VIRTIO_PCI_MEM_BASE; + kinfo.virtio_pci.pf_mem.base =3D GUEST_VIRTIO_PCI_PREFETCH_MEM= _BASE; + } + } + rc =3D dt_property_read_string(node, "xen,enhanced", &dom0less_enhance= d); if ( rc =3D=3D -EILSEQ || rc =3D=3D -ENODATA || diff --git a/xen/arch/arm/include/asm/kernel.h b/xen/arch/arm/include/asm/k= ernel.h index 7e6e3c82a4..2dab2ac88f 100644 --- a/xen/arch/arm/include/asm/kernel.h +++ b/xen/arch/arm/include/asm/kernel.h @@ -29,6 +29,13 @@ #define DOM0LESS_XENSTORE BIT(1, U) #define DOM0LESS_ENHANCED (DOM0LESS_ENHANCED_NO_XS | DOM0LESS_XENST= ORE) =20 +/* virtio-pci types */ +enum virtio_pci_type { + VIRTIO_PCI_NONE, + VIRTIO_PCI, + VIRTIO_PCI_GRANTS, +}; + struct kernel_info { #ifdef CONFIG_ARM_64 enum domain_type type; @@ -62,6 +69,14 @@ struct kernel_info { /* Enable/Disable PV drivers interfaces */ uint16_t dom0less_feature; =20 + struct { + enum virtio_pci_type mode; + struct { + u64 base; + } ecam, mem, pf_mem; + u32 pci_intx_irq_base; + } virtio_pci; + /* GIC phandle */ uint32_t phandle_gic; =20 --=20 2.43.0