From nobody Fri May 3 15:44:47 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510622291967740.4592289779555; Mon, 13 Nov 2017 17:18:11 -0800 (PST) Received: from localhost ([::1]:57145 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEPrU-0004Hn-Tf for importer@patchew.org; Mon, 13 Nov 2017 20:18:00 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35198) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEPpt-0002XC-2t for qemu-devel@nongnu.org; Mon, 13 Nov 2017 20:16:22 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEPpo-0003ck-Po for qemu-devel@nongnu.org; Mon, 13 Nov 2017 20:16:21 -0500 Received: from szxga04-in.huawei.com ([45.249.212.190]:2350) by eggs.gnu.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.71) (envelope-from ) id 1eEPpn-0003Xd-RP; Mon, 13 Nov 2017 20:16:16 -0500 Received: from 172.30.72.60 (EHLO DGGEMS407-HUB.china.huawei.com) ([172.30.72.60]) by dggrg04-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id DKT86839; Tue, 14 Nov 2017 09:16:06 +0800 (CST) Received: from localhost (10.177.27.25) by DGGEMS407-HUB.china.huawei.com (10.3.19.207) with Microsoft SMTP Server id 14.3.361.1; Tue, 14 Nov 2017 09:15:56 +0800 From: To: , , Date: Tue, 14 Nov 2017 09:15:50 +0800 Message-ID: <1510622154-17224-2-git-send-email-zhuyijun@huawei.com> X-Mailer: git-send-email 1.8.4.msysgit.0 In-Reply-To: <1510622154-17224-1-git-send-email-zhuyijun@huawei.com> References: <1510622154-17224-1-git-send-email-zhuyijun@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.27.25] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020205.5A0A43D6.0134, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 4b8fec13b3fdd04b975ecdebe6c4a9bd X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] [fuzzy] X-Received-From: 45.249.212.190 Subject: [Qemu-devel] [RFC 1/5] hw/vfio: Add function for getting reserved_region of device iommu group X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, shameerali.kolothum.thodi@huawei.com, zhaoshenglong@huawei.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Zhu Yijun With kernel 4.11, iommu/smmu will populate the MSI IOVA reserved window and PCI reserved window which has to be excluded from Guest iova allocations. However, If it falls within the Qemu default virtual memory address space, then reserved regions may get allocated for a Guest VF DMA iova and it will fail. So get those reserved regions in this patch and create some holes in the Qe= mu ram address in next patchset. Signed-off-by: Zhu Yijun --- hw/vfio/common.c | 67 +++++++++++++++++++++++++++++++++++++++= ++++ hw/vfio/pci.c | 2 ++ hw/vfio/platform.c | 2 ++ include/exec/memory.h | 7 +++++ include/hw/vfio/vfio-common.h | 3 ++ 5 files changed, 81 insertions(+) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 7b2924c..01bdbbd 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -40,6 +40,8 @@ struct vfio_group_head vfio_group_list =3D QLIST_HEAD_INITIALIZER(vfio_group_list); struct vfio_as_head vfio_address_spaces =3D QLIST_HEAD_INITIALIZER(vfio_address_spaces); +struct reserved_ram_head reserved_ram_regions =3D + QLIST_HEAD_INITIALIZER(reserved_ram_regions); =20 #ifdef CONFIG_KVM /* @@ -52,6 +54,71 @@ struct vfio_as_head vfio_address_spaces =3D static int vfio_kvm_device_fd =3D -1; #endif =20 +void update_reserved_regions(hwaddr addr, hwaddr size) +{ + RAMRegion *reg, *new; + + new =3D g_new(RAMRegion, 1); + new->base =3D addr; + new->size =3D size; + + if (QLIST_EMPTY(&reserved_ram_regions)) { + QLIST_INSERT_HEAD(&reserved_ram_regions, new, next); + return; + } + + /* make the base of reserved_ram_regions increase */ + QLIST_FOREACH(reg, &reserved_ram_regions, next) { + if (addr > (reg->base + reg->size - 1)) { + if (!QLIST_NEXT(reg, next)) { + QLIST_INSERT_AFTER(reg, new, next); + break; + } + continue; + } else if (addr >=3D reg->base && addr <=3D (reg->base + reg->size= - 1)) { + /* discard the duplicate entry */ + if (addr =3D=3D reg->base && size =3D=3D reg->size) { + g_free(new); + break; + } else { + QLIST_INSERT_AFTER(reg, new, next); + break; + } + } else { + QLIST_INSERT_BEFORE(reg, new, next); + break; + } + } +} + +void vfio_get_iommu_group_reserved_region(char *group_path) +{ + char *filename; + FILE *fp; + hwaddr start, end; + char str[10]; + struct stat st; + + filename =3D g_strdup_printf("%s/iommu_group/reserved_regions", group_= path); + if (stat(filename, &st) < 0) { + g_free(filename); + return; + } + + fp =3D fopen(filename, "r"); + if (!fp) { + g_free(filename); + return; + } + + while (fscanf(fp, "%"PRIx64" %"PRIx64" %s", &start, &end, str) !=3D EO= F) { + update_reserved_regions(start, (end - start + 1)); + } + + g_free(filename); + fclose(fp); +} + /* * Common VFIO interrupt disable */ diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index c977ee3..9bffb38 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -2674,6 +2674,8 @@ static void vfio_realize(PCIDevice *pdev, Error **err= p) vdev->vbasedev.type =3D VFIO_DEVICE_TYPE_PCI; vdev->vbasedev.dev =3D &vdev->pdev.qdev; =20 + vfio_get_iommu_group_reserved_region(vdev->vbasedev.sysfsdev); + tmp =3D g_strdup_printf("%s/iommu_group", vdev->vbasedev.sysfsdev); len =3D readlink(tmp, group_path, sizeof(group_path)); g_free(tmp); diff --git a/hw/vfio/platform.c b/hw/vfio/platform.c index da84abf..d5bbc3a 100644 --- a/hw/vfio/platform.c +++ b/hw/vfio/platform.c @@ -578,6 +578,8 @@ static int vfio_base_device_init(VFIODevice *vbasedev, = Error **errp) return -errno; } =20 + vfio_get_iommu_group_reserved_region(vbasedev->sysfsdev); + tmp =3D g_strdup_printf("%s/iommu_group", vbasedev->sysfsdev); len =3D readlink(tmp, group_path, sizeof(group_path)); g_free(tmp); diff --git a/include/exec/memory.h b/include/exec/memory.h index 5ed4042..2bcc83b 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -46,6 +46,13 @@ OBJECT_GET_CLASS(IOMMUMemoryRegionClass, (obj), \ TYPE_IOMMU_MEMORY_REGION) =20 +/* Scattered RAM memory region struct */ +typedef struct RAMRegion { + hwaddr base; + hwaddr size; + QLIST_ENTRY(RAMRegion) next; +} RAMRegion; + typedef struct MemoryRegionOps MemoryRegionOps; typedef struct MemoryRegionMmio MemoryRegionMmio; =20 diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index f3a2ac9..3483ca6 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -161,10 +161,13 @@ VFIOGroup *vfio_get_group(int groupid, AddressSpace *= as, Error **errp); void vfio_put_group(VFIOGroup *group); int vfio_get_device(VFIOGroup *group, const char *name, VFIODevice *vbasedev, Error **errp); +void update_reserved_regions(hwaddr addr, hwaddr size); +void vfio_get_iommu_group_reserved_region(char *group_path); =20 extern const MemoryRegionOps vfio_region_ops; extern QLIST_HEAD(vfio_group_head, VFIOGroup) vfio_group_list; extern QLIST_HEAD(vfio_as_head, VFIOAddressSpace) vfio_address_spaces; +extern QLIST_HEAD(reserved_ram_head, RAMRegion) reserved_ram_regions; =20 #ifdef CONFIG_LINUX int vfio_get_region_info(VFIODevice *vbasedev, int index, --=20 1.8.3.1 From nobody Fri May 3 15:44:47 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510622631312175.05826045360402; Mon, 13 Nov 2017 17:23:51 -0800 (PST) Received: from localhost ([::1]:57175 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEPwt-0000Su-8V for importer@patchew.org; Mon, 13 Nov 2017 20:23:35 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35690) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEPqe-0003yp-5G for qemu-devel@nongnu.org; Mon, 13 Nov 2017 20:17:09 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEPqa-0004A8-V0 for qemu-devel@nongnu.org; Mon, 13 Nov 2017 20:17:08 -0500 Received: from szxga04-in.huawei.com ([45.249.212.190]:2351) by eggs.gnu.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.71) (envelope-from ) id 1eEPqa-00045h-Cc; Mon, 13 Nov 2017 20:17:04 -0500 Received: from 172.30.72.60 (EHLO DGGEMS411-HUB.china.huawei.com) ([172.30.72.60]) by dggrg04-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id DKT86937; Tue, 14 Nov 2017 09:16:54 +0800 (CST) Received: from localhost (10.177.27.25) by DGGEMS411-HUB.china.huawei.com (10.3.19.211) with Microsoft SMTP Server id 14.3.361.1; Tue, 14 Nov 2017 09:15:57 +0800 From: To: , , Date: Tue, 14 Nov 2017 09:15:51 +0800 Message-ID: <1510622154-17224-3-git-send-email-zhuyijun@huawei.com> X-Mailer: git-send-email 1.8.4.msysgit.0 In-Reply-To: <1510622154-17224-1-git-send-email-zhuyijun@huawei.com> References: <1510622154-17224-1-git-send-email-zhuyijun@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.27.25] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020206.5A0A4406.0090, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 6bdc2bd85fb1e5fbaf4796237eb2b147 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] [fuzzy] X-Received-From: 45.249.212.190 Subject: [Qemu-devel] [RFC 2/5] hw/arm/virt: Enable dynamic generation of guest RAM memory regions X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, shameerali.kolothum.thodi@huawei.com, zhaoshenglong@huawei.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Zhu Yijun Register a ram_memory_region_init notify to allocate memory region from system memory and add them as subregions. Signed-off-by: Zhu Yijun --- hw/arm/virt.c | 21 ++++++++++++++++----- include/hw/arm/virt.h | 1 + 2 files changed, 17 insertions(+), 5 deletions(-) diff --git a/hw/arm/virt.c b/hw/arm/virt.c index 9e18b41..ddde5e1 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -1225,6 +1225,19 @@ void virt_machine_done(Notifier *notifier, void *dat= a) virt_build_smbios(vms); } =20 +static void virt_ram_memory_region_init(Notifier *notifier, void *data) +{ + MachineState *machine =3D MACHINE(qdev_get_machine()); + MemoryRegion *sysmem =3D get_system_memory(); + MemoryRegion *ram =3D g_new(MemoryRegion, 1); + VirtMachineState *vms =3D container_of(notifier, VirtMachineState, + ram_memory_region_init); + + memory_region_allocate_system_memory(ram, NULL, "mach-virt.ram", + machine->ram_size); + memory_region_add_subregion(sysmem, vms->memmap[VIRT_MEM].base, ram); +} + static uint64_t virt_cpu_mp_affinity(VirtMachineState *vms, int idx) { uint8_t clustersz =3D ARM_DEFAULT_CPUS_PER_CLUSTER; @@ -1258,7 +1271,6 @@ static void machvirt_init(MachineState *machine) MemoryRegion *sysmem =3D get_system_memory(); MemoryRegion *secure_sysmem =3D NULL; int n, virt_max_cpus; - MemoryRegion *ram =3D g_new(MemoryRegion, 1); bool firmware_loaded =3D bios_name || drive_get(IF_PFLASH, 0, 0); =20 /* We can probe only here because during property set @@ -1409,10 +1421,6 @@ static void machvirt_init(MachineState *machine) fdt_add_cpu_nodes(vms); fdt_add_psci_node(vms); =20 - memory_region_allocate_system_memory(ram, NULL, "mach-virt.ram", - machine->ram_size); - memory_region_add_subregion(sysmem, vms->memmap[VIRT_MEM].base, ram); - create_flash(vms, sysmem, secure_sysmem ? secure_sysmem : sysmem); =20 create_gic(vms, pic); @@ -1462,6 +1470,9 @@ static void machvirt_init(MachineState *machine) * Notifiers are executed in registration reverse order. */ create_platform_bus(vms, pic); + + vms->ram_memory_region_init.notify =3D virt_ram_memory_region_init; + qemu_add_machine_init_done_notifier(&vms->ram_memory_region_init); } =20 static bool virt_get_secure(Object *obj, Error **errp) diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h index 33b0ff3..38906b8 100644 --- a/include/hw/arm/virt.h +++ b/include/hw/arm/virt.h @@ -90,6 +90,7 @@ typedef struct { typedef struct { MachineState parent; Notifier machine_done; + Notifier ram_memory_region_init; FWCfgState *fw_cfg; bool secure; bool highmem; --=20 1.8.3.1 From nobody Fri May 3 15:44:47 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510622407151468.6210909588349; Mon, 13 Nov 2017 17:20:07 -0800 (PST) Received: from localhost ([::1]:57154 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEPtU-0005kh-9c for importer@patchew.org; Mon, 13 Nov 2017 20:20:04 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35196) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEPps-0002XA-Tx for qemu-devel@nongnu.org; Mon, 13 Nov 2017 20:16:22 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEPpo-0003cp-Pu for qemu-devel@nongnu.org; Mon, 13 Nov 2017 20:16:20 -0500 Received: from szxga04-in.huawei.com ([45.249.212.190]:2349) by eggs.gnu.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.71) (envelope-from ) id 1eEPpn-0003XX-Rc; Mon, 13 Nov 2017 20:16:16 -0500 Received: from 172.30.72.59 (EHLO DGGEMS409-HUB.china.huawei.com) ([172.30.72.59]) by dggrg04-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id DKT86830; Tue, 14 Nov 2017 09:16:04 +0800 (CST) Received: from localhost (10.177.27.25) by DGGEMS409-HUB.china.huawei.com (10.3.19.209) with Microsoft SMTP Server id 14.3.361.1; Tue, 14 Nov 2017 09:15:57 +0800 From: To: , , Date: Tue, 14 Nov 2017 09:15:52 +0800 Message-ID: <1510622154-17224-4-git-send-email-zhuyijun@huawei.com> X-Mailer: git-send-email 1.8.4.msysgit.0 In-Reply-To: <1510622154-17224-1-git-send-email-zhuyijun@huawei.com> References: <1510622154-17224-1-git-send-email-zhuyijun@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.27.25] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020205.5A0A43D5.00D9, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 8197e73b3ca87d9a208b7667f2e25e04 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] [fuzzy] X-Received-From: 45.249.212.190 Subject: [Qemu-devel] [RFC 3/5] hw/arm: add scattered RAM memory region support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, shameerali.kolothum.thodi@huawei.com, zhaoshenglong@huawei.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Zhu Yijun Dig out reserved memory holes and collect scattered RAM memory regions by adding mem_list member in arm_boot_info struct. Signed-off-by: Zhu Yijun --- hw/arm/boot.c | 8 ++++ hw/arm/virt.c | 101 +++++++++++++++++++++++++++++++++++++++++++++++= +++- include/hw/arm/arm.h | 1 + 3 files changed, 108 insertions(+), 2 deletions(-) diff --git a/hw/arm/boot.c b/hw/arm/boot.c index c2720c8..30438f4 100644 --- a/hw/arm/boot.c +++ b/hw/arm/boot.c @@ -836,6 +836,14 @@ static void arm_load_kernel_notify(Notifier *notifier,= void *data) */ assert(!(info->secure_board_setup && kvm_enabled())); =20 + /* If machine is not virt, the mem_list will empty. */ + if (QLIST_EMPTY(&vms->bootinfo.mem_list)) { + RAMRegion *new =3D g_new(RAMRegion, 1); + new->base =3D info->loader_start; + new->size =3D info->ram_size; + QLIST_INSERT_HEAD(&vms->bootinfo.mem_list, new, next); + } + info->dtb_filename =3D qemu_opt_get(qemu_get_machine_opts(), "dtb"); =20 /* Load the kernel. */ diff --git a/hw/arm/virt.c b/hw/arm/virt.c index ddde5e1..ff334c1 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -56,6 +56,7 @@ #include "hw/smbios/smbios.h" #include "qapi/visitor.h" #include "standard-headers/linux/input.h" +#include "hw/vfio/vfio-common.h" =20 #define DEFINE_VIRT_MACHINE_LATEST(major, minor, latest) \ static void virt_##major##_##minor##_class_init(ObjectClass *oc, \ @@ -1225,6 +1226,98 @@ void virt_machine_done(Notifier *notifier, void *dat= a) virt_build_smbios(vms); } =20 +static void handle_reserved_ram_region_overlap(void) +{ + hwaddr cur_end, next_end; + RAMRegion *reg, *next_reg, *tmp_reg; + + QLIST_FOREACH(reg, &reserved_ram_regions, next) { + next_reg =3D QLIST_NEXT(reg, next); + + while (next_reg && next_reg->base <=3D (reg->base + reg->size)) { + next_end =3D next_reg->base + next_reg->size; + cur_end =3D reg->base + reg->size; + if (next_end > cur_end) { + reg->size +=3D (next_end - cur_end); + } + + tmp_reg =3D QLIST_NEXT(next_reg, next); + QLIST_REMOVE(next_reg, next); + g_free(next_reg); + next_reg =3D tmp_reg; + } + } +} + +static void update_memory_regions(VirtMachineState *vms, hwaddr ram_size) +{ + + RAMRegion *new, *reg, *last =3D NULL; + hwaddr virt_start, virt_end; + virt_start =3D vms->memmap[VIRT_MEM].base; + virt_end =3D virt_start + ram_size - 1; + + handle_reserved_ram_region_overlap(); + + QLIST_FOREACH(reg, &reserved_ram_regions, next) { + if (reg->base >=3D virt_start && reg->base < virt_end) { + if (reg->base =3D=3D virt_start) { + virt_start +=3D reg->size; + virt_end +=3D reg->size; + continue; + } else { + new =3D g_new(RAMRegion, 1); + new->base =3D virt_start; + new->size =3D reg->base - virt_start; + virt_start =3D reg->base + reg->size; + } + + if (QLIST_EMPTY(&vms->bootinfo.mem_list)) { + QLIST_INSERT_HEAD(&vms->bootinfo.mem_list, new, next); + } else { + QLIST_INSERT_AFTER(last, new, next); + } + + last =3D new; + ram_size -=3D new->size; + virt_end +=3D reg->size; + } + } + + if (ram_size > 0) { + new =3D g_new(RAMRegion, 1); + new->base =3D virt_start; + new->size =3D ram_size; + + if (QLIST_EMPTY(&vms->bootinfo.mem_list)) { + QLIST_INSERT_HEAD(&vms->bootinfo.mem_list, new, next); + } else { + QLIST_INSERT_AFTER(last, new, next); + } + } +} + +static void create_ram_alias(VirtMachineState *vms, + MemoryRegion *sysmem, + MemoryRegion *ram) +{ + RAMRegion *reg; + MemoryRegion *ram_memory; + char *nodename; + hwaddr sz =3D 0; + + QLIST_FOREACH(reg, &vms->bootinfo.mem_list, next) { + nodename =3D g_strdup_printf("ram@%" PRIx64, reg->base); + ram_memory =3D g_new(MemoryRegion, 1); + memory_region_init_alias(ram_memory, NULL, nodename, ram, sz, + reg->size); + memory_region_add_subregion(sysmem, reg->base, ram_memory); + sz +=3D reg->size; + + g_free(nodename); + } +} + static void virt_ram_memory_region_init(Notifier *notifier, void *data) { MachineState *machine =3D MACHINE(qdev_get_machine()); @@ -1232,10 +1325,15 @@ static void virt_ram_memory_region_init(Notifier *n= otifier, void *data) MemoryRegion *ram =3D g_new(MemoryRegion, 1); VirtMachineState *vms =3D container_of(notifier, VirtMachineState, ram_memory_region_init); + RAMRegion *first_mem_reg; =20 memory_region_allocate_system_memory(ram, NULL, "mach-virt.ram", machine->ram_size); - memory_region_add_subregion(sysmem, vms->memmap[VIRT_MEM].base, ram); + update_memory_regions(vms, machine->ram_size); + create_ram_alias(vms, sysmem, ram); + + first_mem_reg =3D QLIST_FIRST(&vms->bootinfo.mem_list); + vms->bootinfo.loader_start =3D first_mem_reg->base; } =20 static uint64_t virt_cpu_mp_affinity(VirtMachineState *vms, int idx) @@ -1458,7 +1556,6 @@ static void machvirt_init(MachineState *machine) vms->bootinfo.initrd_filename =3D machine->initrd_filename; vms->bootinfo.nb_cpus =3D smp_cpus; vms->bootinfo.board_id =3D -1; - vms->bootinfo.loader_start =3D vms->memmap[VIRT_MEM].base; vms->bootinfo.get_dtb =3D machvirt_dtb; vms->bootinfo.firmware_loaded =3D firmware_loaded; arm_load_kernel(ARM_CPU(first_cpu), &vms->bootinfo); diff --git a/include/hw/arm/arm.h b/include/hw/arm/arm.h index ce769bd..d953026 100644 --- a/include/hw/arm/arm.h +++ b/include/hw/arm/arm.h @@ -124,6 +124,7 @@ struct arm_boot_info { bool secure_board_setup; =20 arm_endianness endianness; + QLIST_HEAD(, RAMRegion) mem_list; }; =20 /** --=20 1.8.3.1 From nobody Fri May 3 15:44:47 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510622319151768.0372989253011; Mon, 13 Nov 2017 17:18:39 -0800 (PST) Received: from localhost ([::1]:57146 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEPro-0004X4-9N for importer@patchew.org; Mon, 13 Nov 2017 20:18:20 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35427) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEPqD-0002os-OH for qemu-devel@nongnu.org; Mon, 13 Nov 2017 20:16:43 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEPqC-0003ql-Ck for qemu-devel@nongnu.org; Mon, 13 Nov 2017 20:16:41 -0500 Received: from szxga05-in.huawei.com ([45.249.212.191]:2291) by eggs.gnu.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.71) (envelope-from ) id 1eEPqB-0003pK-GF; Mon, 13 Nov 2017 20:16:40 -0500 Received: from 172.30.72.58 (EHLO DGGEMS405-HUB.china.huawei.com) ([172.30.72.58]) by dggrg05-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id DKW85566; Tue, 14 Nov 2017 09:16:35 +0800 (CST) Received: from localhost (10.177.27.25) by DGGEMS405-HUB.china.huawei.com (10.3.19.205) with Microsoft SMTP Server id 14.3.361.1; Tue, 14 Nov 2017 09:15:58 +0800 From: To: , , Date: Tue, 14 Nov 2017 09:15:53 +0800 Message-ID: <1510622154-17224-5-git-send-email-zhuyijun@huawei.com> X-Mailer: git-send-email 1.8.4.msysgit.0 In-Reply-To: <1510622154-17224-1-git-send-email-zhuyijun@huawei.com> References: <1510622154-17224-1-git-send-email-zhuyijun@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.27.25] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020201.5A0A43F3.0052, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 165b4da8128d720285897c371ffa0918 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] [fuzzy] X-Received-From: 45.249.212.191 Subject: [Qemu-devel] [RFC 4/5] hw/arm/boot: set fdt size cell of memory node from mem_list X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, shameerali.kolothum.thodi@huawei.com, zhaoshenglong@huawei.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Zhu Yijun In case of accessing memory holes, we create memory fdt node by traversing mem_list and adjust memory address if the addr is not belong to RAM memory region. Signed-off-by: Zhu Yijun --- hw/arm/boot.c | 151 ++++++++++++++++++++++++++++++++++++++++++------------= ---- 1 file changed, 109 insertions(+), 42 deletions(-) diff --git a/hw/arm/boot.c b/hw/arm/boot.c index 30438f4..f4ca599 100644 --- a/hw/arm/boot.c +++ b/hw/arm/boot.c @@ -384,6 +384,25 @@ static void set_kernel_args_old(const struct arm_boot_= info *info) } } =20 +static char *create_memory_fdt(void *fdt, uint32_t acells, hwaddr mem_base, + uint32_t scells, hwaddr mem_len) +{ + char *nodename =3D NULL; + int rc; + + nodename =3D g_strdup_printf("/memory@%" PRIx64, mem_base); + qemu_fdt_add_subnode(fdt, nodename); + qemu_fdt_setprop_string(fdt, nodename, "device_type", "memory"); + rc =3D qemu_fdt_setprop_sized_cells(fdt, nodename, "reg", acells, mem_= base, + scells, mem_len); + if (rc < 0) { + fprintf(stderr, "couldn't set %s/reg\n", nodename); + g_free(nodename); + } + + return nodename; +} + /** * load_dtb() - load a device tree binary image into memory * @addr: the address to load the image at @@ -412,7 +431,7 @@ static int load_dtb(hwaddr addr, const struct arm_boot_= info *binfo, uint32_t acells, scells; char *nodename; unsigned int i; - hwaddr mem_base, mem_len; + RAMRegion *reg; =20 if (binfo->dtb_filename) { char *filename; @@ -464,49 +483,65 @@ static int load_dtb(hwaddr addr, const struct arm_boo= t_info *binfo, goto fail; } =20 + /* + * Turn the /memory node created before into a NOP node, then create + * /memory@addr nodes for all numa nodes respectively. + */ + qemu_fdt_nop_node(fdt, "/memory"); + if (nb_numa_nodes > 0) { - /* - * Turn the /memory node created before into a NOP node, then crea= te - * /memory@addr nodes for all numa nodes respectively. - */ - qemu_fdt_nop_node(fdt, "/memory"); - mem_base =3D binfo->loader_start; + hwaddr reg_offset =3D 0; /* region base addr offset */ + hwaddr reg_alloc_size =3D 0; /* region consumed size */ + hwaddr node_mem_size =3D 0; + RAMRegion *tmp_reg =3D QLIST_FIRST(&binfo->mem_list); + for (i =3D 0; i < nb_numa_nodes; i++) { - mem_len =3D numa_info[i].node_mem; - nodename =3D g_strdup_printf("/memory@%" PRIx64, mem_base); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_string(fdt, nodename, "device_type", "memory"= ); - rc =3D qemu_fdt_setprop_sized_cells(fdt, nodename, "reg", - acells, mem_base, - scells, mem_len); - if (rc < 0) { - fprintf(stderr, "couldn't set %s/reg for node %d\n", noden= ame, - i); - goto fail; - } + node_mem_size =3D numa_info[i].node_mem; + QLIST_FOREACH(reg, &binfo->mem_list, next) { + if (reg->base !=3D tmp_reg->base) { + continue; + } =20 - qemu_fdt_setprop_cell(fdt, nodename, "numa-node-id", i); - mem_base +=3D mem_len; - g_free(nodename); - } - } else { - Error *err =3D NULL; + if (node_mem_size >=3D (reg->size - reg_offset)) { + reg_alloc_size =3D reg->size - reg_offset; + } else { + reg_alloc_size =3D node_mem_size; + } =20 - rc =3D fdt_path_offset(fdt, "/memory"); - if (rc < 0) { - qemu_fdt_add_subnode(fdt, "/memory"); - } + nodename =3D create_memory_fdt(fdt, acells, + reg->base + reg_offset, + scells, reg_alloc_size); + if (!nodename) { + goto fail; + } + + qemu_fdt_setprop_cell(fdt, nodename, "numa-node-id", i); + g_free(nodename); =20 - if (!qemu_fdt_getprop(fdt, "/memory", "device_type", NULL, &err)) { - qemu_fdt_setprop_string(fdt, "/memory", "device_type", "memory= "); + node_mem_size -=3D reg_alloc_size; + reg_offset +=3D reg_alloc_size; + tmp_reg =3D reg; + + /* The region is depleted */ + if (reg->size =3D=3D reg_offset) { + reg_offset =3D 0; + tmp_reg =3D QLIST_NEXT(reg, next); + } + + if (node_mem_size =3D=3D 0) { + break; + } + } } + } else { + QLIST_FOREACH(reg, &binfo->mem_list, next) { + nodename =3D create_memory_fdt(fdt, acells, reg->base, + scells, reg->size); + if (!nodename) { + goto fail; + } =20 - rc =3D qemu_fdt_setprop_sized_cells(fdt, "/memory", "reg", - acells, binfo->loader_start, - scells, binfo->ram_size); - if (rc < 0) { - fprintf(stderr, "couldn't set /memory/reg\n"); - goto fail; + g_free(nodename); } } =20 @@ -814,6 +849,35 @@ static uint64_t load_aarch64_image(const char *filenam= e, hwaddr mem_base, return size; } =20 +/* Find the max size memory region after info->initrd_start. + * TODO: we may have a corner case where the memory node size may not corr= ectly + * fit into kernel/initrd/dtb Image sizes. + * */ +static RAMRegion *find_initrd_memregion(const struct arm_boot_info *info, + hwaddr *initrd_start) +{ + RAMRegion *reg, *initrd_reg =3D NULL; + hwaddr max_size =3D 0; + hwaddr addr =3D *initrd_start; + + QLIST_FOREACH(reg, &info->mem_list, next) { + if (addr < reg->base) { + if (max_size < reg->size) { + max_size =3D reg->size; + *initrd_start =3D reg->base; + initrd_reg =3D reg; + } + } else if (addr >=3D reg->base && addr < (reg->base + reg->size)) { + if (max_size < reg->base + reg->size - addr) { + max_size =3D reg->base + reg->size - addr; + initrd_reg =3D reg; + } + } + } + + return initrd_reg; +} + static void arm_load_kernel_notify(Notifier *notifier, void *data) { CPUState *cs; @@ -837,11 +901,11 @@ static void arm_load_kernel_notify(Notifier *notifier= , void *data) assert(!(info->secure_board_setup && kvm_enabled())); =20 /* If machine is not virt, the mem_list will empty. */ - if (QLIST_EMPTY(&vms->bootinfo.mem_list)) { + if (QLIST_EMPTY(&info->mem_list)) { RAMRegion *new =3D g_new(RAMRegion, 1); new->base =3D info->loader_start; new->size =3D info->ram_size; - QLIST_INSERT_HEAD(&vms->bootinfo.mem_list, new, next); + QLIST_INSERT_HEAD(&info->mem_list, new, next); } =20 info->dtb_filename =3D qemu_opt_get(qemu_get_machine_opts(), "dtb"); @@ -973,14 +1037,16 @@ static void arm_load_kernel_notify(Notifier *notifie= r, void *data) uint32_t fixupcontext[FIXUP_MAX]; =20 if (info->initrd_filename) { + RAMRegion *reg =3D find_initrd_memregion(info, &info->initrd_s= tart); + initrd_size =3D load_ramdisk(info->initrd_filename, info->initrd_start, - info->ram_size - + reg->base + reg->size - info->initrd_start); if (initrd_size < 0) { initrd_size =3D load_image_targphys(info->initrd_filename, info->initrd_start, - info->ram_size - + reg->base + reg->size - info->initrd_start); } if (initrd_size < 0) { @@ -1027,7 +1093,8 @@ static void arm_load_kernel_notify(Notifier *notifier= , void *data) } fixupcontext[FIXUP_ARGPTR] =3D dtb_start; } else { - fixupcontext[FIXUP_ARGPTR] =3D info->loader_start + KERNEL_ARG= S_ADDR; + hwaddr kernel_args_addr =3D info->loader_start + KERNEL_ARGS_A= DDR; + fixupcontext[FIXUP_ARGPTR] =3D kernel_args_addr; if (info->ram_size >=3D (1ULL << 32)) { fprintf(stderr, "qemu: RAM size must be less than 4GB to b= oot" " Linux kernel using ATAGS (try passing a device t= ree" --=20 1.8.3.1 From nobody Fri May 3 15:44:47 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510622534746316.82026821483146; Mon, 13 Nov 2017 17:22:14 -0800 (PST) Received: from localhost ([::1]:57169 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEPvX-0007SH-UZ for importer@patchew.org; Mon, 13 Nov 2017 20:22:11 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35413) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEPqC-0002ly-OM for qemu-devel@nongnu.org; Mon, 13 Nov 2017 20:16:41 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEPq9-0003pa-Ga for qemu-devel@nongnu.org; Mon, 13 Nov 2017 20:16:40 -0500 Received: from szxga05-in.huawei.com ([45.249.212.191]:2290) by eggs.gnu.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.71) (envelope-from ) id 1eEPq8-0003nz-TK; Mon, 13 Nov 2017 20:16:37 -0500 Received: from 172.30.72.60 (EHLO DGGEMS412-HUB.china.huawei.com) ([172.30.72.60]) by dggrg05-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id DKW85549; Tue, 14 Nov 2017 09:16:31 +0800 (CST) Received: from localhost (10.177.27.25) by DGGEMS412-HUB.china.huawei.com (10.3.19.212) with Microsoft SMTP Server id 14.3.361.1; Tue, 14 Nov 2017 09:15:59 +0800 From: To: , , Date: Tue, 14 Nov 2017 09:15:54 +0800 Message-ID: <1510622154-17224-6-git-send-email-zhuyijun@huawei.com> X-Mailer: git-send-email 1.8.4.msysgit.0 In-Reply-To: <1510622154-17224-1-git-send-email-zhuyijun@huawei.com> References: <1510622154-17224-1-git-send-email-zhuyijun@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.27.25] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020204.5A0A43EF.00F2, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: fb5d816960baf9cd40ade15de84c2a3e X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] [fuzzy] X-Received-From: 45.249.212.191 Subject: [Qemu-devel] [RFC 5/5] hw/arm/virt-acpi-build: Build srat table according to mem_list X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, shameerali.kolothum.thodi@huawei.com, zhaoshenglong@huawei.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Zhu Yijun In case of accessing memory holes, we build srat table by traversing mem_list. Signed-off-by: Zhu Yijun --- hw/arm/virt-acpi-build.c | 40 +++++++++++++++++++++++++++++++++++----- 1 file changed, 35 insertions(+), 5 deletions(-) diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c index 3d78ff6..e8dd7c9 100644 --- a/hw/arm/virt-acpi-build.c +++ b/hw/arm/virt-acpi-build.c @@ -487,7 +487,12 @@ build_srat(GArray *table_data, BIOSLinker *linker, Vir= tMachineState *vms) AcpiSratProcessorGiccAffinity *core; AcpiSratMemoryAffinity *numamem; int i, srat_start; - uint64_t mem_base; + hwaddr region_offset =3D 0; /* region base addr offset */ + hwaddr region_eat_size =3D 0; /* region consumed size */ + hwaddr node_mem_size =3D 0; + RAMRegion *begin_search_region =3D QLIST_FIRST(&vms->bootinfo.mem_list= ); + RAMRegion *reg; + MachineClass *mc =3D MACHINE_GET_CLASS(vms); const CPUArchIdList *cpu_list =3D mc->possible_cpu_arch_ids(MACHINE(vm= s)); =20 @@ -504,12 +509,37 @@ build_srat(GArray *table_data, BIOSLinker *linker, Vi= rtMachineState *vms) core->flags =3D cpu_to_le32(1); } =20 - mem_base =3D vms->memmap[VIRT_MEM].base; for (i =3D 0; i < nb_numa_nodes; ++i) { numamem =3D acpi_data_push(table_data, sizeof(*numamem)); - build_srat_memory(numamem, mem_base, numa_info[i].node_mem, i, - MEM_AFFINITY_ENABLED); - mem_base +=3D numa_info[i].node_mem; + node_mem_size =3D numa_info[i].node_mem; + QLIST_FOREACH(reg, &vms->bootinfo.mem_list, next) { + if (reg->base !=3D begin_search_region->base) { + continue; + } + + if (node_mem_size >=3D (reg->size - region_offset)) { + region_eat_size =3D reg->size - region_offset; + } else { + region_eat_size =3D node_mem_size; + } + + build_srat_memory(numamem, reg->base + region_offset, + region_eat_size, i, MEM_AFFINITY_ENABLED); + + node_mem_size -=3D region_eat_size; + region_offset +=3D region_eat_size; + begin_search_region =3D reg; + + /* The region is depleted */ + if (reg->size =3D=3D region_offset) { + region_offset =3D 0; + begin_search_region =3D QLIST_NEXT(reg, next); + } + + if (node_mem_size =3D=3D 0) { + break; + } + } } =20 build_header(linker, table_data, (void *)srat, "SRAT", --=20 1.8.3.1