From nobody Fri May 17 03:39:32 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1690768764; cv=none; d=zohomail.com; s=zohoarc; b=ZAamjX+pwU7x2RGHSv/AicnyXd+UV7ICezbR6bnfuFTTCWPoQntoVfpveZK7K98mwGmzjV6pzNdgOmnX4Tzwqgk/fbwNMcTuHgWElv5Qx8SNgr2cMyemnK2yNEoiPFD1dllcYHF/wMpd/az5Po09T1PAZfNE/u3W1DDKDVHcuxY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1690768764; h=Content-Transfer-Encoding:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=c0Cxqmz72nYaiXQBwdNnVf0BlC2egEz3wNcsNlgPysI=; b=n5jOu8H4LeOnnrGTICZ3t1vVxchFnZkbP4qbldDR6WYz3glVM8Gjzx2pAcGTWhcIHQiPSSf/1mTcTw2LY3PCs6EZ8IW0POmn+zlSba6Wc9H2TVTQS3/5mn06tJOjDwPAo0HTNffjsSI4pq9uOTLUCTIBcf8k4N8BU1cyVsoi1hc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1690768764000160.51141012149753; Sun, 30 Jul 2023 18:59:24 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qQIB0-0003ry-4I; Sun, 30 Jul 2023 21:58:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qQIAy-0003iF-AA; Sun, 30 Jul 2023 21:58:24 -0400 Received: from [192.55.52.136] (helo=mgamail.intel.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qQIAv-0007jG-Oz; Sun, 30 Jul 2023 21:58:24 -0400 Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jul 2023 18:51:01 -0700 Received: from wufei-optiplex-7090.sh.intel.com ([10.238.200.247]) by orsmga002.jf.intel.com with ESMTP; 30 Jul 2023 18:50:58 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690768701; x=1722304701; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=lBAI0qFx6IhVQYJzBWvvcX3bLtMyXPAiMtqmarShKro=; b=aB4WQGMN4xy3Ih1dCFpPiYzCWyTZhYqxyOP24tNWWkJ1O3uGVEYlj+Q3 C0bCZZk2YQslbmgyefafFjOw7MlipgBbtVnaXXs9a1Gt9Dv2bH2QVWzpo 6sJKuoqqgzd002cStGpbWtniMyvFP1JQBJUUQQB7lnau+VBISiqBOELt3 8z536kMEGwoY2uAtsjSZIIu7x9Bdcw1CBs/NGRTjT5uhUaxjFBvJ5dRiv 0pKqkRVUpcQv0U1g7sGzwfxi5y5ekhtR5lzIh140u+kRdSnHwhP7TN3d2 vhsOivALv99tY7wOrAndJ+j4eLybW6wBxbLUVZjctyopGiqUDFs8TUuYV g==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="348500841" X-IronPort-AV: E=Sophos;i="6.01,243,1684825200"; d="scan'208";a="348500841" X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="728122173" X-IronPort-AV: E=Sophos;i="6.01,243,1684825200"; d="scan'208";a="728122173" From: Fei Wu To: palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, liweiwei@iscas.ac.cn, dbarboza@ventanamicro.com, zhiwei_liu@linux.alibaba.com, qemu-riscv@nongnu.org, qemu-devel@nongnu.org Cc: Fei Wu , Andrei Warkentin Subject: [PATCH] hw/riscv: split RAM into low and high memory Date: Mon, 31 Jul 2023 09:53:17 +0800 Message-Id: <20230731015317.1026996-1-fei2.wu@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Host-Lookup-Failed: Reverse DNS lookup failed for 192.55.52.136 (failed) Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=192.55.52.136; envelope-from=fei2.wu@intel.com; helo=mgamail.intel.com X-Spam_score_int: -35 X-Spam_score: -3.6 X-Spam_bar: --- X-Spam_report: (-3.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @intel.com) X-ZM-MESSAGEID: 1690768765121100001 Content-Type: text/plain; charset="utf-8" riscv virt platform's memory started at 0x80000000 and straddled the 4GiB boundary. Curiously enough, this choice of a memory layout will prevent from launching a VM with a bit more than 2000MiB and PCIe pass-thru on an x86 host, due to identity mapping requirements for the MSI doorbell on x86, and these (APIC/IOAPIC) live right below 4GiB. So just split the RAM range into two portions: - 1 GiB range from 0x80000000 to 0xc0000000. - The remainder at 0x100000000 ...leaving a hole between the ranges. Signed-off-by: Andrei Warkentin Signed-off-by: Fei Wu --- hw/riscv/virt.c | 74 ++++++++++++++++++++++++++++++++++++----- include/hw/riscv/virt.h | 1 + 2 files changed, 66 insertions(+), 9 deletions(-) diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index d90286dc46..8fbdc7220c 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -75,7 +75,9 @@ #error "Can't accomodate all IMSIC groups in address space" #endif =20 -static const MemMapEntry virt_memmap[] =3D { +#define LOW_MEM (1 * GiB) + +static MemMapEntry virt_memmap[] =3D { [VIRT_DEBUG] =3D { 0x0, 0x100 }, [VIRT_MROM] =3D { 0x1000, 0xf000 }, [VIRT_TEST] =3D { 0x100000, 0x1000 }, @@ -96,6 +98,7 @@ static const MemMapEntry virt_memmap[] =3D { [VIRT_PCIE_ECAM] =3D { 0x30000000, 0x10000000 }, [VIRT_PCIE_MMIO] =3D { 0x40000000, 0x40000000 }, [VIRT_DRAM] =3D { 0x80000000, 0x0 }, + [VIRT_DRAM_HIGH] =3D { 0x100000000, 0x0 }, }; =20 /* PCIe high mmio is fixed for RV32 */ @@ -295,15 +298,12 @@ static void create_fdt_socket_cpus(RISCVVirtState *s,= int socket, } } =20 -static void create_fdt_socket_memory(RISCVVirtState *s, - const MemMapEntry *memmap, int socket) +static void create_fdt_socket_mem_range(RISCVVirtState *s, uint64_t addr, + uint64_t size, int socket) { char *mem_name; - uint64_t addr, size; MachineState *ms =3D MACHINE(s); =20 - addr =3D memmap[VIRT_DRAM].base + riscv_socket_mem_offset(ms, socket); - size =3D riscv_socket_mem_size(ms, socket); mem_name =3D g_strdup_printf("/memory@%lx", (long)addr); qemu_fdt_add_subnode(ms->fdt, mem_name); qemu_fdt_setprop_cells(ms->fdt, mem_name, "reg", @@ -313,6 +313,34 @@ static void create_fdt_socket_memory(RISCVVirtState *s, g_free(mem_name); } =20 +static void create_fdt_socket_memory(RISCVVirtState *s, + const MemMapEntry *memmap, int socket) +{ + uint64_t addr, size; + MachineState *mc =3D MACHINE(s); + uint64_t sock_offset =3D riscv_socket_mem_offset(mc, socket); + uint64_t sock_size =3D riscv_socket_mem_size(mc, socket); + + if (sock_offset < memmap[VIRT_DRAM].size) { + uint64_t low_mem_end =3D memmap[VIRT_DRAM].base + memmap[VIRT_DRAM= ].size; + + addr =3D memmap[VIRT_DRAM].base + sock_offset; + size =3D MIN(low_mem_end - addr, sock_size); + create_fdt_socket_mem_range(s, addr, size, socket); + + size =3D sock_size - size; + if (size > 0) { + create_fdt_socket_mem_range(s, memmap[VIRT_DRAM_HIGH].base, + size, socket); + } + } else { + addr =3D memmap[VIRT_DRAM_HIGH].base + + sock_offset - memmap[VIRT_DRAM].size; + + create_fdt_socket_mem_range(s, addr, sock_size, socket); + } +} + static void create_fdt_socket_clint(RISCVVirtState *s, const MemMapEntry *memmap, int socket, uint32_t *intc_phandles) @@ -1334,10 +1362,12 @@ static void virt_machine_done(Notifier *notifier, v= oid *data) =20 static void virt_machine_init(MachineState *machine) { - const MemMapEntry *memmap =3D virt_memmap; + MemMapEntry *memmap =3D virt_memmap; RISCVVirtState *s =3D RISCV_VIRT_MACHINE(machine); MemoryRegion *system_memory =3D get_system_memory(); MemoryRegion *mask_rom =3D g_new(MemoryRegion, 1); + MemoryRegion *ram_below_4g, *ram_above_4g; + uint64_t ram_size_low, ram_size_high; char *soc_name; DeviceState *mmio_irqchip, *virtio_irqchip, *pcie_irqchip; int i, base_hartid, hart_count; @@ -1448,6 +1478,17 @@ static void virt_machine_init(MachineState *machine) } } =20 + if (machine->ram_size > LOW_MEM) { + ram_size_high =3D machine->ram_size - LOW_MEM; + ram_size_low =3D LOW_MEM; + } else { + ram_size_high =3D 0; + ram_size_low =3D machine->ram_size; + } + + memmap[VIRT_DRAM].size =3D ram_size_low; + memmap[VIRT_DRAM_HIGH].size =3D ram_size_high; + if (riscv_is_32bit(&s->soc[0])) { #if HOST_LONG_BITS =3D=3D 64 /* limit RAM size in a 32-bit system */ @@ -1460,7 +1501,8 @@ static void virt_machine_init(MachineState *machine) virt_high_pcie_memmap.size =3D VIRT32_HIGH_PCIE_MMIO_SIZE; } else { virt_high_pcie_memmap.size =3D VIRT64_HIGH_PCIE_MMIO_SIZE; - virt_high_pcie_memmap.base =3D memmap[VIRT_DRAM].base + machine->r= am_size; + virt_high_pcie_memmap.base =3D memmap[VIRT_DRAM_HIGH].base + + memmap[VIRT_DRAM_HIGH].size; virt_high_pcie_memmap.base =3D ROUND_UP(virt_high_pcie_memmap.base, virt_high_pcie_memmap.siz= e); } @@ -1468,8 +1510,22 @@ static void virt_machine_init(MachineState *machine) s->memmap =3D virt_memmap; =20 /* register system main memory (actual RAM) */ + ram_below_4g =3D g_malloc(sizeof(*ram_below_4g)); + memory_region_init_alias(ram_below_4g, NULL, "ram-below-4g", machine->= ram, + 0, memmap[VIRT_DRAM].size); memory_region_add_subregion(system_memory, memmap[VIRT_DRAM].base, - machine->ram); + ram_below_4g); + + if (memmap[VIRT_DRAM_HIGH].size) { + ram_above_4g =3D g_malloc(sizeof(*ram_above_4g)); + memory_region_init_alias(ram_above_4g, NULL, "ram-above-4g", + machine->ram, + memmap[VIRT_DRAM].size, + memmap[VIRT_DRAM_HIGH].size); + memory_region_add_subregion(system_memory, + memmap[VIRT_DRAM_HIGH].base, + ram_above_4g); + } =20 /* boot rom */ memory_region_init_rom(mask_rom, NULL, "riscv_virt_board.mrom", diff --git a/include/hw/riscv/virt.h b/include/hw/riscv/virt.h index e5c474b26e..36004eb6ef 100644 --- a/include/hw/riscv/virt.h +++ b/include/hw/riscv/virt.h @@ -79,6 +79,7 @@ enum { VIRT_IMSIC_S, VIRT_FLASH, VIRT_DRAM, + VIRT_DRAM_HIGH, VIRT_PCIE_MMIO, VIRT_PCIE_PIO, VIRT_PLATFORM_BUS, --=20 2.34.1