From nobody Thu May 15 02:18:23 2025
Delivered-To: importer@patchew.org
Authentication-Results: mx.zohomail.com;
spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as
permitted sender)
smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org
Return-Path: <qemu-devel-bounces+importer=patchew.org@nongnu.org>
Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by
mx.zohomail.com
with SMTPS id 17338831766270.5774274683092244;
Tue, 10 Dec 2024 18:12:56 -0800 (PST)
Received: from localhost ([::1] helo=lists1p.gnu.org)
by lists.gnu.org with esmtp (Exim 4.90_1)
(envelope-from <qemu-devel-bounces@nongnu.org>)
id 1tLCCl-0006NK-Ip; Tue, 10 Dec 2024 21:11:59 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <wangyuquan1236@phytium.com.cn>)
id 1tLCCj-0006N0-3I; Tue, 10 Dec 2024 21:11:57 -0500
Received: from zg8tmtyylji0my4xnjqumte4.icoremail.net ([162.243.164.118])
by eggs.gnu.org with esmtp (Exim 4.90_1)
(envelope-from <wangyuquan1236@phytium.com.cn>)
id 1tLCCh-0001SX-9n; Tue, 10 Dec 2024 21:11:56 -0500
Received: from prodtpl.icoremail.net (unknown [10.12.1.20])
by hzbj-icmmx-6 (Coremail) with SMTP id AQAAfwDHz9_f9FhnTLJoCA--.30089S2;
Wed, 11 Dec 2024 10:11:43 +0800 (CST)
Received: from phytium.com.cn (unknown [218.76.62.144])
by mail (Coremail) with SMTP id AQAAfwBnv3vO9FhnOi5nAA--.4806S4;
Wed, 11 Dec 2024 10:11:41 +0800 (CST)
From: Yuquan Wang <wangyuquan1236@phytium.com.cn>
To: rad@semihalf.com, peter.maydell@linaro.org, quic_llindhol@quicinc.com,
marcin.juszkiewicz@linaro.org
Cc: Jonathan.Cameron@Huawei.com, chenbaozi@phytium.com.cn,
linux-cxl@vger.kernel.org, qemu-arm@nongnu.org, qemu-devel@nongnu.org,
Yuquan Wang <wangyuquan1236@phytium.com.cn>
Subject: [RFC PATCH v4 1/1] hw/arm/sbsa-ref: Support CXL Host Bridge & CFMW
Date: Wed, 11 Dec 2024 10:11:17 +0800
Message-Id: <20241211021117.120630-2-wangyuquan1236@phytium.com.cn>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20241211021117.120630-1-wangyuquan1236@phytium.com.cn>
References: <20241211021117.120630-1-wangyuquan1236@phytium.com.cn>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
X-CM-TRANSID: AQAAfwBnv3vO9FhnOi5nAA--.4806S4
X-CM-SenderInfo: 5zdqw5pxtxt0arstlqxsk13x1xpou0fpof0/1tbiAQAQAWdYm48BlwAAst
Authentication-Results: hzbj-icmmx-6; spf=neutral smtp.mail=wangyuquan
1236@phytium.com.cn;
X-Coremail-Antispam: 1Uk129KBjvJXoW3Jw4fuF13Gr4rJrW5KrW8tFb_yoW3tw1rpF
43C393Cr48KFs8Arna93WDWF15u34vyFWUC347CF93Cr17Kw1jvFWkKa1UKa45JrWkJa45
u3WDXFyFvw1UZw7anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUj1kv1TuYvTs0mT0YCTnIWj
DUYxn0WfASr-VFAU7a7-sFnT9fnUUIcSsGvfJ3UbIYCTnIWIevJa73UjIFyTuYvj4RJUUU
UUUUU
Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17
as permitted sender) client-ip=209.51.188.17;
envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org;
helo=lists.gnu.org;
Received-SPF: pass client-ip=162.243.164.118;
envelope-from=wangyuquan1236@phytium.com.cn;
helo=zg8tmtyylji0my4xnjqumte4.icoremail.net
X-Spam_score_int: -25
X-Spam_score: -2.6
X-Spam_bar: --
X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7,
RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001,
SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-BeenThere: qemu-devel@nongnu.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <https://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org
Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org
X-ZM-MESSAGEID: 1733883178593116600
Content-Type: text/plain; charset="utf-8"
This creates a specific cxl host bridge (0001:00) with two cxl
root ports on sbsa-ref. And the memory layout provides separate
space windows for the cxl host bridge in the sbsa-ref memmap:
- 64K CXL Host Bridge Component Registers (CHBCR)
- 64K CXL_PIO
- 128M CXL_MMIO
- 256M CXL_ECAM
- 4G CXL_MMIO_HIGH
To provide CFMWs on sbsa-ref, this extends 1TB space from the
hole above RAM Memory [SBSA_MEM] for CXL Fixed Memory Window:
- 1T CXL_FIXED_WINDOW
Signed-off-by: Yuquan Wang <wangyuquan1236@phytium.com.cn>
---
docs/system/arm/sbsa.rst | 4 ++
hw/arm/sbsa-ref.c | 135 ++++++++++++++++++++++++++++++++++++++-
2 files changed, 138 insertions(+), 1 deletion(-)
diff --git a/docs/system/arm/sbsa.rst b/docs/system/arm/sbsa.rst
index 2bf3fc8d59..21b88e88e7 100644
--- a/docs/system/arm/sbsa.rst
+++ b/docs/system/arm/sbsa.rst
@@ -28,6 +28,7 @@ The ``sbsa-ref`` board supports:
- E1000E ethernet card on PCIe bus
- Bochs display adapter on PCIe bus
- A generic SBSA watchdog device
+ - CXL host bridge and CXL fixed memory window
=20
=20
Board to firmware interface
@@ -92,3 +93,6 @@ Platform version changes:
=20
0.4
CPU topology information is present in devicetree.
+
+0.5
+ CXL host bridge and CXL fixed memory window are supported.
diff --git a/hw/arm/sbsa-ref.c b/hw/arm/sbsa-ref.c
index e3195d5449..40aaaef617 100644
--- a/hw/arm/sbsa-ref.c
+++ b/hw/arm/sbsa-ref.c
@@ -36,11 +36,14 @@
#include "hw/arm/smmuv3.h"
#include "hw/block/flash.h"
#include "hw/boards.h"
+#include "hw/cxl/cxl.h"
+#include "hw/cxl/cxl_host.h"
#include "hw/ide/ide-bus.h"
#include "hw/ide/ahci-sysbus.h"
#include "hw/intc/arm_gicv3_common.h"
#include "hw/intc/arm_gicv3_its_common.h"
#include "hw/loader.h"
+#include "hw/pci/pcie_port.h"
#include "hw/pci-host/gpex.h"
#include "hw/qdev-properties.h"
#include "hw/usb.h"
@@ -94,6 +97,13 @@ enum {
SBSA_SECURE_MEM,
SBSA_AHCI,
SBSA_XHCI,
+ SBSA_CXL,
+ SBSA_CXL_CHBCR,
+ SBSA_CXL_MMIO,
+ SBSA_CXL_MMIO_HIGH,
+ SBSA_CXL_PIO,
+ SBSA_CXL_ECAM,
+ SBSA_CXL_FIXED_WINDOW,
};
=20
struct SBSAMachineState {
@@ -105,6 +115,7 @@ struct SBSAMachineState {
int psci_conduit;
DeviceState *gic;
PFlashCFI01 *flash[2];
+ CXLState cxl_devices_state;
};
=20
#define TYPE_SBSA_MACHINE MACHINE_TYPE_NAME("sbsa-ref")
@@ -132,6 +143,14 @@ static const MemMapEntry sbsa_ref_memmap[] =3D {
/* Space here reserved for more SMMUs */
[SBSA_AHCI] =3D { 0x60100000, 0x00010000 },
[SBSA_XHCI] =3D { 0x60110000, 0x00010000 },
+ /* 64K CXL Host Bridge Registers space */
+ [SBSA_CXL_CHBCR] =3D { 0x60200000, 0x00010000 },
+ /* 64K CXL PIO space */
+ [SBSA_CXL_PIO] =3D { 0x60300000, 0x00010000 },
+ /* 128M CXL 32-bit MMIO space */
+ [SBSA_CXL_MMIO] =3D { 0x60400000, 0x08000000 },
+ /* 256M CXL ECAM space */
+ [SBSA_CXL_ECAM] =3D { 0x68500000, 0x10000000 },
/* Space here reserved for other devices */
[SBSA_PCIE_PIO] =3D { 0x7fff0000, 0x00010000 },
/* 32-bit address PCIE MMIO space */
@@ -141,6 +160,10 @@ static const MemMapEntry sbsa_ref_memmap[] =3D {
/* ~1TB PCIE MMIO space (4GB to 1024GB boundary) */
[SBSA_PCIE_MMIO_HIGH] =3D { 0x100000000ULL, 0xFF00000000ULL },
[SBSA_MEM] =3D { 0x10000000000ULL, RAMLIMIT_BYTES },
+ /* 4G CXL 64-bit MMIO space */
+ [SBSA_CXL_MMIO_HIGH] =3D { 0x90000000000ULL, 0x100000000ULL },
+ /* 1TB CXL FIXED WINDOW space */
+ [SBSA_CXL_FIXED_WINDOW] =3D { 0xA0000000000ULL, 0x10000000000ULL },
};
=20
static const int sbsa_ref_irqmap[] =3D {
@@ -154,6 +177,7 @@ static const int sbsa_ref_irqmap[] =3D {
[SBSA_XHCI] =3D 11,
[SBSA_SMMU] =3D 12, /* ... to 15 */
[SBSA_GWDT_WS0] =3D 16,
+ [SBSA_CXL] =3D 17, /* ... to 20 */
};
=20
static uint64_t sbsa_ref_cpu_mp_affinity(SBSAMachineState *sms, int idx)
@@ -216,7 +240,7 @@ static void create_fdt(SBSAMachineState *sms)
* fw compatibility.
*/
qemu_fdt_setprop_cell(fdt, "/", "machine-version-major", 0);
- qemu_fdt_setprop_cell(fdt, "/", "machine-version-minor", 4);
+ qemu_fdt_setprop_cell(fdt, "/", "machine-version-minor", 5);
=20
if (ms->numa_state->have_numa_distance) {
int size =3D nb_numa_nodes * nb_numa_nodes * 3 * sizeof(uint32_t);
@@ -629,6 +653,113 @@ static void create_smmu(const SBSAMachineState *sms, =
PCIBus *bus)
}
}
=20
+static void create_cxl_fixed_window(SBSAMachineState *sms,
+ MemoryRegion *mem, CXLHostBridge *host)
+{
+ PCIBus *cxlbus =3D PCI_HOST_BRIDGE(host)->bus;
+ char *cxl_host =3D object_get_canonical_path(OBJECT(cxlbus));
+ hwaddr base =3D sbsa_ref_memmap[SBSA_CXL_FIXED_WINDOW].base;
+ GList *it;
+ strList host_target =3D { NULL, cxl_host };
+ CXLFixedMemoryWindowOptions sbsa_ref_cfmwoptions =3D {
+ .size =3D 1 * TiB,
+ .has_interleave_granularity =3D false,
+ .targets =3D &host_target,
+ };
+ CXLFixedWindow *fw;
+
+ cxl_fixed_memory_window_config(&sms->cxl_devices_state,
+ &sbsa_ref_cfmwoptions, &error_fatal);
+
+ it =3D sms->cxl_devices_state.fixed_windows;
+ fw =3D it->data;
+ fw->base =3D base;
+ fw->target_chb[0] =3D host;
+
+ memory_region_init_io(&fw->mr, OBJECT(sms), &cfmws_ops, fw,
+ "cxl-fixed-memory-region", fw->size);
+
+ memory_region_add_subregion(mem, fw->base, &fw->mr);
+}
+
+static void create_cxl(SBSAMachineState *sms)
+{
+ hwaddr base_pio =3D sbsa_ref_memmap[SBSA_CXL_PIO].base;
+ int irq =3D sbsa_ref_irqmap[SBSA_CXL];
+ MemoryRegion *mmio_alias, *mmio_alias_high, *mmio_reg;
+ MemoryRegion *ecam_alias, *ecam_reg;
+ MemoryRegion *sysmem =3D get_system_memory();
+ MemoryRegion *chbcr =3D &sms->cxl_devices_state.host_mr;
+ DeviceState *dev;
+ CXLHostBridge *host;
+ PCIHostState *cxl;
+ PCIDevice *cxlrp;
+ PCIEPort *p;
+ PCIESlot *s;
+ int i;
+
+ dev =3D qdev_new(TYPE_CXL_HOST);
+ sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
+ sms->cxl_devices_state.is_enabled =3D true;
+
+ /* Map CXL ECAM space */
+ ecam_alias =3D g_new0(MemoryRegion, 1);
+ ecam_reg =3D sysbus_mmio_get_region(SYS_BUS_DEVICE(dev), 1);
+ memory_region_init_alias(ecam_alias, OBJECT(dev), "cxl-ecam",
+ ecam_reg, 0, sbsa_ref_memmap[SBSA_CXL_ECAM].s=
ize);
+ memory_region_add_subregion(get_system_memory(),
+ sbsa_ref_memmap[SBSA_CXL_ECAM].base, ecam_al=
ias);
+
+ /* Map CXL MMIO space */
+ mmio_alias =3D g_new0(MemoryRegion, 1);
+ mmio_reg =3D sysbus_mmio_get_region(SYS_BUS_DEVICE(dev), 2);
+ memory_region_init_alias(mmio_alias, OBJECT(dev), "cxl-mmio",
+ mmio_reg, sbsa_ref_memmap[SBSA_CXL_MMIO].base,
+ sbsa_ref_memmap[SBSA_CXL_MMIO].size=
);
+ memory_region_add_subregion(get_system_memory(),
+ sbsa_ref_memmap[SBSA_CXL_MMIO].base, mmio_alias=
);
+
+ /* Map CXL MMIO_HIGH space */
+ mmio_alias_high =3D g_new0(MemoryRegion, 1);
+ memory_region_init_alias(mmio_alias_high, OBJECT(dev), "cxl-mmio-high",
+ mmio_reg, sbsa_ref_memmap[SBSA_CXL_MMIO_HIGH].b=
ase,
+ sbsa_ref_memmap[SBSA_CXL_MMIO_HIGH].s=
ize);
+ memory_region_add_subregion(get_system_memory(),
+ sbsa_ref_memmap[SBSA_CXL_MMIO_HIGH].base, mmio_alias_h=
igh);
+
+ /* Map CXL IO port space */
+ sysbus_mmio_map(SYS_BUS_DEVICE(dev), 3, base_pio);
+
+ for (i =3D 0; i < CXL_HOST_NUM_IRQS; i++) {
+ sysbus_connect_irq(SYS_BUS_DEVICE(dev), i,
+ qdev_get_gpio_in(sms->gic, irq + i));
+ cxl_host_set_irq_num(CXL_HOST(dev), i, irq + i);
+ }
+
+ /* Map CXL CHBCR space */
+ memory_region_init(chbcr, OBJECT(sms), "cxl_host_reg",
+ sbsa_ref_memmap[SBSA_CXL_CHBCR].size);
+ memory_region_add_subregion(sysmem, sbsa_ref_memmap[SBSA_CXL_CHBCR].ba=
se,
+ chbcr);
+
+ cxl =3D PCI_HOST_BRIDGE(dev);
+
+ /* Connect two cxl root ports */
+ for (i =3D 0; i < 2; i++) {
+ cxlrp =3D pci_new(-1, "cxl-rp");
+ p =3D PCIE_PORT(cxlrp);
+ s =3D PCIE_SLOT(cxlrp);
+ p->port =3D i;
+ s->slot =3D i;
+ pci_realize_and_unref(cxlrp, cxl->bus, &error_fatal);
+ }
+
+ host =3D CXL_HOST(dev);
+ cxl_host_hook_up_registers(&sms->cxl_devices_state, host, &error_fatal=
);
+
+ create_cxl_fixed_window(sms, sysmem, host);
+}
+
static void create_pcie(SBSAMachineState *sms)
{
hwaddr base_ecam =3D sbsa_ref_memmap[SBSA_PCIE_ECAM].base;
@@ -821,6 +952,8 @@ static void sbsa_ref_init(MachineState *machine)
=20
create_pcie(sms);
=20
+ create_cxl(sms);
+
create_secure_ec(secure_sysmem);
=20
sms->bootinfo.ram_size =3D machine->ram_size;
--=20
2.34.1