From nobody Wed Sep 3 02:27:06 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1649086476; cv=none; d=zohomail.com; s=zohoarc; b=cM5CyqLvg53lAs8yeUNSXsZOT9uz3+/SYUASdGJ7B0WSwqavzPWAMuSkGKh/vEaVo31rFt0gzp53k2WZSFJR0VxtMXQYb6ZB4UhUAikYVuw0G2EmDHrJHoFu1oM9hh0ojSzI96wv9S6XgyFnOqa3/qrvWfqdRVSibOj/yPwU7Vc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1649086476; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=PKEJGwslS48bWwgb2vih7ztGnaHrXUGKiNG398T3ulE=; b=OpcB+ruMpbz/oKN+pwvUEl6mR6IpRRqG9jF2mkJFEmLjcjrKUV0uTCbboT3bCXaEar0gy7GnPP8flCb4bPgx6/ed8ZQhCGIyr7qg/yxUY6YzMMuigfo7pG0NbT/nX5B6bdIMMqkUxUUandOg8ljBeksS7YO3LOUWP7Z5tTXf4s8= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1649086476819748.247201132103; Mon, 4 Apr 2022 08:34:36 -0700 (PDT) Received: from localhost ([::1]:57268 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nbOix-0003Rw-LW for importer@patchew.org; Mon, 04 Apr 2022 11:34:35 -0400 Received: from eggs.gnu.org ([209.51.188.92]:39766) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nbOb6-0004i0-DH for qemu-devel@nongnu.org; Mon, 04 Apr 2022 11:26:28 -0400 Received: from frasgout.his.huawei.com ([185.176.79.56]:2490) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nbOb4-0005vl-3n for qemu-devel@nongnu.org; Mon, 04 Apr 2022 11:26:28 -0400 Received: from fraeml743-chm.china.huawei.com (unknown [172.18.147.207]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4KXF0h4RKLz67Xsq; Mon, 4 Apr 2022 23:23:32 +0800 (CST) Received: from lhreml710-chm.china.huawei.com (10.201.108.61) by fraeml743-chm.china.huawei.com (10.206.15.224) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Mon, 4 Apr 2022 17:26:24 +0200 Received: from SecurePC-101-06.china.huawei.com (10.122.247.231) by lhreml710-chm.china.huawei.com (10.201.108.61) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Mon, 4 Apr 2022 16:26:23 +0100 To: , , =?UTF-8?q?Alex=20Benn=C3=A9e?= , Marcel Apfelbaum , "Michael S . Tsirkin" , Igor Mammedov , Markus Armbruster , "Mark Cave-Ayland" , Adam Manzanares CC: , Ben Widawsky , "Peter Maydell" , Shameerali Kolothum Thodi , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Peter Xu , David Hildenbrand , Paolo Bonzini , Saransh Gupta1 , Shreyas Shah , Chris Browy , "Samarth Saxena" , Dan Williams , "k . jensen @ samsung . com" , Tong Zhang , , Alison Schofield Subject: [PATCH v9 23/45] hw/cxl/component: Implement host bridge MMIO (8.2.5, table 142) Date: Mon, 4 Apr 2022 16:14:23 +0100 Message-ID: <20220404151445.10955-24-Jonathan.Cameron@huawei.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220404151445.10955-1-Jonathan.Cameron@huawei.com> References: <20220404151445.10955-1-Jonathan.Cameron@huawei.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.122.247.231] X-ClientProxiedBy: lhreml717-chm.china.huawei.com (10.201.108.68) To lhreml710-chm.china.huawei.com (10.201.108.61) X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.176.79.56; envelope-from=jonathan.cameron@huawei.com; helo=frasgout.his.huawei.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Reply-to: Jonathan Cameron From: Jonathan Cameron via X-ZM-MESSAGEID: 1649086477891100001 From: Ben Widawsky CXL host bridges themselves may have MMIO. Since host bridges don't have a BAR they are treated as special for MMIO. This patch includes i386/pc support. Also hook up the device reset now that we have have the MMIO space in which the results are visible. Note that we duplicate the PCI express case for the aml_build but the implementations will diverge when the CXL specific _OSC is introduced. Signed-off-by: Ben Widawsky Co-developed-by: Jonathan Cameron Signed-off-by: Jonathan Cameron Reviewed-by: Alex Benn=C3=A9e --- hw/i386/acpi-build.c | 25 ++++++++++- hw/i386/pc.c | 27 +++++++++++- hw/pci-bridge/pci_expander_bridge.c | 65 +++++++++++++++++++++++++---- include/hw/cxl/cxl.h | 14 +++++++ 4 files changed, 121 insertions(+), 10 deletions(-) diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c index dcf6ece3d0..2d81b0f40c 100644 --- a/hw/i386/acpi-build.c +++ b/hw/i386/acpi-build.c @@ -28,6 +28,7 @@ #include "qemu/bitmap.h" #include "qemu/error-report.h" #include "hw/pci/pci.h" +#include "hw/cxl/cxl.h" #include "hw/core/cpu.h" #include "target/i386/cpu.h" #include "hw/misc/pvpanic.h" @@ -1572,10 +1573,21 @@ build_dsdt(GArray *table_data, BIOSLinker *linker, } =20 scope =3D aml_scope("\\_SB"); - dev =3D aml_device("PC%.02X", bus_num); + + if (pci_bus_is_cxl(bus)) { + dev =3D aml_device("CL%.02X", bus_num); + } else { + dev =3D aml_device("PC%.02X", bus_num); + } aml_append(dev, aml_name_decl("_UID", aml_int(bus_num))); aml_append(dev, aml_name_decl("_BBN", aml_int(bus_num))); - if (pci_bus_is_express(bus)) { + if (pci_bus_is_cxl(bus)) { + aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0A08"= ))); + aml_append(dev, aml_name_decl("_CID", aml_eisaid("PNP0A03"= ))); + + /* Expander bridges do not have ACPI PCI Hot-plug enabled = */ + aml_append(dev, build_q35_osc_method(true)); + } else if (pci_bus_is_express(bus)) { aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0A08"= ))); aml_append(dev, aml_name_decl("_CID", aml_eisaid("PNP0A03"= ))); =20 @@ -1595,6 +1607,15 @@ build_dsdt(GArray *table_data, BIOSLinker *linker, aml_append(dev, aml_name_decl("_CRS", crs)); aml_append(scope, dev); aml_append(dsdt, scope); + + /* Handle the ranges for the PXB expanders */ + if (pci_bus_is_cxl(bus)) { + MemoryRegion *mr =3D &machine->cxl_devices_state->host_mr; + uint64_t base =3D mr->addr; + + crs_range_insert(crs_range_set.mem_ranges, base, + base + memory_region_size(mr) - 1); + } } } =20 diff --git a/hw/i386/pc.c b/hw/i386/pc.c index e2849fc741..da74f08f9e 100644 --- a/hw/i386/pc.c +++ b/hw/i386/pc.c @@ -75,6 +75,7 @@ #include "acpi-build.h" #include "hw/mem/pc-dimm.h" #include "hw/mem/nvdimm.h" +#include "hw/cxl/cxl.h" #include "qapi/error.h" #include "qapi/qapi-visit-common.h" #include "qapi/qapi-visit-machine.h" @@ -813,6 +814,7 @@ void pc_memory_init(PCMachineState *pcms, MachineClass *mc =3D MACHINE_GET_CLASS(machine); PCMachineClass *pcmc =3D PC_MACHINE_GET_CLASS(pcms); X86MachineState *x86ms =3D X86_MACHINE(pcms); + hwaddr cxl_base; =20 assert(machine->ram_size =3D=3D x86ms->below_4g_mem_size + x86ms->above_4g_mem_size); @@ -902,6 +904,26 @@ void pc_memory_init(PCMachineState *pcms, &machine->device_memory->mr); } =20 + if (machine->cxl_devices_state->is_enabled) { + MemoryRegion *mr =3D &machine->cxl_devices_state->host_mr; + hwaddr cxl_size =3D MiB; + + if (pcmc->has_reserved_memory && machine->device_memory->base) { + cxl_base =3D machine->device_memory->base; + if (!pcmc->broken_reserved_end) { + cxl_base +=3D memory_region_size(&machine->device_memory->= mr); + } + } else if (pcms->sgx_epc.size !=3D 0) { + cxl_base =3D sgx_epc_above_4g_end(&pcms->sgx_epc); + } else { + cxl_base =3D 0x100000000ULL + x86ms->above_4g_mem_size; + } + + e820_add_entry(cxl_base, cxl_size, E820_RESERVED); + memory_region_init(mr, OBJECT(machine), "cxl_host_reg", cxl_size); + memory_region_add_subregion(system_memory, cxl_base, mr); + } + /* Initialize PC system firmware */ pc_system_firmware_init(pcms, rom_memory); =20 @@ -962,7 +984,10 @@ uint64_t pc_pci_hole64_start(void) X86MachineState *x86ms =3D X86_MACHINE(pcms); uint64_t hole64_start =3D 0; =20 - if (pcmc->has_reserved_memory && ms->device_memory->base) { + if (ms->cxl_devices_state->host_mr.addr) { + hole64_start =3D ms->cxl_devices_state->host_mr.addr + + memory_region_size(&ms->cxl_devices_state->host_mr); + } else if (pcmc->has_reserved_memory && ms->device_memory->base) { hole64_start =3D ms->device_memory->base; if (!pcmc->broken_reserved_end) { hole64_start +=3D memory_region_size(&ms->device_memory->mr); diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expand= er_bridge.c index f762eb4a6e..b2dea027b4 100644 --- a/hw/pci-bridge/pci_expander_bridge.c +++ b/hw/pci-bridge/pci_expander_bridge.c @@ -61,12 +61,6 @@ DECLARE_INSTANCE_CHECKER(PXBDev, PXB_PCIE_DEV, DECLARE_INSTANCE_CHECKER(PXBDev, PXB_CXL_DEV, TYPE_PXB_CXL_DEVICE) =20 -typedef struct CXLHost { - PCIHostState parent_obj; - - CXLComponentState cxl_cstate; -} CXLHost; - struct PXBDev { /*< private >*/ PCIDevice parent_obj; @@ -75,6 +69,9 @@ struct PXBDev { uint8_t bus_nr; uint16_t numa_node; bool bypass_iommu; + struct cxl_dev { + CXLHost *cxl_host_bridge; + } cxl; }; =20 static PXBDev *convert_to_pxb(PCIDevice *dev) @@ -197,6 +194,52 @@ static const TypeInfo pxb_host_info =3D { .class_init =3D pxb_host_class_init, }; =20 +static void pxb_cxl_realize(DeviceState *dev, Error **errp) +{ + MachineState *ms =3D MACHINE(qdev_get_machine()); + SysBusDevice *sbd =3D SYS_BUS_DEVICE(dev); + CXLHost *cxl =3D PXB_CXL_HOST(dev); + CXLComponentState *cxl_cstate =3D &cxl->cxl_cstate; + struct MemoryRegion *mr =3D &cxl_cstate->crb.component_registers; + hwaddr offset; + + cxl_component_register_block_init(OBJECT(dev), cxl_cstate, + TYPE_PXB_CXL_HOST); + sysbus_init_mmio(sbd, mr); + + offset =3D memory_region_size(mr) * ms->cxl_devices_state->next_mr_idx; + if (offset > memory_region_size(&ms->cxl_devices_state->host_mr)) { + error_setg(errp, "Insufficient space for pxb cxl host register spa= ce"); + return; + } + + memory_region_add_subregion(&ms->cxl_devices_state->host_mr, offset, m= r); + ms->cxl_devices_state->next_mr_idx++; +} + +static void pxb_cxl_host_class_init(ObjectClass *class, void *data) +{ + DeviceClass *dc =3D DEVICE_CLASS(class); + PCIHostBridgeClass *hc =3D PCI_HOST_BRIDGE_CLASS(class); + + hc->root_bus_path =3D pxb_host_root_bus_path; + dc->fw_name =3D "cxl"; + dc->realize =3D pxb_cxl_realize; + /* Reason: Internal part of the pxb/pxb-pcie device, not usable by its= elf */ + dc->user_creatable =3D false; +} + +/* + * This is a device to handle the MMIO for a CXL host bridge. It does noth= ing + * else. + */ +static const TypeInfo cxl_host_info =3D { + .name =3D TYPE_PXB_CXL_HOST, + .parent =3D TYPE_PCI_HOST_BRIDGE, + .instance_size =3D sizeof(CXLHost), + .class_init =3D pxb_cxl_host_class_init, +}; + /* * Registers the PXB bus as a child of pci host root bus. */ @@ -245,6 +288,12 @@ static int pxb_map_irq_fn(PCIDevice *pci_dev, int pin) =20 static void pxb_dev_reset(DeviceState *dev) { + CXLHost *cxl =3D PXB_CXL_DEV(dev)->cxl.cxl_host_bridge; + CXLComponentState *cxl_cstate =3D &cxl->cxl_cstate; + uint32_t *reg_state =3D cxl_cstate->crb.cache_mem_registers; + + cxl_component_register_init_common(reg_state, CXL2_ROOT_PORT); + ARRAY_FIELD_DP32(reg_state, CXL_HDM_DECODER_CAPABILITY, TARGET_COUNT, = 8); } =20 static gint pxb_compare(gconstpointer a, gconstpointer b) @@ -281,12 +330,13 @@ static void pxb_dev_realize_common(PCIDevice *dev, en= um BusType type, dev_name =3D dev->qdev.id; } =20 - ds =3D qdev_new(TYPE_PXB_HOST); + ds =3D qdev_new(type =3D=3D CXL ? TYPE_PXB_CXL_HOST : TYPE_PXB_HOST); if (type =3D=3D PCIE) { bus =3D pci_root_bus_new(ds, dev_name, NULL, NULL, 0, TYPE_PXB_PCI= E_BUS); } else if (type =3D=3D CXL) { bus =3D pci_root_bus_new(ds, dev_name, NULL, NULL, 0, TYPE_PXB_CXL= _BUS); bus->flags |=3D PCI_BUS_CXL; + PXB_CXL_DEV(dev)->cxl.cxl_host_bridge =3D PXB_CXL_HOST(ds); } else { bus =3D pci_root_bus_new(ds, "pxb-internal", NULL, NULL, 0, TYPE_P= XB_BUS); bds =3D qdev_new("pci-bridge"); @@ -475,6 +525,7 @@ static void pxb_register_types(void) type_register_static(&pxb_pcie_bus_info); type_register_static(&pxb_cxl_bus_info); type_register_static(&pxb_host_info); + type_register_static(&cxl_host_info); type_register_static(&pxb_dev_info); type_register_static(&pxb_pcie_dev_info); type_register_static(&pxb_cxl_dev_info); diff --git a/include/hw/cxl/cxl.h b/include/hw/cxl/cxl.h index 31af92fd5e..8d1a7245d0 100644 --- a/include/hw/cxl/cxl.h +++ b/include/hw/cxl/cxl.h @@ -10,6 +10,7 @@ #ifndef CXL_H #define CXL_H =20 +#include "hw/pci/pci_host.h" #include "cxl_pci.h" #include "cxl_component.h" #include "cxl_device.h" @@ -17,8 +18,21 @@ #define CXL_COMPONENT_REG_BAR_IDX 0 #define CXL_DEVICE_REG_BAR_IDX 2 =20 +#define CXL_WINDOW_MAX 10 + typedef struct CXLState { bool is_enabled; + MemoryRegion host_mr; + unsigned int next_mr_idx; } CXLState; =20 +struct CXLHost { + PCIHostState parent_obj; + + CXLComponentState cxl_cstate; +}; + +#define TYPE_PXB_CXL_HOST "pxb-cxl-host" +OBJECT_DECLARE_SIMPLE_TYPE(CXLHost, PXB_CXL_HOST) + #endif --=20 2.32.0