From nobody Mon Feb 9 08:57:36 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1654852358; cv=none; d=zohomail.com; s=zohoarc; b=nkH0DQsNHfbm0PGqDw87mdNRbOHBWp6qO/5qTgbxd4aGzcTHUIeO3zX6sXD9HO7r4OyxEMiJgHeutjQRKAcY4wMzE63jh8CrKqpfhT+XOrs/LhQK1GgyW+WSiDfcwqgmO1harTVIlwJt208YS/24lMdtNR+ZeVe8ewZIXWg1Q50= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1654852358; h=Content-Type:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=fF4zkj+lt60cISdNh0bi8uQF74kqrPF+05zKWN2Kk8U=; b=CA5qh1M/HQkr2x00qGpAONUCS/FX18C0U9zqrkTcu2DyUYZLFAUBsHm3G2mPQlJbXXXxTcPIzw9aRpQb50zU1nwjepWMdx8eMyvrBX6vvbMWtQn/dJiNm77JTTAnztGaKHJmkhkpypmCRB6g4UDysvVjLnJkbpXhFli4K2SGYQs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1654852358056410.2603479648526; Fri, 10 Jun 2022 02:12:38 -0700 (PDT) Received: from localhost ([::1]:56642 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzagz-0002B7-Es for importer@patchew.org; Fri, 10 Jun 2022 05:12:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35718) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzZXx-0001u0-6x for qemu-devel@nongnu.org; Fri, 10 Jun 2022 03:59:09 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:45463) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzZXv-0006OX-3c for qemu-devel@nongnu.org; Fri, 10 Jun 2022 03:59:08 -0400 Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-169-INGOVF8zPWa0cSwb7XYp4A-1; Fri, 10 Jun 2022 03:59:05 -0400 Received: by mail-wm1-f69.google.com with SMTP id l4-20020a05600c1d0400b0039c60535405so3369687wms.6 for ; Fri, 10 Jun 2022 00:59:04 -0700 (PDT) Received: from redhat.com ([212.116.178.142]) by smtp.gmail.com with ESMTPSA id p16-20020a05600c205000b0039c5b9e9142sm1953946wmg.17.2022.06.10.00.59.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jun 2022 00:59:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654847946; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=fF4zkj+lt60cISdNh0bi8uQF74kqrPF+05zKWN2Kk8U=; b=Oga1lSpwQ0rQNqdhI4trNrXal9xbCS6h0Z+Be7KeodFS81XkNJo9rV7fD98CyqlrmuR+Fn 8A5WRQOutJRb93HgB+Yax4sXFEzxUL/TD0NRql/qeW1If033CVAk347AJFCgoqPuNr+bPg edEFhbNShKvDEs0Sslx6qCNm+BC3OAM= X-MC-Unique: INGOVF8zPWa0cSwb7XYp4A-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=fF4zkj+lt60cISdNh0bi8uQF74kqrPF+05zKWN2Kk8U=; b=m9EJq/Zo9Fjfm/bSG02xEx7JGhmdjGyFL91H/Sq79ZGEZ4KEQP+MKzVKZCcqh1d9Qn m5/odfDRZdTnt0xRnN7EVh0fIO/6mCgjN1mCqDpPefKpcFLKqJ3Svb7Tl1rhzF5S5fzz HvjJFjPSpgChFolh3uzY6TIcwXfje5BOJrLrHwsKSYkwDj3qERIuJADxl3523YOx8wZe rzEaU4DrNVWuYohI7s5y6HLkvHg3lF0RgG+prFlfOe1jSRyUkxlLs0dYwgmUx/vdeMjA bj+ok1URxnjZUF3QsrnK0hnFqI5gmxUolRPPOp7sFtzPoJc/9loY8diPpbD1psoV0rCA NXIA== X-Gm-Message-State: AOAM532uokK6HACxmYW7chBKDM1CEkIDnxfR27J6ACOK7i67sZ4H9ITW b9bwLDlkMGoZUArrfKHR2sazFLs78DNaStKaYoWPrBwWTPXXU4zogjO66e9+2tavmBWxWjPh5+8 oL7/GnZQG0/b6qjSagjgdMo3PXVFHK2Z1Fmc+tNTPxDW77SeRRNj1Y5RjrzwP X-Received: by 2002:a05:6000:1e0b:b0:219:d06:de6c with SMTP id bj11-20020a0560001e0b00b002190d06de6cmr11041888wrb.529.1654847943548; Fri, 10 Jun 2022 00:59:03 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyng4DGI3RSI9zO00yG8xOaNPQzWrDvVxWXh16Y/OHwcT14F79M0GerlDbeDFFCltKRewymhA== X-Received: by 2002:a05:6000:1e0b:b0:219:d06:de6c with SMTP id bj11-20020a0560001e0b00b002190d06de6cmr11041858wrb.529.1654847943196; Fri, 10 Jun 2022 00:59:03 -0700 (PDT) Date: Fri, 10 Jun 2022 03:59:00 -0400 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Jonathan Cameron , Ben Widawsky , Ben Widawsky , Paolo Bonzini , Richard Henderson , Eduardo Habkost , Marcel Apfelbaum Subject: [PULL 40/54] pci/pci_expander_bridge: For CXL HB delay the HB register memory region setup. Message-ID: <20220610075631.367501-41-mst@redhat.com> References: <20220610075631.367501-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20220610075631.367501-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1654852359545100001 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Jonathan Cameron As the CXLState will no long be accessible via MachineState at time of PXB_CXL realization, come back later from the machine specific code to fill in the missing memory region setup. Only at this stage is it possible to check if cxl=3Don, so that check is moved to this later point. Note that for multiple host bridges, the allocation order of the register spaces is changed. This will be reflected in ACPI CEDT. Stubs are added to handle case of CONFIG_PXB=3Dn for machines that call these functions. The bus walking logic is common to all machines so add a utility function + stub to cxl-host*. Signed-off-by: Jonathan Cameron Reviewed-by: Ben Widawsky Message-Id: <20220608145440.26106-6-Jonathan.Cameron@huawei.com> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- include/hw/cxl/cxl_host.h | 1 + include/hw/pci-bridge/pci_expander_bridge.h | 12 ++++++++ hw/cxl/cxl-host-stubs.c | 1 + hw/cxl/cxl-host.c | 20 +++++++++++++ hw/i386/pc.c | 3 ++ hw/pci-bridge/pci_expander_bridge.c | 32 +++++++++++++-------- hw/pci-bridge/pci_expander_bridge_stubs.c | 14 +++++++++ hw/pci-bridge/meson.build | 5 +++- 8 files changed, 75 insertions(+), 13 deletions(-) create mode 100644 include/hw/pci-bridge/pci_expander_bridge.h create mode 100644 hw/pci-bridge/pci_expander_bridge_stubs.c diff --git a/include/hw/cxl/cxl_host.h b/include/hw/cxl/cxl_host.h index 4d642a81fa..a1b662ce40 100644 --- a/include/hw/cxl/cxl_host.h +++ b/include/hw/cxl/cxl_host.h @@ -16,6 +16,7 @@ =20 void cxl_machine_init(Object *obj, CXLState *state); void cxl_fmws_link_targets(CXLState *stat, Error **errp); +void cxl_hook_up_pxb_registers(PCIBus *bus, CXLState *state, Error **errp); =20 extern const MemoryRegionOps cfmws_ops; =20 diff --git a/include/hw/pci-bridge/pci_expander_bridge.h b/include/hw/pci-b= ridge/pci_expander_bridge.h new file mode 100644 index 0000000000..0b3856d615 --- /dev/null +++ b/include/hw/pci-bridge/pci_expander_bridge.h @@ -0,0 +1,12 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef PCI_EXPANDER_BRIDGE_H +#define PCI_EXPANDER_BRIDGE_H + +#include "hw/cxl/cxl.h" + +void pxb_cxl_hook_up_registers(CXLState *state, PCIBus *bus, Error **errp); + +#endif /* PCI_EXPANDER_BRIDGE_H */ diff --git a/hw/cxl/cxl-host-stubs.c b/hw/cxl/cxl-host-stubs.c index e0d5ec8ad5..cae4afcdde 100644 --- a/hw/cxl/cxl-host-stubs.c +++ b/hw/cxl/cxl-host-stubs.c @@ -10,5 +10,6 @@ =20 void cxl_fmws_link_targets(CXLState *stat, Error **errp) {}; void cxl_machine_init(Object *obj, CXLState *state) {}; +void cxl_hook_up_pxb_registers(PCIBus *bus, CXLState *state, Error **errp)= {}; =20 const MemoryRegionOps cfmws_ops; diff --git a/hw/cxl/cxl-host.c b/hw/cxl/cxl-host.c index 8e7738a65b..efa14908d8 100644 --- a/hw/cxl/cxl-host.c +++ b/hw/cxl/cxl-host.c @@ -20,6 +20,7 @@ #include "hw/pci/pci_bridge.h" #include "hw/pci/pci_host.h" #include "hw/pci/pcie_port.h" +#include "hw/pci-bridge/pci_expander_bridge.h" =20 static void cxl_fixed_memory_window_config(CXLState *cxl_state, CXLFixedMemoryWindowOptions *ob= ject, @@ -280,3 +281,22 @@ void cxl_machine_init(Object *obj, CXLState *state) object_property_set_description(obj, "cxl-fmw", "CXL Fixed Memory Windows (array)"); } + +void cxl_hook_up_pxb_registers(PCIBus *bus, CXLState *state, Error **errp) +{ + /* Walk the pci busses looking for pxb busses to hook up */ + if (bus) { + QLIST_FOREACH(bus, &bus->child, sibling) { + if (!pci_bus_is_root(bus)) { + continue; + } + if (pci_bus_is_cxl(bus)) { + if (!state->is_enabled) { + error_setg(errp, "CXL host bridges present, but cxl=3D= off"); + return; + } + pxb_cxl_hook_up_registers(state, bus, errp); + } + } + } +} diff --git a/hw/i386/pc.c b/hw/i386/pc.c index 6cecd74d58..9f48d02739 100644 --- a/hw/i386/pc.c +++ b/hw/i386/pc.c @@ -37,6 +37,7 @@ #include "hw/ide.h" #include "hw/pci/pci.h" #include "hw/pci/pci_bus.h" +#include "hw/pci-bridge/pci_expander_bridge.h" #include "hw/nvram/fw_cfg.h" #include "hw/timer/hpet.h" #include "hw/firmware/smbios.h" @@ -735,6 +736,8 @@ void pc_machine_done(Notifier *notifier, void *data) MachineState *ms =3D MACHINE(pcms); =20 if (ms->cxl_devices_state) { + cxl_hook_up_pxb_registers(pcms->bus, ms->cxl_devices_state, + &error_fatal); cxl_fmws_link_targets(ms->cxl_devices_state, &error_fatal); } =20 diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expand= er_bridge.c index 02032360f5..c9e817aa58 100644 --- a/hw/pci-bridge/pci_expander_bridge.c +++ b/hw/pci-bridge/pci_expander_bridge.c @@ -17,6 +17,7 @@ #include "hw/pci/pci_host.h" #include "hw/qdev-properties.h" #include "hw/pci/pci_bridge.h" +#include "hw/pci-bridge/pci_expander_bridge.h" #include "hw/cxl/cxl.h" #include "qemu/range.h" #include "qemu/error-report.h" @@ -186,25 +187,38 @@ static const TypeInfo pxb_host_info =3D { =20 static void pxb_cxl_realize(DeviceState *dev, Error **errp) { - MachineState *ms =3D MACHINE(qdev_get_machine()); SysBusDevice *sbd =3D SYS_BUS_DEVICE(dev); CXLHost *cxl =3D PXB_CXL_HOST(dev); CXLComponentState *cxl_cstate =3D &cxl->cxl_cstate; struct MemoryRegion *mr =3D &cxl_cstate->crb.component_registers; - hwaddr offset; =20 cxl_component_register_block_init(OBJECT(dev), cxl_cstate, TYPE_PXB_CXL_HOST); sysbus_init_mmio(sbd, mr); +} =20 - offset =3D memory_region_size(mr) * ms->cxl_devices_state->next_mr_idx; - if (offset > memory_region_size(&ms->cxl_devices_state->host_mr)) { +/* + * Host bridge realization has no means of knowning state associated + * with a particular machine. As such, it is nececssary to delay + * final setup of the host bridge register space until later in the + * machine bring up. + */ +void pxb_cxl_hook_up_registers(CXLState *cxl_state, PCIBus *bus, Error **e= rrp) +{ + PXBDev *pxb =3D PXB_CXL_DEV(pci_bridge_get_device(bus)); + CXLHost *cxl =3D pxb->cxl.cxl_host_bridge; + CXLComponentState *cxl_cstate =3D &cxl->cxl_cstate; + struct MemoryRegion *mr =3D &cxl_cstate->crb.component_registers; + hwaddr offset; + + offset =3D memory_region_size(mr) * cxl_state->next_mr_idx; + if (offset > memory_region_size(&cxl_state->host_mr)) { error_setg(errp, "Insufficient space for pxb cxl host register spa= ce"); return; } =20 - memory_region_add_subregion(&ms->cxl_devices_state->host_mr, offset, m= r); - ms->cxl_devices_state->next_mr_idx++; + memory_region_add_subregion(&cxl_state->host_mr, offset, mr); + cxl_state->next_mr_idx++; } =20 static void pxb_cxl_host_class_init(ObjectClass *class, void *data) @@ -461,17 +475,11 @@ static const TypeInfo pxb_pcie_dev_info =3D { =20 static void pxb_cxl_dev_realize(PCIDevice *dev, Error **errp) { - MachineState *ms =3D MACHINE(qdev_get_machine()); - /* A CXL PXB's parent bus is still PCIe */ if (!pci_bus_is_express(pci_get_bus(dev))) { error_setg(errp, "pxb-cxl devices cannot reside on a PCI bus"); return; } - if (!ms->cxl_devices_state || !ms->cxl_devices_state->is_enabled) { - error_setg(errp, "Machine does not have cxl=3Don"); - return; - } =20 pxb_dev_realize_common(dev, CXL, errp); pxb_dev_reset(DEVICE(dev)); diff --git a/hw/pci-bridge/pci_expander_bridge_stubs.c b/hw/pci-bridge/pci_= expander_bridge_stubs.c new file mode 100644 index 0000000000..b35180311f --- /dev/null +++ b/hw/pci-bridge/pci_expander_bridge_stubs.c @@ -0,0 +1,14 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * + * Stubs for calls made from machines to handle the case where CONFIG_PXB + * is not enabled. + */ +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "hw/pci/pci.h" +#include "hw/pci/pci_bus.h" +#include "hw/pci-bridge/pci_expander_bridge.h" +#include "hw/cxl/cxl.h" + +void pxb_cxl_hook_up_registers(CXLState *state, PCIBus *bus, Error **errp)= {}; diff --git a/hw/pci-bridge/meson.build b/hw/pci-bridge/meson.build index b6d26a03d5..fdbe2e07c5 100644 --- a/hw/pci-bridge/meson.build +++ b/hw/pci-bridge/meson.build @@ -3,7 +3,8 @@ pci_ss.add(files('pci_bridge_dev.c')) pci_ss.add(when: 'CONFIG_I82801B11', if_true: files('i82801b11.c')) pci_ss.add(when: 'CONFIG_IOH3420', if_true: files('ioh3420.c')) pci_ss.add(when: 'CONFIG_PCIE_PORT', if_true: files('pcie_root_port.c', 'g= en_pcie_root_port.c', 'pcie_pci_bridge.c')) -pci_ss.add(when: 'CONFIG_PXB', if_true: files('pci_expander_bridge.c')) +pci_ss.add(when: 'CONFIG_PXB', if_true: files('pci_expander_bridge.c'), + if_false: files('pci_expander_bridge_stubs.= c')) pci_ss.add(when: 'CONFIG_XIO3130', if_true: files('xio3130_upstream.c', 'x= io3130_downstream.c')) pci_ss.add(when: 'CONFIG_CXL', if_true: files('cxl_root_port.c')) =20 @@ -13,3 +14,5 @@ pci_ss.add(when: 'CONFIG_DEC_PCI', if_true: files('dec.c'= )) pci_ss.add(when: 'CONFIG_SIMBA', if_true: files('simba.c')) =20 softmmu_ss.add_all(when: 'CONFIG_PCI', if_true: pci_ss) + +softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('pci_expander_bridge_stu= bs.c')) --=20 MST