From nobody Sun Feb 8 17:21:15 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1655399131; cv=none; d=zohomail.com; s=zohoarc; b=DIKmEdJzPxlKgT/eKZWpFZ0xi1RN1OS3/qyiXjbcCMZ3BO5Bq/Osm/L9Aqw+6tWaUZT3jA7HR04Zy1SXgM0fYWGOi5um24iujPyQiXVZ9SR3+WUr3hTTULJH2l2hy5mOCT1XV2V6ckisEIhMqev2WBAB/fxbGKkEFZ56olZOmt8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1655399131; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=gIfdp2s6V5m9i3e4LI4cBUyn2030MIHw1oZvWnvusE8=; b=gX7KvuERmapbtQC94RsODhFzGYFt3HOp1f6zQFAIRvZRHqBZAU1H6aXTbHNifzwuQhP7I2d7zMIZydzrdVF078CAfpI6tWggRlJ0M8qniY1TZw+9P90/fkLVYNUz+SHZk6kwde0P03RBoRsXwruFn53Vu6GWgR0zEarGFQWLM80= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1655399131329495.8488742615102; Thu, 16 Jun 2022 10:05:31 -0700 (PDT) Received: from localhost ([::1]:55336 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o1svx-0004d6-Jy for importer@patchew.org; Thu, 16 Jun 2022 13:05:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47102) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o1soK-0004Se-Ct for qemu-devel@nongnu.org; Thu, 16 Jun 2022 12:57:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:37985) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o1soH-0001vb-GW for qemu-devel@nongnu.org; Thu, 16 Jun 2022 12:57:35 -0400 Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-661-EhvTj9S0Mo2iDOXQeOYITw-1; Thu, 16 Jun 2022 12:57:31 -0400 Received: by mail-wm1-f71.google.com with SMTP id k15-20020a7bc40f000000b0039c4b7f7d09so1129648wmi.8 for ; Thu, 16 Jun 2022 09:57:31 -0700 (PDT) Received: from redhat.com ([2.53.13.204]) by smtp.gmail.com with ESMTPSA id z16-20020adff1d0000000b0020c5253d927sm2289098wro.115.2022.06.16.09.57.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Jun 2022 09:57:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1655398652; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gIfdp2s6V5m9i3e4LI4cBUyn2030MIHw1oZvWnvusE8=; b=GTjtZ75PEHms78LwTezdJW/3gPnjEEXUn+quc4jScEZ0XZOpx6+LqPDz423+neMWX4MkrO zZD8Q0jGc2/xL+hSzJ4iArJtP/SjeR8oH5eu71zK+sc+rlargg5gGT70VeyNZ1JreBlwQt wXhrvw6/PBF200DGllZKL8bdVBPPXd8= X-MC-Unique: EhvTj9S0Mo2iDOXQeOYITw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=gIfdp2s6V5m9i3e4LI4cBUyn2030MIHw1oZvWnvusE8=; b=jtv6V8d8tw6kZrBcfBboTun28r0ZeewoHQiM4NY2YuYu71/YB0bPizb1LjAMYYCBvO LivLwtQRrD0vERSvJg9rtbjGNZZjpIEMsM4l4m1K+dfRBlAdjSsvIJOw6hXSrcbKi3c5 go0pgGous+xLF3ZRSrDbnTf2u+H/3JxVkXV0cjr97E1YLWz27zVmbPiVZGl+7eYRiUQU 9AYbjczSMHKtE4GST1TiJsKPuhTXP/oMeIfyiEcmLkaBBoi1LFAmH8NXtZLy8cSgSjaK VNL33getT2CW+OgTDsFzsg7PfSaKrIxtW82HSCE5KwJ3tcChv+2H/v5TzZ6KoE77bYpK Ez0Q== X-Gm-Message-State: AJIora+jcM42ssDF3Op+BEbZ57yH50oOJxXE0E4ASs9iV3XVfo5B+/Fx /TXHj10evV3okDHAzDkf1qzA5jg4s7KOeOGF7suKFs254vthPI3+6MGU0SImknYM7fM9/qzPgZw zpc0Ya09+hud2lQLBKbX1GrklOEzODC5RDyZiwSBvmk/6im7i9UPHqTUIQw1p X-Received: by 2002:a05:6000:156d:b0:210:3135:ce1c with SMTP id 13-20020a056000156d00b002103135ce1cmr5573980wrz.409.1655398650186; Thu, 16 Jun 2022 09:57:30 -0700 (PDT) X-Google-Smtp-Source: AGRyM1thVxxsPXhkkEqvcy5epgooOpX9Y0prTcpU6nRqib9YOWfe8SVJ23TL1j74WvLUeqsecP+UXQ== X-Received: by 2002:a05:6000:156d:b0:210:3135:ce1c with SMTP id 13-20020a056000156d00b002103135ce1cmr5573954wrz.409.1655398649879; Thu, 16 Jun 2022 09:57:29 -0700 (PDT) Date: Thu, 16 Jun 2022 12:57:26 -0400 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Jonathan Cameron , Ben Widawsky , Marcel Apfelbaum Subject: [PULL 02/10] pci-bridge/cxl_downstream: Add a CXL switch downstream port Message-ID: <20220616165703.42226-3-mst@redhat.com> References: <20220616165703.42226-1-mst@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: <20220616165703.42226-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1655399132434100001 From: Jonathan Cameron Emulation of a simple CXL Switch downstream port. The Device ID has been allocated for this use. Signed-off-by: Jonathan Cameron Message-Id: <20220616145126.8002-3-Jonathan.Cameron@huawei.com> Signed-off-by: Michael S. Tsirkin Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- hw/cxl/cxl-host.c | 43 +++++- hw/pci-bridge/cxl_downstream.c | 249 +++++++++++++++++++++++++++++++++ hw/pci-bridge/meson.build | 2 +- 3 files changed, 291 insertions(+), 3 deletions(-) create mode 100644 hw/pci-bridge/cxl_downstream.c diff --git a/hw/cxl/cxl-host.c b/hw/cxl/cxl-host.c index efa14908d8..483d8eb13f 100644 --- a/hw/cxl/cxl-host.c +++ b/hw/cxl/cxl-host.c @@ -129,8 +129,9 @@ static bool cxl_hdm_find_target(uint32_t *cache_mem, hw= addr addr, =20 static PCIDevice *cxl_cfmws_find_device(CXLFixedWindow *fw, hwaddr addr) { - CXLComponentState *hb_cstate; + CXLComponentState *hb_cstate, *usp_cstate; PCIHostState *hb; + CXLUpstreamPort *usp; int rb_index; uint32_t *cache_mem; uint8_t target; @@ -164,8 +165,46 @@ static PCIDevice *cxl_cfmws_find_device(CXLFixedWindow= *fw, hwaddr addr) } =20 d =3D pci_bridge_get_sec_bus(PCI_BRIDGE(rp))->devices[0]; + if (!d) { + return NULL; + } =20 - if (!d || !object_dynamic_cast(OBJECT(d), TYPE_CXL_TYPE3)) { + if (object_dynamic_cast(OBJECT(d), TYPE_CXL_TYPE3)) { + return d; + } + + /* + * Could also be a switch. Note only one level of switching currently + * supported. + */ + if (!object_dynamic_cast(OBJECT(d), TYPE_CXL_USP)) { + return NULL; + } + usp =3D CXL_USP(d); + + usp_cstate =3D cxl_usp_to_cstate(usp); + if (!usp_cstate) { + return NULL; + } + + cache_mem =3D usp_cstate->crb.cache_mem_registers; + + target_found =3D cxl_hdm_find_target(cache_mem, addr, &target); + if (!target_found) { + return NULL; + } + + d =3D pcie_find_port_by_pn(&PCI_BRIDGE(d)->sec_bus, target); + if (!d) { + return NULL; + } + + d =3D pci_bridge_get_sec_bus(PCI_BRIDGE(d))->devices[0]; + if (!d) { + return NULL; + } + + if (!object_dynamic_cast(OBJECT(d), TYPE_CXL_TYPE3)) { return NULL; } =20 diff --git a/hw/pci-bridge/cxl_downstream.c b/hw/pci-bridge/cxl_downstream.c new file mode 100644 index 0000000000..a361e519d0 --- /dev/null +++ b/hw/pci-bridge/cxl_downstream.c @@ -0,0 +1,249 @@ +/* + * Emulated CXL Switch Downstream Port + * + * Copyright (c) 2022 Huawei Technologies. + * + * Based on xio3130_downstream.c + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "qemu/log.h" +#include "hw/pci/msi.h" +#include "hw/pci/pcie.h" +#include "hw/pci/pcie_port.h" +#include "qapi/error.h" + +typedef struct CXLDownStreamPort { + /*< private >*/ + PCIESlot parent_obj; + + /*< public >*/ + CXLComponentState cxl_cstate; +} CXLDownstreamPort; + +#define TYPE_CXL_DSP "cxl-downstream" +DECLARE_INSTANCE_CHECKER(CXLDownstreamPort, CXL_DSP, TYPE_CXL_DSP) + +#define CXL_DOWNSTREAM_PORT_MSI_OFFSET 0x70 +#define CXL_DOWNSTREAM_PORT_MSI_NR_VECTOR 1 +#define CXL_DOWNSTREAM_PORT_EXP_OFFSET 0x90 +#define CXL_DOWNSTREAM_PORT_AER_OFFSET 0x100 +#define CXL_DOWNSTREAM_PORT_DVSEC_OFFSET \ + (CXL_DOWNSTREAM_PORT_AER_OFFSET + PCI_ERR_SIZEOF) + +static void latch_registers(CXLDownstreamPort *dsp) +{ + uint32_t *reg_state =3D dsp->cxl_cstate.crb.cache_mem_registers; + uint32_t *write_msk =3D dsp->cxl_cstate.crb.cache_mem_regs_write_mask; + + cxl_component_register_init_common(reg_state, write_msk, + CXL2_DOWNSTREAM_PORT); +} + +/* TODO: Look at sharing this code acorss all CXL port types */ +static void cxl_dsp_dvsec_write_config(PCIDevice *dev, uint32_t addr, + uint32_t val, int len) +{ + CXLDownstreamPort *dsp =3D CXL_DSP(dev); + CXLComponentState *cxl_cstate =3D &dsp->cxl_cstate; + + if (range_contains(&cxl_cstate->dvsecs[EXTENSIONS_PORT_DVSEC], addr)) { + uint8_t *reg =3D &dev->config[addr]; + addr -=3D cxl_cstate->dvsecs[EXTENSIONS_PORT_DVSEC].lob; + if (addr =3D=3D PORT_CONTROL_OFFSET) { + if (pci_get_word(reg) & PORT_CONTROL_UNMASK_SBR) { + /* unmask SBR */ + qemu_log_mask(LOG_UNIMP, "SBR mask control is not supporte= d\n"); + } + if (pci_get_word(reg) & PORT_CONTROL_ALT_MEMID_EN) { + /* Alt Memory & ID Space Enable */ + qemu_log_mask(LOG_UNIMP, + "Alt Memory & ID space is not supported\n"); + + } + } + } +} + +static void cxl_dsp_config_write(PCIDevice *d, uint32_t address, + uint32_t val, int len) +{ + uint16_t slt_ctl, slt_sta; + + pcie_cap_slot_get(d, &slt_ctl, &slt_sta); + pci_bridge_write_config(d, address, val, len); + pcie_cap_flr_write_config(d, address, val, len); + pcie_cap_slot_write_config(d, slt_ctl, slt_sta, address, val, len); + pcie_aer_write_config(d, address, val, len); + + cxl_dsp_dvsec_write_config(d, address, val, len); +} + +static void cxl_dsp_reset(DeviceState *qdev) +{ + PCIDevice *d =3D PCI_DEVICE(qdev); + CXLDownstreamPort *dsp =3D CXL_DSP(qdev); + + pcie_cap_deverr_reset(d); + pcie_cap_slot_reset(d); + pcie_cap_arifwd_reset(d); + pci_bridge_reset(qdev); + + latch_registers(dsp); +} + +static void build_dvsecs(CXLComponentState *cxl) +{ + uint8_t *dvsec; + + dvsec =3D (uint8_t *)&(CXLDVSECPortExtensions){ 0 }; + cxl_component_create_dvsec(cxl, CXL2_DOWNSTREAM_PORT, + EXTENSIONS_PORT_DVSEC_LENGTH, + EXTENSIONS_PORT_DVSEC, + EXTENSIONS_PORT_DVSEC_REVID, dvsec); + + dvsec =3D (uint8_t *)&(CXLDVSECPortFlexBus){ + .cap =3D 0x27, /* Cache, IO, Mem, non-MLD */ + .ctrl =3D 0x02, /* IO always enabled */ + .status =3D 0x26, /* same */ + .rcvd_mod_ts_data_phase1 =3D 0xef, /* WTF? */ + }; + cxl_component_create_dvsec(cxl, CXL2_DOWNSTREAM_PORT, + PCIE_FLEXBUS_PORT_DVSEC_LENGTH_2_0, + PCIE_FLEXBUS_PORT_DVSEC, + PCIE_FLEXBUS_PORT_DVSEC_REVID_2_0, dvsec); + + dvsec =3D (uint8_t *)&(CXLDVSECPortGPF){ + .rsvd =3D 0, + .phase1_ctrl =3D 1, /* 1=CE=BCs timeout */ + .phase2_ctrl =3D 1, /* 1=CE=BCs timeout */ + }; + cxl_component_create_dvsec(cxl, CXL2_DOWNSTREAM_PORT, + GPF_PORT_DVSEC_LENGTH, GPF_PORT_DVSEC, + GPF_PORT_DVSEC_REVID, dvsec); + + dvsec =3D (uint8_t *)&(CXLDVSECRegisterLocator){ + .rsvd =3D 0, + .reg0_base_lo =3D RBI_COMPONENT_REG | CXL_COMPONENT_REG_BAR_IDX, + .reg0_base_hi =3D 0, + }; + cxl_component_create_dvsec(cxl, CXL2_DOWNSTREAM_PORT, + REG_LOC_DVSEC_LENGTH, REG_LOC_DVSEC, + REG_LOC_DVSEC_REVID, dvsec); +} + +static void cxl_dsp_realize(PCIDevice *d, Error **errp) +{ + PCIEPort *p =3D PCIE_PORT(d); + PCIESlot *s =3D PCIE_SLOT(d); + CXLDownstreamPort *dsp =3D CXL_DSP(d); + CXLComponentState *cxl_cstate =3D &dsp->cxl_cstate; + ComponentRegisters *cregs =3D &cxl_cstate->crb; + MemoryRegion *component_bar =3D &cregs->component_registers; + int rc; + + pci_bridge_initfn(d, TYPE_PCIE_BUS); + pcie_port_init_reg(d); + + rc =3D msi_init(d, CXL_DOWNSTREAM_PORT_MSI_OFFSET, + CXL_DOWNSTREAM_PORT_MSI_NR_VECTOR, + true, true, errp); + if (rc) { + assert(rc =3D=3D -ENOTSUP); + goto err_bridge; + } + + rc =3D pcie_cap_init(d, CXL_DOWNSTREAM_PORT_EXP_OFFSET, + PCI_EXP_TYPE_DOWNSTREAM, p->port, + errp); + if (rc < 0) { + goto err_msi; + } + + pcie_cap_flr_init(d); + pcie_cap_deverr_init(d); + pcie_cap_slot_init(d, s); + pcie_cap_arifwd_init(d); + + pcie_chassis_create(s->chassis); + rc =3D pcie_chassis_add_slot(s); + if (rc < 0) { + error_setg(errp, "Can't add chassis slot, error %d", rc); + goto err_pcie_cap; + } + + rc =3D pcie_aer_init(d, PCI_ERR_VER, CXL_DOWNSTREAM_PORT_AER_OFFSET, + PCI_ERR_SIZEOF, errp); + if (rc < 0) { + goto err_chassis; + } + + cxl_cstate->dvsec_offset =3D CXL_DOWNSTREAM_PORT_DVSEC_OFFSET; + cxl_cstate->pdev =3D d; + build_dvsecs(cxl_cstate); + cxl_component_register_block_init(OBJECT(d), cxl_cstate, TYPE_CXL_DSP); + pci_register_bar(d, CXL_COMPONENT_REG_BAR_IDX, + PCI_BASE_ADDRESS_SPACE_MEMORY | + PCI_BASE_ADDRESS_MEM_TYPE_64, + component_bar); + + return; + + err_chassis: + pcie_chassis_del_slot(s); + err_pcie_cap: + pcie_cap_exit(d); + err_msi: + msi_uninit(d); + err_bridge: + pci_bridge_exitfn(d); +} + +static void cxl_dsp_exitfn(PCIDevice *d) +{ + PCIESlot *s =3D PCIE_SLOT(d); + + pcie_aer_exit(d); + pcie_chassis_del_slot(s); + pcie_cap_exit(d); + msi_uninit(d); + pci_bridge_exitfn(d); +} + +static void cxl_dsp_class_init(ObjectClass *oc, void *data) +{ + DeviceClass *dc =3D DEVICE_CLASS(oc); + PCIDeviceClass *k =3D PCI_DEVICE_CLASS(oc); + + k->is_bridge =3D true; + k->config_write =3D cxl_dsp_config_write; + k->realize =3D cxl_dsp_realize; + k->exit =3D cxl_dsp_exitfn; + k->vendor_id =3D 0x19e5; /* Huawei */ + k->device_id =3D 0xa129; /* Emulated CXL Switch Downstream Port */ + k->revision =3D 0; + set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories); + dc->desc =3D "CXL Switch Downstream Port"; + dc->reset =3D cxl_dsp_reset; +} + +static const TypeInfo cxl_dsp_info =3D { + .name =3D TYPE_CXL_DSP, + .instance_size =3D sizeof(CXLDownstreamPort), + .parent =3D TYPE_PCIE_SLOT, + .class_init =3D cxl_dsp_class_init, + .interfaces =3D (InterfaceInfo[]) { + { INTERFACE_PCIE_DEVICE }, + { INTERFACE_CXL_DEVICE }, + { } + }, +}; + +static void cxl_dsp_register_type(void) +{ + type_register_static(&cxl_dsp_info); +} + +type_init(cxl_dsp_register_type); diff --git a/hw/pci-bridge/meson.build b/hw/pci-bridge/meson.build index 6828f0e08d..243ceeda50 100644 --- a/hw/pci-bridge/meson.build +++ b/hw/pci-bridge/meson.build @@ -6,7 +6,7 @@ pci_ss.add(when: 'CONFIG_PCIE_PORT', if_true: files('pcie_r= oot_port.c', 'gen_pci pci_ss.add(when: 'CONFIG_PXB', if_true: files('pci_expander_bridge.c'), if_false: files('pci_expander_bridge_stubs.= c')) pci_ss.add(when: 'CONFIG_XIO3130', if_true: files('xio3130_upstream.c', 'x= io3130_downstream.c')) -pci_ss.add(when: 'CONFIG_CXL', if_true: files('cxl_root_port.c', 'cxl_upst= ream.c')) +pci_ss.add(when: 'CONFIG_CXL', if_true: files('cxl_root_port.c', 'cxl_upst= ream.c', 'cxl_downstream.c')) =20 # NewWorld PowerMac pci_ss.add(when: 'CONFIG_DEC_PCI', if_true: files('dec.c')) --=20 MST