From nobody Sun Apr 12 04:23:09 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1772042896; cv=none; d=zohomail.com; s=zohoarc; b=S/QtyBpdMGnJ6G1mwV2/2KV7QX15/rwzfZyxrfX3kQ4Lxkyn4HvUFLGH7Tl8UhCDeSTBiTY5lyxedhzJi1Dtn6SGoSWqSP+1YT93H7sdFmBU8DFmPY6lObbigHDFrSO/S+e9rK6XG6t/51aAqHMr8N7O4riNCkrEwD71ULL3NZ4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1772042896; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Sender:Subject:Subject:To:To:Message-Id; bh=4L1zmjVLftBPG2oB91HgTaPG2Gx6YO+zvxqr74Qu+R8=; b=GL0OPCrL3pkGyl5B7dgGFC21buCftwc83SNKMVEcmAoz6NAPbaWjQI4+QSWJdCQ+aoYblT3i4NEAzCEtdAko0QOdtXwsYEsSG4knHw4Nr2VPgTZcpnKGcOGqnRYSnD4l73pp2znIbI4IH5IoYSYfDWJgo5FKmOKb8h1faEK3ABo= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1772042896583904.1066414704916; Wed, 25 Feb 2026 10:08:16 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vvJIn-0000i3-Qa; Wed, 25 Feb 2026 13:08:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vvJIm-0000hI-0g for qemu-devel@nongnu.org; Wed, 25 Feb 2026 13:08:00 -0500 Received: from frasgout.his.huawei.com ([185.176.79.56]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vvJIk-0001w1-GE for qemu-devel@nongnu.org; Wed, 25 Feb 2026 13:07:59 -0500 Received: from mail.maildlp.com (unknown [172.18.224.83]) by frasgout.his.huawei.com (SkyGuard) with ESMTPS id 4fLjH71JByzJ467g; Thu, 26 Feb 2026 02:07:31 +0800 (CST) Received: from dubpeml500005.china.huawei.com (unknown [7.214.145.207]) by mail.maildlp.com (Postfix) with ESMTPS id BE20940086; Thu, 26 Feb 2026 02:07:54 +0800 (CST) Received: from a2303103017.china.huawei.com (10.203.177.99) by dubpeml500005.china.huawei.com (7.214.145.207) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Wed, 25 Feb 2026 18:07:53 +0000 To: CC: , , , , , , , , , , , , , , Subject: [PATCH v5 1/3] hw/cxl: Use HPA in cxl_cfmws_find_device() rather than offset in window. Date: Wed, 25 Feb 2026 18:07:16 +0000 Message-ID: <20260225180718.480-2-alireza.sanaee@huawei.com> X-Mailer: git-send-email 2.51.0.windows.2 In-Reply-To: <20260225180718.480-1-alireza.sanaee@huawei.com> References: <20260225180718.480-1-alireza.sanaee@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.203.177.99] X-ClientProxiedBy: lhrpeml500012.china.huawei.com (7.191.174.4) To dubpeml500005.china.huawei.com (7.214.145.207) Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.176.79.56; envelope-from=alireza.sanaee@huawei.com; helo=frasgout.his.huawei.com X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.734, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.78, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Alireza Sanaee From: Alireza Sanaee via qemu development Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1772042898917158500 Content-Type: text/plain; charset="utf-8" This function will shortly be used to help find if there is a route to a device, serving an HPA, under a particular fixed memory window. Rather than having that new use case subtract the base address in the caller, only to add it again in cxl_cfmws_find_device(), push the responsibility for calculating the HPA to the caller. This also reduces the inconsistency in the meaning of the hwaddr addr parameter between this function and the calls made within it that access the HDM decoders that operating on HPA. Reviewed-by: Jonathan Cameron Signed-off-by: Alireza Sanaee --- hw/cxl/cxl-host.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/hw/cxl/cxl-host.c b/hw/cxl/cxl-host.c index f3479b1991..a94b893e99 100644 --- a/hw/cxl/cxl-host.c +++ b/hw/cxl/cxl-host.c @@ -168,9 +168,6 @@ static PCIDevice *cxl_cfmws_find_device(CXLFixedWindow = *fw, hwaddr addr) bool target_found; PCIDevice *rp, *d; =20 - /* Address is relative to memory region. Convert to HPA */ - addr +=3D fw->base; - rb_index =3D (addr / cxl_decode_ig(fw->enc_int_gran)) % fw->num_target= s; hb =3D PCI_HOST_BRIDGE(fw->target_hbs[rb_index]->cxl_host_bridge); if (!hb || !hb->bus || !pci_bus_is_cxl(hb->bus)) { @@ -254,7 +251,7 @@ static MemTxResult cxl_read_cfmws(void *opaque, hwaddr = addr, uint64_t *data, CXLFixedWindow *fw =3D opaque; PCIDevice *d; =20 - d =3D cxl_cfmws_find_device(fw, addr); + d =3D cxl_cfmws_find_device(fw, addr + fw->base); if (d =3D=3D NULL) { *data =3D 0; /* Reads to invalid address return poison */ @@ -271,7 +268,7 @@ static MemTxResult cxl_write_cfmws(void *opaque, hwaddr= addr, CXLFixedWindow *fw =3D opaque; PCIDevice *d; =20 - d =3D cxl_cfmws_find_device(fw, addr); + d =3D cxl_cfmws_find_device(fw, addr + fw->base); if (d =3D=3D NULL) { /* Writes to invalid address are silent */ return MEMTX_OK; --=20 2.43.0 From nobody Sun Apr 12 04:23:09 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1772042934; cv=none; d=zohomail.com; s=zohoarc; b=nmDx5Sl52ZHHCYTjJ9Jq5+joXcZeVYQjSUUEUNNTbz1QCsc7r2cg7IO4GI0HicoE2vi7eLT3vuQAkHY0q3/lBWgBC9G4zgwthy10lRPSyJlOo2I3M0NDJW72NTZKa25PwtNg4aYNPw/NNcCjFvnUk6siLSyuXn+ig42BmZ0q1o0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1772042934; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Sender:Subject:Subject:To:To:Message-Id; bh=YY4x2ySvs/DGAMnNUPQWLoua2QVKt6ySIbhUaU8ZuK0=; b=mf9g6qljP0jWYInCnFaoAHB0IATh/DXS5Vsg4Tl1mBJCL6ck0UP2Imxdpq/G7IgMV3yXkHw90ehpatyvsmTZnfFBorpVDPLE+8AVxJkqYoZ40fDUnQOxJk34O5/x0LyzEdLsxzQjUFPObWh1tN3X71xywiaR9sqUIADZmDfUfDo= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1772042934707588.8651245872858; Wed, 25 Feb 2026 10:08:54 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vvJJJ-0000zU-1L; Wed, 25 Feb 2026 13:08:33 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vvJJG-0000tq-8D for qemu-devel@nongnu.org; Wed, 25 Feb 2026 13:08:31 -0500 Received: from frasgout.his.huawei.com ([185.176.79.56]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vvJJE-0001yN-JL for qemu-devel@nongnu.org; Wed, 25 Feb 2026 13:08:30 -0500 Received: from mail.maildlp.com (unknown [172.18.224.107]) by frasgout.his.huawei.com (SkyGuard) with ESMTPS id 4fLjHl3c0MzJ468g; Thu, 26 Feb 2026 02:08:03 +0800 (CST) Received: from dubpeml500005.china.huawei.com (unknown [7.214.145.207]) by mail.maildlp.com (Postfix) with ESMTPS id 18D8D40570; Thu, 26 Feb 2026 02:08:27 +0800 (CST) Received: from a2303103017.china.huawei.com (10.203.177.99) by dubpeml500005.china.huawei.com (7.214.145.207) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Wed, 25 Feb 2026 18:08:26 +0000 To: CC: , , , , , , , , , , , , , , Subject: [PATCH v5 2/3] hw/cxl: Allow cxl_cfmws_find_device() to filter on whether interleaved paths are accepted Date: Wed, 25 Feb 2026 18:07:17 +0000 Message-ID: <20260225180718.480-3-alireza.sanaee@huawei.com> X-Mailer: git-send-email 2.51.0.windows.2 In-Reply-To: <20260225180718.480-1-alireza.sanaee@huawei.com> References: <20260225180718.480-1-alireza.sanaee@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.203.177.99] X-ClientProxiedBy: lhrpeml500012.china.huawei.com (7.191.174.4) To dubpeml500005.china.huawei.com (7.214.145.207) Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.176.79.56; envelope-from=alireza.sanaee@huawei.com; helo=frasgout.his.huawei.com X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.734, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.78, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Alireza Sanaee From: Alireza Sanaee via qemu development Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1772042937102158500 Content-Type: text/plain; charset="utf-8" Extend cxl_cfmws_find_device() with a parameter that filters on whether the address lies in an interleaved range. For now all callers accept interleave configurations so no functional changes. Signed-off-by: Alireza Sanaee --- hw/cxl/cxl-host.c | 33 ++++++++++++++++++++++++++------- 1 file changed, 26 insertions(+), 7 deletions(-) diff --git a/hw/cxl/cxl-host.c b/hw/cxl/cxl-host.c index a94b893e99..2dc9f77007 100644 --- a/hw/cxl/cxl-host.c +++ b/hw/cxl/cxl-host.c @@ -104,7 +104,7 @@ void cxl_fmws_link_targets(Error **errp) } =20 static bool cxl_hdm_find_target(uint32_t *cache_mem, hwaddr addr, - uint8_t *target) + uint8_t *target, bool *interleaved) { int hdm_inc =3D R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_L= O; unsigned int hdm_count; @@ -138,6 +138,11 @@ static bool cxl_hdm_find_target(uint32_t *cache_mem, h= waddr addr, found =3D true; ig_enc =3D FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, IG); iw_enc =3D FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, IW); + + if (interleaved) { + *interleaved =3D iw_enc !=3D 0; + } + target_idx =3D (addr / cxl_decode_ig(ig_enc)) % (1 << iw_enc); =20 if (target_idx < 4) { @@ -157,7 +162,8 @@ static bool cxl_hdm_find_target(uint32_t *cache_mem, hw= addr addr, return found; } =20 -static PCIDevice *cxl_cfmws_find_device(CXLFixedWindow *fw, hwaddr addr) +static PCIDevice *cxl_cfmws_find_device(CXLFixedWindow *fw, hwaddr addr, + bool allow_interleave) { CXLComponentState *hb_cstate, *usp_cstate; PCIHostState *hb; @@ -165,9 +171,13 @@ static PCIDevice *cxl_cfmws_find_device(CXLFixedWindow= *fw, hwaddr addr) int rb_index; uint32_t *cache_mem; uint8_t target; - bool target_found; + bool target_found, interleaved; PCIDevice *rp, *d; =20 + if ((fw->num_targets > 1) && !allow_interleave) { + return NULL; + } + rb_index =3D (addr / cxl_decode_ig(fw->enc_int_gran)) % fw->num_target= s; hb =3D PCI_HOST_BRIDGE(fw->target_hbs[rb_index]->cxl_host_bridge); if (!hb || !hb->bus || !pci_bus_is_cxl(hb->bus)) { @@ -187,11 +197,16 @@ static PCIDevice *cxl_cfmws_find_device(CXLFixedWindo= w *fw, hwaddr addr) =20 cache_mem =3D hb_cstate->crb.cache_mem_registers; =20 - target_found =3D cxl_hdm_find_target(cache_mem, addr, &target); + target_found =3D cxl_hdm_find_target(cache_mem, addr, &target, + &interleaved); if (!target_found) { return NULL; } =20 + if (interleaved && !allow_interleave) { + return NULL; + } + rp =3D pcie_find_port_by_pn(hb->bus, target); if (!rp) { return NULL; @@ -223,11 +238,15 @@ static PCIDevice *cxl_cfmws_find_device(CXLFixedWindo= w *fw, hwaddr addr) =20 cache_mem =3D usp_cstate->crb.cache_mem_registers; =20 - target_found =3D cxl_hdm_find_target(cache_mem, addr, &target); + target_found =3D cxl_hdm_find_target(cache_mem, addr, &target, &interl= eaved); if (!target_found) { return NULL; } =20 + if (interleaved && !allow_interleave) { + return NULL; + } + d =3D pcie_find_port_by_pn(&PCI_BRIDGE(d)->sec_bus, target); if (!d) { return NULL; @@ -251,7 +270,7 @@ static MemTxResult cxl_read_cfmws(void *opaque, hwaddr = addr, uint64_t *data, CXLFixedWindow *fw =3D opaque; PCIDevice *d; =20 - d =3D cxl_cfmws_find_device(fw, addr + fw->base); + d =3D cxl_cfmws_find_device(fw, addr + fw->base, true); if (d =3D=3D NULL) { *data =3D 0; /* Reads to invalid address return poison */ @@ -268,7 +287,7 @@ static MemTxResult cxl_write_cfmws(void *opaque, hwaddr= addr, CXLFixedWindow *fw =3D opaque; PCIDevice *d; =20 - d =3D cxl_cfmws_find_device(fw, addr + fw->base); + d =3D cxl_cfmws_find_device(fw, addr + fw->base, true); if (d =3D=3D NULL) { /* Writes to invalid address are silent */ return MEMTX_OK; --=20 2.43.0 From nobody Sun Apr 12 04:23:09 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1772042954; cv=none; d=zohomail.com; s=zohoarc; b=CrswfwFJmCVPpfYNGHwMcf15crKKHHsMd1AFdc4XsuSrh0oNfOrXLoncVlAQtsMoFIVAotwMD+uwFDKlkDgSaL8b6ottZeNDCwhwZc9XefvMmPdvXBW+vhsat7MpPzKdJ0MOFwLKg0nfHYgE6D7vU5Knea3rfpkEepo7W5P/0Is= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1772042954; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Sender:Subject:Subject:To:To:Message-Id; bh=M7DiHDo1FBS7EBdPnXWltPZNRzLbXb9d11wM5Jp/lME=; b=Q7bYi4bl3tbOb0S5lh7+3vUcogj6M/fQ+O0y1RZ9hkPf2AKrQhRcEVz4Z1khPlSQHOAc5C5oGGKQiGp2Wq8rJIPViaEEQhLA1MBHIijXE/C259f6usktAw77MV4lNamSPiC5hJSrrYu3KJ0kb0TxADdQYhGV4K5c+R0A75uOXSc= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1772042954550775.6217194818656; Wed, 25 Feb 2026 10:09:14 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vvJJn-0001nF-Ow; Wed, 25 Feb 2026 13:09:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vvJJm-0001n3-RG for qemu-devel@nongnu.org; Wed, 25 Feb 2026 13:09:02 -0500 Received: from frasgout.his.huawei.com ([185.176.79.56]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vvJJk-00020H-O4 for qemu-devel@nongnu.org; Wed, 25 Feb 2026 13:09:02 -0500 Received: from mail.maildlp.com (unknown [172.18.224.83]) by frasgout.his.huawei.com (SkyGuard) with ESMTPS id 4fLjJ10G5FzHnGgd; Thu, 26 Feb 2026 02:08:17 +0800 (CST) Received: from dubpeml500005.china.huawei.com (unknown [7.214.145.207]) by mail.maildlp.com (Postfix) with ESMTPS id 5FB8B40086; Thu, 26 Feb 2026 02:08:59 +0800 (CST) Received: from a2303103017.china.huawei.com (10.203.177.99) by dubpeml500005.china.huawei.com (7.214.145.207) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Wed, 25 Feb 2026 18:08:58 +0000 To: CC: , , , , , , , , , , , , , , Subject: [PATCH v5 3/3] hw/cxl: Add a performant (and correct) path for the non interleaved cases Date: Wed, 25 Feb 2026 18:07:18 +0000 Message-ID: <20260225180718.480-4-alireza.sanaee@huawei.com> X-Mailer: git-send-email 2.51.0.windows.2 In-Reply-To: <20260225180718.480-1-alireza.sanaee@huawei.com> References: <20260225180718.480-1-alireza.sanaee@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.203.177.99] X-ClientProxiedBy: lhrpeml500012.china.huawei.com (7.191.174.4) To dubpeml500005.china.huawei.com (7.214.145.207) Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.176.79.56; envelope-from=alireza.sanaee@huawei.com; helo=frasgout.his.huawei.com X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.734, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.78, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Alireza Sanaee From: Alireza Sanaee via qemu development Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1772042955086158500 Content-Type: text/plain; charset="utf-8" The CXL address to device decoding logic is complex because of the need to correctly decode fine grained interleave. The current implementation prevents use with KVM where executed instructions may reside in that memory and gives very slow performance even in TCG. In many real cases non interleaved memory configurations are useful and for those we can use a more conventional memory region alias allowing similar performance to other memory in the system. Whether this fast path is applicable can be established once the full set of HDM decoders has been committed (in whatever order the guest decides to commit them). As such a check is performed on each commit/uncommit of HDM decoder to establish if the alias should be added or removed. Co-developed-by: Jonathan Cameron Signed-off-by: Jonathan Cameron Signed-off-by: Alireza Sanaee Reported-by: kreview-8842242 --- hw/cxl/cxl-component-utils.c | 6 ++ hw/cxl/cxl-host.c | 189 +++++++++++++++++++++++++++++++++++ hw/mem/cxl_type3.c | 4 + include/hw/cxl/cxl.h | 1 + include/hw/cxl/cxl_device.h | 4 + 5 files changed, 204 insertions(+) diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c index 07aabe331c..a624357978 100644 --- a/hw/cxl/cxl-component-utils.c +++ b/hw/cxl/cxl-component-utils.c @@ -143,6 +143,12 @@ static void dumb_hdm_handler(CXLComponentState *cxl_cs= tate, hwaddr offset, value =3D FIELD_DP32(value, CXL_HDM_DECODER0_CTRL, COMMITTED, 0); } stl_le_p((uint8_t *)cache_mem + offset, value); + + if (should_commit) { + cfmws_update_non_interleaved(true); + } else if (should_uncommit) { + cfmws_update_non_interleaved(false); + } } =20 static void bi_handler(CXLComponentState *cxl_cstate, hwaddr offset, diff --git a/hw/cxl/cxl-host.c b/hw/cxl/cxl-host.c index 2dc9f77007..d51a4dc201 100644 --- a/hw/cxl/cxl-host.c +++ b/hw/cxl/cxl-host.c @@ -264,6 +264,195 @@ static PCIDevice *cxl_cfmws_find_device(CXLFixedWindo= w *fw, hwaddr addr, return d; } =20 +typedef struct CXLDirectPTState { + CXLType3Dev *ct3d; + hwaddr decoder_base; + hwaddr decoder_size; + hwaddr dpa_base; + unsigned int hdm_decoder_idx; +} CXLDirectPTState; + +static void cxl_fmws_direct_passthrough_setup(CXLDirectPTState *state, + CXLFixedWindow *fw) +{ + CXLType3Dev *ct3d =3D state->ct3d; + MemoryRegion *mr =3D NULL; + uint64_t vmr_size =3D 0, pmr_size =3D 0, offset =3D 0; + MemoryRegion *direct_mr; + g_autofree char *direct_mr_name; + unsigned int idx =3D state->hdm_decoder_idx; + + if (ct3d->hostvmem) { + MemoryRegion *vmr =3D host_memory_backend_get_memory(ct3d->hostvme= m); + + vmr_size =3D memory_region_size(vmr); + if (state->dpa_base < vmr_size) { + mr =3D vmr; + offset =3D state->dpa_base; + } + } + if (!mr && ct3d->hostpmem) { + MemoryRegion *pmr =3D host_memory_backend_get_memory(ct3d->hostpme= m); + + pmr_size =3D memory_region_size(pmr); + if (state->dpa_base - vmr_size < pmr_size) { + mr =3D pmr; + offset =3D state->dpa_base - vmr_size; + } + } + if (!mr) { + return; + } + + if (ct3d->direct_mr_fw[idx]) { + return; + } + + direct_mr =3D &ct3d->direct_mr[idx]; + direct_mr_name =3D g_strdup_printf("cxl-direct-mapping-alias-%u", idx); + if (!direct_mr_name) { + return; + } + + memory_region_init_alias(direct_mr, OBJECT(ct3d), direct_mr_name, mr, + offset, state->decoder_size); + memory_region_add_subregion(&fw->mr, + state->decoder_base - fw->base, direct_mr); + ct3d->direct_mr_fw[idx] =3D fw; +} + +static void cxl_fmws_direct_passthrough_remove(CXLType3Dev *ct3d, + uint64_t decoder_base, + unsigned int idx) +{ + CXLFixedWindow *owner_fw =3D ct3d->direct_mr_fw[idx]; + MemoryRegion *direct_mr =3D &ct3d->direct_mr[idx]; + + if (!owner_fw) { + return; + } + + if (!memory_region_is_mapped(direct_mr)) { + return; + } + + if (cxl_cfmws_find_device(owner_fw, decoder_base, false)) { + return; + } + + memory_region_del_subregion(&owner_fw->mr, direct_mr); + ct3d->direct_mr_fw[idx] =3D NULL; +} + +static int cxl_fmws_direct_passthrough(Object *obj, void *opaque) +{ + CXLDirectPTState *state =3D opaque; + CXLFixedWindow *fw; + + if (!object_dynamic_cast(obj, TYPE_CXL_FMW)) { + return 0; + } + + fw =3D CXL_FMW(obj); + + /* Verify not interleaved */ + if (!cxl_cfmws_find_device(fw, state->decoder_base, false)) { + return 0; + } + + cxl_fmws_direct_passthrough_setup(state, fw); + + return 0; +} + +static int update_non_interleaved(Object *obj, void *opaque) +{ + const int hdm_inc =3D R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_= BASE_LO; + bool commit =3D *(bool *)opaque; + CXLType3Dev *ct3d; + uint32_t *cache_mem; + unsigned int hdm_count, i; + uint32_t cap; + uint64_t dpa_base =3D 0; + + if (!object_dynamic_cast(obj, TYPE_CXL_TYPE3)) { + return 0; + } + + ct3d =3D CXL_TYPE3(obj); + cache_mem =3D ct3d->cxl_cstate.crb.cache_mem_registers; + cap =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER_CAPABILITY); + hdm_count =3D cxl_decoder_count_dec(FIELD_EX32(cap, + CXL_HDM_DECODER_CAPABILIT= Y, + DECODER_COUNT)); + for (i =3D 0; i < hdm_count; i++) { + uint64_t decoder_base, decoder_size, skip; + uint32_t hdm_ctrl, low, high; + int iw, committed; + + hdm_ctrl =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + i * hd= m_inc); + committed =3D FIELD_EX32(hdm_ctrl, CXL_HDM_DECODER0_CTRL, COMMITTE= D); + + /* + * Optimization: Looking for a fully committed path; if the type 3= HDM + * decoder is not commmitted, it cannot lie on such a path. + */ + if (commit && !committed) { + return 0; + } + + low =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_DPA_SKIP_LO + + i * hdm_inc); + high =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_DPA_SKIP_HI + + i * hdm_inc); + skip =3D ((uint64_t)high << 32) | (low & 0xf0000000); + dpa_base +=3D skip; + + low =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_LO + i * hdm_= inc); + high =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_HI + i * hdm= _inc); + decoder_size =3D ((uint64_t)high << 32) | (low & 0xf0000000); + + low =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_LO + i * hdm_= inc); + high =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_HI + i * hdm= _inc); + decoder_base =3D ((uint64_t)high << 32) | (low & 0xf0000000); + + iw =3D FIELD_EX32(hdm_ctrl, CXL_HDM_DECODER0_CTRL, IW); + + if (iw =3D=3D 0) { + if (!commit) { + cxl_fmws_direct_passthrough_remove(ct3d, decoder_base, i); + } else { + CXLDirectPTState state =3D { + .ct3d =3D ct3d, + .decoder_base =3D decoder_base, + .decoder_size =3D decoder_size, + .dpa_base =3D dpa_base, + .hdm_decoder_idx =3D i, + }; + + object_child_foreach_recursive(object_get_root(), + cxl_fmws_direct_passthrough, + &state); + } + } + dpa_base +=3D decoder_size / cxl_interleave_ways_dec(iw, &error_fa= tal); + } + + return false; +} + +bool cfmws_update_non_interleaved(bool commit) +{ + /* + * Walk endpoints to find both committed and uncommitted decoders, + * then check if they are not interleaved (but the path is fully set u= p). + */ + object_child_foreach_recursive(object_get_root(), + update_non_interleaved, &commit); + + return false; +} + static MemTxResult cxl_read_cfmws(void *opaque, hwaddr addr, uint64_t *dat= a, unsigned size, MemTxAttrs attrs) { diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index 4739239da3..d9fc0bec8f 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -427,6 +427,8 @@ static void hdm_decoder_commit(CXLType3Dev *ct3d, int w= hich) ctrl =3D FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED, 1); =20 stl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + which * hdm_inc, ctrl); + + cfmws_update_non_interleaved(true); } =20 static void hdm_decoder_uncommit(CXLType3Dev *ct3d, int which) @@ -442,6 +444,8 @@ static void hdm_decoder_uncommit(CXLType3Dev *ct3d, int= which) ctrl =3D FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED, 0); =20 stl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + which * hdm_inc, ctrl); + + cfmws_update_non_interleaved(false); } =20 static int ct3d_qmp_uncor_err_to_cxl(CxlUncorErrorType qmp_err) diff --git a/include/hw/cxl/cxl.h b/include/hw/cxl/cxl.h index 998f495a98..931f5680bd 100644 --- a/include/hw/cxl/cxl.h +++ b/include/hw/cxl/cxl.h @@ -71,4 +71,5 @@ CXLComponentState *cxl_usp_to_cstate(CXLUpstreamPort *usp= ); typedef struct CXLDownstreamPort CXLDownstreamPort; DECLARE_INSTANCE_CHECKER(CXLDownstreamPort, CXL_DSP, TYPE_CXL_DSP) =20 +bool cfmws_update_non_interleaved(bool commit); #endif diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index 393f312217..ba551fa5f9 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -685,6 +685,8 @@ typedef struct CXLSetFeatureInfo { size_t data_size; } CXLSetFeatureInfo; =20 +typedef struct CXLFixedWindow CXLFixedWindow; + struct CXLSanitizeInfo; =20 typedef struct CXLAlertConfig { @@ -712,6 +714,8 @@ struct CXLType3Dev { uint64_t sn; =20 /* State */ + MemoryRegion direct_mr[CXL_HDM_DECODER_COUNT]; + CXLFixedWindow *direct_mr_fw[CXL_HDM_DECODER_COUNT]; AddressSpace hostvmem_as; AddressSpace hostpmem_as; CXLComponentState cxl_cstate; --=20 2.43.0