From nobody Tue Feb 10 16:22:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1694611674; cv=none; d=zohomail.com; s=zohoarc; b=XYaPXPp08IuubH5qIP2We3q7OzwguJuVu+ljLISY4rUnaxKwSgWiW2mLOtwfoT7Ee9gw+5J9grncDwrZurHF0l72A3JGtADG2skmBY7quq+mnwBPxIO5fGdKsHf+HBsa9nEM5KCsxnNhDhhHELCjHs2eLtfqhsPLUpe1yoyyWc4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1694611674; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=P4dhA/CkrZr4CvE4u/QrSco4YlBDRUqb5oxQASppjXU=; b=FoiPhwa9VzZsEIasTETWSuq/3A5bqnYk1a5vpd9+OeXc56/kUAcaxJ0HiJ8l4lEEVvO8rbPoCQwATVUKcftl8hEKYtpfT9gu2jqQe5tcYZMY5hN4Xj01e6cibsXRdtaemg2mj3hYpouPZau6XUaBmMgSYba7HNLKTTfjJLXmS8I= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1694611674782525.7572243339198; Wed, 13 Sep 2023 06:27:54 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qgPu2-0003I3-57; Wed, 13 Sep 2023 09:27:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qgPtz-0003DC-Oa for qemu-devel@nongnu.org; Wed, 13 Sep 2023 09:27:31 -0400 Received: from frasgout.his.huawei.com ([185.176.79.56]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qgPtw-00079q-PH for qemu-devel@nongnu.org; Wed, 13 Sep 2023 09:27:31 -0400 Received: from lhrpeml500005.china.huawei.com (unknown [172.18.147.207]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4Rm1Ss4fP7z6K6T2; Wed, 13 Sep 2023 21:26:53 +0800 (CST) Received: from SecurePC-101-06.china.huawei.com (10.122.247.231) by lhrpeml500005.china.huawei.com (7.191.163.240) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 13 Sep 2023 14:27:26 +0100 To: , Michael Tsirkin , Fan Ni , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , CC: Subject: [PATCH v4 4/4] hw/cxl: Support 4 HDM decoders at all levels of topology Date: Wed, 13 Sep 2023 14:25:23 +0100 Message-ID: <20230913132523.29780-5-Jonathan.Cameron@huawei.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230913132523.29780-1-Jonathan.Cameron@huawei.com> References: <20230913132523.29780-1-Jonathan.Cameron@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.122.247.231] X-ClientProxiedBy: lhrpeml100004.china.huawei.com (7.191.162.219) To lhrpeml500005.china.huawei.com (7.191.163.240) X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.176.79.56; envelope-from=jonathan.cameron@huawei.com; helo=frasgout.his.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Jonathan Cameron From: Jonathan Cameron via Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1694611675623100001 Content-Type: text/plain; charset="utf-8" Support these decoders in CXL host bridges (pxb-cxl), CXL Switch USP and CXL Type 3 end points. Signed-off-by: Jonathan Cameron --- v3: Factor out the hdm_inc changes to previous patch. Fix use of encoded hdm count as if it were decoded in cxl-host. Minor refactoring to make that path and one in cxl_type3.c look more similar. --- include/hw/cxl/cxl_component.h | 10 +++- hw/cxl/cxl-component-utils.c | 7 ++- hw/cxl/cxl-host.c | 67 ++++++++++++++++-------- hw/mem/cxl_type3.c | 96 +++++++++++++++++++++++----------- 4 files changed, 124 insertions(+), 56 deletions(-) diff --git a/include/hw/cxl/cxl_component.h b/include/hw/cxl/cxl_component.h index 7c864d2044..3c795a6278 100644 --- a/include/hw/cxl/cxl_component.h +++ b/include/hw/cxl/cxl_component.h @@ -135,6 +135,10 @@ REG32(CXL_RAS_ERR_HEADER0, CXL_RAS_REGISTERS_OFFSET + = 0x18) REG32(CXL_HDM_DECODER##n##_TARGET_LIST_LO, = \ CXL_HDM_REGISTERS_OFFSET + (0x20 * n) + 0x24) = \ REG32(CXL_HDM_DECODER##n##_TARGET_LIST_HI, = \ + CXL_HDM_REGISTERS_OFFSET + (0x20 * n) + 0x28) = \ + REG32(CXL_HDM_DECODER##n##_DPA_SKIP_LO, = \ + CXL_HDM_REGISTERS_OFFSET + (0x20 * n) + 0x24) = \ + REG32(CXL_HDM_DECODER##n##_DPA_SKIP_HI, = \ CXL_HDM_REGISTERS_OFFSET + (0x20 * n) + 0x28) =20 REG32(CXL_HDM_DECODER_CAPABILITY, CXL_HDM_REGISTERS_OFFSET) @@ -147,9 +151,13 @@ REG32(CXL_HDM_DECODER_GLOBAL_CONTROL, CXL_HDM_REGISTER= S_OFFSET + 4) FIELD(CXL_HDM_DECODER_GLOBAL_CONTROL, POISON_ON_ERR_EN, 0, 1) FIELD(CXL_HDM_DECODER_GLOBAL_CONTROL, HDM_DECODER_ENABLE, 1, 1) =20 +/* Support 4 decoders at all levels of topology */ +#define CXL_HDM_DECODER_COUNT 4 + HDM_DECODER_INIT(0); -/* Only used for HDM decoder registers block address increment */ HDM_DECODER_INIT(1); +HDM_DECODER_INIT(2); +HDM_DECODER_INIT(3); =20 /* 8.2.5.13 - CXL Extended Security Capability Structure (Root complex onl= y) */ #define EXTSEC_ENTRY_MAX 256 diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c index c0630ba5c1..f3bbf0fd13 100644 --- a/hw/cxl/cxl-component-utils.c +++ b/hw/cxl/cxl-component-utils.c @@ -90,6 +90,9 @@ static void dumb_hdm_handler(CXLComponentState *cxl_cstat= e, hwaddr offset, =20 switch (offset) { case A_CXL_HDM_DECODER0_CTRL: + case A_CXL_HDM_DECODER1_CTRL: + case A_CXL_HDM_DECODER2_CTRL: + case A_CXL_HDM_DECODER3_CTRL: should_commit =3D FIELD_EX32(value, CXL_HDM_DECODER0_CTRL, COMMIT); should_uncommit =3D !should_commit; break; @@ -129,7 +132,7 @@ static void cxl_cache_mem_write_reg(void *opaque, hwadd= r offset, uint64_t value, } =20 if (offset >=3D A_CXL_HDM_DECODER_CAPABILITY && - offset <=3D A_CXL_HDM_DECODER0_TARGET_LIST_HI) { + offset <=3D A_CXL_HDM_DECODER3_TARGET_LIST_HI) { dumb_hdm_handler(cxl_cstate, offset, value); } else { cregs->cache_mem_registers[offset / sizeof(*cregs->cache_mem_regis= ters)] =3D value; @@ -209,7 +212,7 @@ static void ras_init_common(uint32_t *reg_state, uint32= _t *write_msk) static void hdm_init_common(uint32_t *reg_state, uint32_t *write_msk, enum reg_type type) { - int decoder_count =3D 1; + int decoder_count =3D CXL_HDM_DECODER_COUNT; int hdm_inc =3D R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_L= O; int i; =20 diff --git a/hw/cxl/cxl-host.c b/hw/cxl/cxl-host.c index 73c5426476..2aa776c79c 100644 --- a/hw/cxl/cxl-host.c +++ b/hw/cxl/cxl-host.c @@ -97,35 +97,58 @@ void cxl_fmws_link_targets(CXLState *cxl_state, Error *= *errp) } } =20 -/* TODO: support, multiple hdm decoders */ static bool cxl_hdm_find_target(uint32_t *cache_mem, hwaddr addr, uint8_t *target) { int hdm_inc =3D R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_L= O; - uint32_t ctrl; - uint32_t ig_enc; - uint32_t iw_enc; - uint32_t target_idx; - int i =3D 0; - - ctrl =3D cache_mem[R_CXL_HDM_DECODER0_CTRL + i * hdm_inc]; - if (!FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED)) { - return false; - } - - ig_enc =3D FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, IG); - iw_enc =3D FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, IW); - target_idx =3D (addr / cxl_decode_ig(ig_enc)) % (1 << iw_enc); + unsigned int hdm_count; + bool found =3D false; + int i; + uint32_t cap; + + cap =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER_CAPABILITY); + hdm_count =3D cxl_decoder_count_dec(FIELD_EX32(cap, + CXL_HDM_DECODER_CAPABILIT= Y, + DECODER_COUNT)); + for (i =3D 0; i < hdm_count; i++) { + uint32_t ctrl, ig_enc, iw_enc, target_idx; + uint32_t low, high; + uint64_t base, size; + + low =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_LO + i * hdm_= inc); + high =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_HI + i * hdm= _inc); + base =3D (low & 0xf0000000) | ((uint64_t)high << 32); + low =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_LO + i * hdm_= inc); + high =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_HI + i * hdm= _inc); + size =3D (low & 0xf0000000) | ((uint64_t)high << 32); + if (addr < base || addr >=3D base + size) { + continue; + } =20 - if (target_idx < 4) { - *target =3D extract32(cache_mem[R_CXL_HDM_DECODER0_TARGET_LIST_LO], - target_idx * 8, 8); - } else { - *target =3D extract32(cache_mem[R_CXL_HDM_DECODER0_TARGET_LIST_HI], - (target_idx - 4) * 8, 8); + ctrl =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + i * hdm_in= c); + if (!FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED)) { + return false; + } + found =3D true; + ig_enc =3D FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, IG); + iw_enc =3D FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, IW); + target_idx =3D (addr / cxl_decode_ig(ig_enc)) % (1 << iw_enc); + + if (target_idx < 4) { + uint32_t val =3D ldl_le_p(cache_mem + + R_CXL_HDM_DECODER0_TARGET_LIST_LO + + i * hdm_inc); + *target =3D extract32(val, target_idx * 8, 8); + } else { + uint32_t val =3D ldl_le_p(cache_mem + + R_CXL_HDM_DECODER0_TARGET_LIST_HI + + i * hdm_inc); + *target =3D extract32(val, (target_idx - 4) * 8, 8); + } + break; } =20 - return true; + return found; } =20 static PCIDevice *cxl_cfmws_find_device(CXLFixedWindow *fw, hwaddr addr) diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index cd92813436..1658e0cc59 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -382,8 +382,6 @@ static void hdm_decoder_commit(CXLType3Dev *ct3d, int w= hich) uint32_t *cache_mem =3D cregs->cache_mem_registers; uint32_t ctrl; =20 - assert(which =3D=3D 0); - ctrl =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + which * hdm_in= c); /* TODO: Sanity checks that the decoder is possible */ ctrl =3D FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, ERR, 0); @@ -399,8 +397,6 @@ static void hdm_decoder_uncommit(CXLType3Dev *ct3d, int= which) uint32_t *cache_mem =3D cregs->cache_mem_registers; uint32_t ctrl; =20 - assert(which =3D=3D 0); - ctrl =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + which * hdm_in= c); =20 ctrl =3D FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, ERR, 0); @@ -489,6 +485,21 @@ static void ct3d_reg_write(void *opaque, hwaddr offset= , uint64_t value, should_uncommit =3D !should_commit; which_hdm =3D 0; break; + case A_CXL_HDM_DECODER1_CTRL: + should_commit =3D FIELD_EX32(value, CXL_HDM_DECODER0_CTRL, COMMIT); + should_uncommit =3D !should_commit; + which_hdm =3D 1; + break; + case A_CXL_HDM_DECODER2_CTRL: + should_commit =3D FIELD_EX32(value, CXL_HDM_DECODER0_CTRL, COMMIT); + should_uncommit =3D !should_commit; + which_hdm =3D 2; + break; + case A_CXL_HDM_DECODER3_CTRL: + should_commit =3D FIELD_EX32(value, CXL_HDM_DECODER0_CTRL, COMMIT); + should_uncommit =3D !should_commit; + which_hdm =3D 3; + break; case A_CXL_RAS_UNC_ERR_STATUS: { uint32_t capctrl =3D ldl_le_p(cache_mem + R_CXL_RAS_ERR_CAP_CTRL); @@ -760,40 +771,63 @@ static void ct3_exit(PCIDevice *pci_dev) } } =20 -/* TODO: Support multiple HDM decoders and DPA skip */ static bool cxl_type3_dpa(CXLType3Dev *ct3d, hwaddr host_addr, uint64_t *d= pa) { int hdm_inc =3D R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_L= O; uint32_t *cache_mem =3D ct3d->cxl_cstate.crb.cache_mem_registers; - uint64_t decoder_base, decoder_size, hpa_offset; - uint32_t hdm0_ctrl; - int ig, iw; - int i =3D 0; - - decoder_base =3D - (((uint64_t)cache_mem[R_CXL_HDM_DECODER0_BASE_HI + i * hdm_inc] <<= 32) | - cache_mem[R_CXL_HDM_DECODER0_BASE_LO + i * hdm_inc]); - if ((uint64_t)host_addr < decoder_base) { - return false; - } - - hpa_offset =3D (uint64_t)host_addr - decoder_base; - - decoder_size =3D - ((uint64_t)cache_mem[R_CXL_HDM_DECODER0_SIZE_HI + i * hdm_inc] << = 32) | - cache_mem[R_CXL_HDM_DECODER0_SIZE_LO + i * hdm_inc]; - if (hpa_offset >=3D decoder_size) { - return false; - } + unsigned int hdm_count; + uint32_t cap; + uint64_t dpa_base =3D 0; + int i; =20 - hdm0_ctrl =3D cache_mem[R_CXL_HDM_DECODER0_CTRL + i * hdm_inc]; - iw =3D FIELD_EX32(hdm0_ctrl, CXL_HDM_DECODER0_CTRL, IW); - ig =3D FIELD_EX32(hdm0_ctrl, CXL_HDM_DECODER0_CTRL, IG); + cap =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER_CAPABILITY); + hdm_count =3D cxl_decoder_count_dec(FIELD_EX32(cap, + CXL_HDM_DECODER_CAPABILIT= Y, + DECODER_COUNT)); + + for (i =3D 0; i < hdm_count; i++) { + uint64_t decoder_base, decoder_size, hpa_offset, skip; + uint32_t hdm_ctrl, low, high; + int ig, iw; + + low =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_LO + i * hdm_= inc); + high =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_HI + i * hdm= _inc); + decoder_base =3D ((uint64_t)high << 32) | (low & 0xf0000000); + + low =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_LO + i * hdm_= inc); + high =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_HI + i * hdm= _inc); + decoder_size =3D ((uint64_t)high << 32) | (low & 0xf0000000); + + low =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_DPA_SKIP_LO + + i * hdm_inc); + high =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_DPA_SKIP_HI + + i * hdm_inc); + skip =3D ((uint64_t)high << 32) | (low & 0xf0000000); + dpa_base +=3D skip; + + hpa_offset =3D (uint64_t)host_addr - decoder_base; + + hdm_ctrl =3D ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + i * hd= m_inc); + iw =3D FIELD_EX32(hdm_ctrl, CXL_HDM_DECODER0_CTRL, IW); + ig =3D FIELD_EX32(hdm_ctrl, CXL_HDM_DECODER0_CTRL, IG); + if (!FIELD_EX32(hdm_ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED)) { + return false; + } + if (((uint64_t)host_addr < decoder_base) || + (hpa_offset >=3D decoder_size)) { + dpa_base +=3D decoder_size / + cxl_interleave_ways_dec(iw, &error_fatal); + continue; + } =20 - *dpa =3D (MAKE_64BIT_MASK(0, 8 + ig) & hpa_offset) | - ((MAKE_64BIT_MASK(8 + ig + iw, 64 - 8 - ig - iw) & hpa_offset) >> = iw); + *dpa =3D dpa_base + + ((MAKE_64BIT_MASK(0, 8 + ig) & hpa_offset) | + ((MAKE_64BIT_MASK(8 + ig + iw, 64 - 8 - ig - iw) & hpa_offset) + >> iw)); =20 - return true; + return true; + } + return false; } =20 static int cxl_type3_hpa_to_as_and_dpa(CXLType3Dev *ct3d, --=20 2.39.2