From nobody Thu May 9 21:45:48 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1709581575; cv=none; d=zohomail.com; s=zohoarc; b=dt1/DDwzkFF7Bizw0+8vf84fUqlt18oIFwhyawSXbX5nocoO18ifpF9lK8eYAwhPaKWE6u7sz3M1hHWkkVPFp8RPZVDzdn4dGCPO5Zckzl+wgHkrSkI+ASMEQhh+UfhAwC+cTHFMOxPEy3cIxbYIcixL9b+XIxYiqBbbaSY6zns= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709581575; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=AkPmI8FuszQzNKc/W7C7bH++18bz1caF1vBxyxTADMI=; b=jyV3MYydZkRiJxgELJU12CweOn2xEldo1pxq1lD6UEj3t1ow+9pSn5yF5xZWNl+oMuSgcQ7djQEIWrqomoGq9Cnd1Vvr3t4RxdmzT2SBemuOQPEMVwRwKzAXkyJLz2c44drK25PChu386fmd+3FaykXmnbhERpUzKPVTp9ylkck= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1709581575205336.43851713019376; Mon, 4 Mar 2024 11:46:15 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rhEEP-0005SK-V0; Mon, 04 Mar 2024 14:44:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhEEL-0005RU-25 for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:10 -0500 Received: from mail-yb1-xb2e.google.com ([2607:f8b0:4864:20::b2e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhEEJ-0006uq-Ku for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:08 -0500 Received: by mail-yb1-xb2e.google.com with SMTP id 3f1490d57ef6-dcc71031680so4588939276.2 for ; Mon, 04 Mar 2024 11:44:07 -0800 (PST) Received: from localhost.localdomain ([50.205.20.42]) by smtp.gmail.com with ESMTPSA id h1-20020a255f41000000b00dc62edd58dasm2282646ybm.40.2024.03.04.11.44.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Mar 2024 11:44:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709581446; x=1710186246; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AkPmI8FuszQzNKc/W7C7bH++18bz1caF1vBxyxTADMI=; b=RNYJQ5UMOFZyfMnu8Y1OyEACreeKPx9gFgDM7sTnHi1yL1/AR32Zbex0yppTGCRtab YpPqO2vhZMPQ4ePDWhvPtNfC1eLemJG498LD034lhyj1OFNOjOeFMDIvM8CjhOVASQU6 vKTJm2WiGGA4IEGbwjLUsYL5vYBpLLcrvR2UJkp+O6+MfBIosOLFxrmFRutGBdaaiCMr l6VQzDSYaJ/znip+VdXTiIq+LmbYO56XC9Y74xjyxzL+cfcgkhFLVkj2MveazkGZIvJo uthfCdaND1FP/Hn5AwlDQF5je9EYvJxcDtAscZmZz8R3NEd2iZntRmo0QJ3bJA1mj3Aq pKwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709581446; x=1710186246; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AkPmI8FuszQzNKc/W7C7bH++18bz1caF1vBxyxTADMI=; b=tR/FuR2ATupImy0vrF1gmwWtl5lreM81iMABE7pE0djUydrm7sG9bNAq9p89QAu5T9 rZLhKiEvnONp1gBvysu0MmVmJa1cT6uIY0Yd2BPzwrelAYC3amYHPqE1X09HpJxuGiD0 Fmjd6adORLniPgGg82AxjQi04rkNiILCh9t+ZYQ7g9SFOsCgchzPPybyEFAQlFphbRVj dcUQpA4VFavDT8hsP+i3P0v0N+xk7j+d8xJSc658vArEIZh2cJV3oJBl1RlBFsXZCp9x TPoC66PEtSBrZzdhdIFhkIt2xue7VsezVN48ldROAOAqmjrMItodHBrxf79K0EDvcmxR oKlA== X-Gm-Message-State: AOJu0YwHcgFyBsP23/EFdWlY9zcRlxjS7U6V4FPxIWydJktBoE+ktYGV Zzc8gsT+iyNp0arPKIge7ZvVLb3WTnrxo3jeFznOA69j4e4HMmOHQs+2gPm/ X-Google-Smtp-Source: AGHT+IF8Ycb0Ig4cvVD+KZC7NiE7FpVTQX5T1d6xPH2bLLUEk6Pr5KEZbkD4FKZ2xmlpNCQuOqLiug== X-Received: by 2002:a25:b109:0:b0:dcc:6112:f90d with SMTP id g9-20020a25b109000000b00dcc6112f90dmr7174783ybj.62.1709581446482; Mon, 04 Mar 2024 11:44:06 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Jorgen.Hansen@wdc.com, wj28.lee@gmail.com, Fan Ni Subject: [PATCH v5 01/13] hw/cxl/cxl-mailbox-utils: Add dc_event_log_size field to output payload of identify memory device command Date: Mon, 4 Mar 2024 11:33:56 -0800 Message-ID: <20240304194331.1586191-2-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240304194331.1586191-1-nifan.cxl@gmail.com> References: <20240304194331.1586191-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::b2e; envelope-from=nifan.cxl@gmail.com; helo=mail-yb1-xb2e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1709581575758100001 Content-Type: text/plain; charset="utf-8" From: Fan Ni Based on CXL spec r3.1 Table 8-127 (Identify Memory Device Output Payload), dynamic capacity event log size should be part of output of the Identify command. Add dc_event_log_size to the output payload for the host to get the info. Signed-off-by: Fan Ni Reviewed-by: Jonathan Cameron --- hw/cxl/cxl-mailbox-utils.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index 4bcd727f4c..ba1d9901df 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -21,6 +21,7 @@ #include "sysemu/hostmem.h" =20 #define CXL_CAPACITY_MULTIPLIER (256 * MiB) +#define CXL_DC_EVENT_LOG_SIZE 8 =20 /* * How to add a new command, example. The command set FOO, with cmd BAR. @@ -780,8 +781,9 @@ static CXLRetCode cmd_identify_memory_device(const stru= ct cxl_cmd *cmd, uint16_t inject_poison_limit; uint8_t poison_caps; uint8_t qos_telemetry_caps; + uint16_t dc_event_log_size; } QEMU_PACKED *id; - QEMU_BUILD_BUG_ON(sizeof(*id) !=3D 0x43); + QEMU_BUILD_BUG_ON(sizeof(*id) !=3D 0x45); CXLType3Dev *ct3d =3D CXL_TYPE3(cci->d); CXLType3Class *cvc =3D CXL_TYPE3_GET_CLASS(ct3d); CXLDeviceState *cxl_dstate =3D &ct3d->cxl_dstate; @@ -807,6 +809,7 @@ static CXLRetCode cmd_identify_memory_device(const stru= ct cxl_cmd *cmd, st24_le_p(id->poison_list_max_mer, 256); /* No limit - so limited by main poison record limit */ stw_le_p(&id->inject_poison_limit, 0); + stw_le_p(&id->dc_event_log_size, CXL_DC_EVENT_LOG_SIZE); =20 *len_out =3D sizeof(*id); return CXL_MBOX_SUCCESS; --=20 2.43.0 From nobody Thu May 9 21:45:48 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1709581594; cv=none; d=zohomail.com; s=zohoarc; b=h/mWMq6JbKC0VCpCq50rhYIw16ZYZcvc/RPawNB7RS84LTdwqJcs+/h9fzN7h4jzO0H+7gduUPpoo2N359B6RBW2n9trzDYqXfXGKA70/kTs4h9QpJ+A8WFH6gF0C9zabqcpyoDOL3DDTaROHTOCjBGmR7Ao9aAdOGL2icvrW4A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709581594; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=9sd2BgVEGhVtZ8GHzjE9iedUGsXSGw1JofJzU7n5NJk=; b=Iz+rT0nAfP2GIE4CWG24+xxWxUxnoBkpbn49b0+0jTvFX8aH3gykg4R1QjBy73gsExTlJc+3Gzop3oFhiBeBs6S2LK9KVCUPcD6AssJEC53DzNTBXBYRQ5+MvEdFGmoir/I8xrazqME73v6zyAD80Ar7TK/K9k/WA6of5H7Jw0A= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1709581594791262.7604665879936; Mon, 4 Mar 2024 11:46:34 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rhEEQ-0005SV-Ry; Mon, 04 Mar 2024 14:44:14 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhEEN-0005Rx-PI for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:11 -0500 Received: from mail-yw1-x1134.google.com ([2607:f8b0:4864:20::1134]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhEEM-0006v7-0J for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:11 -0500 Received: by mail-yw1-x1134.google.com with SMTP id 00721157ae682-608e0b87594so50524277b3.1 for ; Mon, 04 Mar 2024 11:44:09 -0800 (PST) Received: from localhost.localdomain ([50.205.20.42]) by smtp.gmail.com with ESMTPSA id h1-20020a255f41000000b00dc62edd58dasm2282646ybm.40.2024.03.04.11.44.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Mar 2024 11:44:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709581449; x=1710186249; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9sd2BgVEGhVtZ8GHzjE9iedUGsXSGw1JofJzU7n5NJk=; b=hpDUHRvojAkW8o8RtjaBgYJdEIZLRDjJHUWudZpCuCxO7599AIU99m1j+WN/GKefri kSLirSRSupiMn97hEQ26QDeMeCVBvQutp3eTZywbu4nvO1Na4tm0qZU/sqfe43yW/a+W UGe29hekuE5YNUNmk2WQh01vCLPOe3qZcJM7SAVq2Kmlb/nhlQCdlXHWI6gCF3JaYahY qz9rstMXuiJaIT9gFharyJFn63Iup9elJmXe/iDoKVIWCd3MZbiDcABmgCO6SKpHjn1V 1ozi5jcQTYyidvzftptyhAizxl7QeNIscw4Z7CMsvJT6ksIrC9zeQxPk4LE5VJkmSt/3 Ctww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709581449; x=1710186249; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9sd2BgVEGhVtZ8GHzjE9iedUGsXSGw1JofJzU7n5NJk=; b=E/9A3AEAWzHwOq93oTCtUHWNvKp10df41u5GRflY0P7kqgSafOfQcxSZa/g9VQIYer ddC3r22GlIdJVc/kvkbZpIUPwh1oR5gxOYXZNI0IgB1NU/MpF60o68s3T0nJL708O7O3 AdQM2XVh9eFOex/+0c59sDkZyPKzksNfGPxXB1np2ith5vfVFpx2rPrAf7JGdcTMZu7M txgYFIhdwAxqbx3PlWiRnPQg6rnXcxBj7/3lvhQENIWFuFHJH5mxMcQmi/vaAkdn3T5+ ZSXNL3mtvWR4K+xTDHaa94WRrmCWyWByOO5XN0Ah6VVuqJ+c8C4Bwx0YIvYK6N1wGseU bouw== X-Gm-Message-State: AOJu0YzyKhR8OruU+0Jv0g/JAnI797RHQIG6T8R6meYCzLACDBjLD5TV iL/z31GoYI60itdRL5YLPptnELSiUZc+SH0tjz234ZFHQVAJEzl5Tgb62/6v X-Google-Smtp-Source: AGHT+IG39RPA5p0WekIO5yNVRaTUVzpbb/YPybrWX41+xzC4a8iu9naWr1DSiius45eB4Q8FHykk6A== X-Received: by 2002:a5b:f05:0:b0:dcc:8c5e:7c9b with SMTP id x5-20020a5b0f05000000b00dcc8c5e7c9bmr7046054ybr.57.1709581448659; Mon, 04 Mar 2024 11:44:08 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Jorgen.Hansen@wdc.com, wj28.lee@gmail.com, Fan Ni Subject: [PATCH v5 02/13] hw/cxl/cxl-mailbox-utils: Add dynamic capacity region representative and mailbox command support Date: Mon, 4 Mar 2024 11:33:57 -0800 Message-ID: <20240304194331.1586191-3-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240304194331.1586191-1-nifan.cxl@gmail.com> References: <20240304194331.1586191-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::1134; envelope-from=nifan.cxl@gmail.com; helo=mail-yw1-x1134.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1709581596097100001 Content-Type: text/plain; charset="utf-8" From: Fan Ni Per cxl spec r3.1, add dynamic capacity region representative based on Table 8-165 and extend the cxl type3 device definition to include dc region information. Also, based on info in 8.2.9.9.9.1, add 'Get Dynamic Capacity Configuration' mailbox support. Note: we store region decode length as byte-wise length on the device, which should be divided by 256 * MiB before being returned to the host for "Get Dynamic Capacity Configuration" mailbox command per specification. Signed-off-by: Fan Ni Reviewed-by: Jonathan Cameron --- hw/cxl/cxl-mailbox-utils.c | 99 +++++++++++++++++++++++++++++++++++++ include/hw/cxl/cxl_device.h | 16 ++++++ 2 files changed, 115 insertions(+) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index ba1d9901df..5792010c12 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -22,6 +22,8 @@ =20 #define CXL_CAPACITY_MULTIPLIER (256 * MiB) #define CXL_DC_EVENT_LOG_SIZE 8 +#define CXL_NUM_EXTENTS_SUPPORTED 512 +#define CXL_NUM_TAGS_SUPPORTED 0 =20 /* * How to add a new command, example. The command set FOO, with cmd BAR. @@ -80,6 +82,8 @@ enum { #define GET_POISON_LIST 0x0 #define INJECT_POISON 0x1 #define CLEAR_POISON 0x2 + DCD_CONFIG =3D 0x48, + #define GET_DC_CONFIG 0x0 PHYSICAL_SWITCH =3D 0x51, #define IDENTIFY_SWITCH_DEVICE 0x0 #define GET_PHYSICAL_PORT_STATE 0x1 @@ -1238,6 +1242,91 @@ static CXLRetCode cmd_media_clear_poison(const struc= t cxl_cmd *cmd, return CXL_MBOX_SUCCESS; } =20 +/* + * CXL r3.1 section 8.2.9.9.9.1: Get Dynamic Capacity Configuration + * (Opcode: 4800h) + */ +static CXLRetCode cmd_dcd_get_dyn_cap_config(const struct cxl_cmd *cmd, + uint8_t *payload_in, + size_t len_in, + uint8_t *payload_out, + size_t *len_out, + CXLCCI *cci) +{ + CXLType3Dev *ct3d =3D CXL_TYPE3(cci->d); + struct { + uint8_t region_cnt; + uint8_t start_region_id; + } QEMU_PACKED *in; + struct { + uint8_t num_regions; + uint8_t regions_returned; + uint8_t rsvd1[6]; + struct { + uint64_t base; + uint64_t decode_len; + uint64_t region_len; + uint64_t block_size; + uint32_t dsmadhandle; + uint8_t flags; + uint8_t rsvd2[3]; + } QEMU_PACKED records[]; + } QEMU_PACKED *out; + struct { + uint32_t num_extents_supported; + uint32_t num_extents_available; + uint32_t num_tags_supported; + uint32_t num_tags_available; + } QEMU_PACKED *extra_out; + uint16_t record_count; + uint16_t i; + uint16_t out_pl_len; + uint8_t start_region_id; + + in =3D (void *)payload_in; + out =3D (void *)payload_out; + start_region_id =3D in->start_region_id; + if (start_region_id >=3D ct3d->dc.num_regions) { + return CXL_MBOX_INVALID_INPUT; + } + + record_count =3D MIN(ct3d->dc.num_regions - in->start_region_id, + in->region_cnt); + + out_pl_len =3D sizeof(*out) + record_count * sizeof(out->records[0]); + extra_out =3D (void *)(payload_out + out_pl_len); + out_pl_len +=3D sizeof(*extra_out); + assert(out_pl_len <=3D CXL_MAILBOX_MAX_PAYLOAD_SIZE); + + out->num_regions =3D ct3d->dc.num_regions; + out->regions_returned =3D record_count; + for (i =3D 0; i < record_count; i++) { + stq_le_p(&out->records[i].base, + ct3d->dc.regions[start_region_id + i].base); + stq_le_p(&out->records[i].decode_len, + ct3d->dc.regions[start_region_id + i].decode_len / + CXL_CAPACITY_MULTIPLIER); + stq_le_p(&out->records[i].region_len, + ct3d->dc.regions[start_region_id + i].len); + stq_le_p(&out->records[i].block_size, + ct3d->dc.regions[start_region_id + i].block_size); + stl_le_p(&out->records[i].dsmadhandle, + ct3d->dc.regions[start_region_id + i].dsmadhandle); + out->records[i].flags =3D ct3d->dc.regions[start_region_id + i].fl= ags; + } + /* + * TODO: will assign proper values when extents and tags are introduced + * to use. + */ + stl_le_p(&extra_out->num_extents_supported, CXL_NUM_EXTENTS_SUPPORTED); + stl_le_p(&extra_out->num_extents_available, CXL_NUM_EXTENTS_SUPPORTED); + stl_le_p(&extra_out->num_tags_supported, CXL_NUM_TAGS_SUPPORTED); + stl_le_p(&extra_out->num_tags_available, CXL_NUM_TAGS_SUPPORTED); + + *len_out =3D out_pl_len; + return CXL_MBOX_SUCCESS; +} + #define IMMEDIATE_CONFIG_CHANGE (1 << 1) #define IMMEDIATE_DATA_CHANGE (1 << 2) #define IMMEDIATE_POLICY_CHANGE (1 << 3) @@ -1282,6 +1371,11 @@ static const struct cxl_cmd cxl_cmd_set[256][256] = =3D { cmd_media_clear_poison, 72, 0 }, }; =20 +static const struct cxl_cmd cxl_cmd_set_dcd[256][256] =3D { + [DCD_CONFIG][GET_DC_CONFIG] =3D { "DCD_GET_DC_CONFIG", + cmd_dcd_get_dyn_cap_config, 2, 0 }, +}; + static const struct cxl_cmd cxl_cmd_set_sw[256][256] =3D { [INFOSTAT][IS_IDENTIFY] =3D { "IDENTIFY", cmd_infostat_identify, 0, 0 = }, [INFOSTAT][BACKGROUND_OPERATION_STATUS] =3D { "BACKGROUND_OPERATION_ST= ATUS", @@ -1487,7 +1581,12 @@ void cxl_initialize_mailbox_swcci(CXLCCI *cci, Devic= eState *intf, =20 void cxl_initialize_mailbox_t3(CXLCCI *cci, DeviceState *d, size_t payload= _max) { + CXLType3Dev *ct3d =3D CXL_TYPE3(d); + cxl_copy_cci_commands(cci, cxl_cmd_set); + if (ct3d->dc.num_regions) { + cxl_copy_cci_commands(cci, cxl_cmd_set_dcd); + } cci->d =3D d; =20 /* No separation for PCI MB as protocol handled in PCI device */ diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index 3cf3077afa..93ce047b28 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -422,6 +422,17 @@ typedef struct CXLPoison { typedef QLIST_HEAD(, CXLPoison) CXLPoisonList; #define CXL_POISON_LIST_LIMIT 256 =20 +#define DCD_MAX_NUM_REGION 8 + +typedef struct CXLDCRegion { + uint64_t base; /* aligned to 256*MiB */ + uint64_t decode_len; /* aligned to 256*MiB */ + uint64_t len; + uint64_t block_size; + uint32_t dsmadhandle; + uint8_t flags; +} CXLDCRegion; + struct CXLType3Dev { /* Private */ PCIDevice parent_obj; @@ -454,6 +465,11 @@ struct CXLType3Dev { unsigned int poison_list_cnt; bool poison_list_overflowed; uint64_t poison_list_overflow_ts; + + struct dynamic_capacity { + uint8_t num_regions; /* 0-8 regions */ + CXLDCRegion regions[DCD_MAX_NUM_REGION]; + } dc; }; =20 #define TYPE_CXL_TYPE3 "cxl-type3" --=20 2.43.0 From nobody Thu May 9 21:45:48 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1709581626; cv=none; d=zohomail.com; s=zohoarc; b=ANiYG8r3Xt3a8IMHnlhP7K1Lkr+B5340m4XC4mpSKR827InKy6ZwE5cRJqxzhnpdwBrYFudPLryLA/KEiJPOKKwHN3MPR6pvcnAPBDQTx5Q9yvuT+qivww4kiDd1mtg4//fSezLGJV2qTVXYOIb2N+W53OGQv8xnylLr/d9FAPw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709581626; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=2InpeJRt+f0kCa8qliojbJhzR8/UO/mLPd8lEue23eI=; b=kfXp3S3YJg+8tibiHSnI38KI+nFUN0EvvDVjK9t2F+y1MHPtKvuJYh6Vb4qjalak/67l4veDcjhog8R65T4wJRsrbjFe8/BbUiMq9pUfdjuIa2prAMxVtO8vI55aJIfklvtzjUVTdFR8PGXxWiaRIXZvoUO81nZztkW8+BPXhCk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 170958162647487.8638592658549; Mon, 4 Mar 2024 11:47:06 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rhEER-0005Ss-N4; Mon, 04 Mar 2024 14:44:15 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhEEQ-0005SN-0h for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:14 -0500 Received: from mail-yb1-xb36.google.com ([2607:f8b0:4864:20::b36]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhEEO-0006vE-3C for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:13 -0500 Received: by mail-yb1-xb36.google.com with SMTP id 3f1490d57ef6-dc74435c428so5362259276.2 for ; Mon, 04 Mar 2024 11:44:11 -0800 (PST) Received: from localhost.localdomain ([50.205.20.42]) by smtp.gmail.com with ESMTPSA id h1-20020a255f41000000b00dc62edd58dasm2282646ybm.40.2024.03.04.11.44.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Mar 2024 11:44:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709581451; x=1710186251; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2InpeJRt+f0kCa8qliojbJhzR8/UO/mLPd8lEue23eI=; b=KzbM50V7t/K1GRGJ9/eGnPo0jLKH7MYGng6e0nFcTvCk4fArMh3z91P7eys6U2lWJV edrtM9BcSKDWUNyYq1VdXX10q7mKjMzeaAE0+++Y4aSTXZFFtdqugTR8AMOA9ArOT7bK Ge2kHeNEH73N4ngg4pc34o/XhxzDePhrrWJOHsf26U8dczdFk6ZXzNDYLe8kni6OV2SR M+LgNrtOmT6ZIa1Wt/le6+YhBp5CM6DD6pQs6/maRaWo7dVHFbhxUGB3VH6ZtmVfb7hs ESn6frRV6sFpYVjSV2zyP2kYELJrCDned4+mbZbcwHxBRSoXebM1OxXTyukW0UQ44KBS N4tQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709581451; x=1710186251; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2InpeJRt+f0kCa8qliojbJhzR8/UO/mLPd8lEue23eI=; b=Aoxm+MdawKhw8IrpCVCv+QSju67H2MG88eYotIV+Upj8kME+7Zu6Qga49043I7z1sn BvqdO3q9aMg3N/2CO/dVLD8QH0cDneJEUsit0xpkaX2wp1tYY3nFR73UE0KDsmgVhuoB jeT2nrmf1SyXKLRUM4/rgUwJyL/sSoyL3qKAZmxODFPbT/VwFgV6NqI0PpmFn+HoWnrJ QuWq3+rKURChL/X2PtYkwnm2fDp4DwyUoBelLTYX+Km1AF5hTL4fF3z6zmPjZ6vwtBXm U+EHYL7T9AWwnFEGld2NpVYxejz8MZCN3KEd3Y1dx2ttWLxeR+k34btwhLAqHJEt6Z3I mbMw== X-Gm-Message-State: AOJu0YxE6chN2GWynBtI4S7IOy6wsNJG4JvAuptqMZYSI1us1e8e3qsQ EvJ1VIEKLGS62oyDy+vYXW1MYOOzSA8fKK2FYgm9XlgyulRGwkLlsBYawBOn X-Google-Smtp-Source: AGHT+IEklaIYAzLXknCQyGLMTYvknIhiLaansPxR45rNdr+9VPvP4NwG4q+IH2RxW6udFYT1qcQEPQ== X-Received: by 2002:a25:aa8b:0:b0:dc2:2b0d:613e with SMTP id t11-20020a25aa8b000000b00dc22b0d613emr7102455ybi.10.1709581450845; Mon, 04 Mar 2024 11:44:10 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Jorgen.Hansen@wdc.com, wj28.lee@gmail.com, Fan Ni Subject: [PATCH v5 03/13] include/hw/cxl/cxl_device: Rename mem_size as static_mem_size for type3 memory devices Date: Mon, 4 Mar 2024 11:33:58 -0800 Message-ID: <20240304194331.1586191-4-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240304194331.1586191-1-nifan.cxl@gmail.com> References: <20240304194331.1586191-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::b36; envelope-from=nifan.cxl@gmail.com; helo=mail-yb1-xb36.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1709581628065100003 Content-Type: text/plain; charset="utf-8" From: Fan Ni Rename mem_size as static_mem_size for type3 memdev to cover static RAM and pmem capacity, preparing for the introduction of dynamic capacity to support dynamic capacity devices. Signed-off-by: Fan Ni Reviewed-by: Jonathan Cameron --- hw/cxl/cxl-mailbox-utils.c | 4 ++-- hw/mem/cxl_type3.c | 8 ++++---- include/hw/cxl/cxl_device.h | 2 +- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index 5792010c12..853dadba39 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -803,7 +803,7 @@ static CXLRetCode cmd_identify_memory_device(const stru= ct cxl_cmd *cmd, snprintf(id->fw_revision, 0x10, "BWFW VERSION %02d", 0); =20 stq_le_p(&id->total_capacity, - cxl_dstate->mem_size / CXL_CAPACITY_MULTIPLIER); + cxl_dstate->static_mem_size / CXL_CAPACITY_MULTIPLIER); stq_le_p(&id->persistent_capacity, cxl_dstate->pmem_size / CXL_CAPACITY_MULTIPLIER); stq_le_p(&id->volatile_capacity, @@ -1179,7 +1179,7 @@ static CXLRetCode cmd_media_clear_poison(const struct= cxl_cmd *cmd, struct clear_poison_pl *in =3D (void *)payload_in; =20 dpa =3D ldq_le_p(&in->dpa); - if (dpa + CXL_CACHE_LINE_SIZE > cxl_dstate->mem_size) { + if (dpa + CXL_CACHE_LINE_SIZE > cxl_dstate->static_mem_size) { return CXL_MBOX_INVALID_PA; } =20 diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index e8801805b9..244d2b5fd5 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -608,7 +608,7 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error *= *errp) } address_space_init(&ct3d->hostvmem_as, vmr, v_name); ct3d->cxl_dstate.vmem_size =3D memory_region_size(vmr); - ct3d->cxl_dstate.mem_size +=3D memory_region_size(vmr); + ct3d->cxl_dstate.static_mem_size +=3D memory_region_size(vmr); g_free(v_name); } =20 @@ -631,7 +631,7 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error *= *errp) } address_space_init(&ct3d->hostpmem_as, pmr, p_name); ct3d->cxl_dstate.pmem_size =3D memory_region_size(pmr); - ct3d->cxl_dstate.mem_size +=3D memory_region_size(pmr); + ct3d->cxl_dstate.static_mem_size +=3D memory_region_size(pmr); g_free(p_name); } =20 @@ -837,7 +837,7 @@ static int cxl_type3_hpa_to_as_and_dpa(CXLType3Dev *ct3= d, return -EINVAL; } =20 - if (*dpa_offset > ct3d->cxl_dstate.mem_size) { + if (*dpa_offset > ct3d->cxl_dstate.static_mem_size) { return -EINVAL; } =20 @@ -1010,7 +1010,7 @@ static bool set_cacheline(CXLType3Dev *ct3d, uint64_t= dpa_offset, uint8_t *data) return false; } =20 - if (dpa_offset + CXL_CACHE_LINE_SIZE > ct3d->cxl_dstate.mem_size) { + if (dpa_offset + CXL_CACHE_LINE_SIZE > ct3d->cxl_dstate.static_mem_siz= e) { return false; } =20 diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index 93ce047b28..f82d018422 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -234,7 +234,7 @@ typedef struct cxl_device_state { } timestamp; =20 /* memory region size, HDM */ - uint64_t mem_size; + uint64_t static_mem_size; uint64_t pmem_size; uint64_t vmem_size; =20 --=20 2.43.0 From nobody Thu May 9 21:45:48 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1709581581; cv=none; d=zohomail.com; s=zohoarc; b=JjiQ4lMZCLYqfDwi6uiDo2UI5+LmmUYCURKeirv2CGvZpz06XcR2tGbW8RaU6lm+bghbqBdJBPDaV6hTx8e1dAQhBWZsC8e3z2t2bkegIRg6cNhwHxc8n6vjyYHlr3r5C1piCvXLpb2glfnm203cTw48Es4jv9XXonFLtqwMkkE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709581581; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=qbOcnbLy9moWsSBbE+Q42MVGWhpKJBtzuH2spFlbBIc=; b=PPh5W6REtxRPZWi1pq6HdcZxcS6s7uKy+qj+O2qpZBRdoe1gfyD8KDZQCV30gZHMfjE1ubJVaJPMhckykuq/AIT6NTQorvQrNeOYtGYmnLGYIoHdPEzkD94eAlvxhHnqcg7GpS/rg4EFct/mc2me6MbRAYTE0RspoZ/h8DESbn0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1709581581931833.0630206797699; Mon, 4 Mar 2024 11:46:21 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rhEES-0005TP-L0; Mon, 04 Mar 2024 14:44:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhEER-0005Sr-JV for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:15 -0500 Received: from mail-yb1-xb2c.google.com ([2607:f8b0:4864:20::b2c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhEEP-0006vR-QI for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:15 -0500 Received: by mail-yb1-xb2c.google.com with SMTP id 3f1490d57ef6-dcd7c526cc0so5184544276.1 for ; Mon, 04 Mar 2024 11:44:13 -0800 (PST) Received: from localhost.localdomain ([50.205.20.42]) by smtp.gmail.com with ESMTPSA id h1-20020a255f41000000b00dc62edd58dasm2282646ybm.40.2024.03.04.11.44.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Mar 2024 11:44:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709581453; x=1710186253; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qbOcnbLy9moWsSBbE+Q42MVGWhpKJBtzuH2spFlbBIc=; b=F+NqVZkan2g6rTkEtl2ksPATYTmVx/hF1j849qakco0Z7AqUSP9lAsc1nFMzYtn/Qk KwtB+TClJ5KMDeN0ryUy4DlmmxLlPKdH5kPABbyrRJz29hR0ePd2FTB4R8CxtBVRdjaC 4r37NgEbO3TM/IRH/qhGAAzl4ocajaZWGBm3WcDLTcc0E5FpyWnNEh663+kOM2C4SlZz uDuKYqEp4SfllPglUe1i6tDRjHCI+uJN12a+WqaK7BF/MyKe8Em/KUGXXOScwvS3FLcc I5HFWrZISL2Wmz8bwgXItQKOipFwLJqBKbDyav8rnYO+eaOC+GQnNFCFYl7DT9fUm597 KB9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709581453; x=1710186253; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qbOcnbLy9moWsSBbE+Q42MVGWhpKJBtzuH2spFlbBIc=; b=keQx2gReSyZsanffI9Yk69NEFeRs8OjDlr1eZDodaK9dwlAe9YGOruTQ2gz0Dh4ibH DzDxzSNrLiYCtfAp+3WxHygAF4GpTFArKNmGmI7q3rl3g0eLONZt3XqUhiWQTvCjjitN C9RYJs6cNm3Je2P40DzBoTjAJat2WYbEEdnHKvu7Ur2b+YUln60+Djo1TTYdJbI4s5JE rcoM6qMwMkCFt+si50FmHQA3rCJUGuB6jjrA/RbOZLVi9M2sUG3ZTsj3PV2FntI4+mmV P8dFepl2IeZHgsqRduDNnXDYOCY6of/jfusaBDUeHU+KsDnxhIXX5VfeC2wHfBRimNtF Wf0w== X-Gm-Message-State: AOJu0YwAONjf1l0s0L3BqeBYp7XOhQtGe41mG87cHpNiYqjjVlR1/Yg4 725OiBUPTncecbHvtv9/ncIOUGECjm6qJj9GaD1bWyvJnUzPwkX+RC1vT38u X-Google-Smtp-Source: AGHT+IFj/6AKOb01CS8iZ0cx8I6hTpu+7AdNBUdow4ScUYPauwu6XEYgtcZPPXwXUBAP+6x/56QZMw== X-Received: by 2002:a25:9d88:0:b0:dcb:aa26:50fe with SMTP id v8-20020a259d88000000b00dcbaa2650femr6868080ybp.15.1709581452753; Mon, 04 Mar 2024 11:44:12 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Jorgen.Hansen@wdc.com, wj28.lee@gmail.com, Fan Ni Subject: [PATCH v5 04/13] hw/mem/cxl_type3: Add support to create DC regions to type3 memory devices Date: Mon, 4 Mar 2024 11:33:59 -0800 Message-ID: <20240304194331.1586191-5-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240304194331.1586191-1-nifan.cxl@gmail.com> References: <20240304194331.1586191-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::b2c; envelope-from=nifan.cxl@gmail.com; helo=mail-yb1-xb2c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1709581583834100015 Content-Type: text/plain; charset="utf-8" From: Fan Ni With the change, when setting up memory for type3 memory device, we can create DC regions. A property 'num-dc-regions' is added to ct3_props to allow users to pass the number of DC regions to create. To make it easier, other region parameters like region base, length, and block size are hard coded. If needed, these parameters can be added easily. With the change, we can create DC regions with proper kernel side support like below: region=3D$(cat /sys/bus/cxl/devices/decoder0.0/create_dc_region) echo $region > /sys/bus/cxl/devices/decoder0.0/create_dc_region echo 256 > /sys/bus/cxl/devices/$region/interleave_granularity echo 1 > /sys/bus/cxl/devices/$region/interleave_ways echo "dc0" >/sys/bus/cxl/devices/decoder2.0/mode echo 0x40000000 >/sys/bus/cxl/devices/decoder2.0/dpa_size echo 0x40000000 > /sys/bus/cxl/devices/$region/size echo "decoder2.0" > /sys/bus/cxl/devices/$region/target0 echo 1 > /sys/bus/cxl/devices/$region/commit echo $region > /sys/bus/cxl/drivers/cxl_region/bind Signed-off-by: Fan Ni Reviewed-by: Jonathan Cameron --- hw/mem/cxl_type3.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index 244d2b5fd5..a191211009 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -30,6 +30,7 @@ #include "hw/pci/msix.h" =20 #define DWORD_BYTE 4 +#define CXL_CAPACITY_MULTIPLIER (256 * MiB) =20 /* Default CDAT entries for a memory region */ enum { @@ -567,6 +568,45 @@ static void ct3d_reg_write(void *opaque, hwaddr offset= , uint64_t value, } } =20 +/* + * TODO: dc region configuration will be updated once host backend and add= ress + * space support is added for DCD. + */ +static bool cxl_create_dc_regions(CXLType3Dev *ct3d, Error **errp) +{ + int i; + uint64_t region_base =3D 0; + uint64_t region_len =3D 2 * GiB; + uint64_t decode_len =3D 2 * GiB; + uint64_t blk_size =3D 2 * MiB; + CXLDCRegion *region; + MemoryRegion *mr; + + if (ct3d->hostvmem) { + mr =3D host_memory_backend_get_memory(ct3d->hostvmem); + region_base +=3D memory_region_size(mr); + } + if (ct3d->hostpmem) { + mr =3D host_memory_backend_get_memory(ct3d->hostpmem); + region_base +=3D memory_region_size(mr); + } + assert(region_base % CXL_CAPACITY_MULTIPLIER =3D=3D 0); + + for (i =3D 0; i < ct3d->dc.num_regions; i++) { + region =3D &ct3d->dc.regions[i]; + region->base =3D region_base; + region->decode_len =3D decode_len; + region->len =3D region_len; + region->block_size =3D blk_size; + /* dsmad_handle is set when creating cdat table entries */ + region->flags =3D 0; + + region_base +=3D region->len; + } + + return true; +} + static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp) { DeviceState *ds =3D DEVICE(ct3d); @@ -635,6 +675,11 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error = **errp) g_free(p_name); } =20 + if (!cxl_create_dc_regions(ct3d, errp)) { + error_setg(errp, "setup DC regions failed"); + return false; + } + return true; } =20 @@ -930,6 +975,7 @@ static Property ct3_props[] =3D { HostMemoryBackend *), DEFINE_PROP_UINT64("sn", CXLType3Dev, sn, UI64_NULL), DEFINE_PROP_STRING("cdat", CXLType3Dev, cxl_cstate.cdat.filename), + DEFINE_PROP_UINT8("num-dc-regions", CXLType3Dev, dc.num_regions, 0), DEFINE_PROP_END_OF_LIST(), }; =20 --=20 2.43.0 From nobody Thu May 9 21:45:48 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1709581513; cv=none; d=zohomail.com; s=zohoarc; b=aIoOjHzErJVQ7B6g1xHVaqtPcMgba23gMUbj1nFzY2+fylRjzbqO4KDgn6P2si73DLtN5wqXCmCrIu7e+f/co+d72rDb+oDkl5BQQVG+eItywLWkS/aXejfpZIOfcdLPOPJUrOxg77dMx+ud143kalCXgtSxuHXPSJjrhSdw4b8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709581513; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=XOJjP1Hc+ZaCeqP1kL3i8IBpTTnk/Z2dT1K8zGM/Ls8=; b=Px4ltCjIAbGowsUfRRwQRhgZKL9p013okfQr/hSvpUL0BQj34fnA8KnU0aQ1CkR7UeBoyg7rBhZZTx6Hhu1zo4nzCX4Jkw9F8zbNzC25w85BX4qhdOg55KjnOhejD4ok+rkWiAxSWuaVE9e+UsVFwgozkdFXCQy5jbmR7hCBGZE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1709581513591588.2767511327996; Mon, 4 Mar 2024 11:45:13 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rhEEU-0005UO-Ms; Mon, 04 Mar 2024 14:44:18 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhEET-0005Th-41 for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:17 -0500 Received: from mail-yb1-xb2b.google.com ([2607:f8b0:4864:20::b2b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhEER-0006vb-De for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:16 -0500 Received: by mail-yb1-xb2b.google.com with SMTP id 3f1490d57ef6-dc74e33fe1bso4614021276.0 for ; Mon, 04 Mar 2024 11:44:15 -0800 (PST) Received: from localhost.localdomain ([50.205.20.42]) by smtp.gmail.com with ESMTPSA id h1-20020a255f41000000b00dc62edd58dasm2282646ybm.40.2024.03.04.11.44.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Mar 2024 11:44:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709581454; x=1710186254; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XOJjP1Hc+ZaCeqP1kL3i8IBpTTnk/Z2dT1K8zGM/Ls8=; b=mugcwJ7Zk6hwV9p6Avz+B53IuKPvABehX6wjH/0+2UFsrjqCWUAvyRbFQJjjPcXb8l aO+KvkjlN5Zc8KAzhxuWpYCJ3WvrSlRdL4igeY7S8B/NYP7qfZ8brEFVwVCbHBYg5YhM SoNMn8LLA+Y3be1PWiUYEWpMs+osOEcCOrHC2ERTcxepkeqN+/zI4z7/Vlsdg2OKsEHF F4ifB46edJ+Y679OH+RrrtZ6ox2gmlK6kxzBZNwNEpXtVB4GvzQt+NNT4dKdOgzoDLwQ 1vyqfCYXct1rCn03BhbB8b7+Polid18BjMR86bJeipEGsDNsDgAwSXIpKIYmAPYTVotp Tuuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709581454; x=1710186254; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XOJjP1Hc+ZaCeqP1kL3i8IBpTTnk/Z2dT1K8zGM/Ls8=; b=xGrJ65ZqnVanDTMBzocvSkSGh7f8CwCE1viIXCRw3mgfYZ5Igqc0GFUuvbMvqe1LZj K9BVNmDDzR66g8KeSRJqr38Q8EQo+HmP23NDlpyjOWFVuhnOHdj5jLG3ywpwJfVmH1QR Z8WAlZWLwYojZnuPmJkVIMOqiserjZEh4rJ3RFMO4u+3rkFuDzUdIOjzl7cBURjoMqny eQb34ahNhGM2mKaCFhIHcLYufdqBjxx7I30a6Pa/9lnGfMhJBH+0nQp/DrvLFd18jr9r 1FJtxRiUTp5QcUb9JMK4t4FN0FSWehwUwAmR3cRKM5sdRTdOlUHD3wqJ5JQ90aqE9vvp n7IQ== X-Gm-Message-State: AOJu0YxO+21CYDnmMymVIU+9032pGraL9cd7oqTmDxDNZCikGgQSELJS GncDZqjREhziSgW2kyLnAu3N9WPqP82vwfyhIXkCzlkzHVfmadm73iKIw6lh X-Google-Smtp-Source: AGHT+IFL7lKbuf/q1X2xmA0sVMudvhJVA07sf4CQSydEOjZZtUETHFFOZl201CiddOHVU9PW/PCFSQ== X-Received: by 2002:a05:6902:2306:b0:dc7:496e:42e1 with SMTP id do6-20020a056902230600b00dc7496e42e1mr7811375ybb.51.1709581454151; Mon, 04 Mar 2024 11:44:14 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Jorgen.Hansen@wdc.com, wj28.lee@gmail.com, Fan Ni Subject: [PATCH v5 05/13] hw/mem/cxl-type3: Refactor ct3_build_cdat_entries_for_mr to take mr size insead of mr as argument Date: Mon, 4 Mar 2024 11:34:00 -0800 Message-ID: <20240304194331.1586191-6-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240304194331.1586191-1-nifan.cxl@gmail.com> References: <20240304194331.1586191-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::b2b; envelope-from=nifan.cxl@gmail.com; helo=mail-yb1-xb2b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1709581515745100005 Content-Type: text/plain; charset="utf-8" From: Fan Ni The function ct3_build_cdat_entries_for_mr only uses size of the passed memory region argument, refactor the function definition to make the passed arguments more specific. Signed-off-by: Fan Ni Reviewed-by: Jonathan Cameron --- hw/mem/cxl_type3.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index a191211009..c045fee32d 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -44,7 +44,7 @@ enum { }; =20 static void ct3_build_cdat_entries_for_mr(CDATSubHeader **cdat_table, - int dsmad_handle, MemoryRegion *= mr, + int dsmad_handle, uint64_t size, bool is_pmem, uint64_t dpa_base) { g_autofree CDATDsmas *dsmas =3D NULL; @@ -63,7 +63,7 @@ static void ct3_build_cdat_entries_for_mr(CDATSubHeader *= *cdat_table, .DSMADhandle =3D dsmad_handle, .flags =3D is_pmem ? CDAT_DSMAS_FLAG_NV : 0, .DPA_base =3D dpa_base, - .DPA_length =3D memory_region_size(mr), + .DPA_length =3D size, }; =20 /* For now, no memory side cache, plausiblish numbers */ @@ -132,7 +132,7 @@ static void ct3_build_cdat_entries_for_mr(CDATSubHeader= **cdat_table, */ .EFI_memory_type_attr =3D is_pmem ? 2 : 1, .DPA_offset =3D 0, - .DPA_length =3D memory_region_size(mr), + .DPA_length =3D size, }; =20 /* Header always at start of structure */ @@ -149,6 +149,7 @@ static int ct3_build_cdat_table(CDATSubHeader ***cdat_t= able, void *priv) g_autofree CDATSubHeader **table =3D NULL; CXLType3Dev *ct3d =3D priv; MemoryRegion *volatile_mr =3D NULL, *nonvolatile_mr =3D NULL; + uint64_t vmr_size =3D 0, pmr_size =3D 0; int dsmad_handle =3D 0; int cur_ent =3D 0; int len =3D 0; @@ -163,6 +164,7 @@ static int ct3_build_cdat_table(CDATSubHeader ***cdat_t= able, void *priv) return -EINVAL; } len +=3D CT3_CDAT_NUM_ENTRIES; + vmr_size =3D memory_region_size(volatile_mr); } =20 if (ct3d->hostpmem) { @@ -171,21 +173,22 @@ static int ct3_build_cdat_table(CDATSubHeader ***cdat= _table, void *priv) return -EINVAL; } len +=3D CT3_CDAT_NUM_ENTRIES; + pmr_size =3D memory_region_size(nonvolatile_mr); } =20 table =3D g_malloc0(len * sizeof(*table)); =20 /* Now fill them in */ if (volatile_mr) { - ct3_build_cdat_entries_for_mr(table, dsmad_handle++, volatile_mr, + ct3_build_cdat_entries_for_mr(table, dsmad_handle++, vmr_size, false, 0); cur_ent =3D CT3_CDAT_NUM_ENTRIES; } =20 if (nonvolatile_mr) { - uint64_t base =3D volatile_mr ? memory_region_size(volatile_mr) : = 0; + uint64_t base =3D vmr_size; ct3_build_cdat_entries_for_mr(&(table[cur_ent]), dsmad_handle++, - nonvolatile_mr, true, base); + pmr_size, true, base); cur_ent +=3D CT3_CDAT_NUM_ENTRIES; } assert(len =3D=3D cur_ent); --=20 2.43.0 From nobody Thu May 9 21:45:48 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1709581575; cv=none; d=zohomail.com; s=zohoarc; b=aVBHAe+gK/Ae1O98zeHOb7XSRgjsdr6MCZAHXiOZYenQVfLuSmCPS2YigKMkYp6VJNH2eTtZW38+HnSrVAw414xNFWv4XL+5YwlZuCrqeVix5wKicGXa/VELwPlGEa0+O7YmmiOLfLqs3qNlV6C6/MDmH5v9U1agAC9ZxKGx/c4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709581575; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=PfECBT/sLbntByDSR3pn5gaUEK0qP+L5lceYFnDxHOg=; b=kFl5tJ5ymGKxoame79PiS5EohY7UtShLRhjk7rXa5A/8O2Hvj9PYk4WM3TasEQjrjKMKzflyPCJgbFCJ11o98xzAzeQeA/L1R7OkSv22TjwW1wS0nLg0TRhNAfLMt1gx5kpm+beIc0gYsmEiRRzvhTvzSdjdm/h1PG77IQQXVMQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1709581575500614.5194540305321; Mon, 4 Mar 2024 11:46:15 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rhEEk-0005XD-WC; Mon, 04 Mar 2024 14:44:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhEEd-0005VC-Jr for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:29 -0500 Received: from mail-yw1-x112c.google.com ([2607:f8b0:4864:20::112c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhEEU-0006vo-MF for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:20 -0500 Received: by mail-yw1-x112c.google.com with SMTP id 00721157ae682-6098b9ed2a3so25014537b3.0 for ; Mon, 04 Mar 2024 11:44:17 -0800 (PST) Received: from localhost.localdomain ([50.205.20.42]) by smtp.gmail.com with ESMTPSA id h1-20020a255f41000000b00dc62edd58dasm2282646ybm.40.2024.03.04.11.44.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Mar 2024 11:44:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709581456; x=1710186256; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PfECBT/sLbntByDSR3pn5gaUEK0qP+L5lceYFnDxHOg=; b=dnm6gqMVDEQuTYlg1/TnrQDzgWIE+ay7uYuQyJ4FYQHrWYriYZMMhUF7B7PVvJQUM7 USTxJJo8ip1IxuRFmNQ1D4QkXfiupnfzvIXmUGhm7cKTXXBSyuzI01PUXLCoyoSiaIqo d6KJAx5KYIsGO8myCnnemOt/97xzVcWd9dBOLZjpXRO1LZlxfEUV+VtEf/KrgYdwOR5G KwxqsZBPa4WDuKpKdethsmpAM31fJyvRCkc485c5Zo6Av6vbVuAnL36MadBuR1ujxIZY ONYxixtv+q3GvjxnDZfgjrrCX96981/VVnPDthWCMiDVjOMSzqJ4DIezPDwwhbfPdV8X vjnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709581456; x=1710186256; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PfECBT/sLbntByDSR3pn5gaUEK0qP+L5lceYFnDxHOg=; b=iOr5VoYnf3Qxxj/fzIz4eWrGoJVPbx31ic/Cz983rmBCc8kOaCfrsTYMAggXxGhi4v ejQ8Q4vp6KHknSX51iLOqQWYYYa5wOr6dF4OAc3Pu7PriveI4DozfuTYUn1i4s5ZyoTF BmRDRMg6RdQZxBFZZEpaCH7W/minnedLXl3ZNnrDAsjQwXN1dMKNabgcKlRONQBnx4Ra tc4vr8e6vkmmN3Pg90zXnyE/46FvW3fzCQvsyn4uTIcRmNgCYfjPXR8raFCzMa06Jfi3 eLmvPCdNSENc5d1hBUzjXQkJLg1GhvbJ4bqmP5UD0qli8eJMuCciK+yKVyYVf9X7VXak bWkw== X-Gm-Message-State: AOJu0YzEtW0UxyEgN5E1j1FCBJHaX69IbZniKof2OfR2LFTTe4Ng6QNI C4cgqG709SdbjtMfrvtRbSNTg0/BWAb4AGsC0XBGsTSo5eP2FaisihlNkI1n X-Google-Smtp-Source: AGHT+IH57Jlo7NmUoCdrFXG2iZYWUpRvZrlyGp9KTCdn+o1CPYk8N74rPmo/Nkf3DTxZgvVi/v4qvQ== X-Received: by 2002:a25:f44c:0:b0:dcd:4257:45e6 with SMTP id p12-20020a25f44c000000b00dcd425745e6mr6725453ybe.35.1709581456146; Mon, 04 Mar 2024 11:44:16 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Jorgen.Hansen@wdc.com, wj28.lee@gmail.com, Fan Ni Subject: [PATCH v5 06/13] hw/mem/cxl_type3: Add host backend and address space handling for DC regions Date: Mon, 4 Mar 2024 11:34:01 -0800 Message-ID: <20240304194331.1586191-7-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240304194331.1586191-1-nifan.cxl@gmail.com> References: <20240304194331.1586191-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::112c; envelope-from=nifan.cxl@gmail.com; helo=mail-yw1-x112c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1709581577829100007 Content-Type: text/plain; charset="utf-8" From: Fan Ni Add (file/memory backed) host backend, all the dynamic capacity regions will share a single, large enough host backend. Set up address space for DC regions to support read/write operations to dynamic capacity for DCD. With the change, following supports are added: 1. Add a new property to type3 device "volatile-dc-memdev" to point to host memory backend for dynamic capacity. Currently, all dc regions share one host backend. 2. Add namespace for dynamic capacity for read/write support; 3. Create cdat entries for each dynamic capacity region; 4. Fix dvsec range registers to include DC regions. Signed-off-by: Fan Ni --- hw/cxl/cxl-mailbox-utils.c | 16 ++- hw/mem/cxl_type3.c | 189 +++++++++++++++++++++++++++++------- include/hw/cxl/cxl_device.h | 4 + 3 files changed, 170 insertions(+), 39 deletions(-) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index 853dadba39..8309f27a2b 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -622,7 +622,8 @@ static CXLRetCode cmd_firmware_update_get_info(const st= ruct cxl_cmd *cmd, size_t *len_out, CXLCCI *cci) { - CXLDeviceState *cxl_dstate =3D &CXL_TYPE3(cci->d)->cxl_dstate; + CXLType3Dev *ct3d =3D CXL_TYPE3(cci->d); + CXLDeviceState *cxl_dstate =3D &ct3d->cxl_dstate; struct { uint8_t slots_supported; uint8_t slot_info; @@ -636,7 +637,8 @@ static CXLRetCode cmd_firmware_update_get_info(const st= ruct cxl_cmd *cmd, QEMU_BUILD_BUG_ON(sizeof(*fw_info) !=3D 0x50); =20 if ((cxl_dstate->vmem_size < CXL_CAPACITY_MULTIPLIER) || - (cxl_dstate->pmem_size < CXL_CAPACITY_MULTIPLIER)) { + (cxl_dstate->pmem_size < CXL_CAPACITY_MULTIPLIER) || + (ct3d->dc.total_capacity < CXL_CAPACITY_MULTIPLIER)) { return CXL_MBOX_INTERNAL_ERROR; } =20 @@ -793,7 +795,8 @@ static CXLRetCode cmd_identify_memory_device(const stru= ct cxl_cmd *cmd, CXLDeviceState *cxl_dstate =3D &ct3d->cxl_dstate; =20 if ((!QEMU_IS_ALIGNED(cxl_dstate->vmem_size, CXL_CAPACITY_MULTIPLIER))= || - (!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER))= ) { + (!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER))= || + (!QEMU_IS_ALIGNED(ct3d->dc.total_capacity, CXL_CAPACITY_MULTIPLIER= ))) { return CXL_MBOX_INTERNAL_ERROR; } =20 @@ -835,9 +838,11 @@ static CXLRetCode cmd_ccls_get_partition_info(const st= ruct cxl_cmd *cmd, uint64_t next_pmem; } QEMU_PACKED *part_info =3D (void *)payload_out; QEMU_BUILD_BUG_ON(sizeof(*part_info) !=3D 0x20); + CXLType3Dev *ct3d =3D container_of(cxl_dstate, CXLType3Dev, cxl_dstate= ); =20 if ((!QEMU_IS_ALIGNED(cxl_dstate->vmem_size, CXL_CAPACITY_MULTIPLIER))= || - (!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER))= ) { + (!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER))= || + (!QEMU_IS_ALIGNED(ct3d->dc.total_capacity, CXL_CAPACITY_MULTIPLIER= ))) { return CXL_MBOX_INTERNAL_ERROR; } =20 @@ -1179,7 +1184,8 @@ static CXLRetCode cmd_media_clear_poison(const struct= cxl_cmd *cmd, struct clear_poison_pl *in =3D (void *)payload_in; =20 dpa =3D ldq_le_p(&in->dpa); - if (dpa + CXL_CACHE_LINE_SIZE > cxl_dstate->static_mem_size) { + if (dpa + CXL_CACHE_LINE_SIZE > cxl_dstate->static_mem_size + + ct3d->dc.total_capacity) { return CXL_MBOX_INVALID_PA; } =20 diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index c045fee32d..2b380a260b 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -45,7 +45,8 @@ enum { =20 static void ct3_build_cdat_entries_for_mr(CDATSubHeader **cdat_table, int dsmad_handle, uint64_t size, - bool is_pmem, uint64_t dpa_base) + bool is_pmem, bool is_dynamic, + uint64_t dpa_base) { g_autofree CDATDsmas *dsmas =3D NULL; g_autofree CDATDslbis *dslbis0 =3D NULL; @@ -61,7 +62,8 @@ static void ct3_build_cdat_entries_for_mr(CDATSubHeader *= *cdat_table, .length =3D sizeof(*dsmas), }, .DSMADhandle =3D dsmad_handle, - .flags =3D is_pmem ? CDAT_DSMAS_FLAG_NV : 0, + .flags =3D (is_pmem ? CDAT_DSMAS_FLAG_NV : 0) | + (is_dynamic ? CDAT_DSMAS_FLAG_DYNAMIC_CAP : 0), .DPA_base =3D dpa_base, .DPA_length =3D size, }; @@ -149,12 +151,13 @@ static int ct3_build_cdat_table(CDATSubHeader ***cdat= _table, void *priv) g_autofree CDATSubHeader **table =3D NULL; CXLType3Dev *ct3d =3D priv; MemoryRegion *volatile_mr =3D NULL, *nonvolatile_mr =3D NULL; + MemoryRegion *dc_mr =3D NULL; uint64_t vmr_size =3D 0, pmr_size =3D 0; int dsmad_handle =3D 0; int cur_ent =3D 0; int len =3D 0; =20 - if (!ct3d->hostpmem && !ct3d->hostvmem) { + if (!ct3d->hostpmem && !ct3d->hostvmem && !ct3d->dc.num_regions) { return 0; } =20 @@ -176,21 +179,55 @@ static int ct3_build_cdat_table(CDATSubHeader ***cdat= _table, void *priv) pmr_size =3D memory_region_size(nonvolatile_mr); } =20 + if (ct3d->dc.num_regions) { + if (ct3d->dc.host_dc) { + dc_mr =3D host_memory_backend_get_memory(ct3d->dc.host_dc); + if (!dc_mr) { + return -EINVAL; + } + len +=3D CT3_CDAT_NUM_ENTRIES * ct3d->dc.num_regions; + } else { + return -EINVAL; + } + } + table =3D g_malloc0(len * sizeof(*table)); =20 /* Now fill them in */ if (volatile_mr) { ct3_build_cdat_entries_for_mr(table, dsmad_handle++, vmr_size, - false, 0); + false, false, 0); cur_ent =3D CT3_CDAT_NUM_ENTRIES; } =20 if (nonvolatile_mr) { uint64_t base =3D vmr_size; ct3_build_cdat_entries_for_mr(&(table[cur_ent]), dsmad_handle++, - pmr_size, true, base); + pmr_size, true, false, base); cur_ent +=3D CT3_CDAT_NUM_ENTRIES; } + + if (dc_mr) { + int i; + uint64_t region_base =3D vmr_size + pmr_size; + + /* + * TODO: we assume the dynamic capacity to be volatile for now, + * non-volatile dynamic capacity will be added if needed in the + * future. + */ + for (i =3D 0; i < ct3d->dc.num_regions; i++) { + ct3_build_cdat_entries_for_mr(&(table[cur_ent]), + dsmad_handle++, + ct3d->dc.regions[i].len, + false, true, region_base); + ct3d->dc.regions[i].dsmadhandle =3D dsmad_handle - 1; + + cur_ent +=3D CT3_CDAT_NUM_ENTRIES; + region_base +=3D ct3d->dc.regions[i].len; + } + } + assert(len =3D=3D cur_ent); =20 *cdat_table =3D g_steal_pointer(&table); @@ -300,11 +337,24 @@ static void build_dvsecs(CXLType3Dev *ct3d) range2_size_hi =3D ct3d->hostpmem->size >> 32; range2_size_lo =3D (2 << 5) | (2 << 2) | 0x3 | (ct3d->hostpmem->size & 0xF0000000); + } else if (ct3d->dc.host_dc) { + range2_size_hi =3D ct3d->dc.host_dc->size >> 32; + range2_size_lo =3D (2 << 5) | (2 << 2) | 0x3 | + (ct3d->dc.host_dc->size & 0xF0000000); } - } else { + } else if (ct3d->hostpmem) { range1_size_hi =3D ct3d->hostpmem->size >> 32; range1_size_lo =3D (2 << 5) | (2 << 2) | 0x3 | (ct3d->hostpmem->size & 0xF0000000); + if (ct3d->dc.host_dc) { + range2_size_hi =3D ct3d->dc.host_dc->size >> 32; + range2_size_lo =3D (2 << 5) | (2 << 2) | 0x3 | + (ct3d->dc.host_dc->size & 0xF0000000); + } + } else { + range1_size_hi =3D ct3d->dc.host_dc->size >> 32; + range1_size_lo =3D (2 << 5) | (2 << 2) | 0x3 | + (ct3d->dc.host_dc->size & 0xF0000000); } =20 dvsec =3D (uint8_t *)&(CXLDVSECDevice){ @@ -579,11 +629,27 @@ static bool cxl_create_dc_regions(CXLType3Dev *ct3d, = Error **errp) { int i; uint64_t region_base =3D 0; - uint64_t region_len =3D 2 * GiB; - uint64_t decode_len =3D 2 * GiB; + uint64_t region_len; + uint64_t decode_len; uint64_t blk_size =3D 2 * MiB; CXLDCRegion *region; MemoryRegion *mr; + uint64_t dc_size; + + mr =3D host_memory_backend_get_memory(ct3d->dc.host_dc); + dc_size =3D memory_region_size(mr); + region_len =3D DIV_ROUND_UP(dc_size, ct3d->dc.num_regions); + + if (region_len * ct3d->dc.num_regions > dc_size) { + error_setg(errp, "host backend size must be multiples of region le= n"); + return false; + } + if (region_len % CXL_CAPACITY_MULTIPLIER !=3D 0) { + error_setg(errp, "DC region size is unaligned to %lx", + CXL_CAPACITY_MULTIPLIER); + return false; + } + decode_len =3D region_len; =20 if (ct3d->hostvmem) { mr =3D host_memory_backend_get_memory(ct3d->hostvmem); @@ -605,6 +671,7 @@ static bool cxl_create_dc_regions(CXLType3Dev *ct3d, Er= ror **errp) region->flags =3D 0; =20 region_base +=3D region->len; + ct3d->dc.total_capacity +=3D region->len; } =20 return true; @@ -614,7 +681,8 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error *= *errp) { DeviceState *ds =3D DEVICE(ct3d); =20 - if (!ct3d->hostmem && !ct3d->hostvmem && !ct3d->hostpmem) { + if (!ct3d->hostmem && !ct3d->hostvmem && !ct3d->hostpmem + && !ct3d->dc.num_regions) { error_setg(errp, "at least one memdev property must be set"); return false; } else if (ct3d->hostmem && ct3d->hostpmem) { @@ -678,9 +746,41 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error = **errp) g_free(p_name); } =20 - if (!cxl_create_dc_regions(ct3d, errp)) { - error_setg(errp, "setup DC regions failed"); - return false; + ct3d->dc.total_capacity =3D 0; + if (ct3d->dc.num_regions) { + MemoryRegion *dc_mr; + char *dc_name; + + if (!ct3d->dc.host_dc) { + error_setg(errp, "dynamic capacity must have a backing device"= ); + return false; + } + + dc_mr =3D host_memory_backend_get_memory(ct3d->dc.host_dc); + if (!dc_mr) { + error_setg(errp, "dynamic capacity must have a backing device"= ); + return false; + } + + /* + * TODO: set dc as volatile for now, non-volatile support can be a= dded + * in the future if needed. + */ + memory_region_set_nonvolatile(dc_mr, false); + memory_region_set_enabled(dc_mr, true); + host_memory_backend_set_mapped(ct3d->dc.host_dc, true); + if (ds->id) { + dc_name =3D g_strdup_printf("cxl-dcd-dpa-dc-space:%s", ds->id); + } else { + dc_name =3D g_strdup("cxl-dcd-dpa-dc-space"); + } + address_space_init(&ct3d->dc.host_dc_as, dc_mr, dc_name); + g_free(dc_name); + + if (!cxl_create_dc_regions(ct3d, errp)) { + error_setg(errp, "setup DC regions failed"); + return false; + } } =20 return true; @@ -772,6 +872,9 @@ err_release_cdat: err_free_special_ops: g_free(regs->special_ops); err_address_space_free: + if (ct3d->dc.host_dc) { + address_space_destroy(&ct3d->dc.host_dc_as); + } if (ct3d->hostpmem) { address_space_destroy(&ct3d->hostpmem_as); } @@ -790,6 +893,9 @@ static void ct3_exit(PCIDevice *pci_dev) pcie_aer_exit(pci_dev); cxl_doe_cdat_release(cxl_cstate); g_free(regs->special_ops); + if (ct3d->dc.host_dc) { + address_space_destroy(&ct3d->dc.host_dc_as); + } if (ct3d->hostpmem) { address_space_destroy(&ct3d->hostpmem_as); } @@ -868,16 +974,24 @@ static int cxl_type3_hpa_to_as_and_dpa(CXLType3Dev *c= t3d, AddressSpace **as, uint64_t *dpa_offset) { - MemoryRegion *vmr =3D NULL, *pmr =3D NULL; + MemoryRegion *vmr =3D NULL, *pmr =3D NULL, *dc_mr =3D NULL; + uint64_t vmr_size =3D 0, pmr_size =3D 0, dc_size =3D 0; =20 if (ct3d->hostvmem) { vmr =3D host_memory_backend_get_memory(ct3d->hostvmem); + vmr_size =3D memory_region_size(vmr); } if (ct3d->hostpmem) { pmr =3D host_memory_backend_get_memory(ct3d->hostpmem); + pmr_size =3D memory_region_size(pmr); + } + if (ct3d->dc.host_dc) { + dc_mr =3D host_memory_backend_get_memory(ct3d->dc.host_dc); + /* Do we want dc_size to be dc_mr->size or not?? */ + dc_size =3D ct3d->dc.total_capacity; } =20 - if (!vmr && !pmr) { + if (!vmr && !pmr && !dc_mr) { return -ENODEV; } =20 @@ -885,19 +999,18 @@ static int cxl_type3_hpa_to_as_and_dpa(CXLType3Dev *c= t3d, return -EINVAL; } =20 - if (*dpa_offset > ct3d->cxl_dstate.static_mem_size) { + if (*dpa_offset >=3D vmr_size + pmr_size + dc_size) { return -EINVAL; } =20 - if (vmr) { - if (*dpa_offset < memory_region_size(vmr)) { - *as =3D &ct3d->hostvmem_as; - } else { - *as =3D &ct3d->hostpmem_as; - *dpa_offset -=3D memory_region_size(vmr); - } - } else { + if (*dpa_offset < vmr_size) { + *as =3D &ct3d->hostvmem_as; + } else if (*dpa_offset < vmr_size + pmr_size) { *as =3D &ct3d->hostpmem_as; + *dpa_offset -=3D vmr_size; + } else { + *as =3D &ct3d->dc.host_dc_as; + *dpa_offset -=3D (vmr_size + pmr_size); } =20 return 0; @@ -979,6 +1092,8 @@ static Property ct3_props[] =3D { DEFINE_PROP_UINT64("sn", CXLType3Dev, sn, UI64_NULL), DEFINE_PROP_STRING("cdat", CXLType3Dev, cxl_cstate.cdat.filename), DEFINE_PROP_UINT8("num-dc-regions", CXLType3Dev, dc.num_regions, 0), + DEFINE_PROP_LINK("volatile-dc-memdev", CXLType3Dev, dc.host_dc, + TYPE_MEMORY_BACKEND, HostMemoryBackend *), DEFINE_PROP_END_OF_LIST(), }; =20 @@ -1045,33 +1160,39 @@ static void set_lsa(CXLType3Dev *ct3d, const void *= buf, uint64_t size, =20 static bool set_cacheline(CXLType3Dev *ct3d, uint64_t dpa_offset, uint8_t = *data) { - MemoryRegion *vmr =3D NULL, *pmr =3D NULL; + MemoryRegion *vmr =3D NULL, *pmr =3D NULL, *dc_mr =3D NULL; AddressSpace *as; + uint64_t vmr_size =3D 0, pmr_size =3D 0, dc_size =3D 0; =20 if (ct3d->hostvmem) { vmr =3D host_memory_backend_get_memory(ct3d->hostvmem); + vmr_size =3D memory_region_size(vmr); } if (ct3d->hostpmem) { pmr =3D host_memory_backend_get_memory(ct3d->hostpmem); + pmr_size =3D memory_region_size(pmr); } + if (ct3d->dc.host_dc) { + dc_mr =3D host_memory_backend_get_memory(ct3d->dc.host_dc); + dc_size =3D ct3d->dc.total_capacity; + } =20 - if (!vmr && !pmr) { + if (!vmr && !pmr && !dc_mr) { return false; } =20 - if (dpa_offset + CXL_CACHE_LINE_SIZE > ct3d->cxl_dstate.static_mem_siz= e) { + if (dpa_offset + CXL_CACHE_LINE_SIZE > vmr_size + pmr_size + dc_size) { return false; } =20 - if (vmr) { - if (dpa_offset < memory_region_size(vmr)) { - as =3D &ct3d->hostvmem_as; - } else { - as =3D &ct3d->hostpmem_as; - dpa_offset -=3D memory_region_size(vmr); - } - } else { + if (dpa_offset < vmr_size) { + as =3D &ct3d->hostvmem_as; + } else if (dpa_offset < vmr_size + pmr_size) { as =3D &ct3d->hostpmem_as; + dpa_offset -=3D vmr_size; + } else { + as =3D &ct3d->dc.host_dc_as; + dpa_offset -=3D (vmr_size + pmr_size); } =20 address_space_write(as, dpa_offset, MEMTXATTRS_UNSPECIFIED, &data, diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index f82d018422..265679302c 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -467,6 +467,10 @@ struct CXLType3Dev { uint64_t poison_list_overflow_ts; =20 struct dynamic_capacity { + HostMemoryBackend *host_dc; + AddressSpace host_dc_as; + uint64_t total_capacity; /* 256M aligned */ + uint8_t num_regions; /* 0-8 regions */ CXLDCRegion regions[DCD_MAX_NUM_REGION]; } dc; --=20 2.43.0 From nobody Thu May 9 21:45:48 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1709581515; cv=none; d=zohomail.com; s=zohoarc; b=Y4uYJegJDPVIU1YlzzoTxWZqAihFzeJDeBGSUXWyeeXJdXEGYTPMDHnLya2enLEQgAoPJjAjUnH3IeSkjZcp38YvE85faPI8OPqha0EVf+LBrOKzmnI4SL6hQSgtZtso/obq3gythciawWOphC1DMRPpQ5Eig4QUFe8UBrZ2knQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709581515; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=g8lPD938tgQqBzbyLXCNS0GDAHx1dZoU8O4sZyz4+ao=; b=GgJgxs4iFc24Ic9vMEillIpqrYMjQrtWFFbGMG+8ELVzU12+Ttp585D7I68H7zucRmPNcmZRYvzrYQ2Nwt0Afi7MCphLQYnC/6EcH1OnNkeusLZZCZ9rtEtaMiNRi+blWe49HtN/eyJ6skwofObQVX2fmkF00I6h4MV6pbYjXBE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1709581515923631.6195771788346; Mon, 4 Mar 2024 11:45:15 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rhEEh-0005WN-Ey; Mon, 04 Mar 2024 14:44:31 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhEEf-0005VR-1y for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:29 -0500 Received: from mail-yw1-x1131.google.com ([2607:f8b0:4864:20::1131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhEEW-0006wX-Jn for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:28 -0500 Received: by mail-yw1-x1131.google.com with SMTP id 00721157ae682-607c5679842so48932017b3.2 for ; Mon, 04 Mar 2024 11:44:20 -0800 (PST) Received: from localhost.localdomain ([50.205.20.42]) by smtp.gmail.com with ESMTPSA id h1-20020a255f41000000b00dc62edd58dasm2282646ybm.40.2024.03.04.11.44.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Mar 2024 11:44:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709581459; x=1710186259; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=g8lPD938tgQqBzbyLXCNS0GDAHx1dZoU8O4sZyz4+ao=; b=Y66kDvOWkiZH98Cxn30Z8/RKSgR67scwQ6Nppy2EdP14DY8aZuQGFOA01dM5N6/y0l q7b4sLfWqe+unM5bjStTJNyShfDbyqRKdwptSt3UIqOc1ZZGggtvX5Lihc8UJfxRRFpS LJh86TULZbpL9WeMg0FvodBm0XFgyyo8l6SU5p6qI6Osn26YjLZclTAeP4KOynfM22Xn flBj9qovCAPAtCYjS2b4ZvAkmLEyvorSc7jHdfXo5YsZdKubFRBJ3Y6X0oKYvPm5mX1C Kcmfetj7DlFLD/AXzHoJ2YDCsYhXbEN1Bzz8P1DzBwTf+MQHu3Ph7nJOxnfrEjhRC8pw coAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709581459; x=1710186259; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=g8lPD938tgQqBzbyLXCNS0GDAHx1dZoU8O4sZyz4+ao=; b=ggYmC+PZDy5NLy/EMSe6r5QPp5x/lcH/+DfAtKcMVhjPp/xDO6yEExooNpV3UTooZH 2f47j68wpJ2wYrfOpMWWj/nJwr2RYFIrTtSxqOSZD65EDn8e5KyD4oJC3IjacDWlDGym Kp3C5gbqZFtzBGYOn59FDyRV6ERmrkR1zMDvub4KUxjq+GyYPE/xYGurW1slsaSRgHME QDHqUKf7u/HgBoQ2XmE/pF1KtrUCF8O07uX73bWUJ7+DwvTUaz0i16/Qevv96Wv09Ez+ lRhUldCyJuwKcJQgMW4nNSimHhdmgcQHoeZUcKVqmF8sR8jSA6lr3D5uR18RRiTIQTqa uyrw== X-Gm-Message-State: AOJu0Yx6R+Wk8P+1nq7AsdkDCtXL6G05SrE4ZlFAfWH7VoBVlXCSy/Gx HJ4Gj6q8EaYQybfkdM87NpUZNsvZdWGxuV/yD/hckHt2Rj5QcmNh1C5zPyUk X-Google-Smtp-Source: AGHT+IGMxwD7M3abEPAmA4B7Id72YrzI5ZSRaZbmHCQNrQ/PXng4IFuOLlXZLROHo70LffTiLhR0wg== X-Received: by 2002:a25:5f48:0:b0:dbf:6267:eba4 with SMTP id h8-20020a255f48000000b00dbf6267eba4mr7589407ybm.27.1709581458654; Mon, 04 Mar 2024 11:44:18 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Jorgen.Hansen@wdc.com, wj28.lee@gmail.com, Fan Ni Subject: [PATCH v5 07/13] hw/mem/cxl_type3: Add DC extent list representative and get DC extent list mailbox support Date: Mon, 4 Mar 2024 11:34:02 -0800 Message-ID: <20240304194331.1586191-8-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240304194331.1586191-1-nifan.cxl@gmail.com> References: <20240304194331.1586191-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::1131; envelope-from=nifan.cxl@gmail.com; helo=mail-yw1-x1131.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1709581517620100011 Content-Type: text/plain; charset="utf-8" From: Fan Ni Add dynamic capacity extent list representative to the definition of CXLType3Dev and add get DC extent list mailbox command per CXL.spec.3.1:.8.2.9.9.9.2. Signed-off-by: Fan Ni --- hw/cxl/cxl-mailbox-utils.c | 71 ++++++++++++++++++++++++++++++++++++- hw/mem/cxl_type3.c | 1 + include/hw/cxl/cxl_device.h | 22 ++++++++++++ 3 files changed, 93 insertions(+), 1 deletion(-) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index 8309f27a2b..425b378a2c 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -84,6 +84,7 @@ enum { #define CLEAR_POISON 0x2 DCD_CONFIG =3D 0x48, #define GET_DC_CONFIG 0x0 + #define GET_DYN_CAP_EXT_LIST 0x1 PHYSICAL_SWITCH =3D 0x51, #define IDENTIFY_SWITCH_DEVICE 0x0 #define GET_PHYSICAL_PORT_STATE 0x1 @@ -1325,7 +1326,8 @@ static CXLRetCode cmd_dcd_get_dyn_cap_config(const st= ruct cxl_cmd *cmd, * to use. */ stl_le_p(&extra_out->num_extents_supported, CXL_NUM_EXTENTS_SUPPORTED); - stl_le_p(&extra_out->num_extents_available, CXL_NUM_EXTENTS_SUPPORTED); + stl_le_p(&extra_out->num_extents_available, CXL_NUM_EXTENTS_SUPPORTED - + ct3d->dc.total_extent_count); stl_le_p(&extra_out->num_tags_supported, CXL_NUM_TAGS_SUPPORTED); stl_le_p(&extra_out->num_tags_available, CXL_NUM_TAGS_SUPPORTED); =20 @@ -1333,6 +1335,70 @@ static CXLRetCode cmd_dcd_get_dyn_cap_config(const s= truct cxl_cmd *cmd, return CXL_MBOX_SUCCESS; } =20 +/* + * CXL r3.1 section 8.2.9.9.9.2: + * Get Dynamic Capacity Extent List (Opcode 4801h) + */ +static CXLRetCode cmd_dcd_get_dyn_cap_ext_list(const struct cxl_cmd *cmd, + uint8_t *payload_in, + size_t len_in, + uint8_t *payload_out, + size_t *len_out, + CXLCCI *cci) +{ + CXLType3Dev *ct3d =3D CXL_TYPE3(cci->d); + struct { + uint32_t extent_cnt; + uint32_t start_extent_id; + } QEMU_PACKED *in =3D (void *)payload_in; + struct { + uint32_t count; + uint32_t total_extents; + uint32_t generation_num; + uint8_t rsvd[4]; + CXLDCExtentRaw records[]; + } QEMU_PACKED *out =3D (void *)payload_out; + uint16_t record_count =3D 0, i =3D 0, record_done =3D 0; + uint16_t out_pl_len; + uint32_t start_extent_id =3D in->start_extent_id; + CXLDCExtentList *extent_list =3D &ct3d->dc.extents; + CXLDCExtent *ent; + + if (start_extent_id > ct3d->dc.total_extent_count) { + return CXL_MBOX_INVALID_INPUT; + } + + record_count =3D MIN(in->extent_cnt, + ct3d->dc.total_extent_count - start_extent_id); + + out_pl_len =3D sizeof(*out) + record_count * sizeof(out->records[0]); + assert(out_pl_len <=3D CXL_MAILBOX_MAX_PAYLOAD_SIZE); + + stl_le_p(&out->count, record_count); + stl_le_p(&out->total_extents, ct3d->dc.total_extent_count); + stl_le_p(&out->generation_num, ct3d->dc.ext_list_gen_seq); + + if (record_count > 0) { + QTAILQ_FOREACH(ent, extent_list, node) { + if (i++ < start_extent_id) { + continue; + } + stq_le_p(&out->records[record_done].start_dpa, ent->start_dpa); + stq_le_p(&out->records[record_done].len, ent->len); + memcpy(&out->records[record_done].tag, ent->tag, 0x10); + stw_le_p(&out->records[record_done].shared_seq, ent->shared_se= q); + + record_done++; + if (record_done =3D=3D record_count) { + break; + } + } + } + + *len_out =3D out_pl_len; + return CXL_MBOX_SUCCESS; +} + #define IMMEDIATE_CONFIG_CHANGE (1 << 1) #define IMMEDIATE_DATA_CHANGE (1 << 2) #define IMMEDIATE_POLICY_CHANGE (1 << 3) @@ -1380,6 +1446,9 @@ static const struct cxl_cmd cxl_cmd_set[256][256] =3D= { static const struct cxl_cmd cxl_cmd_set_dcd[256][256] =3D { [DCD_CONFIG][GET_DC_CONFIG] =3D { "DCD_GET_DC_CONFIG", cmd_dcd_get_dyn_cap_config, 2, 0 }, + [DCD_CONFIG][GET_DYN_CAP_EXT_LIST] =3D { + "DCD_GET_DYNAMIC_CAPACITY_EXTENT_LIST", cmd_dcd_get_dyn_cap_ext_li= st, + 8, 0 }, }; =20 static const struct cxl_cmd cxl_cmd_set_sw[256][256] =3D { diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index 2b380a260b..102fa8151e 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -673,6 +673,7 @@ static bool cxl_create_dc_regions(CXLType3Dev *ct3d, Er= ror **errp) region_base +=3D region->len; ct3d->dc.total_capacity +=3D region->len; } + QTAILQ_INIT(&ct3d->dc.extents); =20 return true; } diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index 265679302c..8148bcc34b 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -424,6 +424,25 @@ typedef QLIST_HEAD(, CXLPoison) CXLPoisonList; =20 #define DCD_MAX_NUM_REGION 8 =20 +typedef struct CXLDCExtentRaw { + uint64_t start_dpa; + uint64_t len; + uint8_t tag[0x10]; + uint16_t shared_seq; + uint8_t rsvd[0x6]; +} QEMU_PACKED CXLDCExtentRaw; + +typedef struct CXLDCExtent { + uint64_t start_dpa; + uint64_t len; + uint8_t tag[0x10]; + uint16_t shared_seq; + uint8_t rsvd[0x6]; + + QTAILQ_ENTRY(CXLDCExtent) node; +} CXLDCExtent; +typedef QTAILQ_HEAD(, CXLDCExtent) CXLDCExtentList; + typedef struct CXLDCRegion { uint64_t base; /* aligned to 256*MiB */ uint64_t decode_len; /* aligned to 256*MiB */ @@ -470,6 +489,9 @@ struct CXLType3Dev { HostMemoryBackend *host_dc; AddressSpace host_dc_as; uint64_t total_capacity; /* 256M aligned */ + CXLDCExtentList extents; + uint32_t total_extent_count; + uint32_t ext_list_gen_seq; =20 uint8_t num_regions; /* 0-8 regions */ CXLDCRegion regions[DCD_MAX_NUM_REGION]; --=20 2.43.0 From nobody Thu May 9 21:45:48 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1709581513; cv=none; d=zohomail.com; s=zohoarc; b=TLTVcym0rZEuPHFiMi/Tq6/T7SnnjgdhLqMhvbDvmLpx2tPzL6KnDYVBXlZFkyY6ARmMY4NtVy5yWxK9bzLOnNu+WWJwSXP2mGv4zebtYSBJ2SeW3u+95tlLpuwseVI957sujWNS1z+K5x4YtSf72amHFsnoD1inGfXauVIIz7o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709581513; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=nsJMpRBsiDoinCTIfmIF1KNqDp8uRG05Rbh+KLifL8w=; b=fUl1/T53nr52mKynTTKgjKoYUIEsLKcKioaZpp4EqyM6DlM4Hs6iKMYWLuKRfE8udpVSQ0EW9q9YlGO5yWlJrTWeyEBTwM8iHmPXTpMuIhsDfg/9tyucP60D45J8xAp+7izp2MOzWbCAZ0J84UanOqCitdOxyTWq9Hr72RaT6to= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1709581513660175.12450721843732; Mon, 4 Mar 2024 11:45:13 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rhEEs-0005Ym-CF; Mon, 04 Mar 2024 14:44:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhEEf-0005VT-Ea for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:29 -0500 Received: from mail-yb1-xb32.google.com ([2607:f8b0:4864:20::b32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhEEd-0006xE-Bu for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:29 -0500 Received: by mail-yb1-xb32.google.com with SMTP id 3f1490d57ef6-dc23bf7e5aaso5404159276.0 for ; Mon, 04 Mar 2024 11:44:22 -0800 (PST) Received: from localhost.localdomain ([50.205.20.42]) by smtp.gmail.com with ESMTPSA id h1-20020a255f41000000b00dc62edd58dasm2282646ybm.40.2024.03.04.11.44.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Mar 2024 11:44:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709581461; x=1710186261; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nsJMpRBsiDoinCTIfmIF1KNqDp8uRG05Rbh+KLifL8w=; b=CpMLRzPOrDUM5ljvxsf++FHkHt5eYEFnXekmJecb1aRcy7sOUeKr0pglWskvq6i/Ol vavHNmeAD1VDloHmXR28pbk2UF9XwaeUsL0bwSGEA4wf8YJZXNNlIz9nivSUuWE4Oje8 DUS/nkxRAxj5qeOXeCYNdufLIEkUJPAtW0Z5xAuYpfxfOGqDMWX37aKH4HOFM41p6f+2 3QSPLn5OMKWw2leWHHzSdRU06fSWCJ9UUWU3OIYd8gjuEHvFJo54Uhiizdv89tjF0kk3 kpx7gC5VErMNnQ8/dwMbkdggq7IaBA/HtGlRHooyy61gpmDfFXFyuo5BOA4Y8M90Tlkp rlDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709581461; x=1710186261; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nsJMpRBsiDoinCTIfmIF1KNqDp8uRG05Rbh+KLifL8w=; b=RiJ7ZGG9IZnAMHNsgID1enpz+CYNAZNTJCey+GbK8W2Cd5pUHqeXrW1iuaIRoyvI/6 d72UqLtX6+F7vVywsHs2KDDWR7y6uM9SVP5b5GtL0SlmXj2Tw8NBz5nNvRBTqRCaeowL iL2hujHW3tfFQpTE4zko8UYZ7b5xXnfVZF3dBFWGvGgjC3mu+4zvfJyRfSLjNR8y99AP AO5fO9AJd9jYStuoWchdXF/b6HuEHMWM6wsWfdJWNpRM4oUJgA9Z2PPHmKnJyw2AiobQ 4GpoEnrWBHx974/fn4etRzKOcosaas/jQTRafD+TW8iaI7ilVxOoC4M3XgGFAzkJCLKu xnmw== X-Gm-Message-State: AOJu0Yw0nL0M+gKOs+aB6dyg0KSYclRbPpz238Z1YPQpXVO8AaPcQAz0 06RRV6p5gsOvObuSh52iFO6sQLdSdp4oODm4AeBnToLH6uWq1q9N86dr7T0U X-Google-Smtp-Source: AGHT+IEywjb671TDLGTuxKZsMTEqBabdGbzhbtW53K0+2HFJPtOshNZi3a6jONCN9Wy4Ur+BDkvDlw== X-Received: by 2002:a05:6902:1890:b0:dcd:49d0:eece with SMTP id cj16-20020a056902189000b00dcd49d0eecemr9446480ybb.60.1709581461549; Mon, 04 Mar 2024 11:44:21 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Jorgen.Hansen@wdc.com, wj28.lee@gmail.com, Fan Ni Subject: [PATCH v5 08/13] hw/cxl/cxl-mailbox-utils: Add mailbox commands to support add/release dynamic capacity response Date: Mon, 4 Mar 2024 11:34:03 -0800 Message-ID: <20240304194331.1586191-9-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240304194331.1586191-1-nifan.cxl@gmail.com> References: <20240304194331.1586191-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::b32; envelope-from=nifan.cxl@gmail.com; helo=mail-yb1-xb32.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1709581515775100006 Content-Type: text/plain; charset="utf-8" From: Fan Ni Per CXL spec 3.1, two mailbox commands are implemented: Add Dynamic Capacity Response (Opcode 4802h) 8.2.9.9.9.3, and Release Dynamic Capacity (Opcode 4803h) 8.2.9.9.9.4. Signed-off-by: Fan Ni --- hw/cxl/cxl-mailbox-utils.c | 310 ++++++++++++++++++++++++++++++++++++ hw/mem/cxl_type3.c | 12 ++ include/hw/cxl/cxl_device.h | 4 + 3 files changed, 326 insertions(+) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index 425b378a2c..8c59635a9f 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -85,6 +85,8 @@ enum { DCD_CONFIG =3D 0x48, #define GET_DC_CONFIG 0x0 #define GET_DYN_CAP_EXT_LIST 0x1 + #define ADD_DYN_CAP_RSP 0x2 + #define RELEASE_DYN_CAP 0x3 PHYSICAL_SWITCH =3D 0x51, #define IDENTIFY_SWITCH_DEVICE 0x0 #define GET_PHYSICAL_PORT_STATE 0x1 @@ -1399,6 +1401,308 @@ static CXLRetCode cmd_dcd_get_dyn_cap_ext_list(cons= t struct cxl_cmd *cmd, return CXL_MBOX_SUCCESS; } =20 +/* + * Check whether any bit between addr[nr, nr+size) is set, + * return true if any bit is set, otherwise return false + */ +static bool test_any_bits_set(const unsigned long *addr, unsigned long nr, + unsigned long size) +{ + unsigned long res =3D find_next_bit(addr, size + nr, nr); + + return res < nr + size; +} + +CXLDCRegion *cxl_find_dc_region(CXLType3Dev *ct3d, uint64_t dpa, uint64_t = len) +{ + int i; + CXLDCRegion *region =3D &ct3d->dc.regions[0]; + + if (dpa < region->base || + dpa >=3D region->base + ct3d->dc.total_capacity) { + return NULL; + } + + /* + * CXL r3.1 section 9.13.3: Dynamic Capacity Device (DCD) + * + * Regions are used in increasing-DPA order, with Region 0 being used = for + * the lowest DPA of Dynamic Capacity and Region 7 for the highest DPA. + * So check from the last region to find where the dpa belongs. Extent= s that + * cross multiple regions are not allowed. + */ + for (i =3D ct3d->dc.num_regions - 1; i >=3D 0; i--) { + region =3D &ct3d->dc.regions[i]; + if (dpa >=3D region->base) { + if (dpa + len > region->base + region->len) { + return NULL; + } + return region; + } + } + + return NULL; +} + +static void cxl_insert_extent_to_extent_list(CXLDCExtentList *list, + uint64_t dpa, + uint64_t len, + uint8_t *tag, + uint16_t shared_seq) +{ + CXLDCExtent *extent; + + extent =3D g_new0(CXLDCExtent, 1); + extent->start_dpa =3D dpa; + extent->len =3D len; + if (tag) { + memcpy(extent->tag, tag, 0x10); + } + extent->shared_seq =3D shared_seq; + + QTAILQ_INSERT_TAIL(list, extent, node); +} + +void cxl_remove_extent_from_extent_list(CXLDCExtentList *list, + CXLDCExtent *extent) +{ + QTAILQ_REMOVE(list, extent, node); + g_free(extent); +} + +/* + * CXL r3.1 Table 8-168: Add Dynamic Capacity Response Input Payload + * CXL r3.1 Table 8-170: Release Dynamic Capacity Input Payload + */ +typedef struct CXLUpdateDCExtentListInPl { + uint32_t num_entries_updated; + uint8_t flags; + uint8_t rsvd[3]; + /* CXL r3.1 Table 8-169: Updated Extent */ + struct { + uint64_t start_dpa; + uint64_t len; + uint8_t rsvd[8]; + } QEMU_PACKED updated_entries[]; +} QEMU_PACKED CXLUpdateDCExtentListInPl; + +/* + * For the extents in the extent list to operate, check whether they are v= alid + * 1. The extent should be in the range of a valid DC region; + * 2. The extent should not cross multiple regions; + * 3. The start DPA and the length of the extent should align with the blo= ck + * size of the region; + * 4. The address range of multiple extents in the list should not overlap. + */ +static CXLRetCode cxl_detect_malformed_extent_list(CXLType3Dev *ct3d, + const CXLUpdateDCExtentListInPl *in) +{ + uint64_t min_block_size =3D UINT64_MAX; + CXLDCRegion *region =3D &ct3d->dc.regions[0]; + CXLDCRegion *lastregion =3D &ct3d->dc.regions[ct3d->dc.num_regions - 1= ]; + g_autofree unsigned long *blk_bitmap =3D NULL; + uint64_t dpa, len; + uint32_t i; + + for (i =3D 0; i < ct3d->dc.num_regions; i++) { + region =3D &ct3d->dc.regions[i]; + min_block_size =3D MIN(min_block_size, region->block_size); + } + + blk_bitmap =3D bitmap_new((lastregion->base + lastregion->len - + ct3d->dc.regions[0].base) / min_block_size); + + for (i =3D 0; i < in->num_entries_updated; i++) { + dpa =3D in->updated_entries[i].start_dpa; + len =3D in->updated_entries[i].len; + + region =3D cxl_find_dc_region(ct3d, dpa, len); + if (!region) { + return CXL_MBOX_INVALID_PA; + } + + dpa -=3D ct3d->dc.regions[0].base; + if (dpa % region->block_size || len % region->block_size) { + return CXL_MBOX_INVALID_EXTENT_LIST; + } + /* the dpa range already covered by some other extents in the list= */ + if (test_any_bits_set(blk_bitmap, dpa / min_block_size, + len / min_block_size)) { + return CXL_MBOX_INVALID_EXTENT_LIST; + } + bitmap_set(blk_bitmap, dpa / min_block_size, len / min_block_size); + } + + return CXL_MBOX_SUCCESS; +} + +/* + * CXL r3.1 section 8.2.9.9.9.3: Add Dynamic Capacity Response (Opcode 480= 2h) + * An extent is added to the extent list and becomes usable only after the + * response is processed successfully + */ +static CXLRetCode cmd_dcd_add_dyn_cap_rsp(const struct cxl_cmd *cmd, + uint8_t *payload_in, + size_t len_in, + uint8_t *payload_out, + size_t *len_out, + CXLCCI *cci) +{ + CXLUpdateDCExtentListInPl *in =3D (void *)payload_in; + CXLType3Dev *ct3d =3D CXL_TYPE3(cci->d); + CXLDCExtentList *extent_list =3D &ct3d->dc.extents; + CXLDCExtent *ent; + uint32_t i; + uint64_t dpa, len; + CXLRetCode ret; + + if (in->num_entries_updated =3D=3D 0) { + return CXL_MBOX_SUCCESS; + } + + /* Adding extents causes exceeding device's extent tracking ability. */ + if (in->num_entries_updated + ct3d->dc.total_extent_count > + CXL_NUM_EXTENTS_SUPPORTED) { + return CXL_MBOX_RESOURCES_EXHAUSTED; + } + + ret =3D cxl_detect_malformed_extent_list(ct3d, in); + if (ret !=3D CXL_MBOX_SUCCESS) { + return ret; + } + + for (i =3D 0; i < in->num_entries_updated; i++) { + dpa =3D in->updated_entries[i].start_dpa; + len =3D in->updated_entries[i].len; + + /* + * Check if the DPA range of the to-be-added extent overlaps with + * existing extent list maintained by the device. + */ + QTAILQ_FOREACH(ent, extent_list, node) { + if (ent->start_dpa <=3D dpa && + dpa + len <=3D ent->start_dpa + ent->len) { + return CXL_MBOX_INVALID_PA; + /* Overlapping one end of the other */ + } else if ((dpa < ent->start_dpa + ent->len && + dpa + len > ent->start_dpa + ent->len) || + (dpa < ent->start_dpa && dpa + len > ent->start_dpa= )) { + return CXL_MBOX_INVALID_PA; + } + } + + /* + * TODO: we will add a pending extent list based on event log reco= rd + * and verify the input response; also, the "More" flag is not + * considered at the moment. + */ + + cxl_insert_extent_to_extent_list(extent_list, dpa, len, NULL, 0); + ct3d->dc.total_extent_count +=3D 1; + } + + return CXL_MBOX_SUCCESS; +} + +/* + * CXL r3.1 section 8.2.9.9.9.4: Release Dynamic Capacity (Opcode 4803h) + */ +static CXLRetCode cmd_dcd_release_dyn_cap(const struct cxl_cmd *cmd, + uint8_t *payload_in, + size_t len_in, + uint8_t *payload_out, + size_t *len_out, + CXLCCI *cci) +{ + CXLUpdateDCExtentListInPl *in =3D (void *)payload_in; + CXLType3Dev *ct3d =3D CXL_TYPE3(cci->d); + CXLDCExtentList *extent_list =3D &ct3d->dc.extents; + CXLDCExtent *ent; + uint32_t i; + uint64_t dpa, len; + CXLRetCode ret; + + if (in->num_entries_updated =3D=3D 0) { + return CXL_MBOX_INVALID_INPUT; + } + + ret =3D cxl_detect_malformed_extent_list(ct3d, in); + if (ret !=3D CXL_MBOX_SUCCESS) { + return ret; + } + + for (i =3D 0; i < in->num_entries_updated; i++) { + bool found =3D false; + + dpa =3D in->updated_entries[i].start_dpa; + len =3D in->updated_entries[i].len; + + QTAILQ_FOREACH(ent, extent_list, node) { + /* Found the extent overlapping with */ + if (ent->start_dpa <=3D dpa && dpa < ent->start_dpa + ent->len= ) { + if (dpa + len <=3D ent->start_dpa + ent->len) { + /* + * The incoming extent covers a portion of an extent + * in the device extent list, remove only the overlapp= ing + * portion, meaning + * 1. the portions that are not covered by the incoming + * extent at both end of the original extent will b= ecome + * new extents and inserted to the extent list; and + * 2. the original extent is removed from the extent l= ist; + * 3. DC extent count is updated accordingly. + */ + uint64_t ent_start_dpa =3D ent->start_dpa; + uint64_t ent_len =3D ent->len; + uint64_t len1 =3D dpa - ent_start_dpa; + uint64_t len2 =3D ent_start_dpa + ent_len - dpa - len; + + /* + * TODO: checking for possible extent overflow, will be + * moved into a dedicated function of detecting extent + * overflow. + */ + if (len1 && len2 && ct3d->dc.total_extent_count =3D=3D + CXL_NUM_EXTENTS_SUPPORTED) { + return CXL_MBOX_RESOURCES_EXHAUSTED; + } + + found =3D true; + cxl_remove_extent_from_extent_list(extent_list, ent); + ct3d->dc.total_extent_count -=3D 1; + + if (len1) { + cxl_insert_extent_to_extent_list(extent_list, + ent_start_dpa, le= n1, + NULL, 0); + ct3d->dc.total_extent_count +=3D 1; + } + if (len2) { + cxl_insert_extent_to_extent_list(extent_list, dpa = + len, + len2, NULL, 0); + ct3d->dc.total_extent_count +=3D 1; + } + break; + } else { + /* + * TODO: we reject the attempt to remove an extent that + * overlaps with multiple extents in the device for no= w, + * once the bitmap indicating whether a DPA range is + * covered by valid extents is introduced, will allow = it. + */ + return CXL_MBOX_INVALID_PA; + } + } + } + + if (!found) { + /* Try to remove a non-existing extent. */ + return CXL_MBOX_INVALID_PA; + } + } + + return CXL_MBOX_SUCCESS; +} + #define IMMEDIATE_CONFIG_CHANGE (1 << 1) #define IMMEDIATE_DATA_CHANGE (1 << 2) #define IMMEDIATE_POLICY_CHANGE (1 << 3) @@ -1449,6 +1753,12 @@ static const struct cxl_cmd cxl_cmd_set_dcd[256][256= ] =3D { [DCD_CONFIG][GET_DYN_CAP_EXT_LIST] =3D { "DCD_GET_DYNAMIC_CAPACITY_EXTENT_LIST", cmd_dcd_get_dyn_cap_ext_li= st, 8, 0 }, + [DCD_CONFIG][ADD_DYN_CAP_RSP] =3D { + "DCD_ADD_DYNAMIC_CAPACITY_RESPONSE", cmd_dcd_add_dyn_cap_rsp, + ~0, IMMEDIATE_DATA_CHANGE }, + [DCD_CONFIG][RELEASE_DYN_CAP] =3D { + "DCD_RELEASE_DYNAMIC_CAPACITY", cmd_dcd_release_dyn_cap, + ~0, IMMEDIATE_DATA_CHANGE }, }; =20 static const struct cxl_cmd cxl_cmd_set_sw[256][256] =3D { diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index 102fa8151e..dccfaaad3a 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -678,6 +678,16 @@ static bool cxl_create_dc_regions(CXLType3Dev *ct3d, E= rror **errp) return true; } =20 +static void cxl_destroy_dc_regions(CXLType3Dev *ct3d) +{ + CXLDCExtent *ent; + + while (!QTAILQ_EMPTY(&ct3d->dc.extents)) { + ent =3D QTAILQ_FIRST(&ct3d->dc.extents); + cxl_remove_extent_from_extent_list(&ct3d->dc.extents, ent); + } +} + static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp) { DeviceState *ds =3D DEVICE(ct3d); @@ -874,6 +884,7 @@ err_free_special_ops: g_free(regs->special_ops); err_address_space_free: if (ct3d->dc.host_dc) { + cxl_destroy_dc_regions(ct3d); address_space_destroy(&ct3d->dc.host_dc_as); } if (ct3d->hostpmem) { @@ -895,6 +906,7 @@ static void ct3_exit(PCIDevice *pci_dev) cxl_doe_cdat_release(cxl_cstate); g_free(regs->special_ops); if (ct3d->dc.host_dc) { + cxl_destroy_dc_regions(ct3d); address_space_destroy(&ct3d->dc.host_dc_as); } if (ct3d->hostpmem) { diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index 8148bcc34b..341260e6e4 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -547,4 +547,8 @@ void cxl_event_irq_assert(CXLType3Dev *ct3d); =20 void cxl_set_poison_list_overflowed(CXLType3Dev *ct3d); =20 +CXLDCRegion *cxl_find_dc_region(CXLType3Dev *ct3d, uint64_t dpa, uint64_t = len); + +void cxl_remove_extent_from_extent_list(CXLDCExtentList *list, + CXLDCExtent *extent); #endif --=20 2.43.0 From nobody Thu May 9 21:45:48 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1709581595; cv=none; d=zohomail.com; s=zohoarc; b=klA5mhTjRCxoSNtsD4PZw9QaDFG7dYLt3Bf/HVBO3kCXY75H+im/g6cWaLzDCDE3PqZAtAbr7zH/Nui9dC/BZ6LonZHMwcMkRsIbqelOX5ByvtEk1ErlJdNyen9rbIhuwbxpXbOyU3C3PIOJq1UknA148QsLavqkKGu8HyqSGCk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709581595; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=9IBOSIGSvnMADzgZ+mW+5zQVhNDHXh+pawO9uLnLzyc=; b=Dlqn+pZts5DipkwmC8ndcZtpgemj1+Ww5QXxyYvjsMgdrJOsHPaBY1vEbIevbYe+jnTqxr6E4/01PuJ0XABWuw4wHOk8eGsHEyPYS7Mil2B3NiM6OnLsuqF564jHFWZPwDdlNje4RdsVkKNdo5diecNQMC1R+7Pg8HX6P3gWOE4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1709581595305532.9716747164822; Mon, 4 Mar 2024 11:46:35 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rhEFD-0005m9-9D; Mon, 04 Mar 2024 14:45:04 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhEEh-0005WT-8V for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:31 -0500 Received: from mail-yb1-xb2c.google.com ([2607:f8b0:4864:20::b2c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhEEd-0006xK-D5 for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:30 -0500 Received: by mail-yb1-xb2c.google.com with SMTP id 3f1490d57ef6-dbed179f0faso4172142276.1 for ; Mon, 04 Mar 2024 11:44:26 -0800 (PST) Received: from localhost.localdomain ([50.205.20.42]) by smtp.gmail.com with ESMTPSA id h1-20020a255f41000000b00dc62edd58dasm2282646ybm.40.2024.03.04.11.44.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Mar 2024 11:44:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709581466; x=1710186266; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9IBOSIGSvnMADzgZ+mW+5zQVhNDHXh+pawO9uLnLzyc=; b=WVSgvrk2nvhxOgLZ5zWcR2Nde5j/s5mLqr3lhM4fmEjYGwwQmMmGjNzsGPm7I1KUg7 RU+7fkIpI1Ed0CuR16nY5zeiyFCmVG28QcIVhgX0Q2hNZk9mqoTQRJvkgSxeiBKAiaus RKkRZLQKf9SI2Ex1GkXMIEQoAKA5lOv7dz7D3SspfjVk9dmZ9/06tjBob9kMkanpiGhw xeHR7npgVqnt7udwqWWnsK6R1VtM49qkKIvV0JvzN6ncqnIvjd9F+EwPjWkFQ/yryRUY LzYSxGv03a5S6eF+UcQ+A7zVWTlYfTkMEmpbrvoV/ErYmRHXpHZwP77MxQFFzpHYH4vo /n6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709581466; x=1710186266; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9IBOSIGSvnMADzgZ+mW+5zQVhNDHXh+pawO9uLnLzyc=; b=TAfLnH96BeMzqIvSqSENHEtYgAED5B1Wh678HjGuSc2CtueCWlXDrNfME/lBDBUREM jALGZhNUxdjZRF+xgFJqPeV6oYOwN0wkFSdM0g8z6Ubiyw5eK1WG3sgXGDN4UPWfByN8 nz+UXfZaEBY00R6+iy/ZtKgedW3XmmTjleYO0X3H+hLJOjB2gaXCvSxKnLx+re7OOSCo 66/VTeNzr1SYU+iUhrYu+rqsdfuA+vfYXNhF43DX6pxT/VuTDPTRqIT2t+XiyA2HWbWe h5TyQf9uHmlV4ArXB3EiItAitZGnWb1Fe5OyfmxSjANEWMhA+qj+TQp3fowwvkgjbSne T+Cg== X-Gm-Message-State: AOJu0YydoMD3ZfdpDbxU4UGBXqkTnwP0SMau0xGjdp8A7OHiI2LuxOZT JvxgUtal1dqj7Bz7cfEy4aywCtZucQ5tUqlyLMTadN09R1Q8IdqlYHHkN9gg X-Google-Smtp-Source: AGHT+IHKrxBL4gj+jTPCrIcVjmwDt26gJvL7p07d8UAeOHaLX1Zkbnj3sPAl0YyHB5/As3vhFckawQ== X-Received: by 2002:a5b:10f:0:b0:dc7:4265:1e92 with SMTP id 15-20020a5b010f000000b00dc742651e92mr448450ybx.23.1709581465690; Mon, 04 Mar 2024 11:44:25 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Jorgen.Hansen@wdc.com, wj28.lee@gmail.com, Fan Ni Subject: [PATCH v5 09/13] hw/cxl/events: Add qmp interfaces to add/release dynamic capacity extents Date: Mon, 4 Mar 2024 11:34:04 -0800 Message-ID: <20240304194331.1586191-10-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240304194331.1586191-1-nifan.cxl@gmail.com> References: <20240304194331.1586191-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::b2c; envelope-from=nifan.cxl@gmail.com; helo=mail-yb1-xb2c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1709581596105100003 Content-Type: text/plain; charset="utf-8" From: Fan Ni Since fabric manager emulation is not supported yet, the change implements the functions to add/release dynamic capacity extents as QMP interfaces. Note: we skips any FM issued extent release request if the exact extent does not exist in the extent list of the device. We will loose the restriction later once we have partial release support in the kernel. 1. Add dynamic capacity extents: For example, the command to add two continuous extents (each 128MiB long) to region 0 (starting at DPA offset 0) looks like below: { "execute": "qmp_capabilities" } { "execute": "cxl-add-dynamic-capacity", "arguments": { "path": "/machine/peripheral/cxl-dcd0", "region-id": 0, "extents": [ { "dpa": 0, "len": 134217728 }, { "dpa": 134217728, "len": 134217728 } ] } } 2. Release dynamic capacity extents: For example, the command to release an extent of size 128MiB from region 0 (DPA offset 128MiB) look like below: { "execute": "cxl-release-dynamic-capacity", "arguments": { "path": "/machine/peripheral/cxl-dcd0", "region-id": 0, "extents": [ { "dpa": 134217728, "len": 134217728 } ] } } Signed-off-by: Fan Ni --- hw/cxl/cxl-mailbox-utils.c | 26 ++-- hw/mem/cxl_type3.c | 245 +++++++++++++++++++++++++++++++++++- hw/mem/cxl_type3_stubs.c | 14 +++ include/hw/cxl/cxl_device.h | 6 + include/hw/cxl/cxl_events.h | 18 +++ qapi/cxl.json | 61 ++++++++- 6 files changed, 361 insertions(+), 9 deletions(-) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index 8c59635a9f..53ebc526ae 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -1405,7 +1405,7 @@ static CXLRetCode cmd_dcd_get_dyn_cap_ext_list(const = struct cxl_cmd *cmd, * Check whether any bit between addr[nr, nr+size) is set, * return true if any bit is set, otherwise return false */ -static bool test_any_bits_set(const unsigned long *addr, unsigned long nr, +bool test_any_bits_set(const unsigned long *addr, unsigned long nr, unsigned long size) { unsigned long res =3D find_next_bit(addr, size + nr, nr); @@ -1444,7 +1444,7 @@ CXLDCRegion *cxl_find_dc_region(CXLType3Dev *ct3d, ui= nt64_t dpa, uint64_t len) return NULL; } =20 -static void cxl_insert_extent_to_extent_list(CXLDCExtentList *list, +void cxl_insert_extent_to_extent_list(CXLDCExtentList *list, uint64_t dpa, uint64_t len, uint8_t *tag, @@ -1591,16 +1591,28 @@ static CXLRetCode cmd_dcd_add_dyn_cap_rsp(const str= uct cxl_cmd *cmd, } } =20 - /* - * TODO: we will add a pending extent list based on event log reco= rd - * and verify the input response; also, the "More" flag is not - * considered at the moment. - */ + QTAILQ_FOREACH(ent, &ct3d->dc.extents_pending_to_add, node) { + if (ent->start_dpa <=3D dpa && + dpa + len <=3D ent->start_dpa + ent->len) { + break; + } + } + if (!ent) { + return CXL_MBOX_INVALID_PA; + } + + cxl_remove_extent_from_extent_list(&ct3d->dc.extents_pending_to_ad= d, + ent); =20 cxl_insert_extent_to_extent_list(extent_list, dpa, len, NULL, 0); ct3d->dc.total_extent_count +=3D 1; } =20 + /* + * TODO: extents_pending_to_add needs to be cleared so the extents not + * accepted can be reclaimed base on spec r3.1: 8.2.9.9.9.3 + */ + return CXL_MBOX_SUCCESS; } =20 diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index dccfaaad3a..e9c8994cdb 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -674,6 +674,7 @@ static bool cxl_create_dc_regions(CXLType3Dev *ct3d, Er= ror **errp) ct3d->dc.total_capacity +=3D region->len; } QTAILQ_INIT(&ct3d->dc.extents); + QTAILQ_INIT(&ct3d->dc.extents_pending_to_add); =20 return true; } @@ -686,6 +687,12 @@ static void cxl_destroy_dc_regions(CXLType3Dev *ct3d) ent =3D QTAILQ_FIRST(&ct3d->dc.extents); cxl_remove_extent_from_extent_list(&ct3d->dc.extents, ent); } + + while (!QTAILQ_EMPTY(&ct3d->dc.extents_pending_to_add)) { + ent =3D QTAILQ_FIRST(&ct3d->dc.extents_pending_to_add); + cxl_remove_extent_from_extent_list(&ct3d->dc.extents_pending_to_ad= d, + ent); + } } =20 static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp) @@ -1451,7 +1458,8 @@ static int ct3d_qmp_cxl_event_log_enc(CxlEventLog log) return CXL_EVENT_TYPE_FAIL; case CXL_EVENT_LOG_FATAL: return CXL_EVENT_TYPE_FATAL; -/* DCD not yet supported */ + case CXL_EVENT_LOG_DYNCAP: + return CXL_EVENT_TYPE_DYNAMIC_CAP; default: return -EINVAL; } @@ -1702,6 +1710,241 @@ void qmp_cxl_inject_memory_module_event(const char = *path, CxlEventLog log, } } =20 +/* CXL r3.1 Table 8-50: Dynamic Capacity Event Record */ +static const QemuUUID dynamic_capacity_uuid =3D { + .data =3D UUID(0xca95afa7, 0xf183, 0x4018, 0x8c, 0x2f, + 0x95, 0x26, 0x8e, 0x10, 0x1a, 0x2a), +}; + +typedef enum CXLDCEventType { + DC_EVENT_ADD_CAPACITY =3D 0x0, + DC_EVENT_RELEASE_CAPACITY =3D 0x1, + DC_EVENT_FORCED_RELEASE_CAPACITY =3D 0x2, + DC_EVENT_REGION_CONFIG_UPDATED =3D 0x3, + DC_EVENT_ADD_CAPACITY_RSP =3D 0x4, + DC_EVENT_CAPACITY_RELEASED =3D 0x5, +} CXLDCEventType; + +/* + * Check whether the exact extent exists in the list + * Return value: the extent pointer in the list; else null + */ +static CXLDCExtent *cxl_dc_extent_exists(CXLDCExtentList *list, + CXLDCExtentRaw *ext) +{ + CXLDCExtent *ent; + + if (!ext || !list) { + return NULL; + } + + QTAILQ_FOREACH(ent, list, node) { + if (ent->start_dpa !=3D ext->start_dpa) { + continue; + } + + /* Found exact extent */ + return ent->len =3D=3D ext->len ? ent : NULL; + } + + return NULL; +} + +/* + * The main function to process dynamic capacity event. Currently DC exten= ts + * add/release requests are processed. + */ +static void qmp_cxl_process_dynamic_capacity(const char *path, CxlEventLog= log, + CXLDCEventType type, uint16_t= hid, + uint8_t rid, + CXLDCExtentRecordList *record= s, + Error **errp) +{ + Object *obj; + CXLEventDynamicCapacity dCap =3D {}; + CXLEventRecordHdr *hdr =3D &dCap.hdr; + CXLType3Dev *dcd; + uint8_t flags =3D 1 << CXL_EVENT_TYPE_INFO; + uint32_t num_extents =3D 0; + CXLDCExtentRecordList *list; + g_autofree CXLDCExtentRaw *extents =3D NULL; + uint8_t enc_log; + uint64_t offset, len, block_size; + int i; + int rc; + g_autofree unsigned long *blk_bitmap =3D NULL; + + obj =3D object_resolve_path(path, NULL); + if (!obj) { + error_setg(errp, "Unable to resolve path"); + return; + } + if (!object_dynamic_cast(obj, TYPE_CXL_TYPE3)) { + error_setg(errp, "Path not point to a valid CXL type3 device"); + return; + } + + dcd =3D CXL_TYPE3(obj); + if (!dcd->dc.num_regions) { + error_setg(errp, "No dynamic capacity support from the device"); + return; + } + + rc =3D ct3d_qmp_cxl_event_log_enc(log); + if (rc < 0) { + error_setg(errp, "Unhandled error log type"); + return; + } + enc_log =3D rc; + + if (rid >=3D dcd->dc.num_regions) { + error_setg(errp, "region id is too large"); + return; + } + block_size =3D dcd->dc.regions[rid].block_size; + + /* Sanity check and count the extents */ + list =3D records; + while (list) { + offset =3D list->value->offset; + len =3D list->value->len; + + if (len =3D=3D 0) { + error_setg(errp, "extent with 0 length is not allowed"); + return; + } + + if (offset % block_size || len % block_size) { + error_setg(errp, "dpa or len is not aligned to region block si= ze"); + return; + } + + if (offset + len > dcd->dc.regions[rid].len) { + error_setg(errp, "extent range is beyond the region end"); + return; + } + + num_extents++; + list =3D list->next; + } + if (num_extents =3D=3D 0) { + error_setg(errp, "No extents found in the command"); + return; + } + + blk_bitmap =3D bitmap_new(dcd->dc.regions[rid].len / block_size); + + /* Create Extent list for event being passed to host */ + i =3D 0; + list =3D records; + extents =3D g_new0(CXLDCExtentRaw, num_extents); + while (list) { + CXLDCExtent *ent; + bool skip_extent =3D false; + + offset =3D list->value->offset; + len =3D list->value->len; + + extents[i].start_dpa =3D offset + dcd->dc.regions[rid].base; + extents[i].len =3D len; + memset(extents[i].tag, 0, 0x10); + extents[i].shared_seq =3D 0; + + if (type =3D=3D DC_EVENT_RELEASE_CAPACITY || + type =3D=3D DC_EVENT_FORCED_RELEASE_CAPACITY) { + /* + * if the extent is still pending to be added to the host, + * remove it from the pending extent list, so later when the a= dd + * response for the extent arrives, the device can reject the + * extent as it is not in the pending list. + */ + ent =3D cxl_dc_extent_exists(&dcd->dc.extents_pending_to_add, + &extents[i]); + if (ent) { + QTAILQ_REMOVE(&dcd->dc.extents_pending_to_add, ent, node); + g_free(ent); + skip_extent =3D true; + } else if (!cxl_dc_extent_exists(&dcd->dc.extents, &extents[i]= )) { + /* If the exact extent is not in the accepted list, skip */ + skip_extent =3D true; + } + } + + /* No duplicate or overlapped extents are allowed */ + if (test_any_bits_set(blk_bitmap, offset / block_size, + len / block_size)) { + error_setg(errp, "duplicate or overlapped extents are detected= "); + return; + } + bitmap_set(blk_bitmap, offset / block_size, len / block_size); + + list =3D list->next; + if (!skip_extent) { + i++; + } + } + num_extents =3D i; + + /* + * CXL r3.1 section 8.2.9.2.1.6: Dynamic Capacity Event Record + * + * All Dynamic Capacity event records shall set the Event Record Sever= ity + * field in the Common Event Record Format to Informational Event. All + * Dynamic Capacity related events shall be logged in the Dynamic Capa= city + * Event Log. + */ + cxl_assign_event_header(hdr, &dynamic_capacity_uuid, flags, sizeof(dCa= p), + cxl_device_get_timestamp(&dcd->cxl_dstate)); + + dCap.type =3D type; + /* FIXME: for now, validity flag is cleared */ + dCap.validity_flags =3D 0; + stw_le_p(&dCap.host_id, hid); + /* only valid for DC_REGION_CONFIG_UPDATED event */ + dCap.updated_region_id =3D 0; + /* + * FIXME: for now, the "More" flag is cleared as there is only one + * extent associating with each record and tag-based release is + * not supported. + */ + dCap.flags =3D 0; + for (i =3D 0; i < num_extents; i++) { + memcpy(&dCap.dynamic_capacity_extent, &extents[i], + sizeof(CXLDCExtentRaw)); + + if (type =3D=3D DC_EVENT_ADD_CAPACITY) { + cxl_insert_extent_to_extent_list(&dcd->dc.extents_pending_to_a= dd, + extents[i].start_dpa, + extents[i].len, + extents[i].tag, + extents[i].shared_seq); + } + + if (cxl_event_insert(&dcd->cxl_dstate, enc_log, + (CXLEventRecordRaw *)&dCap)) { + cxl_event_irq_assert(dcd); + } + } +} + +void qmp_cxl_add_dynamic_capacity(const char *path, uint8_t region_id, + CXLDCExtentRecordList *records, + Error **errp) +{ + qmp_cxl_process_dynamic_capacity(path, CXL_EVENT_LOG_DYNCAP, + DC_EVENT_ADD_CAPACITY, 0, + region_id, records, errp); +} + +void qmp_cxl_release_dynamic_capacity(const char *path, uint8_t region_id, + CXLDCExtentRecordList *records, + Error **errp) +{ + qmp_cxl_process_dynamic_capacity(path, CXL_EVENT_LOG_DYNCAP, + DC_EVENT_RELEASE_CAPACITY, 0, + region_id, records, errp); +} + static void ct3_class_init(ObjectClass *oc, void *data) { DeviceClass *dc =3D DEVICE_CLASS(oc); diff --git a/hw/mem/cxl_type3_stubs.c b/hw/mem/cxl_type3_stubs.c index 3e1851e32b..d913b11b4d 100644 --- a/hw/mem/cxl_type3_stubs.c +++ b/hw/mem/cxl_type3_stubs.c @@ -67,3 +67,17 @@ void qmp_cxl_inject_correctable_error(const char *path, = CxlCorErrorType type, { error_setg(errp, "CXL Type 3 support is not compiled in"); } + +void qmp_cxl_add_dynamic_capacity(const char *path, uint8_t region_id, + CXLDCExtentRecordList *records, + Error **errp) +{ + error_setg(errp, "CXL Type 3 support is not compiled in"); +} + +void qmp_cxl_release_dynamic_capacity(const char *path, uint8_t region_id, + CXLDCExtentRecordList *records, + Error **errp) +{ + error_setg(errp, "CXL Type 3 support is not compiled in"); +} diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index 341260e6e4..b524c5e699 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -490,6 +490,7 @@ struct CXLType3Dev { AddressSpace host_dc_as; uint64_t total_capacity; /* 256M aligned */ CXLDCExtentList extents; + CXLDCExtentList extents_pending_to_add; uint32_t total_extent_count; uint32_t ext_list_gen_seq; =20 @@ -551,4 +552,9 @@ CXLDCRegion *cxl_find_dc_region(CXLType3Dev *ct3d, uint= 64_t dpa, uint64_t len); =20 void cxl_remove_extent_from_extent_list(CXLDCExtentList *list, CXLDCExtent *extent); +void cxl_insert_extent_to_extent_list(CXLDCExtentList *list, uint64_t dpa, + uint64_t len, uint8_t *tag, + uint16_t shared_seq); +bool test_any_bits_set(const unsigned long *addr, unsigned long nr, + unsigned long size); #endif diff --git a/include/hw/cxl/cxl_events.h b/include/hw/cxl/cxl_events.h index 5170b8dbf8..38cadaa0f3 100644 --- a/include/hw/cxl/cxl_events.h +++ b/include/hw/cxl/cxl_events.h @@ -166,4 +166,22 @@ typedef struct CXLEventMemoryModule { uint8_t reserved[0x3d]; } QEMU_PACKED CXLEventMemoryModule; =20 +/* + * CXL r3.1 section Table 8-50: Dynamic Capacity Event Record + * All fields little endian. + */ +typedef struct CXLEventDynamicCapacity { + CXLEventRecordHdr hdr; + uint8_t type; + uint8_t validity_flags; + uint16_t host_id; + uint8_t updated_region_id; + uint8_t flags; + uint8_t reserved2[2]; + uint8_t dynamic_capacity_extent[0x28]; /* defined in cxl_device.h */ + uint8_t reserved[0x18]; + uint32_t extents_avail; + uint32_t tags_avail; +} QEMU_PACKED CXLEventDynamicCapacity; + #endif /* CXL_EVENTS_H */ diff --git a/qapi/cxl.json b/qapi/cxl.json index 8cc4c72fa9..2645004666 100644 --- a/qapi/cxl.json +++ b/qapi/cxl.json @@ -19,13 +19,16 @@ # # @fatal: Fatal Event Log # +# @dyncap: Dynamic Capacity Event Log +# # Since: 8.1 ## { 'enum': 'CxlEventLog', 'data': ['informational', 'warning', 'failure', - 'fatal'] + 'fatal', + 'dyncap'] } =20 ## @@ -361,3 +364,59 @@ ## {'command': 'cxl-inject-correctable-error', 'data': {'path': 'str', 'type': 'CxlCorErrorType'}} + +## +# @CXLDCExtentRecord: +# +# Record of a single extent to add/release +# +# @offset: offset to the start of the region where the extent to be operat= ed +# @len: length of the extent +# +# Since: 9.0 +## +{ 'struct': 'CXLDCExtentRecord', + 'data': { + 'offset':'uint64', + 'len': 'uint64' + } +} + +## +# @cxl-add-dynamic-capacity: +# +# Command to start add dynamic capacity extents flow. The device will +# have to acknowledged the acceptance of the extents before they are usabl= e. +# +# @path: CXL DCD canonical QOM path +# @region-id: id of the region where the extent to add +# @extents: Extents to add +# +# Since : 9.0 +## +{ 'command': 'cxl-add-dynamic-capacity', + 'data': { 'path': 'str', + 'region-id': 'uint8', + 'extents': [ 'CXLDCExtentRecord' ] + } +} + +## +# @cxl-release-dynamic-capacity: +# +# Command to start release dynamic capacity extents flow. The host will +# need to respond to indicate that it has released the capacity before it +# is made unavailable for read and write and can be re-added. +# +# @path: CXL DCD canonical QOM path +# @region-id: id of the region where the extent to release +# @extents: Extents to release +# +# Since : 9.0 +## +{ 'command': 'cxl-release-dynamic-capacity', + 'data': { 'path': 'str', + 'region-id': 'uint8', + 'extents': [ 'CXLDCExtentRecord' ] + } +} --=20 2.43.0 From nobody Thu May 9 21:45:48 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1709581579; cv=none; d=zohomail.com; s=zohoarc; b=OF0xOga/RPC1MQ21mw+1t0BZTFUbWGuE5piay1uHov1ThoXvESF/Tz94I35Cycb7PcmlBuJgJmPXX0tvk8FIwvoCucdCgzHZXxz4y0FkDVh/XXe8PLgn7VSh9MfAnpJ5a0v4eZRU9gAarowlQ1x0YigiTMrTZZxLxDKgRpiXk4w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709581579; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=zJmbpLHmX91RSD+FS2bbI4ESdtGu0icFFQqbt0j7En8=; b=LIgV3rCYG4S1s6maE6PmYvJAtI2VlzzTAOHpi6RPn9Sfa+D1YpKVDfzjhpXw1yuF8qVkchvd5KfB2VU24yB5ZdikKSAEIwGW4GHe4ON1FPhWBV3e5anyCVxZ/DtAEvyTFo2BiMjduIXbYFFyU09vvRqnx9YuggOQHXvIP0zvl4w= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1709581579126336.5089539114682; Mon, 4 Mar 2024 11:46:19 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rhEEk-0005XU-V6; Mon, 04 Mar 2024 14:44:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhEEh-0005Wp-JI for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:31 -0500 Received: from mail-yb1-xb31.google.com ([2607:f8b0:4864:20::b31]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhEEf-0006xb-Ny for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:31 -0500 Received: by mail-yb1-xb31.google.com with SMTP id 3f1490d57ef6-dcc80d6006aso5068175276.0 for ; Mon, 04 Mar 2024 11:44:29 -0800 (PST) Received: from localhost.localdomain ([50.205.20.42]) by smtp.gmail.com with ESMTPSA id h1-20020a255f41000000b00dc62edd58dasm2282646ybm.40.2024.03.04.11.44.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Mar 2024 11:44:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709581468; x=1710186268; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zJmbpLHmX91RSD+FS2bbI4ESdtGu0icFFQqbt0j7En8=; b=QOW14upCueKGmhl7kqpIQXMJonky45CZaSrT9w5rW5nkEfu1vgILKNPuApPVDm4xTu r3gLa5E2dB7DgARnsnMFJmPLQYCzibL9ShQj5eE0VMMckagmL72spLd8qSETgTM2HfEW xqsNuRIQ0JZAB4bX9NtpEmH0AKruvwni4SvCCZoNwMsyS+DBDH7xFuIL3YlykZZRpXyW YufYMZf0/y1jzoK8AmeWYri/oP6PHdI7wQmpLZrgOU7pohqzJo9qUqSofXTKzbho+UMC OY9ZCZ1dQDUDIyVTx5Ezzvnex6qVSfTlS+iVvYIhmIF2NhA0vaI2UpINLIbgZOUt1/yl zh2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709581468; x=1710186268; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zJmbpLHmX91RSD+FS2bbI4ESdtGu0icFFQqbt0j7En8=; b=YicfiJ8Mk/WwgR+WERI9dCWIrbejJJ/wtMLL30W9+Xh9ZPmc8ZtDaXwEc6yq5eO9OO nJH2kRR6I8OWcssLXc+FGKAyLCplrbxdprBkiDkOgYbIqtcb/7m/rgNOts4PJ2n/i59E xeprHxTLOtJD6O9M7zl1iedgDT92S/NbdMWAyfris0SSxvdjblRfg3vxdbCPeG0rYkGz ej4ZWo4MSGNgy0r2445T/LJ+fHQMxswlaH8gcZ/9AHOZQlrMl7jQyC6uypcgjAjfbKCq GtqIYVpNLa4c/Aof1fnO6ULHksl/03XsmSW9vn7bxFQuvFOLJKJbEdBlWgGhjpYeO+sa ecqQ== X-Gm-Message-State: AOJu0Yx7RX6UcMksNQymiak9UuIHH7nIdNDPKvvIu6bOs+zwqiCxzmF5 nfVx46CXdUhqqNM3Qva6svMebvcPzRcr4RFkJg3bb7sOlzWNVQtg8jQYrKf9 X-Google-Smtp-Source: AGHT+IFgVk4gFsTBw619+d2h77tRgbIEeyXBAg4E1CIdEk27elXaNIecJfX30qCONXyB3gh3/cBbGg== X-Received: by 2002:a5b:644:0:b0:dc2:41de:b744 with SMTP id o4-20020a5b0644000000b00dc241deb744mr7144765ybq.32.1709581468003; Mon, 04 Mar 2024 11:44:28 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Jorgen.Hansen@wdc.com, wj28.lee@gmail.com, Fan Ni Subject: [PATCH v5 10/13] hw/mem/cxl_type3: Add dpa range validation for accesses to DC regions Date: Mon, 4 Mar 2024 11:34:05 -0800 Message-ID: <20240304194331.1586191-11-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240304194331.1586191-1-nifan.cxl@gmail.com> References: <20240304194331.1586191-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::b31; envelope-from=nifan.cxl@gmail.com; helo=mail-yb1-xb31.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1709581579822100009 Content-Type: text/plain; charset="utf-8" From: Fan Ni Not all dpa range in the DC regions is valid to access until an extent covering the range has been added. Add a bitmap for each region to record whether a DC block in the region has been backed by DC extent. For the bitmap, a bit in the bitmap represents a DC block. When a DC extent is added, all the bits of the blocks in the extent will be set, which will be cleared when the extent is released. Signed-off-by: Fan Ni Reviewed-by: Jonathan Cameron --- hw/cxl/cxl-mailbox-utils.c | 4 ++ hw/mem/cxl_type3.c | 76 +++++++++++++++++++++++++++++++++++++ include/hw/cxl/cxl_device.h | 7 ++++ 3 files changed, 87 insertions(+) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index 53ebc526ae..b538297bb5 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -1606,6 +1606,7 @@ static CXLRetCode cmd_dcd_add_dyn_cap_rsp(const struc= t cxl_cmd *cmd, =20 cxl_insert_extent_to_extent_list(extent_list, dpa, len, NULL, 0); ct3d->dc.total_extent_count +=3D 1; + ct3_set_region_block_backed(ct3d, dpa, len); } =20 /* @@ -1681,17 +1682,20 @@ static CXLRetCode cmd_dcd_release_dyn_cap(const str= uct cxl_cmd *cmd, found =3D true; cxl_remove_extent_from_extent_list(extent_list, ent); ct3d->dc.total_extent_count -=3D 1; + ct3_clear_region_block_backed(ct3d, ent_start_dpa, ent= _len); =20 if (len1) { cxl_insert_extent_to_extent_list(extent_list, ent_start_dpa, le= n1, NULL, 0); ct3d->dc.total_extent_count +=3D 1; + ct3_set_region_block_backed(ct3d, ent_start_dpa, l= en1); } if (len2) { cxl_insert_extent_to_extent_list(extent_list, dpa = + len, len2, NULL, 0); ct3d->dc.total_extent_count +=3D 1; + ct3_set_region_block_backed(ct3d, dpa + len, len2); } break; } else { diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index e9c8994cdb..c164cf4580 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -672,6 +672,7 @@ static bool cxl_create_dc_regions(CXLType3Dev *ct3d, Er= ror **errp) =20 region_base +=3D region->len; ct3d->dc.total_capacity +=3D region->len; + region->blk_bitmap =3D bitmap_new(region->len / region->block_size= ); } QTAILQ_INIT(&ct3d->dc.extents); QTAILQ_INIT(&ct3d->dc.extents_pending_to_add); @@ -682,6 +683,8 @@ static bool cxl_create_dc_regions(CXLType3Dev *ct3d, Er= ror **errp) static void cxl_destroy_dc_regions(CXLType3Dev *ct3d) { CXLDCExtent *ent; + int i; + CXLDCRegion *region; =20 while (!QTAILQ_EMPTY(&ct3d->dc.extents)) { ent =3D QTAILQ_FIRST(&ct3d->dc.extents); @@ -693,6 +696,11 @@ static void cxl_destroy_dc_regions(CXLType3Dev *ct3d) cxl_remove_extent_from_extent_list(&ct3d->dc.extents_pending_to_ad= d, ent); } + + for (i =3D 0; i < ct3d->dc.num_regions; i++) { + region =3D &ct3d->dc.regions[i]; + g_free(region->blk_bitmap); + } } =20 static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp) @@ -924,6 +932,70 @@ static void ct3_exit(PCIDevice *pci_dev) } } =20 +/* + * Mark the DPA range [dpa, dap + len) to be backed and accessible. This + * happens when a DC extent is added and accepted by the host. + */ +void ct3_set_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa, + uint64_t len) +{ + CXLDCRegion *region; + + region =3D cxl_find_dc_region(ct3d, dpa, len); + if (!region) { + return; + } + + bitmap_set(region->blk_bitmap, (dpa - region->base) / region->block_si= ze, + len / region->block_size); +} + +/* + * Check whether the DPA range [dpa, dpa + len) is backed with DC extents. + * Used when validating read/write to dc regions + */ +bool ct3_test_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa, + uint64_t len) +{ + CXLDCRegion *region; + uint64_t nbits; + long nr; + + region =3D cxl_find_dc_region(ct3d, dpa, len); + if (!region) { + return false; + } + + nr =3D (dpa - region->base) / region->block_size; + nbits =3D DIV_ROUND_UP(len, region->block_size); + /* + * if bits between [dpa, dpa + len) are all 1s, meaning the DPA range = is + * backed with DC extents, return true; else return false. + */ + return find_next_zero_bit(region->blk_bitmap, nr + nbits, nr) =3D=3D n= r + nbits; +} + +/* + * Mark the DPA range [dpa, dap + len) to be unbacked and inaccessible. Th= is + * happens when a dc extent is released by the host. + */ +void ct3_clear_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa, + uint64_t len) +{ + CXLDCRegion *region; + uint64_t nbits; + long nr; + + region =3D cxl_find_dc_region(ct3d, dpa, len); + if (!region) { + return; + } + + nr =3D (dpa - region->base) / region->block_size; + nbits =3D len / region->block_size; + bitmap_clear(region->blk_bitmap, nr, nbits); +} + static bool cxl_type3_dpa(CXLType3Dev *ct3d, hwaddr host_addr, uint64_t *d= pa) { int hdm_inc =3D R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_L= O; @@ -1029,6 +1101,10 @@ static int cxl_type3_hpa_to_as_and_dpa(CXLType3Dev *= ct3d, *as =3D &ct3d->hostpmem_as; *dpa_offset -=3D vmr_size; } else { + if (!ct3_test_region_block_backed(ct3d, *dpa_offset, size)) { + return -ENODEV; + } + *as =3D &ct3d->dc.host_dc_as; *dpa_offset -=3D (vmr_size + pmr_size); } diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index b524c5e699..b213149de2 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -450,6 +450,7 @@ typedef struct CXLDCRegion { uint64_t block_size; uint32_t dsmadhandle; uint8_t flags; + unsigned long *blk_bitmap; } CXLDCRegion; =20 struct CXLType3Dev { @@ -557,4 +558,10 @@ void cxl_insert_extent_to_extent_list(CXLDCExtentList = *list, uint64_t dpa, uint16_t shared_seq); bool test_any_bits_set(const unsigned long *addr, unsigned long nr, unsigned long size); +void ct3_set_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa, + uint64_t len); +void ct3_clear_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa, + uint64_t len); +bool ct3_test_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa, + uint64_t len); #endif --=20 2.43.0 From nobody Thu May 9 21:45:48 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1709581590; cv=none; d=zohomail.com; s=zohoarc; b=VDiXAPADtCIjgFqdhbhAOiVbRc9qi9BN3lMPsMGCWF6s9+XvbvkLgzA2kYMbIFFqGhcUupSqH4dUzJLlQFzqZJpADkrDu3ek7AuLnD86YyBicvUcafFUN6+oyW/RmIY3heVPU8e71mVmP6/7ltn077QyD2Rd1CqYQkIkoJWXpDA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709581590; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=283fafKoCAhZyon638Mj48RQ1VuKIm6CVjiYXSW1c5Y=; b=HmWUi1alNyoeLRS695GAf1NzOf5uGw/Iyjgt9uCFse1vxNIC5h4cJqkHQwctO9Uyxm+a/Z8NDtmS/IFC/e3XgtbnAGWiw1IH/UUmWq0j3DbDmz62Ei+Dk1/FTmIGc/hk5dhPCjMN4SLq4OslA//yDOXfHX2Ozd0LPTxM8l6TuN4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1709581590356701.8108176337904; Mon, 4 Mar 2024 11:46:30 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rhEFG-0005sK-R8; Mon, 04 Mar 2024 14:45:06 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhEEl-0005XY-IK for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:36 -0500 Received: from mail-yb1-xb2a.google.com ([2607:f8b0:4864:20::b2a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhEEj-0006yG-Na for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:35 -0500 Received: by mail-yb1-xb2a.google.com with SMTP id 3f1490d57ef6-dc238cb1b17so5221363276.0 for ; Mon, 04 Mar 2024 11:44:32 -0800 (PST) Received: from localhost.localdomain ([50.205.20.42]) by smtp.gmail.com with ESMTPSA id h1-20020a255f41000000b00dc62edd58dasm2282646ybm.40.2024.03.04.11.44.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Mar 2024 11:44:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709581471; x=1710186271; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=283fafKoCAhZyon638Mj48RQ1VuKIm6CVjiYXSW1c5Y=; b=SuzMFmzlEkvDTt/LOSPaTMBG7QDpwl1W4rbbUnitwLHJ54Iw5yBwrGwqB9lAMqi6PY ptiTUDUXuv/nrkN+Swx24fJcFlFeV2opWnOxrfk57KwT9tWHQsEFpn4AdOicFdOsnOHQ tM9Zrzm0P5ZmAull+5v649I6EYjTZXXHNYsOAw9F8+L/TeuwI/UhCt46S4/djyPPi2CI EgNXc/QZ1hJ3v1ZTPbINtfWue2WBpA/YeaVEyZXcvlirIkVgJdNTyyqfB7gcm/0XvpLD 0L13PRZK9iSsEjAbi96S+aXCOlqYmDAYbVPvtKYIMqpqh4VeKShzz13yFUrag+0uuy84 d9Kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709581471; x=1710186271; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=283fafKoCAhZyon638Mj48RQ1VuKIm6CVjiYXSW1c5Y=; b=HJDV3f8n1G0YVUiAtZ0bxVJIj2mNlxXC2n9UFUlvN0WGf0IXQdE5KqAdzF2DgM0fuB DPtzGLsnUMgwkgBGOa8RbJK+G73cqw0AnWO8M06jfgkuX2SXVdyos/f7qWryI7PmDhCC E2qwPtB8hgOBsD7qvOWvGASZe9KG4nlfyX/WghcQmkTfGQtf8u3YP40fzeJqPH3Doozo 923USUcZYJgpnK59GOa7zSyO7Opf6D3M0Te//tMzTYRnN14sU+8A8Ew5cN16AKf1xgfu YHoUoqw4udpOLq4QU6l4B1GHLLFtmXKWErD72oBTlhddpobTMc6ty1V72AYBACYnVgXa 4L4Q== X-Gm-Message-State: AOJu0YxYL5fCDZigbpt3KbBP2Xhp8Ui+0MDhv/omRLjscF9B8pCJHFg4 YKlcnlj70Yt4xGzxwohLTgITdhSq1zBjVPPNr9FD5AUuQmS+J/t6EOqwP0/Q X-Google-Smtp-Source: AGHT+IFK+f2/iwwB2tpKeab2vQ3PeNmoLam8Uf4HcS3wFtyitfX3H5sfkoJ3VHSCHggO5FT0xZVaWw== X-Received: by 2002:a25:7412:0:b0:dcc:79ab:e51a with SMTP id p18-20020a257412000000b00dcc79abe51amr8056756ybc.57.1709581471312; Mon, 04 Mar 2024 11:44:31 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Jorgen.Hansen@wdc.com, wj28.lee@gmail.com, Fan Ni Subject: [PATCH v5 11/13] hw/cxl/cxl-mailbox-utils: Add partial and superset extent release mailbox support Date: Mon, 4 Mar 2024 11:34:06 -0800 Message-ID: <20240304194331.1586191-12-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240304194331.1586191-1-nifan.cxl@gmail.com> References: <20240304194331.1586191-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::b2a; envelope-from=nifan.cxl@gmail.com; helo=mail-yb1-xb2a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1709581591917100001 Content-Type: text/plain; charset="utf-8" From: Fan Ni With the change, we extend the extent release mailbox command processing to allow more flexible release. As long as the DPA range of the extent to release is covered by valid extent(s) in the device, the release can be performed. Signed-off-by: Fan Ni --- hw/cxl/cxl-mailbox-utils.c | 211 +++++++++++++++++++++++++++++++++---- 1 file changed, 188 insertions(+), 23 deletions(-) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index b538297bb5..eaff5c4c93 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -1617,6 +1617,155 @@ static CXLRetCode cmd_dcd_add_dyn_cap_rsp(const str= uct cxl_cmd *cmd, return CXL_MBOX_SUCCESS; } =20 +/* + * Return value: the id of the DC region that covers the DPA range + * [dpa, dpa+len) The assumption is that the range is valid and within + * a DC region. + */ +static uint8_t cxl_find_dc_region_id(const CXLType3Dev *ct3d, uint64_t dpa, + uint64_t len) +{ + int i; + const CXLDCRegion *region; + + for (i =3D ct3d->dc.num_regions - 1; i >=3D 0; i--) { + region =3D &ct3d->dc.regions[i]; + if (dpa >=3D region->base) { + break; + } + } + return i; +} + +/* + * Copy extent list from src to dst + * Return value: number of extents copied + */ +static uint32_t copy_extent_list(CXLDCExtentList *dst, + const CXLDCExtentList *src) +{ + uint32_t cnt =3D 0; + CXLDCExtent *ent; + + if (!dst || !src) { + return 0; + } + + QTAILQ_FOREACH(ent, src, node) { + cxl_insert_extent_to_extent_list(dst, ent->start_dpa, ent->len, + ent->tag, ent->shared_seq); + cnt++; + } + return cnt; +} + +/* + * Detect potential extent overflow caused by extent split during processi= ng + * extent release requests, also allow releasing superset of extents where= the + * extent to release covers the range of multiple extents in the device. + * Note: + * 1.we will reject releasing an extent if some portion of its rang is + * not covered by valid extents. + * 2.This function is called after cxl_detect_malformed_extent_list so che= cks + * already performed there will be skipped. + */ +static CXLRetCode cxl_detect_extent_overflow(const CXLType3Dev *ct3d, + const CXLUpdateDCExtentListInPl *in) +{ + uint64_t nbits, offset; + const CXLDCRegion *region; + unsigned long **bitmaps_copied; + uint64_t dpa, len; + int i, rid; + CXLRetCode ret =3D CXL_MBOX_SUCCESS; + long extent_cnt_delta =3D 0; + CXLDCExtentList tmp_list; + CXLDCExtent *ent; + + QTAILQ_INIT(&tmp_list); + copy_extent_list(&tmp_list, &ct3d->dc.extents); + + bitmaps_copied =3D g_new0(unsigned long *, ct3d->dc.num_regions); + for (i =3D 0; i < ct3d->dc.num_regions; i++) { + region =3D &ct3d->dc.regions[i]; + nbits =3D region->len / region->block_size; + bitmaps_copied[i] =3D bitmap_new(nbits); + bitmap_copy(bitmaps_copied[i], region->blk_bitmap, nbits); + } + + for (i =3D 0; i < in->num_entries_updated; i++) { + dpa =3D in->updated_entries[i].start_dpa; + len =3D in->updated_entries[i].len; + + rid =3D cxl_find_dc_region_id(ct3d, dpa, len); + region =3D &ct3d->dc.regions[rid]; + offset =3D (dpa - region->base) / region->block_size; + nbits =3D len / region->block_size; + + /* Check whether range [dpa, dpa + len) is covered by valid range = */ + if (find_next_zero_bit(bitmaps_copied[rid], offset + nbits, offset= ) < + offset + nbits) { + ret =3D CXL_MBOX_INVALID_PA; + goto free_and_exit; + } + + QTAILQ_FOREACH(ent, &tmp_list, node) { + /* Only split within an extent can cause extent count increase= */ + if (ent->start_dpa <=3D dpa && + dpa + len <=3D ent->start_dpa + ent->len) { + uint64_t ent_start_dpa =3D ent->start_dpa; + uint64_t ent_len =3D ent->len; + uint64_t len1 =3D dpa - ent_start_dpa; + uint64_t len2 =3D ent_start_dpa + ent_len - dpa - len; + + extent_cnt_delta +=3D len1 && len2 ? 2 : (len1 || len2 ? 1= : 0); + extent_cnt_delta -=3D 1; + if (ct3d->dc.total_extent_count + extent_cnt_delta > + CXL_NUM_EXTENTS_SUPPORTED) { + ret =3D CXL_MBOX_RESOURCES_EXHAUSTED; + goto free_and_exit; + } + + offset =3D (ent->start_dpa - region->base) / region->block= _size; + nbits =3D ent->len / region->block_size; + bitmap_clear(bitmaps_copied[rid], offset, nbits); + cxl_remove_extent_from_extent_list(&tmp_list, ent); + + if (len1) { + offset =3D (dpa - region->base) / region->block_size; + nbits =3D len1 / region->block_size; + bitmap_set(bitmaps_copied[rid], offset, nbits); + cxl_insert_extent_to_extent_list(&tmp_list, + ent_start_dpa, len1, + NULL, 0); + } + + if (len2) { + offset =3D (dpa + len - region->base) / region->block_= size; + nbits =3D len2 / region->block_size; + bitmap_set(bitmaps_copied[rid], offset, nbits); + cxl_insert_extent_to_extent_list(&tmp_list, dpa + len, + len2, NULL, 0); + } + break; + } + } + } + +free_and_exit: + for (i =3D 0; i < ct3d->dc.num_regions; i++) { + g_free(bitmaps_copied[i]); + } + g_free(bitmaps_copied); + + while (!QTAILQ_EMPTY(&tmp_list)) { + ent =3D QTAILQ_FIRST(&tmp_list); + cxl_remove_extent_from_extent_list(&tmp_list, ent); + } + + return ret; +} + /* * CXL r3.1 section 8.2.9.9.9.4: Release Dynamic Capacity (Opcode 4803h) */ @@ -1644,15 +1793,28 @@ static CXLRetCode cmd_dcd_release_dyn_cap(const str= uct cxl_cmd *cmd, return ret; } =20 - for (i =3D 0; i < in->num_entries_updated; i++) { - bool found =3D false; + ret =3D cxl_detect_extent_overflow(ct3d, in); + if (ret !=3D CXL_MBOX_SUCCESS) { + return ret; + } =20 + /* + * After this point, it is guaranteed that the extents in the + * updated extent list to release is valid, that means: + * 1. All extents in the list have no overlaps; + * 2. Each extent belongs to a valid DC region; + * 3. The DPA range of each extent is covered by valid extent + * in the device. + */ + for (i =3D 0; i < in->num_entries_updated; i++) { dpa =3D in->updated_entries[i].start_dpa; len =3D in->updated_entries[i].len; =20 +process_leftover: QTAILQ_FOREACH(ent, extent_list, node) { /* Found the extent overlapping with */ if (ent->start_dpa <=3D dpa && dpa < ent->start_dpa + ent->len= ) { + /* Case 1: The to-release extent is subset of ent */ if (dpa + len <=3D ent->start_dpa + ent->len) { /* * The incoming extent covers a portion of an extent @@ -1669,17 +1831,6 @@ static CXLRetCode cmd_dcd_release_dyn_cap(const stru= ct cxl_cmd *cmd, uint64_t len1 =3D dpa - ent_start_dpa; uint64_t len2 =3D ent_start_dpa + ent_len - dpa - len; =20 - /* - * TODO: checking for possible extent overflow, will be - * moved into a dedicated function of detecting extent - * overflow. - */ - if (len1 && len2 && ct3d->dc.total_extent_count =3D=3D - CXL_NUM_EXTENTS_SUPPORTED) { - return CXL_MBOX_RESOURCES_EXHAUSTED; - } - - found =3D true; cxl_remove_extent_from_extent_list(extent_list, ent); ct3d->dc.total_extent_count -=3D 1; ct3_clear_region_block_backed(ct3d, ent_start_dpa, ent= _len); @@ -1700,20 +1851,34 @@ static CXLRetCode cmd_dcd_release_dyn_cap(const str= uct cxl_cmd *cmd, break; } else { /* - * TODO: we reject the attempt to remove an extent that - * overlaps with multiple extents in the device for no= w, - * once the bitmap indicating whether a DPA range is - * covered by valid extents is introduced, will allow = it. + * Case 2: the to-release extent overlaps with multiple + * extents, including the superset case */ - return CXL_MBOX_INVALID_PA; + uint64_t ent_start_dpa =3D ent->start_dpa; + uint64_t ent_len =3D ent->len; + uint64_t len1 =3D dpa - ent_start_dpa; + + cxl_remove_extent_from_extent_list(extent_list, ent); + ct3d->dc.total_extent_count -=3D 1; + ct3_clear_region_block_backed(ct3d, ent_start_dpa, ent= _len); + + if (len1) { + cxl_insert_extent_to_extent_list(extent_list, + ent_start_dpa, le= n1, + NULL, 0); + ct3d->dc.total_extent_count +=3D 1; + ct3_set_region_block_backed(ct3d, ent_start_dpa, l= en1); + } + /* + * processing the portion of the range following curre= nt + * extent + */ + len =3D dpa + len - ent_start_dpa - ent_len; + dpa =3D ent_start_dpa + ent_len; + goto process_leftover; } } } - - if (!found) { - /* Try to remove a non-existing extent. */ - return CXL_MBOX_INVALID_PA; - } } =20 return CXL_MBOX_SUCCESS; --=20 2.43.0 From nobody Thu May 9 21:45:48 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1709581606; cv=none; d=zohomail.com; s=zohoarc; b=Tg3ZFiUUUbjj2fnEGw37nkGf6wz39BdOtUpqi4SAaLzlRZwDUJ4eMT8c2E9+DDtODUvnCxVw3+VcI/c6Ptba8tKFqVL8Ydh4ezkVt9Iq8D5QG7rsyeCGDWJgBUoxa7R8wKOgfqwH9lSqwnOrtsJKnG4Xi1eQXCSDaf5JCK8NtK4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709581606; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=AaOcSj55m2vuHo7NOezYbOQrG1G43UvBtyGzB+x7te4=; b=Rs0yMqrubUysrRto2IIKCFx48azqaOJHkMjKPJgJoc4jxyr4GYkM4fLFN0bacK8kgbZ8qmtEnJ2nv0l8o6IV82szVi70q2cFEe6rIZidIioy0bYByWC2cQ2rUmK5herOK9cUssEDD8phDimMNZvvVtYk5r05nEMFnQcZhAENtjw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1709581606630628.7592399808628; Mon, 4 Mar 2024 11:46:46 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rhEFE-0005oG-VW; Mon, 04 Mar 2024 14:45:05 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhEEm-0005Xj-E6 for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:36 -0500 Received: from mail-yw1-x1129.google.com ([2607:f8b0:4864:20::1129]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhEEk-0006yo-Ld for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:36 -0500 Received: by mail-yw1-x1129.google.com with SMTP id 00721157ae682-6095dfcb461so46337797b3.2 for ; Mon, 04 Mar 2024 11:44:34 -0800 (PST) Received: from localhost.localdomain ([50.205.20.42]) by smtp.gmail.com with ESMTPSA id h1-20020a255f41000000b00dc62edd58dasm2282646ybm.40.2024.03.04.11.44.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Mar 2024 11:44:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709581473; x=1710186273; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AaOcSj55m2vuHo7NOezYbOQrG1G43UvBtyGzB+x7te4=; b=Hft/mq88jQrBqtcWqgdNOxOs64LYOIrQmJ24uwy/qtzqq3a4x3NlUvMGSo60B4vcMc +QiEtA92kI/s7n+uPRSxslW9mvEqsV+BdNg1It4/fAu6weGgoCekPRHp3UNNMaiU9lGc VrsrkTTVtwqqxN4WyKF5mu8QBgm8hcbRcD//D42Irxd80tw892KCb++Xhpnb4S5kB52e qgYu1xlRCjBwfChEv+2e+SPy3OZtWYYWbytiAAMT8pg8GoP003j+tnYc3/o0ki0sdsyC VK2QS4328Qd+LAe+plWaAgH0RYbd/dRs7dO27+KQUM/2a8+654/UDqCrAK9kpo5fTdG1 xhqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709581473; x=1710186273; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AaOcSj55m2vuHo7NOezYbOQrG1G43UvBtyGzB+x7te4=; b=uj2WNob5KPx2fe/ysqgOeGAW1El/XpiKoYxyvO+OEBAXeWRDkTjy/rXNf7b1bBI8I1 6xLwOmRZK1NZwbKX4V20Fg18NxPw6DQrb765V0CrZfDJGi6lfZWCddIuA4Yz/wiyMYkd FnV9qmjVt8FWSoTpfz4OPzclXjcRCm1x3XeFsNzqJDuGaYF/mGs0IPu7p/It7TXBX+d1 XCNArmIyf6GbmJLlKigm6MS6PcJAK5u823cqrzq8+X+KqOL+dQ9NSYUPPukFXskRqdzd ktkhdFH+fh3Sv9cIUYu84WDM/aYx0F7Shfui/7OrO6UrmfI3hd86ne1b+NAmlz6WBbNU XhgA== X-Gm-Message-State: AOJu0YxgKApGuiUTriiATOU61++OkCkprv7DtgTLCXparnze1oekcwMo zeZBuZ9h0aTLq7NBBLutlQQ7ELffbBrcVujjPH0HNqFjv5WGS4E4G+g7Qes1 X-Google-Smtp-Source: AGHT+IGAUQ4G1g2oUYmUyuIpiZpE/G0LLDXICATQRHvrTwniiC32r7Y2rbFBwsEBLZHMsINmZ5Ysdg== X-Received: by 2002:a5b:cd1:0:b0:dc2:1f53:3a4f with SMTP id e17-20020a5b0cd1000000b00dc21f533a4fmr7160003ybr.5.1709581473236; Mon, 04 Mar 2024 11:44:33 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Jorgen.Hansen@wdc.com, wj28.lee@gmail.com, Fan Ni Subject: [PATCH v5 12/13] hw/mem/cxl_type3: Allow to release partial extent and extent superset in QMP interface Date: Mon, 4 Mar 2024 11:34:07 -0800 Message-ID: <20240304194331.1586191-13-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240304194331.1586191-1-nifan.cxl@gmail.com> References: <20240304194331.1586191-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::1129; envelope-from=nifan.cxl@gmail.com; helo=mail-yw1-x1129.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1709581607952100003 Content-Type: text/plain; charset="utf-8" From: Fan Ni Before the change, the QMP interface used for add/release DC extents only allows to release extents that exist in either pending-to-add list or accepted list in the device, which means the DPA range of the extent must match exactly that of an extent in either list. Otherwise, the release request will be ignored. With the change, we relax the constraints. As long as the DPA range of the extent to release is covered by extents in one of the two lists mentioned above, we allow the release. Signed-off-by: Fan Ni --- hw/mem/cxl_type3.c | 110 ++++++++++++++++++++++++++++++++++++--------- 1 file changed, 89 insertions(+), 21 deletions(-) diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index c164cf4580..5bd64e604e 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -1802,28 +1802,79 @@ typedef enum CXLDCEventType { } CXLDCEventType; =20 /* - * Check whether the exact extent exists in the list - * Return value: the extent pointer in the list; else null + * Testing whether the DPA range [dpa, dpa + len) is covered by + * extents in the list. */ -static CXLDCExtent *cxl_dc_extent_exists(CXLDCExtentList *list, - CXLDCExtentRaw *ext) +static bool cxl_test_dpa_range_covered_by_extents(CXLDCExtentList *list, + uint64_t dpa, uint64_t l= en) { CXLDCExtent *ent; =20 - if (!ext || !list) { - return NULL; + if (!list) { + return false; } =20 - QTAILQ_FOREACH(ent, list, node) { - if (ent->start_dpa !=3D ext->start_dpa) { - continue; - } + while (len) { + bool has_leftover =3D false; =20 - /* Found exact extent */ - return ent->len =3D=3D ext->len ? ent : NULL; + QTAILQ_FOREACH(ent, list, node) { + if (ent->start_dpa <=3D dpa && dpa < ent->start_dpa + ent->len= ) { + if (dpa + len <=3D ent->start_dpa + ent->len) { + return true; + } else { + len =3D dpa + len - ent->start_dpa - ent->len; + dpa =3D ent->start_dpa + ent->len; + has_leftover =3D true; + break; + } + } + } + if (!has_leftover) { + break; + } } + return false; +} + +/* + * Remove all extents whose DPA range has overlaps with the DPA range + * [dpa, dpa + len) from the list, and delete the overlapped portion. + * Note: + * 1. If the removed extents is fully within the DPA range, delete the ext= ent; + * 2. Otherwise, keep the portion that does not overlap, insert new extent= s to + * the list if needed for the un-coverlapped part. + */ +static void cxl_delist_extent_by_dpa_range(CXLDCExtentList *list, + uint64_t dpa, uint64_t len) +{ + CXLDCExtent *ent; =20 - return NULL; +process_leftover: + QTAILQ_FOREACH(ent, list, node) { + if (ent->start_dpa <=3D dpa && dpa < ent->start_dpa + ent->len) { + uint64_t ent_start_dpa =3D ent->start_dpa; + uint64_t ent_len =3D ent->len; + uint64_t len1 =3D dpa - ent_start_dpa; + + cxl_remove_extent_from_extent_list(list, ent); + if (len1) { + cxl_insert_extent_to_extent_list(list, ent_start_dpa, + len1, NULL, 0); + } + + if (dpa + len <=3D ent_start_dpa + ent_len) { + uint64_t len2 =3D ent_start_dpa + ent_len - dpa - len; + if (len2) { + cxl_insert_extent_to_extent_list(list, dpa + len, + len2, NULL, 0); + } + } else { + len =3D dpa + len - ent_start_dpa - ent_len; + dpa =3D ent_start_dpa + ent_len; + goto process_leftover; + } + } + } } =20 /* @@ -1915,8 +1966,8 @@ static void qmp_cxl_process_dynamic_capacity(const ch= ar *path, CxlEventLog log, list =3D records; extents =3D g_new0(CXLDCExtentRaw, num_extents); while (list) { - CXLDCExtent *ent; bool skip_extent =3D false; + CXLDCExtentList *extent_list; =20 offset =3D list->value->offset; len =3D list->value->len; @@ -1933,15 +1984,32 @@ static void qmp_cxl_process_dynamic_capacity(const = char *path, CxlEventLog log, * remove it from the pending extent list, so later when the a= dd * response for the extent arrives, the device can reject the * extent as it is not in the pending list. + * Now, we can handle the case where the extent covers the DPA + * range of multiple extents in the pending_to_add list. + * TODO: we do not allow the extent covers range of extents in + * pending_to_add list and accepted list at the same time for = now. */ - ent =3D cxl_dc_extent_exists(&dcd->dc.extents_pending_to_add, - &extents[i]); - if (ent) { - QTAILQ_REMOVE(&dcd->dc.extents_pending_to_add, ent, node); - g_free(ent); + extent_list =3D &dcd->dc.extents_pending_to_add; + if (cxl_test_dpa_range_covered_by_extents(extent_list, + extents[i].start_dpa, + extents[i].len)) { + cxl_delist_extent_by_dpa_range(extent_list, + extents[i].start_dpa, + extents[i].len); + } else if (!ct3_test_region_block_backed(dcd, extents[i].start= _dpa, + extents[i].len)) { + /* + * If the DPA range of the extent is not covered by extents + * in the accepted list, skip + */ skip_extent =3D true; - } else if (!cxl_dc_extent_exists(&dcd->dc.extents, &extents[i]= )) { - /* If the exact extent is not in the accepted list, skip */ + } + } else if (type =3D=3D DC_EVENT_ADD_CAPACITY) { + extent_list =3D &dcd->dc.extents; + /* If the extent is ready pending to add, skip */ + if (cxl_test_dpa_range_covered_by_extents(extent_list, + extents[i].start_dpa, + extents[i].len)) { skip_extent =3D true; } } --=20 2.43.0 From nobody Thu May 9 21:45:48 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1709581533; cv=none; d=zohomail.com; s=zohoarc; b=UGP0R7yihWOX2HiG9I74etBZRtYEfmlzZdi483yvSNz5QXRZvuGq9KWtpa+vaLQUzjH08kmedd4rwgTOUx9zE+QQfOQi07Ai4nelVehRWIn1jjMd+9thuq8uypejdMvjOmUrh9cbyA65uGVDjJNkm7jnVPkBrINZN0oi3yBdZYo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709581533; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=TKU7EMmrCEGIbEXLnUmzxmQlpe4p33ksgB+5YiuAYXs=; b=G3ODD3n/sEf5LvbEu59f+o7R7b9izWEQZ+uW/huHlnmJJ2lnkOIX9Hi3CCfUdDF0GptTZ1D7ig+bGLSX7mCEpP72czZndDFcPAC69vKmUvoOY/rcTwr9o7b2X14/9yynn5swYry74tS2ZjNe/FxJqFzCAL0WCqzcmBZ+K21bRT4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 170958153396123.79482303038685; Mon, 4 Mar 2024 11:45:33 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rhEFF-0005q0-AM; Mon, 04 Mar 2024 14:45:05 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhEEq-0005dI-MJ for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:42 -0500 Received: from mail-yb1-xb36.google.com ([2607:f8b0:4864:20::b36]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhEEp-0006z9-2Z for qemu-devel@nongnu.org; Mon, 04 Mar 2024 14:44:40 -0500 Received: by mail-yb1-xb36.google.com with SMTP id 3f1490d57ef6-db4364ecd6aso5048718276.2 for ; Mon, 04 Mar 2024 11:44:37 -0800 (PST) Received: from localhost.localdomain ([50.205.20.42]) by smtp.gmail.com with ESMTPSA id h1-20020a255f41000000b00dc62edd58dasm2282646ybm.40.2024.03.04.11.44.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Mar 2024 11:44:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709581476; x=1710186276; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TKU7EMmrCEGIbEXLnUmzxmQlpe4p33ksgB+5YiuAYXs=; b=ApR0SxyZ0y2x1m+wVRGfSeaCxU2+FrFRRXht3z2fEsAbauv5juhk0ly4XIPvFjHYT6 NgJPcbiwEnYPrpAZiSpz11fpbj+6nxInqXzjh4Qcq15qhactaA+/oENfVzitSGUZ9XFI VE93BTDDDLSyEB1LZW9UoNHun4JXEaoNAd5peBs5RhTO3Kb1WOsa0ZQhrAKxj25APGs7 ZIh1ps5odkXwXPdnYF3bsg77hv4+3iMVIKZLr95m+szK8P3W2Ptzg16Uf5LRnLzv4cTz y6Mo/fQs+wKelU5tzL6GaOFDIY1EzCp3TnpwK3xOy6DkWcMfzS8IlrZRU5Bxt94iSg3Q xGSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709581476; x=1710186276; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TKU7EMmrCEGIbEXLnUmzxmQlpe4p33ksgB+5YiuAYXs=; b=JS69OqcJC6bN25s+krw2ODMlgxh5fJR4tdbOrtHpsU+MvmKzsjGQT7iWgHO92oGLuy 8xqhDWOjUaK3UU8KlHG4IETiK8DvPjt5Rkf4NNV8kAJc/Ni/IA6phBdVGVzrAu5lkNaz +0AL0EqErEA2ynwJe31n/8PpAV10OGMtFAsjNAKs61N5pwmMthmZaTYPcrv1F+0whlLM rc87ZAICRrdt8OLHkLkGqe73ymc59GHNJ2hjWCTe8/wP1PSxTh8zAQhSJ6iLua3q0/mQ QZ9j3O1/mwAzC7WCIipxaLAOCjYaD2yN82dyT0VAjHzKo6/J1xvWRCU+MlP+uZSqCRd4 2+tg== X-Gm-Message-State: AOJu0Yy2y6214AiK4MxPWQpD16ifDBQXMfNyvsfnF4NP/AfFD4GR/NIo SLM/M8XmBppDzSoTJ7tf4Lw4W62cWsz5hS5FGvfFpFRR5rc2f7yBxiesTBxl X-Google-Smtp-Source: AGHT+IGJaenBrz1oSac79n8j93VzbYMt0qiN1tC/RpBnCVT3yoX1a0NYTi0pRDY7jv1mKlEMaKgrEw== X-Received: by 2002:a25:2690:0:b0:dc6:ff12:13d7 with SMTP id m138-20020a252690000000b00dc6ff1213d7mr5343408ybm.60.1709581476386; Mon, 04 Mar 2024 11:44:36 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Jorgen.Hansen@wdc.com, wj28.lee@gmail.com, Fan Ni Subject: [PATCH v5 13/13] qapi/cxl.json: Add QMP interfaces to print out accepted and pending DC extents Date: Mon, 4 Mar 2024 11:34:08 -0800 Message-ID: <20240304194331.1586191-14-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240304194331.1586191-1-nifan.cxl@gmail.com> References: <20240304194331.1586191-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::b36; envelope-from=nifan.cxl@gmail.com; helo=mail-yb1-xb36.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1709581535674100001 Content-Type: text/plain; charset="utf-8" From: Fan Ni With the change, we add the following two QMP interfaces to print out extents information in the device, 1. cxl-display-accepted-dc-extents: print out the accepted DC extents in the device; 2. cxl-display-pending-to-add-dc-extents: print out the pending-to-add DC extents in the device; The output is appended to a file passed to the command and by default it is /tmp/dc-extent.txt. Signed-off-by: Fan Ni --- hw/mem/cxl_type3.c | 80 ++++++++++++++++++++++++++++++++++++++++ hw/mem/cxl_type3_stubs.c | 12 ++++++ qapi/cxl.json | 32 ++++++++++++++++ 3 files changed, 124 insertions(+) diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index 5bd64e604e..6a08e7ae40 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -2089,6 +2089,86 @@ void qmp_cxl_release_dynamic_capacity(const char *pa= th, uint8_t region_id, region_id, records, errp); } =20 +static void cxl_dcd_display_extent_list(const CXLType3Dev *dcd, const char= *f, + bool accepted_list, Error **errp) +{ + const CXLDCExtentList *list; + CXLDCExtent *ent; + FILE *fp =3D NULL; + int i =3D 0; + + if (!dcd->dc.num_regions) { + error_setg(errp, "No dynamic capacity support from the device"); + return; + } + + if (!f) { + fp =3D fopen("/tmp/dc-extent.txt", "a+"); + } else { + fp =3D fopen(f, "a+"); + } + + if (!fp) { + error_setg(errp, "Open log file failed"); + return; + } + if (accepted_list) { + list =3D &dcd->dc.extents; + fprintf(fp, "Print accepted extent info:\n"); + } else { + list =3D &dcd->dc.extents_pending_to_add; + fprintf(fp, "Print pending-to-add extent info:\n"); + } + + QTAILQ_FOREACH(ent, list, node) { + fprintf(fp, "%d: [0x%lx - 0x%lx]\n", i++, ent->start_dpa, + ent->start_dpa + ent->len); + } + fprintf(fp, "In total, %d extents printed!\n", i); + fclose(fp); +} + +void qmp_cxl_display_accepted_dc_extents(const char *path, const char *f, + Error **errp) +{ + Object *obj; + CXLType3Dev *dcd; + + obj =3D object_resolve_path(path, NULL); + if (!obj) { + error_setg(errp, "Unable to resolve path"); + return; + } + if (!object_dynamic_cast(obj, TYPE_CXL_TYPE3)) { + error_setg(errp, "Path not point to a valid CXL type3 device"); + return; + } + + dcd =3D CXL_TYPE3(obj); + cxl_dcd_display_extent_list(dcd, f, true, errp); +} + +void qmp_cxl_display_pending_to_add_dc_extents(const char *path, const cha= r *f, + Error **errp) +{ + Object *obj; + CXLType3Dev *dcd; + + obj =3D object_resolve_path(path, NULL); + if (!obj) { + error_setg(errp, "Unable to resolve path"); + return; + } + if (!object_dynamic_cast(obj, TYPE_CXL_TYPE3)) { + error_setg(errp, "Path not point to a valid CXL type3 device"); + return; + } + + + dcd =3D CXL_TYPE3(obj); + cxl_dcd_display_extent_list(dcd, f, false, errp); +} + static void ct3_class_init(ObjectClass *oc, void *data) { DeviceClass *dc =3D DEVICE_CLASS(oc); diff --git a/hw/mem/cxl_type3_stubs.c b/hw/mem/cxl_type3_stubs.c index d913b11b4d..d896758301 100644 --- a/hw/mem/cxl_type3_stubs.c +++ b/hw/mem/cxl_type3_stubs.c @@ -81,3 +81,15 @@ void qmp_cxl_release_dynamic_capacity(const char *path, = uint8_t region_id, { error_setg(errp, "CXL Type 3 support is not compiled in"); } + +void qmp_cxl_display_accepted_dc_extents(const char *path, const char *f, + Error **errp) +{ + error_setg(errp, "CXL Type 3 support is not compiled in"); +} + +void qmp_cxl_display_pending_to_add_dc_extents(const char *path, const cha= r *f, + Error **errp) +{ + error_setg(errp, "CXL Type 3 support is not compiled in"); +} diff --git a/qapi/cxl.json b/qapi/cxl.json index 2645004666..6f10300ec6 100644 --- a/qapi/cxl.json +++ b/qapi/cxl.json @@ -420,3 +420,35 @@ 'extents': [ 'CXLDCExtentRecord' ] } } + +## +# @cxl-display-accepted-dc-extents: +# +# Command to print out all the accepted DC extents in the device +# +# @path: CXL DCD canonical QOM path +# @output: path of output file to dump the results to +# +# Since : 9.0 +## +{ 'command': 'cxl-display-accepted-dc-extents', + 'data': { 'path': 'str', + 'output': 'str' + } +} + +## +# @cxl-display-pending-to-add-dc-extents: +# +# Command to print out all the pending-to-add DC extents in the device +# +# @path: CXL DCD canonical QOM path +# @output: path of output file to dump the results to +# +# Since : 9.0 +## +{ 'command': 'cxl-display-pending-to-add-dc-extents', + 'data': { 'path': 'str', + 'output': 'str' + } +} --=20 2.43.0