From nobody Tue Nov 26 08:37:50 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1708539797; cv=none; d=zohomail.com; s=zohoarc; b=XTFO2Z9lapmWMmW6c1STb+Of8QP5RcISTU5s0JSK5k1MhyCc79nDemxOWhoB/g1ysc/iep85xIFXZuJPuQd0eDQZE5FDSyghZmY5aIxLlyXafFD+PuENv9WerACpiau54q0qFRkKj17IMQmNtxOJapIowjZhv45O61cjluH4XT4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708539797; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=AkPmI8FuszQzNKc/W7C7bH++18bz1caF1vBxyxTADMI=; b=bpb1jJ7Mc5+IrTr3xzLtbpx3JFH6OB+NP9Fo+Z73nPhdIc6Cai/Vg8jFsXwMfHWkmyK8gYzeSDN71pyezvE86SeE0yAvpCUW03kNgZHp/vhakKjEtMNCZHuptTRML8u4Hn/T+0vyxnbKelQG7yAHpr52ZUbe8IPWr4ZRHHby1sE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708539797313423.44299138069243; Wed, 21 Feb 2024 10:23:17 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rcrDg-0007t1-B7; Wed, 21 Feb 2024 13:21:24 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rcrDY-0007ft-Qt for qemu-devel@nongnu.org; Wed, 21 Feb 2024 13:21:16 -0500 Received: from mail-pj1-x1029.google.com ([2607:f8b0:4864:20::1029]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rcrDW-0003J1-8c for qemu-devel@nongnu.org; Wed, 21 Feb 2024 13:21:16 -0500 Received: by mail-pj1-x1029.google.com with SMTP id 98e67ed59e1d1-29026523507so5241576a91.0 for ; Wed, 21 Feb 2024 10:21:13 -0800 (PST) Received: from localhost.localdomain ([2601:641:300:14de:5692:7e41:13a2:69a]) by smtp.gmail.com with ESMTPSA id cs16-20020a17090af51000b002992f49922csm9979009pjb.25.2024.02.21.10.21.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Feb 2024 10:21:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708539672; x=1709144472; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AkPmI8FuszQzNKc/W7C7bH++18bz1caF1vBxyxTADMI=; b=bcWptzMSWKntLYZZJ525r6JVGX9N1dCimKy8I6N69cAt3ZHgxpPgX+5DPSwCn1L3sV WkYVlq82dMTJy4Xhuxgi6xuxcETqBNDbg85ql6qplBAzRIJjP0STei9o8Y8I545r+jhx OesGa1rUfScC5e/ni5sqC7TRMyl4zdi1eQbhqzCfUTAdVNtj8s2BFpgAztT+XznhKqIS h7I7GLm8+JhKkopRnpv1PUt7eUcCjpMTsINCooI7IuPBQqBtV+CTKeSYrBXgErza/aU/ JuVMb+tfNEM2kXWHej1mTCdZzyL+UEFwR5aM2GloKW8Kzu8COGAK8G6Ah61w8kmT7LVE eoUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708539672; x=1709144472; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AkPmI8FuszQzNKc/W7C7bH++18bz1caF1vBxyxTADMI=; b=tLr75g4//UECRsWJuTNZg3Qnsi4L9RcfPTCa8BYhlIDLaNRZJI5L4wBQk9kYsXr3T4 xF23qVmjAjS87IkoOcyWViJNe7NysCs2kv6+yxiV9b6FRIsGsYOfuOtC8OcZ8lf4irpz Btow3wmUTILfv5T5TRI2o9vEBTYjuA7A0Pbf5mw7hFSPKLt4cqwT4FI4Nm44wJXnC25f RvDyZXeFS5gT+A87Oy2fno3D78Lzt/mK9KOuJeDsg4FI5PAihLu3Cr0vi3Kzrwedcbw2 qJLPmIgyQJOAMCkcD63521nYAeRydNFTPirsYqMbhQJM+6Dmk724VxzH3uGdZ6FnI/lp aHIQ== X-Gm-Message-State: AOJu0YwF0oKgb2Z5Crut1rwc12yolt2N/w8wMz9iZ7ZY+LJ4SHl7X2D1 PFHxZGkUxfF3j0PDO+7xMaDVGiD3FlZoL68TFcigN+6mzozOxH53RcKnmj2p X-Google-Smtp-Source: AGHT+IFMcnQh5f80Wkda3K0vH+ueICO5TJY8WkNFGkF0tVy5RzTaPhNg7QC/g2xsj8usX4PLltMs+Q== X-Received: by 2002:a17:90b:1e47:b0:299:2241:820b with SMTP id pi7-20020a17090b1e4700b002992241820bmr14945559pjb.37.1708539672173; Wed, 21 Feb 2024 10:21:12 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Fan Ni Subject: [PATCH v4 01/10] hw/cxl/cxl-mailbox-utils: Add dc_event_log_size field to output payload of identify memory device command Date: Wed, 21 Feb 2024 10:15:54 -0800 Message-ID: <20240221182020.1086096-2-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240221182020.1086096-1-nifan.cxl@gmail.com> References: <20240221182020.1086096-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::1029; envelope-from=nifan.cxl@gmail.com; helo=mail-pj1-x1029.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1708539799509100003 Content-Type: text/plain; charset="utf-8" From: Fan Ni Based on CXL spec r3.1 Table 8-127 (Identify Memory Device Output Payload), dynamic capacity event log size should be part of output of the Identify command. Add dc_event_log_size to the output payload for the host to get the info. Signed-off-by: Fan Ni --- hw/cxl/cxl-mailbox-utils.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index 4bcd727f4c..ba1d9901df 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -21,6 +21,7 @@ #include "sysemu/hostmem.h" =20 #define CXL_CAPACITY_MULTIPLIER (256 * MiB) +#define CXL_DC_EVENT_LOG_SIZE 8 =20 /* * How to add a new command, example. The command set FOO, with cmd BAR. @@ -780,8 +781,9 @@ static CXLRetCode cmd_identify_memory_device(const stru= ct cxl_cmd *cmd, uint16_t inject_poison_limit; uint8_t poison_caps; uint8_t qos_telemetry_caps; + uint16_t dc_event_log_size; } QEMU_PACKED *id; - QEMU_BUILD_BUG_ON(sizeof(*id) !=3D 0x43); + QEMU_BUILD_BUG_ON(sizeof(*id) !=3D 0x45); CXLType3Dev *ct3d =3D CXL_TYPE3(cci->d); CXLType3Class *cvc =3D CXL_TYPE3_GET_CLASS(ct3d); CXLDeviceState *cxl_dstate =3D &ct3d->cxl_dstate; @@ -807,6 +809,7 @@ static CXLRetCode cmd_identify_memory_device(const stru= ct cxl_cmd *cmd, st24_le_p(id->poison_list_max_mer, 256); /* No limit - so limited by main poison record limit */ stw_le_p(&id->inject_poison_limit, 0); + stw_le_p(&id->dc_event_log_size, CXL_DC_EVENT_LOG_SIZE); =20 *len_out =3D sizeof(*id); return CXL_MBOX_SUCCESS; --=20 2.43.0 From nobody Tue Nov 26 08:37:50 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1708539749; cv=none; d=zohomail.com; s=zohoarc; b=RfpC5TXyQLQoxoJ+NR5cpBHuSH4sR0NtGGD9TH8OP6tY1LpqvQhxm/ru465XPPVBebvEPCw+vQ+VgNUYb3gq4SDhGiDa1UTLeA4ELOOxduaRym5hGI6wrhiM7OGHGEjjVvnIVPpK85fLBcIRKY0s0hNGUsIfLL6FSC5jE3UKdwI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708539749; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=un/ALJ33UJ/4j/7Dwtf0xAxrV2NGLPOZb9LxVVKMbdE=; b=OHZsfuzvQCn+LBCcvogddZfus8A+EYy210BGbQMEshZbJGFuzMpKUT/wX3HgW7NAJVQzLGZab8z7gSaSeSS+Nj3h3zASSjHvjSQP56e4FOT3dY2EdzBnYLd8jwSGuuUn6tUjwfpKKMNRmLucpEuIjC0Rbv98khEipZ49e0V9idU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708539749907359.2699439202604; Wed, 21 Feb 2024 10:22:29 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rcrDb-0007hc-QC; Wed, 21 Feb 2024 13:21:20 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rcrDZ-0007g6-0i for qemu-devel@nongnu.org; Wed, 21 Feb 2024 13:21:17 -0500 Received: from mail-pg1-x530.google.com ([2607:f8b0:4864:20::530]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rcrDX-0003J8-1Y for qemu-devel@nongnu.org; Wed, 21 Feb 2024 13:21:16 -0500 Received: by mail-pg1-x530.google.com with SMTP id 41be03b00d2f7-53fa455cd94so4867318a12.2 for ; Wed, 21 Feb 2024 10:21:14 -0800 (PST) Received: from localhost.localdomain ([2601:641:300:14de:5692:7e41:13a2:69a]) by smtp.gmail.com with ESMTPSA id cs16-20020a17090af51000b002992f49922csm9979009pjb.25.2024.02.21.10.21.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Feb 2024 10:21:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708539673; x=1709144473; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=un/ALJ33UJ/4j/7Dwtf0xAxrV2NGLPOZb9LxVVKMbdE=; b=Vbhr/9JoLBzc5Uns1bH6sx9fYjh6jt31ROhH5tcwRZStdirtb9u6cShs+mR9IvXpmN bxi7kRbpmzyFUfv3yJ4InslbYcQKyLACHHvJfQ5C/u+dWAUIHyWi5TrF9joX1uiD74P+ Ac1ZFTDUmN2Lt6m5EGDjkOt0Vs8DJfySzquFtG3inPa1FwUA04u8akZ56FiQSpYwK24M 0O1TFwRTxo8Nt7nAmOpklWZWw1doAvP8Q55+OVWnedTcs068pTYgi9DTWC8/Ea8W71vl qbyY2wX8ltA24nbjvcK/gwR05qlxm53XhEsfIgaNI9M4dgSGRB+dkESgfFAytas3ggm3 xm4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708539673; x=1709144473; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=un/ALJ33UJ/4j/7Dwtf0xAxrV2NGLPOZb9LxVVKMbdE=; b=UGjN+bVvdCUVzcjxIx7ggXyFYbOD9UdvXDesBuQGnFm6Hxd8BuUHNEiZ2kCPTccaVk 8XxXo/10yjU0bA8q8cpxO1hHSE10kxyVYd1LSjjfGretR2PEEJSIKq6vPT3XtQFFDI26 UX3/YtBGZTB8n35lkb1IEsuUaokqbOx/aWIPlnR3d9/B5ASKl9U8GHlEdTr0x00T1wX2 PaycoF2woTZGk5stEsfUDQu/xJE2scbWnwqlaN7r4hjltUayhjbbPoqVOgyxfUcXvWzw 0ZXicms5hN3uzLkfS/WG+2kkAYOjya3IPfccCGuytTEeK2duFGiGVJKhtN0m9xWNN8ql 2QBw== X-Gm-Message-State: AOJu0YwkVt/sO0KS3dIhMwjxgTLiKpJ1OukGRqw85gLt7Mmvb49054Au U8QibqttMDApkccLpI6nZAYXPtMsLuyi8SX1yPeHxapRgH0qzTXJH+gWRgmb X-Google-Smtp-Source: AGHT+IHZJskgqEMippermGM7haozIZGUI/e1pO3JbuOpb/ZzYlJZuIuyb9dQ1vzD1Ic/ZVpDa2Qq3w== X-Received: by 2002:a17:90a:10d0:b0:299:bade:ed0b with SMTP id b16-20020a17090a10d000b00299badeed0bmr6854885pje.45.1708539673292; Wed, 21 Feb 2024 10:21:13 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Fan Ni Subject: [PATCH v4 02/10] hw/cxl/cxl-mailbox-utils: Add dynamic capacity region representative and mailbox command support Date: Wed, 21 Feb 2024 10:15:55 -0800 Message-ID: <20240221182020.1086096-3-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240221182020.1086096-1-nifan.cxl@gmail.com> References: <20240221182020.1086096-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::530; envelope-from=nifan.cxl@gmail.com; helo=mail-pg1-x530.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1708539751363100009 Content-Type: text/plain; charset="utf-8" From: Fan Ni Per cxl spec r3.1, add dynamic capacity region representative based on Table 8-165 and extend the cxl type3 device definition to include dc region information. Also, based on info in 8.2.9.9.9.1, add 'Get Dynamic Capacity Configuration' mailbox support. Note: decode_len of a dc region is aligned to 256*MiB, divided by 256 * MiB before returned to the host for "Get Dynamic Capacity Configurati= on" mailbox command. Signed-off-by: Fan Ni --- hw/cxl/cxl-mailbox-utils.c | 110 ++++++++++++++++++++++++++++++++++++ include/hw/cxl/cxl_device.h | 16 ++++++ 2 files changed, 126 insertions(+) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index ba1d9901df..88e3b733e3 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -22,6 +22,7 @@ =20 #define CXL_CAPACITY_MULTIPLIER (256 * MiB) #define CXL_DC_EVENT_LOG_SIZE 8 +#define CXL_SPEC_AFTER_R30 =20 /* * How to add a new command, example. The command set FOO, with cmd BAR. @@ -80,6 +81,8 @@ enum { #define GET_POISON_LIST 0x0 #define INJECT_POISON 0x1 #define CLEAR_POISON 0x2 + DCD_CONFIG =3D 0x48, + #define GET_DC_CONFIG 0x0 PHYSICAL_SWITCH =3D 0x51, #define IDENTIFY_SWITCH_DEVICE 0x0 #define GET_PHYSICAL_PORT_STATE 0x1 @@ -1238,6 +1241,103 @@ static CXLRetCode cmd_media_clear_poison(const stru= ct cxl_cmd *cmd, return CXL_MBOX_SUCCESS; } =20 +/* + * CXL r3.1 section 8.2.9.9.9.1: Get Dynamic Capacity Configuration + * (Opcode: 4800h) + */ +static CXLRetCode cmd_dcd_get_dyn_cap_config(const struct cxl_cmd *cmd, + uint8_t *payload_in, + size_t len_in, + uint8_t *payload_out, + size_t *len_out, + CXLCCI *cci) +{ + CXLType3Dev *ct3d =3D CXL_TYPE3(cci->d); + struct get_dyn_cap_config_in_pl { + uint8_t region_cnt; + uint8_t start_region_id; + } QEMU_PACKED; + + struct get_dyn_cap_config_out_pl { + uint8_t num_regions; + uint8_t regions_returned; + uint8_t rsvd1[6]; + struct { + uint64_t base; + uint64_t decode_len; + uint64_t region_len; + uint64_t block_size; + uint32_t dsmadhandle; + uint8_t flags; + uint8_t rsvd2[3]; + } QEMU_PACKED records[]; + /* + * if cxl spec version >=3D 3.1, extra output payload as defined + * in struct get_dyn_cap_config_out_pl_extra comes here. + */ + } QEMU_PACKED; + + struct get_dyn_cap_config_in_pl *in =3D (void *)payload_in; + struct get_dyn_cap_config_out_pl *out =3D (void *)payload_out; + uint16_t record_count =3D 0; + uint16_t i; + uint16_t out_pl_len; + uint8_t start_region_id =3D in->start_region_id; +#ifdef CXL_SPEC_AFTER_R30 + struct get_dyn_cap_config_out_pl_extra { + uint32_t num_extents_supported; + uint32_t num_extents_available; + uint32_t num_tags_supported; + uint32_t num_tags_available; + } QEMU_PACKED; + struct get_dyn_cap_config_out_pl_extra *extra_out; +#endif + + if (start_region_id >=3D ct3d->dc.num_regions) { + return CXL_MBOX_INVALID_INPUT; + } + + record_count =3D MIN(ct3d->dc.num_regions - in->start_region_id, + in->region_cnt); + + out_pl_len =3D sizeof(*out) + record_count * sizeof(out->records[0]); +#ifdef CXL_SPEC_AFTER_R30 + extra_out =3D (struct get_dyn_cap_config_out_pl_extra *)(payload_out + + out_pl_len); + out_pl_len +=3D sizeof(struct get_dyn_cap_config_out_pl_extra); +#endif + assert(out_pl_len <=3D CXL_MAILBOX_MAX_PAYLOAD_SIZE); + + out->num_regions =3D ct3d->dc.num_regions; +#ifdef CXL_SPEC_AFTER_R30 + out->regions_returned =3D record_count; +#endif + for (i =3D 0; i < record_count; i++) { + stq_le_p(&out->records[i].base, + ct3d->dc.regions[start_region_id + i].base); + stq_le_p(&out->records[i].decode_len, + ct3d->dc.regions[start_region_id + i].decode_len / + CXL_CAPACITY_MULTIPLIER); + stq_le_p(&out->records[i].region_len, + ct3d->dc.regions[start_region_id + i].len); + stq_le_p(&out->records[i].block_size, + ct3d->dc.regions[start_region_id + i].block_size); + stl_le_p(&out->records[i].dsmadhandle, + ct3d->dc.regions[start_region_id + i].dsmadhandle); + out->records[i].flags =3D ct3d->dc.regions[start_region_id + i].fl= ags; + } +#ifdef CXL_SPEC_AFTER_R30 + /*FIXME: need to set valid values in the future*/ + stq_le_p(&extra_out->num_extents_supported, 0); + stq_le_p(&extra_out->num_extents_available, 0); + stq_le_p(&extra_out->num_tags_supported, 0); + stq_le_p(&extra_out->num_tags_available, 0); +#endif + + *len_out =3D out_pl_len; + return CXL_MBOX_SUCCESS; +} + #define IMMEDIATE_CONFIG_CHANGE (1 << 1) #define IMMEDIATE_DATA_CHANGE (1 << 2) #define IMMEDIATE_POLICY_CHANGE (1 << 3) @@ -1282,6 +1382,11 @@ static const struct cxl_cmd cxl_cmd_set[256][256] = =3D { cmd_media_clear_poison, 72, 0 }, }; =20 +static const struct cxl_cmd cxl_cmd_set_dcd[256][256] =3D { + [DCD_CONFIG][GET_DC_CONFIG] =3D { "DCD_GET_DC_CONFIG", + cmd_dcd_get_dyn_cap_config, 2, 0 }, +}; + static const struct cxl_cmd cxl_cmd_set_sw[256][256] =3D { [INFOSTAT][IS_IDENTIFY] =3D { "IDENTIFY", cmd_infostat_identify, 0, 0 = }, [INFOSTAT][BACKGROUND_OPERATION_STATUS] =3D { "BACKGROUND_OPERATION_ST= ATUS", @@ -1487,7 +1592,12 @@ void cxl_initialize_mailbox_swcci(CXLCCI *cci, Devic= eState *intf, =20 void cxl_initialize_mailbox_t3(CXLCCI *cci, DeviceState *d, size_t payload= _max) { + CXLType3Dev *ct3d =3D CXL_TYPE3(d); + cxl_copy_cci_commands(cci, cxl_cmd_set); + if (ct3d->dc.num_regions) { + cxl_copy_cci_commands(cci, cxl_cmd_set_dcd); + } cci->d =3D d; =20 /* No separation for PCI MB as protocol handled in PCI device */ diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index 3cf3077afa..6df7fecdf1 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -422,6 +422,17 @@ typedef struct CXLPoison { typedef QLIST_HEAD(, CXLPoison) CXLPoisonList; #define CXL_POISON_LIST_LIMIT 256 =20 +#define DCD_MAX_REGION_NUM 8 + +typedef struct CXLDCDRegion { + uint64_t base; + uint64_t decode_len; /* aligned to 256*MiB */ + uint64_t len; + uint64_t block_size; + uint32_t dsmadhandle; + uint8_t flags; +} CXLDCDRegion; + struct CXLType3Dev { /* Private */ PCIDevice parent_obj; @@ -454,6 +465,11 @@ struct CXLType3Dev { unsigned int poison_list_cnt; bool poison_list_overflowed; uint64_t poison_list_overflow_ts; + + struct dynamic_capacity { + uint8_t num_regions; /* 0-8 regions */ + CXLDCDRegion regions[DCD_MAX_REGION_NUM]; + } dc; }; =20 #define TYPE_CXL_TYPE3 "cxl-type3" --=20 2.43.0 From nobody Tue Nov 26 08:37:50 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1708539746; cv=none; d=zohomail.com; s=zohoarc; b=nQScfdtogAXq/t8pB8uGcxfVgcX8+uaGmQ19lk8O7axlj0FVa5yOmm+toCXwyzYVhTnYyzE4QOG81B5drYGDAkmYKxi5ijOXxdZr5FoyyWYxxzTq8mR03tRV9wh2Wn4DSgGXmbV8LVfO9wu6JqB6LiNOk369hen2hYmibPZmI1Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708539746; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=jnjf2LLXHCSGzRVtlUxFuz40qj2WpBzq7LFxL5kuDZQ=; b=k/5QQfITBnbEiCDbrnTLpuK9Vui4FwhkhsU1NQ/TedDW6IUlEzjYEN9VVYrDMdAoXIA7wnq3hR1w8ajWBxrjUtOW2LED9SVPkbcXkm7XphVLxg86NVPl8g9RwbuOzXZeTBKRr4fCuc4NXVWuOZK938IE54Ito2X1G8abBaiLAxQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708539746801816.6512036007988; Wed, 21 Feb 2024 10:22:26 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rcrDg-0007tw-Ax; Wed, 21 Feb 2024 13:21:24 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rcrDa-0007hW-Cu for qemu-devel@nongnu.org; Wed, 21 Feb 2024 13:21:19 -0500 Received: from mail-pg1-x52f.google.com ([2607:f8b0:4864:20::52f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rcrDY-0003JT-J4 for qemu-devel@nongnu.org; Wed, 21 Feb 2024 13:21:18 -0500 Received: by mail-pg1-x52f.google.com with SMTP id 41be03b00d2f7-5dbcfa0eb5dso6499864a12.3 for ; Wed, 21 Feb 2024 10:21:15 -0800 (PST) Received: from localhost.localdomain ([2601:641:300:14de:5692:7e41:13a2:69a]) by smtp.gmail.com with ESMTPSA id cs16-20020a17090af51000b002992f49922csm9979009pjb.25.2024.02.21.10.21.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Feb 2024 10:21:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708539674; x=1709144474; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jnjf2LLXHCSGzRVtlUxFuz40qj2WpBzq7LFxL5kuDZQ=; b=lfw7tUJB7p1/DMHdjnxk3SZVUryFR0Z85kUk0Lbk3t/9evocGFtk3CAlHCUMdoaq6v XD7RSjxG1bKORkUauRKWh7ySy4BtKuLgAhniZ0qvwy8X2jA8ITseG2/2SuX/wU6/yPgM 9Kga6Xo0wFZS/DD0gHGH4rVaelbD67BP7SBeUYrPiZ2aFUEQ6MKVJzgXPiqYgLMnvPNb Q7uzn1kggPrx8Kp9S26DqX55dbhfVsgMrtLohX8+UA4fjslz+DopKHY9ULSDvBZBGhbq ckNfgAMEq8m656064T5qepy6yYsfc7wSOJKvvFsC67J6TaLnB0k1dcNlTrZx1E+fXRny MJzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708539674; x=1709144474; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jnjf2LLXHCSGzRVtlUxFuz40qj2WpBzq7LFxL5kuDZQ=; b=YTmpbnAUjNIyeZRGyRFnmnzdlRFde5HhcvhAWRiEgQbQVN/+yOVH091kqfTnbrC5tx PQo9Q2XaAQWXvtoasjJxhw0cU9zWklJt+pR1GzlKMpXdMyy+wl+gKcTD1e/CkBOJc4ip ILsC5nF8VFr+wU8jXjilDIznQbRthgkAmkUQZMRVe0v5GL2WE7KF87SpesjuUAtjZPmL /W5AjZJFKa+0iS3ILMwmKTbe61oYWFuXSOiczyz/lDqUwB6UQxITp2AgF1GKMJcluYjy +tx11TJwO2Wru4RtkRonxC3cPY/IU+wvgDr7gN7l2fEc1Ip+5RE2dnZix0jL4HVPO3La 2pJQ== X-Gm-Message-State: AOJu0Yzw3Sp76I9r1A5ejSViQFVZYdCyq37TiRHsAse0fiNbYl8Rp0d3 gCYbDL0BTQGnfYQVBfx9+/krd+ScuhrQB1FGdB6PNJXatG9OX4eXGNGkeN+N X-Google-Smtp-Source: AGHT+IHROlOq4gqtw0lnNWWoRjKU1S6sa8+quFg1tqY8CpOuHQhfr0DhIXnQnZ4RDbnFAreDF8H/lA== X-Received: by 2002:a17:90a:ae0d:b0:299:c344:2b5 with SMTP id t13-20020a17090aae0d00b00299c34402b5mr6298324pjq.40.1708539674434; Wed, 21 Feb 2024 10:21:14 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Fan Ni Subject: [PATCH v4 03/10] include/hw/cxl/cxl_device: Rename mem_size as static_mem_size for type3 memory devices Date: Wed, 21 Feb 2024 10:15:56 -0800 Message-ID: <20240221182020.1086096-4-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240221182020.1086096-1-nifan.cxl@gmail.com> References: <20240221182020.1086096-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::52f; envelope-from=nifan.cxl@gmail.com; helo=mail-pg1-x52f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1708539747310100001 Content-Type: text/plain; charset="utf-8" From: Fan Ni Rename mem_size as static_mem_size for type3 memdev to cover static RAM and pmem capacity, preparing for the introduction of dynamic capacity to support dynamic capacity devices. Signed-off-by: Fan Ni --- hw/cxl/cxl-mailbox-utils.c | 4 ++-- hw/mem/cxl_type3.c | 8 ++++---- include/hw/cxl/cxl_device.h | 2 +- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index 88e3b733e3..7d2b74c9c5 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -802,7 +802,7 @@ static CXLRetCode cmd_identify_memory_device(const stru= ct cxl_cmd *cmd, snprintf(id->fw_revision, 0x10, "BWFW VERSION %02d", 0); =20 stq_le_p(&id->total_capacity, - cxl_dstate->mem_size / CXL_CAPACITY_MULTIPLIER); + cxl_dstate->static_mem_size / CXL_CAPACITY_MULTIPLIER); stq_le_p(&id->persistent_capacity, cxl_dstate->pmem_size / CXL_CAPACITY_MULTIPLIER); stq_le_p(&id->volatile_capacity, @@ -1178,7 +1178,7 @@ static CXLRetCode cmd_media_clear_poison(const struct= cxl_cmd *cmd, struct clear_poison_pl *in =3D (void *)payload_in; =20 dpa =3D ldq_le_p(&in->dpa); - if (dpa + CXL_CACHE_LINE_SIZE > cxl_dstate->mem_size) { + if (dpa + CXL_CACHE_LINE_SIZE > cxl_dstate->static_mem_size) { return CXL_MBOX_INVALID_PA; } =20 diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index e8801805b9..244d2b5fd5 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -608,7 +608,7 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error *= *errp) } address_space_init(&ct3d->hostvmem_as, vmr, v_name); ct3d->cxl_dstate.vmem_size =3D memory_region_size(vmr); - ct3d->cxl_dstate.mem_size +=3D memory_region_size(vmr); + ct3d->cxl_dstate.static_mem_size +=3D memory_region_size(vmr); g_free(v_name); } =20 @@ -631,7 +631,7 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error *= *errp) } address_space_init(&ct3d->hostpmem_as, pmr, p_name); ct3d->cxl_dstate.pmem_size =3D memory_region_size(pmr); - ct3d->cxl_dstate.mem_size +=3D memory_region_size(pmr); + ct3d->cxl_dstate.static_mem_size +=3D memory_region_size(pmr); g_free(p_name); } =20 @@ -837,7 +837,7 @@ static int cxl_type3_hpa_to_as_and_dpa(CXLType3Dev *ct3= d, return -EINVAL; } =20 - if (*dpa_offset > ct3d->cxl_dstate.mem_size) { + if (*dpa_offset > ct3d->cxl_dstate.static_mem_size) { return -EINVAL; } =20 @@ -1010,7 +1010,7 @@ static bool set_cacheline(CXLType3Dev *ct3d, uint64_t= dpa_offset, uint8_t *data) return false; } =20 - if (dpa_offset + CXL_CACHE_LINE_SIZE > ct3d->cxl_dstate.mem_size) { + if (dpa_offset + CXL_CACHE_LINE_SIZE > ct3d->cxl_dstate.static_mem_siz= e) { return false; } =20 diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index 6df7fecdf1..80188db670 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -234,7 +234,7 @@ typedef struct cxl_device_state { } timestamp; =20 /* memory region size, HDM */ - uint64_t mem_size; + uint64_t static_mem_size; uint64_t pmem_size; uint64_t vmem_size; =20 --=20 2.43.0 From nobody Tue Nov 26 08:37:50 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1708539808; cv=none; d=zohomail.com; s=zohoarc; b=d/xX0FSB1CWTHNrUK9qxiNWvINhULioSoQOTQtl3MU/lO0eT0uABHnDAVx/qoOVYM7NZxC3AHgoJWjHFSTG7rV0j61BCVWOCVW1pJrlE4fdQh/9Wzh1AbjfjIa1MRvbmehVIPEpDeD6kCnjBPannnOTCnpNYiqzJhVInou5f4No= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708539808; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=WJqQjnlAGwd774HUXqTHZfBD58qfYj95z3AiQp1cluE=; b=T+I1KhBYfCl4f6mueZxRukxD0yZvrgtVmTfH2e55qdxOy1DOfGsT0tA6/+jyEo5uJyFUq74SFVKemfkW0EGqLHhbGSqbpVzBOn3IF5z93djhnzCTODcYcS+IxfCm5II+x4DJvNUm9aEeBEEIqTfFmog1RmYjczMX/dDiJsBzTf4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708539808173166.8995009591307; Wed, 21 Feb 2024 10:23:28 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rcrDd-0007oW-Fp; Wed, 21 Feb 2024 13:21:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rcrDa-0007he-TJ for qemu-devel@nongnu.org; Wed, 21 Feb 2024 13:21:19 -0500 Received: from mail-pj1-x102c.google.com ([2607:f8b0:4864:20::102c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rcrDZ-0003Kf-2h for qemu-devel@nongnu.org; Wed, 21 Feb 2024 13:21:18 -0500 Received: by mail-pj1-x102c.google.com with SMTP id 98e67ed59e1d1-299c5a23f40so2220605a91.3 for ; Wed, 21 Feb 2024 10:21:16 -0800 (PST) Received: from localhost.localdomain ([2601:641:300:14de:5692:7e41:13a2:69a]) by smtp.gmail.com with ESMTPSA id cs16-20020a17090af51000b002992f49922csm9979009pjb.25.2024.02.21.10.21.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Feb 2024 10:21:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708539675; x=1709144475; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WJqQjnlAGwd774HUXqTHZfBD58qfYj95z3AiQp1cluE=; b=aw13FB7EPJYLeNKS5Wf677NyX0ey2+gQizAANuA4Y8XJ2TpH+JjmJL0IoUHmroHCEn AN/HOw8BoGptnupvxSmEy+qEcKeBTWMUdXzm1WBEc2ORAZX+ioutlgWJGgGbYVYLqf5b PF0VT3eq/X3zfbmO0WToiVZgEk/7FMYbHnWnVTg6buesjqDxZp5fiIBDbZsRyrPDbukT wrKE1B0sU2OMFbsPfdjRFAdyAhxnfz6X42KaDydUQbNcjKW9ZeRZiUxjfIORkwcDfgCF IHVV+JYBZaDIeKC/wMLYQeYlllfag33w5/8v2YBU1GUHPaTFy2OizNr8piGWpzrfHk0P /Haw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708539675; x=1709144475; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WJqQjnlAGwd774HUXqTHZfBD58qfYj95z3AiQp1cluE=; b=tJJrQ6cz27JtuNIDG0cIcSqPOgDua+Vw+Ddvg95zKp8IHPcc41HDRyrSCtotW/tQIN M6ksF81t+L/qRIljAiWbxOK3RMUZFth304MiFU0F3iqsNpkiLRwrVIS56PLE6anSmo54 prQOEIC+7U4vwW8aEX+kAiH0UYPRtKwJBs2vTw0VyOo6gCqyEW1aEtLJiLEWHkQt5pQV DTq/Uz3djuyP/B0uebniHNDBd4gH1Cj1Mqfoi3yAPa3DCvBm0OU9GSvGzYx5Ot9X3Mxm YsFBigkQXR77ljOscAt4WvLcvkKMEnNqgyj+N75H7OdkFeY91p8Wp7JEtzgUbDZhKKH1 Pzww== X-Gm-Message-State: AOJu0Yy15+agjWZIVw9wC1rTowSkqq/3MU3XkvQAnMxX70u6TZ/CAO97 WbaNSWNdDKVjJydmiGPb3HF47GdtkrKvDMgD2AyGawbrjKwWHYff8B8zvpiL X-Google-Smtp-Source: AGHT+IGIpQfawDAr7HNOdT8nJf9XYpRgVO5h7M9s1ju3JADRvCiD6h78Yzt+I16QV7f+Bfep1X3C2g== X-Received: by 2002:a17:90a:fa0c:b0:299:2ed1:d462 with SMTP id cm12-20020a17090afa0c00b002992ed1d462mr13082477pjb.6.1708539675466; Wed, 21 Feb 2024 10:21:15 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Fan Ni Subject: [PATCH v4 04/10] hw/mem/cxl_type3: Add support to create DC regions to type3 memory devices Date: Wed, 21 Feb 2024 10:15:57 -0800 Message-ID: <20240221182020.1086096-5-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240221182020.1086096-1-nifan.cxl@gmail.com> References: <20240221182020.1086096-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::102c; envelope-from=nifan.cxl@gmail.com; helo=mail-pj1-x102c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1708539809494100001 Content-Type: text/plain; charset="utf-8" From: Fan Ni With the change, when setting up memory for type3 memory device, we can create DC regions. A property 'num-dc-regions' is added to ct3_props to allow users to pass the number of DC regions to create. To make it easier, other region parameters like region base, length, and block size are hard coded. If needed, these parameters can be added easily. With the change, we can create DC regions with proper kernel side support like below: region=3D$(cat /sys/bus/cxl/devices/decoder0.0/create_dc_region) echo $region > /sys/bus/cxl/devices/decoder0.0/create_dc_region echo 256 > /sys/bus/cxl/devices/$region/interleave_granularity echo 1 > /sys/bus/cxl/devices/$region/interleave_ways echo "dc0" >/sys/bus/cxl/devices/decoder2.0/mode echo 0x40000000 >/sys/bus/cxl/devices/decoder2.0/dpa_size echo 0x40000000 > /sys/bus/cxl/devices/$region/size echo "decoder2.0" > /sys/bus/cxl/devices/$region/target0 echo 1 > /sys/bus/cxl/devices/$region/commit echo $region > /sys/bus/cxl/drivers/cxl_region/bind However, we cannot really read/write to the DC regions due to lack of 1. host backend and address space setup for DC regions; 2. mailbox command support for adding/releasing DC extents. Signed-off-by: Fan Ni --- hw/mem/cxl_type3.c | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index 244d2b5fd5..c61cd2b5ac 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -567,6 +567,40 @@ static void ct3d_reg_write(void *opaque, hwaddr offset= , uint64_t value, } } =20 +/* + * TODO: dc region configuration will be updated once host backend and add= ress + * space support is added for DCD. + */ +static bool cxl_create_dc_regions(CXLType3Dev *ct3d, Error **errp) +{ + int i; + uint64_t region_base =3D 0; + uint64_t region_len =3D 2 * GiB; + uint64_t decode_len =3D 2 * GiB; + uint64_t blk_size =3D 2 * MiB; + CXLDCDRegion *region; + + if (ct3d->hostvmem) { + region_base +=3D ct3d->hostvmem->size; + } + if (ct3d->hostpmem) { + region_base +=3D ct3d->hostpmem->size; + } + for (i =3D 0; i < ct3d->dc.num_regions; i++) { + region =3D &ct3d->dc.regions[i]; + region->base =3D region_base; + region->decode_len =3D decode_len; + region->len =3D region_len; + region->block_size =3D blk_size; + /* dsmad_handle is set when creating cdat table entries */ + region->flags =3D 0; + + region_base +=3D region->len; + } + + return true; +} + static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp) { DeviceState *ds =3D DEVICE(ct3d); @@ -635,6 +669,11 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error = **errp) g_free(p_name); } =20 + if (!cxl_create_dc_regions(ct3d, errp)) { + error_setg(errp, "setup DC regions failed"); + return false; + } + return true; } =20 @@ -930,6 +969,7 @@ static Property ct3_props[] =3D { HostMemoryBackend *), DEFINE_PROP_UINT64("sn", CXLType3Dev, sn, UI64_NULL), DEFINE_PROP_STRING("cdat", CXLType3Dev, cxl_cstate.cdat.filename), + DEFINE_PROP_UINT8("num-dc-regions", CXLType3Dev, dc.num_regions, 0), DEFINE_PROP_END_OF_LIST(), }; =20 --=20 2.43.0 From nobody Tue Nov 26 08:37:50 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1708539732; cv=none; d=zohomail.com; s=zohoarc; b=OoS6SZr1KO41iTirJxD/8EvbVO9qs+9Kn9lV2toy4PGmCgPMj3kLcCCzxLDr/d61PUHyR5DOuZHNDKO/leYAt/zFMtWS0SL791+6+8z4bL1OThu4VbAPzibtQipiboWgkJNp3sx6rYPkerNUF8G2LUO7+Ko47GzeQxlqIxRWIPw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708539732; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=eiSoVZJtjOJJ7IGlGmk8cRI2lGtWO4AhdMc312mNoV4=; b=a3sNqP+/4DAiAnWbfXraCr07vTLf0ba2vuu1xiSPbU34KTweURmHXNSBFzvOWZXIwLeyh8nNso0sKjvgvNH0DlUu1Ey8w8auhY9a/YOccP61xRb1xLzy73wL09EM175bOUF9OxyCDpwqzwqtU1BJXpbDpjqX4OSiRBd5SRNzVQk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708539732681937.6144051601027; Wed, 21 Feb 2024 10:22:12 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rcrDh-0007uj-2s; Wed, 21 Feb 2024 13:21:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rcrDe-0007oZ-2F for qemu-devel@nongnu.org; Wed, 21 Feb 2024 13:21:22 -0500 Received: from mail-pg1-x531.google.com ([2607:f8b0:4864:20::531]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rcrDa-0003L1-Cd for qemu-devel@nongnu.org; Wed, 21 Feb 2024 13:21:21 -0500 Received: by mail-pg1-x531.google.com with SMTP id 41be03b00d2f7-5c6bd3100fcso4062433a12.3 for ; Wed, 21 Feb 2024 10:21:17 -0800 (PST) Received: from localhost.localdomain ([2601:641:300:14de:5692:7e41:13a2:69a]) by smtp.gmail.com with ESMTPSA id cs16-20020a17090af51000b002992f49922csm9979009pjb.25.2024.02.21.10.21.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Feb 2024 10:21:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708539677; x=1709144477; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eiSoVZJtjOJJ7IGlGmk8cRI2lGtWO4AhdMc312mNoV4=; b=YG+8fVm/E44UqIyCWSzozjy4kjxet1xwDUvsUQTpz5Spo2aTSglOXMNoTINqfKCyCZ axbgT6Blk+YOPabp2rq0KqVdz/N/wyoL7uxfj91THQjW8yvOmDneE/zJkTOXRqqlzFLF WBQoFhtsuzPM9zJwP+18GserGma37frF8DBY2jCpSqCLWOhdjN3yUzYY9CbSQ0VaDlvI lPl3fLouujtgLsXJpMRkrPi8D7mjkStzNldYN/Nac/3ztO0FOmr2KSp0/NLlBa7qjqOz rvAPkIqzFpR+EwWdvNfgJ/eYtoUJTZ/CZAQsFt586sSPdrXw7dcIH/vfd0htc9OajidZ N8iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708539677; x=1709144477; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eiSoVZJtjOJJ7IGlGmk8cRI2lGtWO4AhdMc312mNoV4=; b=C+LntND50bce9vDUHsHFCX9wqC6X0SFTwZa/TKhLOm7oRUtRv9rGzuOjtAcgfzm5dt oXcXJEvr72DOJws/AQMePVF8yg/k0wR9fcFZCIrfIbDH8JJ8qvLYNhpBp0v7EnALylAU eM8QKyu9FFhFyhSRUfnutWk/hK4kxnA+ebcDskedvm7FdFRuztgMGfsMfKLoT7JTKmdW a79brOm0CbUrkW5M5/e/iDTbHAM9apAmGqZF0GFJQnCcggWFRhFPAVY3o1DgiBx/d39H 9G955GFxlL3C4xjSEKpCxKA0PNAgvbFOE8O3gsoGIKPAg2AlWg9UyKKDvGkzAYsvr3tb oCSw== X-Gm-Message-State: AOJu0YzD/whJODG0QoseR6EvBr9qhw+whyqLlcuyjB/zHxoIA7HwyEbA ysr77zOlGFdAy+9f2vRQUKi3dGDU2vupHpUVAIrU3rGoVlt96uo0vRplFYQe X-Google-Smtp-Source: AGHT+IGAk48olbaYcijRDumKRKOL973AAWX5mM+93KfBcG5wDx9QsekZGw40m+D/x9wD0JEaDPXDfg== X-Received: by 2002:a17:90a:cc9:b0:297:24da:887e with SMTP id 9-20020a17090a0cc900b0029724da887emr15889680pjt.18.1708539676735; Wed, 21 Feb 2024 10:21:16 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Fan Ni Subject: [PATCH v4 05/10] hw/mem/cxl-type3: Refactor ct3_build_cdat_entries_for_mr to take mr size insead of mr as argument Date: Wed, 21 Feb 2024 10:15:58 -0800 Message-ID: <20240221182020.1086096-6-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240221182020.1086096-1-nifan.cxl@gmail.com> References: <20240221182020.1086096-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::531; envelope-from=nifan.cxl@gmail.com; helo=mail-pg1-x531.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1708539733276100001 Content-Type: text/plain; charset="utf-8" From: Fan Ni The function ct3_build_cdat_entries_for_mr only uses size of the passed memory region argument, refactor the function definition to make the passed arguments more specific. Signed-off-by: Fan Ni --- hw/mem/cxl_type3.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index c61cd2b5ac..6e5f908fb1 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -43,7 +43,7 @@ enum { }; =20 static void ct3_build_cdat_entries_for_mr(CDATSubHeader **cdat_table, - int dsmad_handle, MemoryRegion *= mr, + int dsmad_handle, uint64_t size, bool is_pmem, uint64_t dpa_base) { g_autofree CDATDsmas *dsmas =3D NULL; @@ -62,7 +62,7 @@ static void ct3_build_cdat_entries_for_mr(CDATSubHeader *= *cdat_table, .DSMADhandle =3D dsmad_handle, .flags =3D is_pmem ? CDAT_DSMAS_FLAG_NV : 0, .DPA_base =3D dpa_base, - .DPA_length =3D memory_region_size(mr), + .DPA_length =3D size, }; =20 /* For now, no memory side cache, plausiblish numbers */ @@ -131,7 +131,7 @@ static void ct3_build_cdat_entries_for_mr(CDATSubHeader= **cdat_table, */ .EFI_memory_type_attr =3D is_pmem ? 2 : 1, .DPA_offset =3D 0, - .DPA_length =3D memory_region_size(mr), + .DPA_length =3D size, }; =20 /* Header always at start of structure */ @@ -148,6 +148,7 @@ static int ct3_build_cdat_table(CDATSubHeader ***cdat_t= able, void *priv) g_autofree CDATSubHeader **table =3D NULL; CXLType3Dev *ct3d =3D priv; MemoryRegion *volatile_mr =3D NULL, *nonvolatile_mr =3D NULL; + uint64_t vmr_size =3D 0, pmr_size =3D 0; int dsmad_handle =3D 0; int cur_ent =3D 0; int len =3D 0; @@ -162,6 +163,7 @@ static int ct3_build_cdat_table(CDATSubHeader ***cdat_t= able, void *priv) return -EINVAL; } len +=3D CT3_CDAT_NUM_ENTRIES; + vmr_size =3D memory_region_size(volatile_mr); } =20 if (ct3d->hostpmem) { @@ -170,21 +172,22 @@ static int ct3_build_cdat_table(CDATSubHeader ***cdat= _table, void *priv) return -EINVAL; } len +=3D CT3_CDAT_NUM_ENTRIES; + pmr_size =3D memory_region_size(nonvolatile_mr); } =20 table =3D g_malloc0(len * sizeof(*table)); =20 /* Now fill them in */ if (volatile_mr) { - ct3_build_cdat_entries_for_mr(table, dsmad_handle++, volatile_mr, + ct3_build_cdat_entries_for_mr(table, dsmad_handle++, vmr_size, false, 0); cur_ent =3D CT3_CDAT_NUM_ENTRIES; } =20 if (nonvolatile_mr) { - uint64_t base =3D volatile_mr ? memory_region_size(volatile_mr) : = 0; + uint64_t base =3D vmr_size; ct3_build_cdat_entries_for_mr(&(table[cur_ent]), dsmad_handle++, - nonvolatile_mr, true, base); + pmr_size, true, base); cur_ent +=3D CT3_CDAT_NUM_ENTRIES; } assert(len =3D=3D cur_ent); --=20 2.43.0 From nobody Tue Nov 26 08:37:50 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1708539722; cv=none; d=zohomail.com; s=zohoarc; b=RN0ZsvcEYjVsZG5QxlcPX7XpKlbeddMc2RCnIAIaqGf/NiqPrsn2/CdonvHvmljyV9JgN7EKr+3kUVyGqGn1RGDSEso5Ft7eGUS+lppddfepf/ovIZjMzWJ3LlzjPsrEHU5HOd4uE0YNlW4ZRcePkwSywu7h/DjclkI/Smdaw7s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708539722; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ByXkFoZstt7Taile7J7sAjWvfPjaE3AcMIHhZiOAS+I=; b=DMcXRGt19skyeOELMaY9bJH3lAbBrQ3iDFPVKH/M86WAZIxLu3vN+bFH0Lo9Jjyi5zd1LKELs3Vbaj8GbiRho2RZ86ww8cgKmzhJl6c0BEvJoQuFpBSJsXP9fO7FBrxRQUZf4BWNyA6tMJ7tsB/34j1DXCo3uDA4sZoMrN4PYek= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708539722841272.1383006934901; Wed, 21 Feb 2024 10:22:02 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rcrDi-00080W-Qu; Wed, 21 Feb 2024 13:21:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rcrDg-0007t2-1X for qemu-devel@nongnu.org; Wed, 21 Feb 2024 13:21:24 -0500 Received: from mail-pg1-x532.google.com ([2607:f8b0:4864:20::532]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rcrDc-0003LJ-CQ for qemu-devel@nongnu.org; Wed, 21 Feb 2024 13:21:23 -0500 Received: by mail-pg1-x532.google.com with SMTP id 41be03b00d2f7-517ab9a4a13so700784a12.1 for ; Wed, 21 Feb 2024 10:21:19 -0800 (PST) Received: from localhost.localdomain ([2601:641:300:14de:5692:7e41:13a2:69a]) by smtp.gmail.com with ESMTPSA id cs16-20020a17090af51000b002992f49922csm9979009pjb.25.2024.02.21.10.21.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Feb 2024 10:21:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708539678; x=1709144478; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ByXkFoZstt7Taile7J7sAjWvfPjaE3AcMIHhZiOAS+I=; b=SrTpuUvHmJGHMZIl/vy2ZkmaDF1ngR7c417aEjeEmMXx4pWREo0ZTc2AuqUeM+wONH d0IH1E60WxGLr32Da96QURuuSq+IMRsXOIhyxVCCSXvcTJrroMxmH1HixGLaXfUmrnTA YcSz6pn9LOe3oJ4hgT3wBouRpaTWhZKcBtHzKT5up3T1EYlFc8QxAnHE+PYFaZ1xeKfp eZj+f5Q6/Wy0S9ERO1e9gXlo0Dpy1eaYKbtveyLzEsNDRSvmjojvPU19qWViVDGwfyvT ptPdNruGDV4p0NIYd1+5STM72h4u5cXg3yJBD2zA1Np77x34eSYL6BHn2cNT46kE5Qoz F4EQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708539678; x=1709144478; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ByXkFoZstt7Taile7J7sAjWvfPjaE3AcMIHhZiOAS+I=; b=UvHT6zTI/K93W+P3TWTZR8jumpD0oYYPmrDuwmymnpP97dlUPdvvsrjDA3QDTTMClE PxVJn1ukGY2zMIUXnS/p75FL27ggFlxvD6GNRhFX1o1TMtuaSxaZJWEqy90/sBi3/rHY 9XBUAUtFrWNh72/002+M3fOwExjidAq7GW0liMK2ZDsFuqxrhXsR6Triq1BoSsi2XwR1 4BaYqPC3aLqcehvjXc8k4mtKhRGwiADSbhjlmnYlXMu7qW+PKAY34nkCDan6Qx/adOFo 7YmLRrbIlk1O6JBaXH14BrEAHVP9PhbMZWFySTVhvrASbsr+AdocyvtY0jl2ZkKCrCuZ tfbA== X-Gm-Message-State: AOJu0YwNK+T6YrZASCgqplFBE3HW+Ap2iA1lOYKgby5/SXkpyy2d6P0E fj/1qCMEx89w17LiicdrweyE02W8dOwZvjn/7+T290mFOiiA3TmuFaCKulbC X-Google-Smtp-Source: AGHT+IEDrY9CBvXUVAZCTG6XPyMxCbOD4bhgwDQW7wmUAU1B3xHNd5EHMNgjtfioPjh2SMGZMZxwqw== X-Received: by 2002:a17:90a:b014:b0:299:6e88:7b6a with SMTP id x20-20020a17090ab01400b002996e887b6amr10884499pjq.36.1708539678029; Wed, 21 Feb 2024 10:21:18 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Fan Ni Subject: [PATCH v4 06/10] hw/mem/cxl_type3: Add host backend and address space handling for DC regions Date: Wed, 21 Feb 2024 10:15:59 -0800 Message-ID: <20240221182020.1086096-7-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240221182020.1086096-1-nifan.cxl@gmail.com> References: <20240221182020.1086096-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::532; envelope-from=nifan.cxl@gmail.com; helo=mail-pg1-x532.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1708539723460100001 Content-Type: text/plain; charset="utf-8" From: Fan Ni Add (file/memory backed) host backend, all the dynamic capacity regions will share a single, large enough host backend. Set up address space for DC regions to support read/write operations to dynamic capacity for DCD. With the change, following supports are added: 1. Add a new property to type3 device "volatile-dc-memdev" to point to host memory backend for dynamic capacity. Currently, all dc regions share one one host backend. 2. Add namespace for dynamic capacity for read/write support; 3. Create cdat entries for each dynamic capacity region; 4. Fix dvsec range registers to include DC regions. Signed-off-by: Fan Ni --- hw/cxl/cxl-mailbox-utils.c | 16 ++- hw/mem/cxl_type3.c | 188 ++++++++++++++++++++++++++++-------- include/hw/cxl/cxl_device.h | 4 + 3 files changed, 165 insertions(+), 43 deletions(-) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index 7d2b74c9c5..f95e417683 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -621,7 +621,8 @@ static CXLRetCode cmd_firmware_update_get_info(const st= ruct cxl_cmd *cmd, size_t *len_out, CXLCCI *cci) { - CXLDeviceState *cxl_dstate =3D &CXL_TYPE3(cci->d)->cxl_dstate; + CXLType3Dev *ct3d =3D CXL_TYPE3(cci->d); + CXLDeviceState *cxl_dstate =3D &ct3d->cxl_dstate; struct { uint8_t slots_supported; uint8_t slot_info; @@ -635,7 +636,8 @@ static CXLRetCode cmd_firmware_update_get_info(const st= ruct cxl_cmd *cmd, QEMU_BUILD_BUG_ON(sizeof(*fw_info) !=3D 0x50); =20 if ((cxl_dstate->vmem_size < CXL_CAPACITY_MULTIPLIER) || - (cxl_dstate->pmem_size < CXL_CAPACITY_MULTIPLIER)) { + (cxl_dstate->pmem_size < CXL_CAPACITY_MULTIPLIER) || + (ct3d->dc.total_capacity < CXL_CAPACITY_MULTIPLIER)) { return CXL_MBOX_INTERNAL_ERROR; } =20 @@ -792,7 +794,8 @@ static CXLRetCode cmd_identify_memory_device(const stru= ct cxl_cmd *cmd, CXLDeviceState *cxl_dstate =3D &ct3d->cxl_dstate; =20 if ((!QEMU_IS_ALIGNED(cxl_dstate->vmem_size, CXL_CAPACITY_MULTIPLIER))= || - (!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER))= ) { + (!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER))= || + (!QEMU_IS_ALIGNED(ct3d->dc.total_capacity, CXL_CAPACITY_MULTIPLIER= ))) { return CXL_MBOX_INTERNAL_ERROR; } =20 @@ -834,9 +837,11 @@ static CXLRetCode cmd_ccls_get_partition_info(const st= ruct cxl_cmd *cmd, uint64_t next_pmem; } QEMU_PACKED *part_info =3D (void *)payload_out; QEMU_BUILD_BUG_ON(sizeof(*part_info) !=3D 0x20); + CXLType3Dev *ct3d =3D container_of(cxl_dstate, CXLType3Dev, cxl_dstate= ); =20 if ((!QEMU_IS_ALIGNED(cxl_dstate->vmem_size, CXL_CAPACITY_MULTIPLIER))= || - (!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER))= ) { + (!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER))= || + (!QEMU_IS_ALIGNED(ct3d->dc.total_capacity, CXL_CAPACITY_MULTIPLIER= ))) { return CXL_MBOX_INTERNAL_ERROR; } =20 @@ -1178,7 +1183,8 @@ static CXLRetCode cmd_media_clear_poison(const struct= cxl_cmd *cmd, struct clear_poison_pl *in =3D (void *)payload_in; =20 dpa =3D ldq_le_p(&in->dpa); - if (dpa + CXL_CACHE_LINE_SIZE > cxl_dstate->static_mem_size) { + if (dpa + CXL_CACHE_LINE_SIZE > cxl_dstate->static_mem_size + + ct3d->dc.total_capacity) { return CXL_MBOX_INVALID_PA; } =20 diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index 6e5f908fb1..b966fa4f10 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -30,6 +30,7 @@ #include "hw/pci/msix.h" =20 #define DWORD_BYTE 4 +#define CXL_CAPACITY_MULTIPLIER (256 * MiB) =20 /* Default CDAT entries for a memory region */ enum { @@ -44,7 +45,8 @@ enum { =20 static void ct3_build_cdat_entries_for_mr(CDATSubHeader **cdat_table, int dsmad_handle, uint64_t size, - bool is_pmem, uint64_t dpa_base) + bool is_pmem, bool is_dynamic, + uint64_t dpa_base) { g_autofree CDATDsmas *dsmas =3D NULL; g_autofree CDATDslbis *dslbis0 =3D NULL; @@ -60,7 +62,8 @@ static void ct3_build_cdat_entries_for_mr(CDATSubHeader *= *cdat_table, .length =3D sizeof(*dsmas), }, .DSMADhandle =3D dsmad_handle, - .flags =3D is_pmem ? CDAT_DSMAS_FLAG_NV : 0, + .flags =3D (is_pmem ? CDAT_DSMAS_FLAG_NV : 0) | + (is_dynamic ? CDAT_DSMAS_FLAG_DYNAMIC_CAP : 0), .DPA_base =3D dpa_base, .DPA_length =3D size, }; @@ -148,12 +151,13 @@ static int ct3_build_cdat_table(CDATSubHeader ***cdat= _table, void *priv) g_autofree CDATSubHeader **table =3D NULL; CXLType3Dev *ct3d =3D priv; MemoryRegion *volatile_mr =3D NULL, *nonvolatile_mr =3D NULL; + MemoryRegion *dc_mr =3D NULL; uint64_t vmr_size =3D 0, pmr_size =3D 0; int dsmad_handle =3D 0; int cur_ent =3D 0; int len =3D 0; =20 - if (!ct3d->hostpmem && !ct3d->hostvmem) { + if (!ct3d->hostpmem && !ct3d->hostvmem && !ct3d->dc.num_regions) { return 0; } =20 @@ -175,21 +179,51 @@ static int ct3_build_cdat_table(CDATSubHeader ***cdat= _table, void *priv) pmr_size =3D memory_region_size(nonvolatile_mr); } =20 + if (ct3d->dc.num_regions) { + if (ct3d->dc.host_dc) { + dc_mr =3D host_memory_backend_get_memory(ct3d->dc.host_dc); + if (!dc_mr) { + return -EINVAL; + } + len +=3D CT3_CDAT_NUM_ENTRIES * ct3d->dc.num_regions; + } else { + return -EINVAL; + } + } + table =3D g_malloc0(len * sizeof(*table)); =20 /* Now fill them in */ if (volatile_mr) { ct3_build_cdat_entries_for_mr(table, dsmad_handle++, vmr_size, - false, 0); + false, false, 0); cur_ent =3D CT3_CDAT_NUM_ENTRIES; } =20 if (nonvolatile_mr) { uint64_t base =3D vmr_size; ct3_build_cdat_entries_for_mr(&(table[cur_ent]), dsmad_handle++, - pmr_size, true, base); + pmr_size, true, false, base); cur_ent +=3D CT3_CDAT_NUM_ENTRIES; } + + if (dc_mr) { + int i; + uint64_t region_base =3D vmr_size + pmr_size; + + /* FIXME: Currently we assume the dynamic capacity to be volatile.= */ + for (i =3D 0; i < ct3d->dc.num_regions; i++) { + ct3_build_cdat_entries_for_mr(&(table[cur_ent]), + dsmad_handle++, + ct3d->dc.regions[i].len, + false, true, region_base); + ct3d->dc.regions[i].dsmadhandle =3D dsmad_handle - 1; + + cur_ent +=3D CT3_CDAT_NUM_ENTRIES; + region_base +=3D ct3d->dc.regions[i].len; + } + } + assert(len =3D=3D cur_ent); =20 *cdat_table =3D g_steal_pointer(&table); @@ -299,11 +333,24 @@ static void build_dvsecs(CXLType3Dev *ct3d) range2_size_hi =3D ct3d->hostpmem->size >> 32; range2_size_lo =3D (2 << 5) | (2 << 2) | 0x3 | (ct3d->hostpmem->size & 0xF0000000); + } else if (ct3d->dc.host_dc) { + range2_size_hi =3D ct3d->dc.host_dc->size >> 32; + range2_size_lo =3D (2 << 5) | (2 << 2) | 0x3 | + (ct3d->dc.host_dc->size & 0xF0000000); } - } else { + } else if (ct3d->hostpmem) { range1_size_hi =3D ct3d->hostpmem->size >> 32; range1_size_lo =3D (2 << 5) | (2 << 2) | 0x3 | (ct3d->hostpmem->size & 0xF0000000); + if (ct3d->dc.host_dc) { + range2_size_hi =3D ct3d->dc.host_dc->size >> 32; + range2_size_lo =3D (2 << 5) | (2 << 2) | 0x3 | + (ct3d->dc.host_dc->size & 0xF0000000); + } + } else { + range1_size_hi =3D ct3d->dc.host_dc->size >> 32; + range1_size_lo =3D (2 << 5) | (2 << 2) | 0x3 | + (ct3d->dc.host_dc->size & 0xF0000000); } =20 dvsec =3D (uint8_t *)&(CXLDVSECDevice){ @@ -570,19 +617,32 @@ static void ct3d_reg_write(void *opaque, hwaddr offse= t, uint64_t value, } } =20 -/* - * TODO: dc region configuration will be updated once host backend and add= ress - * space support is added for DCD. - */ static bool cxl_create_dc_regions(CXLType3Dev *ct3d, Error **errp) { int i; + MemoryRegion *dc_mr; + uint64_t dc_size =3D 0; uint64_t region_base =3D 0; - uint64_t region_len =3D 2 * GiB; - uint64_t decode_len =3D 2 * GiB; + uint64_t region_len; + uint64_t decode_len; uint64_t blk_size =3D 2 * MiB; CXLDCDRegion *region; =20 + dc_mr =3D host_memory_backend_get_memory(ct3d->dc.host_dc); + dc_size =3D memory_region_size(dc_mr); + region_len =3D DIV_ROUND_UP(dc_size, ct3d->dc.num_regions); + + if (region_len * ct3d->dc.num_regions > dc_size) { + error_setg(errp, "host backend size must be multiples of region le= n"); + return false; + } + if (region_len % CXL_CAPACITY_MULTIPLIER !=3D 0) { + error_setg(errp, "DC region size is unaligned to %lx", + CXL_CAPACITY_MULTIPLIER); + return false; + } + decode_len =3D region_len; + if (ct3d->hostvmem) { region_base +=3D ct3d->hostvmem->size; } @@ -599,6 +659,7 @@ static bool cxl_create_dc_regions(CXLType3Dev *ct3d, Er= ror **errp) region->flags =3D 0; =20 region_base +=3D region->len; + ct3d->dc.total_capacity +=3D region->len; } =20 return true; @@ -608,7 +669,8 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error *= *errp) { DeviceState *ds =3D DEVICE(ct3d); =20 - if (!ct3d->hostmem && !ct3d->hostvmem && !ct3d->hostpmem) { + if (!ct3d->hostmem && !ct3d->hostvmem && !ct3d->hostpmem + && !ct3d->dc.num_regions) { error_setg(errp, "at least one memdev property must be set"); return false; } else if (ct3d->hostmem && ct3d->hostpmem) { @@ -672,9 +734,38 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error = **errp) g_free(p_name); } =20 - if (!cxl_create_dc_regions(ct3d, errp)) { - error_setg(errp, "setup DC regions failed"); - return false; + ct3d->dc.total_capacity =3D 0; + if (ct3d->dc.num_regions) { + MemoryRegion *dc_mr; + char *dc_name; + + if (!ct3d->dc.host_dc) { + error_setg(errp, "dynamic capacity must have a backing device"= ); + return false; + } + + dc_mr =3D host_memory_backend_get_memory(ct3d->dc.host_dc); + if (!dc_mr) { + error_setg(errp, "dynamic capacity must have a backing device"= ); + return false; + } + + /* FIXME: set dc as volatile for now */ + memory_region_set_nonvolatile(dc_mr, false); + memory_region_set_enabled(dc_mr, true); + host_memory_backend_set_mapped(ct3d->dc.host_dc, true); + if (ds->id) { + dc_name =3D g_strdup_printf("cxl-dcd-dpa-dc-space:%s", ds->id); + } else { + dc_name =3D g_strdup("cxl-dcd-dpa-dc-space"); + } + address_space_init(&ct3d->dc.host_dc_as, dc_mr, dc_name); + g_free(dc_name); + + if (!cxl_create_dc_regions(ct3d, errp)) { + error_setg(errp, "setup DC regions failed"); + return false; + } } =20 return true; @@ -766,6 +857,9 @@ err_release_cdat: err_free_special_ops: g_free(regs->special_ops); err_address_space_free: + if (ct3d->dc.host_dc) { + address_space_destroy(&ct3d->dc.host_dc_as); + } if (ct3d->hostpmem) { address_space_destroy(&ct3d->hostpmem_as); } @@ -784,6 +878,9 @@ static void ct3_exit(PCIDevice *pci_dev) pcie_aer_exit(pci_dev); cxl_doe_cdat_release(cxl_cstate); g_free(regs->special_ops); + if (ct3d->dc.host_dc) { + address_space_destroy(&ct3d->dc.host_dc_as); + } if (ct3d->hostpmem) { address_space_destroy(&ct3d->hostpmem_as); } @@ -862,16 +959,24 @@ static int cxl_type3_hpa_to_as_and_dpa(CXLType3Dev *c= t3d, AddressSpace **as, uint64_t *dpa_offset) { - MemoryRegion *vmr =3D NULL, *pmr =3D NULL; + MemoryRegion *vmr =3D NULL, *pmr =3D NULL, *dc_mr =3D NULL; + uint64_t vmr_size =3D 0, pmr_size =3D 0, dc_size =3D 0; =20 if (ct3d->hostvmem) { vmr =3D host_memory_backend_get_memory(ct3d->hostvmem); + vmr_size =3D memory_region_size(vmr); } if (ct3d->hostpmem) { pmr =3D host_memory_backend_get_memory(ct3d->hostpmem); + pmr_size =3D memory_region_size(pmr); + } + if (ct3d->dc.host_dc) { + dc_mr =3D host_memory_backend_get_memory(ct3d->dc.host_dc); + /* Do we want dc_size to be dc_mr->size or not?? */ + dc_size =3D ct3d->dc.total_capacity; } =20 - if (!vmr && !pmr) { + if (!vmr && !pmr && !dc_mr) { return -ENODEV; } =20 @@ -879,19 +984,18 @@ static int cxl_type3_hpa_to_as_and_dpa(CXLType3Dev *c= t3d, return -EINVAL; } =20 - if (*dpa_offset > ct3d->cxl_dstate.static_mem_size) { + if (*dpa_offset >=3D vmr_size + pmr_size + dc_size) { return -EINVAL; } =20 - if (vmr) { - if (*dpa_offset < memory_region_size(vmr)) { - *as =3D &ct3d->hostvmem_as; - } else { - *as =3D &ct3d->hostpmem_as; - *dpa_offset -=3D memory_region_size(vmr); - } - } else { + if (*dpa_offset < vmr_size) { + *as =3D &ct3d->hostvmem_as; + } else if (*dpa_offset < vmr_size + pmr_size) { *as =3D &ct3d->hostpmem_as; + *dpa_offset -=3D vmr_size; + } else { + *as =3D &ct3d->dc.host_dc_as; + *dpa_offset -=3D (vmr_size + pmr_size); } =20 return 0; @@ -973,6 +1077,8 @@ static Property ct3_props[] =3D { DEFINE_PROP_UINT64("sn", CXLType3Dev, sn, UI64_NULL), DEFINE_PROP_STRING("cdat", CXLType3Dev, cxl_cstate.cdat.filename), DEFINE_PROP_UINT8("num-dc-regions", CXLType3Dev, dc.num_regions, 0), + DEFINE_PROP_LINK("volatile-dc-memdev", CXLType3Dev, dc.host_dc, + TYPE_MEMORY_BACKEND, HostMemoryBackend *), DEFINE_PROP_END_OF_LIST(), }; =20 @@ -1039,33 +1145,39 @@ static void set_lsa(CXLType3Dev *ct3d, const void *= buf, uint64_t size, =20 static bool set_cacheline(CXLType3Dev *ct3d, uint64_t dpa_offset, uint8_t = *data) { - MemoryRegion *vmr =3D NULL, *pmr =3D NULL; + MemoryRegion *vmr =3D NULL, *pmr =3D NULL, *dc_mr =3D NULL; AddressSpace *as; + uint64_t vmr_size =3D 0, pmr_size =3D 0, dc_size =3D 0; =20 if (ct3d->hostvmem) { vmr =3D host_memory_backend_get_memory(ct3d->hostvmem); + vmr_size =3D memory_region_size(vmr); } if (ct3d->hostpmem) { pmr =3D host_memory_backend_get_memory(ct3d->hostpmem); + pmr_size =3D memory_region_size(pmr); } + if (ct3d->dc.host_dc) { + dc_mr =3D host_memory_backend_get_memory(ct3d->dc.host_dc); + dc_size =3D ct3d->dc.total_capacity; + } =20 - if (!vmr && !pmr) { + if (!vmr && !pmr && !dc_mr) { return false; } =20 - if (dpa_offset + CXL_CACHE_LINE_SIZE > ct3d->cxl_dstate.static_mem_siz= e) { + if (dpa_offset + CXL_CACHE_LINE_SIZE > vmr_size + pmr_size + dc_size) { return false; } =20 - if (vmr) { - if (dpa_offset < memory_region_size(vmr)) { - as =3D &ct3d->hostvmem_as; - } else { - as =3D &ct3d->hostpmem_as; - dpa_offset -=3D memory_region_size(vmr); - } - } else { + if (dpa_offset < vmr_size) { + as =3D &ct3d->hostvmem_as; + } else if (dpa_offset < vmr_size + pmr_size) { as =3D &ct3d->hostpmem_as; + dpa_offset -=3D vmr_size; + } else { + as =3D &ct3d->dc.host_dc_as; + dpa_offset -=3D (vmr_size + pmr_size); } =20 address_space_write(as, dpa_offset, MEMTXATTRS_UNSPECIFIED, &data, diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index 80188db670..2f244da9a1 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -467,6 +467,10 @@ struct CXLType3Dev { uint64_t poison_list_overflow_ts; =20 struct dynamic_capacity { + HostMemoryBackend *host_dc; + AddressSpace host_dc_as; + uint64_t total_capacity; /* 256M aligned */ + uint8_t num_regions; /* 0-8 regions */ CXLDCDRegion regions[DCD_MAX_REGION_NUM]; } dc; --=20 2.43.0 From nobody Tue Nov 26 08:37:50 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1708539714; cv=none; d=zohomail.com; s=zohoarc; b=b7Dt0fJgjKTUOBMj3oN1hFyQjL6cbkLo8xNEq0SxO67koQaEUqF6mnt+42pBEDyHVuNS59Zg81CA5O8xX2PHFjeNcICvxTzj1ilcUtUyRya5tUJchKo95wA+vEimSAS4Yw8ZlM5k24LWtE7+gdlQ2yFulbmG5q0QXdKpIcXf+Eg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708539714; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=NMsgYchrz+Nvp+KunIns7+nQr9EgC+4zBQ7CzSVSfR4=; b=i1NmqrD552hCPFGUZz79fVjiNrYa/TKnip24owMuBk7M9rNq5KzqAC8R7MIStn/4W8dUArzi3hndgMTpU184k68JULG78g7IHoaXL7ZbHIAYyMyBpk2O5J8nurBgSIGNgn2t4gN+o3kTJd0WmX1/ueF88J+LXfY378BG3jGPwfg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708539714051240.106412173389; Wed, 21 Feb 2024 10:21:54 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rcrDk-000888-J7; Wed, 21 Feb 2024 13:21:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rcrDh-0007wG-Ht for qemu-devel@nongnu.org; Wed, 21 Feb 2024 13:21:25 -0500 Received: from mail-pg1-x529.google.com ([2607:f8b0:4864:20::529]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rcrDf-0003LT-NE for qemu-devel@nongnu.org; Wed, 21 Feb 2024 13:21:25 -0500 Received: by mail-pg1-x529.google.com with SMTP id 41be03b00d2f7-5cddfe0cb64so5359955a12.0 for ; Wed, 21 Feb 2024 10:21:22 -0800 (PST) Received: from localhost.localdomain ([2601:641:300:14de:5692:7e41:13a2:69a]) by smtp.gmail.com with ESMTPSA id cs16-20020a17090af51000b002992f49922csm9979009pjb.25.2024.02.21.10.21.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Feb 2024 10:21:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708539679; x=1709144479; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NMsgYchrz+Nvp+KunIns7+nQr9EgC+4zBQ7CzSVSfR4=; b=C20VSSLFbkinDsECXyzFp9cp9Y2v1XGDX0DaH6v2neIxFOEGhX3ega5xvJMeCwMiok uUx3MLsJfQY/oW5MkbizdfDLfqT35ZumfYfWS5yH61m8b0I1pYJkyEeq4G0JBtP5Wz1+ AqTsONLH/jUdaZk/1hF0fgCfWav/U5TzyJFQ07Otk9DR6bvxIfMUar7PVOD6b3GbWBjC +u+MuRlp1jJZvIGIvUlF7a/P/j+fld4afFEn2lHAZ6fOP8N1I3cM4FugWT+5pN4m10xl c0pNk7LEV7Czp5dTJGlffnjkeMS0tae7Oimj9AAGTa+B8p9h6R1BsR9fyjlgkh9SrzMt tGzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708539679; x=1709144479; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NMsgYchrz+Nvp+KunIns7+nQr9EgC+4zBQ7CzSVSfR4=; b=BtwAh2ADI5aW1hP6vZhl2ChuTs1GWknEtslWZAByjoNl2yeNVo5iivRkcrHtE8Bb0W tShZERTjQdpU1QwlAZCggQAbj6xicnYhGyVCSOvTyucspE+iGyLB+KU+YkJ3X5j6kUA3 HTVE+6lNkZkVT94UIpEx45Ii4DEE4ooN9GL/3d/lCTyDVLtiHu/cXDIke07uFfmCAIm2 e6Er8RqnBOiA0mf7cTNHw+Pu07BqVdgc9DWdgQX9k9AufYJtchyHfKZ0KTu88YhQtouZ winNZIZE0gctw1fr6HoezXvQU7G99dM6TxnGqlQ6L1AfIyiOjQyToqPPFjP/sfmGoFtv jhvw== X-Gm-Message-State: AOJu0YyK89RURoNF9pr1ySvZPOtru3TPAHIsNMrVWkE7KBFcyGBn5Qfg 81JFxH1n+pz9h52nQTXw+FyKes9PK+kx4YJxTyJYVtV5CQaPiM5hNxO/cNcX X-Google-Smtp-Source: AGHT+IE55Txhzt5qTIZN0Q8RXv9M2omyIKe/vPoOqMifiqePhZSO9K1zIAHiHrBjclNSMl0HCCDkNQ== X-Received: by 2002:a17:90b:4b4c:b0:299:73b3:cf15 with SMTP id mi12-20020a17090b4b4c00b0029973b3cf15mr8589636pjb.12.1708539679129; Wed, 21 Feb 2024 10:21:19 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Fan Ni Subject: [PATCH v4 07/10] hw/mem/cxl_type3: Add DC extent list representative and get DC extent list mailbox support Date: Wed, 21 Feb 2024 10:16:00 -0800 Message-ID: <20240221182020.1086096-8-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240221182020.1086096-1-nifan.cxl@gmail.com> References: <20240221182020.1086096-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::529; envelope-from=nifan.cxl@gmail.com; helo=mail-pg1-x529.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1708539715218100001 Content-Type: text/plain; charset="utf-8" From: Fan Ni Add dynamic capacity extent list representative to the definition of CXLType3Dev and add get DC extent list mailbox command per CXL.spec.3.1:.8.2.9.9.9.2. Signed-off-by: Fan Ni --- hw/cxl/cxl-mailbox-utils.c | 71 +++++++++++++++++++++++++++++++++++++ hw/mem/cxl_type3.c | 1 + include/hw/cxl/cxl_device.h | 23 ++++++++++++ 3 files changed, 95 insertions(+) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index f95e417683..dae7fe00ed 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -83,6 +83,7 @@ enum { #define CLEAR_POISON 0x2 DCD_CONFIG =3D 0x48, #define GET_DC_CONFIG 0x0 + #define GET_DYN_CAP_EXT_LIST 0x1 PHYSICAL_SWITCH =3D 0x51, #define IDENTIFY_SWITCH_DEVICE 0x0 #define GET_PHYSICAL_PORT_STATE 0x1 @@ -1344,6 +1345,73 @@ static CXLRetCode cmd_dcd_get_dyn_cap_config(const s= truct cxl_cmd *cmd, return CXL_MBOX_SUCCESS; } =20 +/* + * CXL r3.1 section 8.2.9.9.9.2: + * Get Dynamic Capacity Extent List (Opcode 4801h) + */ +static CXLRetCode cmd_dcd_get_dyn_cap_ext_list(const struct cxl_cmd *cmd, + uint8_t *payload_in, + size_t len_in, + uint8_t *payload_out, + size_t *len_out, + CXLCCI *cci) +{ + CXLType3Dev *ct3d =3D CXL_TYPE3(cci->d); + struct get_dyn_cap_ext_list_in_pl { + uint32_t extent_cnt; + uint32_t start_extent_id; + } QEMU_PACKED; + + struct get_dyn_cap_ext_list_out_pl { + uint32_t count; + uint32_t total_extents; + uint32_t generation_num; + uint8_t rsvd[4]; + CXLDCExtentRaw records[]; + } QEMU_PACKED; + + struct get_dyn_cap_ext_list_in_pl *in =3D (void *)payload_in; + struct get_dyn_cap_ext_list_out_pl *out =3D (void *)payload_out; + uint16_t record_count =3D 0, i =3D 0, record_done =3D 0; + CXLDCExtentList *extent_list =3D &ct3d->dc.extents; + CXLDCExtent *ent; + uint16_t out_pl_len; + uint32_t start_extent_id =3D in->start_extent_id; + + if (start_extent_id > ct3d->dc.total_extent_count) { + return CXL_MBOX_INVALID_INPUT; + } + + record_count =3D MIN(in->extent_cnt, + ct3d->dc.total_extent_count - start_extent_id); + + out_pl_len =3D sizeof(*out) + record_count * sizeof(out->records[0]); + assert(out_pl_len <=3D CXL_MAILBOX_MAX_PAYLOAD_SIZE); + + stl_le_p(&out->count, record_count); + stl_le_p(&out->total_extents, ct3d->dc.total_extent_count); + stl_le_p(&out->generation_num, ct3d->dc.ext_list_gen_seq); + + if (record_count > 0) { + QTAILQ_FOREACH(ent, extent_list, node) { + if (i++ < start_extent_id) { + continue; + } + stq_le_p(&out->records[record_done].start_dpa, ent->start_dpa); + stq_le_p(&out->records[record_done].len, ent->len); + memcpy(&out->records[record_done].tag, ent->tag, 0x10); + stw_le_p(&out->records[record_done].shared_seq, ent->shared_se= q); + record_done++; + if (record_done =3D=3D record_count) { + break; + } + } + } + + *len_out =3D out_pl_len; + return CXL_MBOX_SUCCESS; +} + #define IMMEDIATE_CONFIG_CHANGE (1 << 1) #define IMMEDIATE_DATA_CHANGE (1 << 2) #define IMMEDIATE_POLICY_CHANGE (1 << 3) @@ -1391,6 +1459,9 @@ static const struct cxl_cmd cxl_cmd_set[256][256] =3D= { static const struct cxl_cmd cxl_cmd_set_dcd[256][256] =3D { [DCD_CONFIG][GET_DC_CONFIG] =3D { "DCD_GET_DC_CONFIG", cmd_dcd_get_dyn_cap_config, 2, 0 }, + [DCD_CONFIG][GET_DYN_CAP_EXT_LIST] =3D { + "DCD_GET_DYNAMIC_CAPACITY_EXTENT_LIST", cmd_dcd_get_dyn_cap_ext_li= st, + 8, 0 }, }; =20 static const struct cxl_cmd cxl_cmd_set_sw[256][256] =3D { diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index b966fa4f10..f4edada303 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -661,6 +661,7 @@ static bool cxl_create_dc_regions(CXLType3Dev *ct3d, Er= ror **errp) region_base +=3D region->len; ct3d->dc.total_capacity +=3D region->len; } + QTAILQ_INIT(&ct3d->dc.extents); =20 return true; } diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index 2f244da9a1..12a6fb47a9 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -424,6 +424,25 @@ typedef QLIST_HEAD(, CXLPoison) CXLPoisonList; =20 #define DCD_MAX_REGION_NUM 8 =20 +typedef struct CXLDCExtentRaw { + uint64_t start_dpa; + uint64_t len; + uint8_t tag[0x10]; + uint16_t shared_seq; + uint8_t rsvd[0x6]; +} QEMU_PACKED CXLDCExtentRaw; + +typedef struct CXLDCExtent { + uint64_t start_dpa; + uint64_t len; + uint8_t tag[0x10]; + uint16_t shared_seq; + uint8_t rsvd[0x6]; + + QTAILQ_ENTRY(CXLDCExtent) node; +} CXLDCExtent; +typedef QTAILQ_HEAD(, CXLDCExtent) CXLDCExtentList; + typedef struct CXLDCDRegion { uint64_t base; uint64_t decode_len; /* aligned to 256*MiB */ @@ -470,6 +489,10 @@ struct CXLType3Dev { HostMemoryBackend *host_dc; AddressSpace host_dc_as; uint64_t total_capacity; /* 256M aligned */ + CXLDCExtentList extents; + + uint32_t total_extent_count; + uint32_t ext_list_gen_seq; =20 uint8_t num_regions; /* 0-8 regions */ CXLDCDRegion regions[DCD_MAX_REGION_NUM]; --=20 2.43.0 From nobody Tue Nov 26 08:37:50 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1708539754; cv=none; d=zohomail.com; s=zohoarc; b=W3Nti4ZgitwORm6EL6p90vnLgSzed6uvs78ZRHZ7+oBYODPZTQd4I4udnFLlGNCut7uvE+jcchauxXB8B5ipnwsq2lIah3iCOJUHbDy4/U81iojS8pnJAfJdEf8j97B/eBIggPntwzA8Qp748wxiOuHlPSuO4n/oJ5d5snaT+HM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708539754; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=g6VL3c1gI2rozkVMagqzxRAFNNaT8HWGIMmWFgrUxvc=; b=NnIIP9v/afHTTDma7GnpAXzQgWTTVKNKlCoNv1AuYC29rCPOr5Yi/1jMn0KCSqID8JBCXxvk2wruEtwrZWACvz3UOieCa/2BTHw398qT/8fJq2enFON0KTAC3PKnmPDFWfI42/8U79p0dROBcShv/Re1IF5UIFf5dSwEzTw3Dks= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708539754849209.76401551179515; Wed, 21 Feb 2024 10:22:34 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rcrDx-0008P9-Is; Wed, 21 Feb 2024 13:21:41 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rcrDv-0008Nc-AM for qemu-devel@nongnu.org; Wed, 21 Feb 2024 13:21:39 -0500 Received: from mail-pg1-x52a.google.com ([2607:f8b0:4864:20::52a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rcrDg-0003Le-4G for qemu-devel@nongnu.org; Wed, 21 Feb 2024 13:21:38 -0500 Received: by mail-pg1-x52a.google.com with SMTP id 41be03b00d2f7-5cddfe0cb64so5359979a12.0 for ; Wed, 21 Feb 2024 10:21:23 -0800 (PST) Received: from localhost.localdomain ([2601:641:300:14de:5692:7e41:13a2:69a]) by smtp.gmail.com with ESMTPSA id cs16-20020a17090af51000b002992f49922csm9979009pjb.25.2024.02.21.10.21.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Feb 2024 10:21:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708539680; x=1709144480; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=g6VL3c1gI2rozkVMagqzxRAFNNaT8HWGIMmWFgrUxvc=; b=H6E9IgKFjPtILFDnZiPI3yQSi5kY3UVMd1Yjg3z959osXpQ+SR0gCHvP6mBJDXRZh5 GJhjYukx492za0BqXQeXbLkEIe+ZDv5shGEFtyo1DfeOttgqed/7A1oltcCkTuD9rPjK YYIj4jwCFTblNUTJTerjheNieli/ZNj9gV7hLsxup8kfXH25Xgv1bMdqQA4pi5KXRmZn /37Tn9flpDKFdYXrjrj7MIv1TCm/R8FKEUax6j0RzYSR2qoNSH9fDfiVZKTBxDGHIiDM 57f4CYc/yGdCO8qRE5kbDt92hsJmftL+m3Wryjwi9Pm60c3THLyLrZm9DDo5xjvu54ga ezVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708539680; x=1709144480; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=g6VL3c1gI2rozkVMagqzxRAFNNaT8HWGIMmWFgrUxvc=; b=CK6S4D6o9lXvniYvPvz3Vnqw0pkLX3Ak/rPGkF/zYOncw1QUu/cStNDULF8zOzT4QO KhrPNU5JWwd5Cx21cQsiyJbKnyJ5DiS/9nBF7O+312tovvH72vyRnVj706wUM4h2z2YO 6JQR81d1WSe0tH1P+zzFMkGlwrEREXZXrdBIG94taIXZQ4t7T0iAUu6x2+0S7McxUgNM C6lqvPkxm8q+yl+vFd31NjYTCrpU/rYY6dZ/+/k/gL1P7Ur7OT5V7NTRPpjO7sOE69mQ qkXaHW/A9AG0xnVvSF/10niFaWDtbyWoSVL3gxk3lfNnICuTLWJh+2o7SUuXE+G0YWbn o7qQ== X-Gm-Message-State: AOJu0YxBNatp3OWWBneBamsls/wgdglkgGBZV5yovglI3WRRmKe3YTRF pmr1unJ3RScb27hO6EduanALRArf6mOX8nqSozPXGs0Wj4vTpeDLITzqE8Pw X-Google-Smtp-Source: AGHT+IFBGjAFI6oKAXvbH/bbeOvJ8OH4HKQWlcBjiXs+PMXpY/3xEh0k13+5pB5wI3icc4F5ofUS2A== X-Received: by 2002:a17:90a:c706:b0:297:4f1:8a19 with SMTP id o6-20020a17090ac70600b0029704f18a19mr17216669pjt.23.1708539680417; Wed, 21 Feb 2024 10:21:20 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Fan Ni Subject: [PATCH v4 08/10] hw/cxl/cxl-mailbox-utils: Add mailbox commands to support add/release dynamic capacity response Date: Wed, 21 Feb 2024 10:16:01 -0800 Message-ID: <20240221182020.1086096-9-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240221182020.1086096-1-nifan.cxl@gmail.com> References: <20240221182020.1086096-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::52a; envelope-from=nifan.cxl@gmail.com; helo=mail-pg1-x52a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1708539755442100001 Content-Type: text/plain; charset="utf-8" From: Fan Ni Per CXL spec 3.1, two mailbox commands are implemented: Add Dynamic Capacity Response (Opcode 4802h) 8.2.9.9.9.3, and Release Dynamic Capacity (Opcode 4803h) 8.2.9.9.9.4. Signed-off-by: Fan Ni --- hw/cxl/cxl-mailbox-utils.c | 288 ++++++++++++++++++++++++++++++++++++ include/hw/cxl/cxl_device.h | 2 + 2 files changed, 290 insertions(+) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index dae7fe00ed..65ed28f700 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -84,6 +84,8 @@ enum { DCD_CONFIG =3D 0x48, #define GET_DC_CONFIG 0x0 #define GET_DYN_CAP_EXT_LIST 0x1 + #define ADD_DYN_CAP_RSP 0x2 + #define RELEASE_DYN_CAP 0x3 PHYSICAL_SWITCH =3D 0x51, #define IDENTIFY_SWITCH_DEVICE 0x0 #define GET_PHYSICAL_PORT_STATE 0x1 @@ -1412,6 +1414,286 @@ static CXLRetCode cmd_dcd_get_dyn_cap_ext_list(cons= t struct cxl_cmd *cmd, return CXL_MBOX_SUCCESS; } =20 +/* + * Check whether any bit between addr[nr, nr+size) is set, + * return true if any bit is set, otherwise return false + */ +static bool test_any_bits_set(const unsigned long *addr, int nr, int size) +{ + unsigned long res =3D find_next_bit(addr, size + nr, nr); + + return res < nr + size; +} + +CXLDCDRegion *cxl_find_dc_region(CXLType3Dev *ct3d, uint64_t dpa, uint64_t= len) +{ + int i; + CXLDCDRegion *region =3D &ct3d->dc.regions[0]; + + if (dpa < region->base || + dpa >=3D region->base + ct3d->dc.total_capacity) { + return NULL; + } + + /* + * CXL r3.1 section 9.13.3: Dynamic Capacity Device (DCD) + * + * Regions are used in increasing-DPA order, with Region 0 being used = for + * the lowest DPA of Dynamic Capacity and Region 7 for the highest DPA. + * So check from the last region to find where the dpa belongs. Extent= s that + * cross multiple regions are not allowed. + */ + for (i =3D ct3d->dc.num_regions - 1; i >=3D 0; i--) { + region =3D &ct3d->dc.regions[i]; + if (dpa >=3D region->base) { + if (dpa + len > region->base + region->len) { + return NULL; + } + return region; + } + } + + return NULL; +} + +static void cxl_insert_extent_to_extent_list(CXLDCExtentList *list, + uint64_t dpa, + uint64_t len, + uint8_t *tag, + uint16_t shared_seq) +{ + CXLDCExtent *extent; + + extent =3D g_new0(CXLDCExtent, 1); + extent->start_dpa =3D dpa; + extent->len =3D len; + if (tag) { + memcpy(extent->tag, tag, 0x10); + } + extent->shared_seq =3D shared_seq; + + QTAILQ_INSERT_TAIL(list, extent, node); +} + +static void cxl_remove_extent_from_extent_list(CXLDCExtentList *list, + CXLDCExtent *extent) +{ + QTAILQ_REMOVE(list, extent, node); + g_free(extent); +} + +/* + * CXL r3.1 Table 8-168: Add Dynamic Capacity Response Input Payload + * CXL r3.1 Table 8-170: Release Dynamic Capacity Input Payload + */ +typedef struct CXLUpdateDCExtentListInPl { + uint32_t num_entries_updated; + uint8_t flags; + uint8_t rsvd[3]; + /* CXL r3.1 Table 8-169: Updated Extent List */ + struct { + uint64_t start_dpa; + uint64_t len; + uint8_t rsvd[8]; + } QEMU_PACKED updated_entries[]; +} QEMU_PACKED CXLUpdateDCExtentListInPl; + +/* + * For the extents in the extent list to operate, check whether they are v= alid + * 1. The extent should be in the range of a valid DC region; + * 2. The extent should not cross multiple regions; + * 3. The start DPA and the length of the extent should align with the blo= ck + * size of the region; + * 4. The address range of multiple extents in the list should not overlap. + */ +static CXLRetCode cxl_detect_malformed_extent_list(CXLType3Dev *ct3d, + const CXLUpdateDCExtentListInPl *in) +{ + uint64_t min_block_size =3D UINT64_MAX; + CXLDCDRegion *region =3D &ct3d->dc.regions[0]; + CXLDCDRegion *lastregion =3D &ct3d->dc.regions[ct3d->dc.num_regions - = 1]; + g_autofree unsigned long *blk_bitmap =3D NULL; + uint64_t dpa, len; + uint32_t i; + + for (i =3D 0; i < ct3d->dc.num_regions; i++) { + region =3D &ct3d->dc.regions[i]; + min_block_size =3D MIN(min_block_size, region->block_size); + } + + blk_bitmap =3D bitmap_new((lastregion->base + lastregion->len - + ct3d->dc.regions[0].base) / min_block_size); + + for (i =3D 0; i < in->num_entries_updated; i++) { + dpa =3D in->updated_entries[i].start_dpa; + len =3D in->updated_entries[i].len; + + region =3D cxl_find_dc_region(ct3d, dpa, len); + if (!region) { + return CXL_MBOX_INVALID_PA; + } + + dpa -=3D ct3d->dc.regions[0].base; + if (dpa % region->block_size || len % region->block_size) { + return CXL_MBOX_INVALID_EXTENT_LIST; + } + /* the dpa range already covered by some other extents in the list= */ + if (test_any_bits_set(blk_bitmap, dpa / min_block_size, + len / min_block_size)) { + return CXL_MBOX_INVALID_EXTENT_LIST; + } + bitmap_set(blk_bitmap, dpa / min_block_size, len / min_block_size); + } + + return CXL_MBOX_SUCCESS; +} + +/* + * CXL r3.1 section 8.2.9.9.9.3: Add Dynamic Capacity Response (opcode 480= 2h) + * An extent is added to the extent list and becomes usable only after the + * response is processed successfully + */ +static CXLRetCode cmd_dcd_add_dyn_cap_rsp(const struct cxl_cmd *cmd, + uint8_t *payload_in, + size_t len_in, + uint8_t *payload_out, + size_t *len_out, + CXLCCI *cci) +{ + CXLUpdateDCExtentListInPl *in =3D (void *)payload_in; + CXLType3Dev *ct3d =3D CXL_TYPE3(cci->d); + CXLDCExtentList *extent_list =3D &ct3d->dc.extents; + CXLDCExtent *ent; + uint32_t i; + uint64_t dpa, len; + CXLRetCode ret; + + if (in->num_entries_updated =3D=3D 0) { + return CXL_MBOX_SUCCESS; + } + + ret =3D cxl_detect_malformed_extent_list(ct3d, in); + if (ret !=3D CXL_MBOX_SUCCESS) { + return ret; + } + + for (i =3D 0; i < in->num_entries_updated; i++) { + dpa =3D in->updated_entries[i].start_dpa; + len =3D in->updated_entries[i].len; + + /* + * Check if the DPA range of the to-be-added extent overlaps with + * existing extent list maintained by the device. + */ + QTAILQ_FOREACH(ent, extent_list, node) { + if (ent->start_dpa <=3D dpa && + dpa + len <=3D ent->start_dpa + ent->len) { + return CXL_MBOX_INVALID_PA; + /* Overlapping one end of the other */ + } else if ((dpa < ent->start_dpa + ent->len && + dpa + len > ent->start_dpa + ent->len) || + (dpa < ent->start_dpa && dpa + len > ent->start_dpa= )) { + return CXL_MBOX_INVALID_PA; + } + } + + /* + * TODO: we will add a pending extent list based on event log reco= rd + * and verify the input response; also, the "More" flag is not + * considered at the moment. + */ + + cxl_insert_extent_to_extent_list(extent_list, dpa, len, NULL, 0); + ct3d->dc.total_extent_count +=3D 1; + } + + return CXL_MBOX_SUCCESS; +} + +/* + * CXL r3.1 section 8.2.9.9.9.4: Release Dynamic Capacity (opcode 4803h) + */ +static CXLRetCode cmd_dcd_release_dyn_cap(const struct cxl_cmd *cmd, + uint8_t *payload_in, + size_t len_in, + uint8_t *payload_out, + size_t *len_out, + CXLCCI *cci) +{ + CXLUpdateDCExtentListInPl *in =3D (void *)payload_in; + CXLType3Dev *ct3d =3D CXL_TYPE3(cci->d); + CXLDCExtentList *extent_list =3D &ct3d->dc.extents; + CXLDCExtent *ent; + uint32_t i; + uint64_t dpa, len; + CXLRetCode ret; + + if (in->num_entries_updated =3D=3D 0) { + return CXL_MBOX_INVALID_INPUT; + } + + ret =3D cxl_detect_malformed_extent_list(ct3d, in); + if (ret !=3D CXL_MBOX_SUCCESS) { + return ret; + } + + for (i =3D 0; i < in->num_entries_updated; i++) { + bool found =3D false; + + dpa =3D in->updated_entries[i].start_dpa; + len =3D in->updated_entries[i].len; + + QTAILQ_FOREACH(ent, extent_list, node) { + if (ent->start_dpa <=3D dpa && + dpa + len <=3D ent->start_dpa + ent->len) { + /* + * If an incoming extent covers a portion of an extent + * in the device extent list, remove only the overlapping + * portion, meaning + * 1. the portions that are not covered by the incoming + * extent at both end of the original extent will become + * new extents and inserted to the extent list; and + * 2. the original extent is removed from the extent list; + * 3. dc extent count is updated accordingly. + */ + uint64_t ent_start_dpa =3D ent->start_dpa; + uint64_t ent_len =3D ent->len; + uint64_t len1 =3D dpa - ent_start_dpa; + uint64_t len2 =3D ent_start_dpa + ent_len - dpa - len; + + found =3D true; + cxl_remove_extent_from_extent_list(extent_list, ent); + ct3d->dc.total_extent_count -=3D 1; + + if (len1) { + cxl_insert_extent_to_extent_list(extent_list, + ent_start_dpa, len1, + NULL, 0); + ct3d->dc.total_extent_count +=3D 1; + } + if (len2) { + cxl_insert_extent_to_extent_list(extent_list, dpa + le= n, + len2, NULL, 0); + ct3d->dc.total_extent_count +=3D 1; + } + break; + /*Currently we reject the attempt to remove a superset*/ + } else if ((dpa < ent->start_dpa + ent->len && + dpa + len > ent->start_dpa + ent->len) || + (dpa < ent->start_dpa && dpa + len > ent->start_dpa= )) { + return CXL_MBOX_INVALID_EXTENT_LIST; + } + } + + if (!found) { + /* Try to remove a non-existing extent */ + return CXL_MBOX_INVALID_PA; + } + } + + return CXL_MBOX_SUCCESS; +} + #define IMMEDIATE_CONFIG_CHANGE (1 << 1) #define IMMEDIATE_DATA_CHANGE (1 << 2) #define IMMEDIATE_POLICY_CHANGE (1 << 3) @@ -1462,6 +1744,12 @@ static const struct cxl_cmd cxl_cmd_set_dcd[256][256= ] =3D { [DCD_CONFIG][GET_DYN_CAP_EXT_LIST] =3D { "DCD_GET_DYNAMIC_CAPACITY_EXTENT_LIST", cmd_dcd_get_dyn_cap_ext_li= st, 8, 0 }, + [DCD_CONFIG][ADD_DYN_CAP_RSP] =3D { + "ADD_DCD_DYNAMIC_CAPACITY_RESPONSE", cmd_dcd_add_dyn_cap_rsp, + ~0, IMMEDIATE_DATA_CHANGE }, + [DCD_CONFIG][RELEASE_DYN_CAP] =3D { + "RELEASE_DCD_DYNAMIC_CAPACITY", cmd_dcd_release_dyn_cap, + ~0, IMMEDIATE_DATA_CHANGE }, }; =20 static const struct cxl_cmd cxl_cmd_set_sw[256][256] =3D { diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index 12a6fb47a9..6178416cbb 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -548,4 +548,6 @@ void cxl_event_irq_assert(CXLType3Dev *ct3d); =20 void cxl_set_poison_list_overflowed(CXLType3Dev *ct3d); =20 +CXLDCDRegion *cxl_find_dc_region(CXLType3Dev *ct3d, uint64_t dpa, uint64_t= len); + #endif --=20 2.43.0 From nobody Tue Nov 26 08:37:50 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1708539749; cv=none; d=zohomail.com; s=zohoarc; b=VM/dVgBmhZRUhSKUv1WqcmFF07yDB9BBvMIKZ0oU+pcujSu/h+tE7Tm+2VkY8FLQjv6/eISfRE/TmF+2DkikvIUVBNWdGFQLkX5svyZUm/Z5irMyxVOsIRBfpjuro0AjjDTjuyndeY55ZJqDo5ncCSkugyhtir3RlxioFrrF+LU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708539749; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=qYU6nCebrZao7rRGfPAJpKxrYnxAjdViLCf8sgh5PNA=; b=XOaM2cEvwEg6rrF2cykvV+Y+FQ4wBjSIo/QZEgvB/PaceuvrVXFKseHEC04leVnventRuZNoPdyy/ExJUxl1NcjiiJxRKG6Fioguu9+gJMpGjpPuri5SXPKssnjrtjxpshlNlHbH9GdA40AaX6laiYx+4ChcCZeyQaSWG0yi3X8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708539749495664.2022575139716; Wed, 21 Feb 2024 10:22:29 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rcrDr-0008IJ-3p; Wed, 21 Feb 2024 13:21:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rcrDo-0008BO-EJ for qemu-devel@nongnu.org; Wed, 21 Feb 2024 13:21:32 -0500 Received: from mail-pj1-x1032.google.com ([2607:f8b0:4864:20::1032]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rcrDg-0003Lq-FS for qemu-devel@nongnu.org; Wed, 21 Feb 2024 13:21:32 -0500 Received: by mail-pj1-x1032.google.com with SMTP id 98e67ed59e1d1-2906bcae4feso3621545a91.3 for ; Wed, 21 Feb 2024 10:21:24 -0800 (PST) Received: from localhost.localdomain ([2601:641:300:14de:5692:7e41:13a2:69a]) by smtp.gmail.com with ESMTPSA id cs16-20020a17090af51000b002992f49922csm9979009pjb.25.2024.02.21.10.21.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Feb 2024 10:21:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708539682; x=1709144482; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qYU6nCebrZao7rRGfPAJpKxrYnxAjdViLCf8sgh5PNA=; b=ipK7oAgTEkap6wBzfDwBNwSWXSoNyu+3YzOEN7WdHAxX4r1EYWZ6rUd/sHzu5zFk8w 6ip/jOuGrLazSgXQMmX8Cl/02dRiAQFgrPEzZGHDPIKisr9pCHtDwHgmFzDc8PRgljDJ HFaN3gSZhUFaH9yWP/KDUATWoy58s6yb6BL0Rx4JEPi+lIdiFRXv7BJ4IASNvjEF/fYQ 7dL4u/or9GDly0pdBADRoxGPgxOBfsnTLdQe/jAU4EJDlB/d+zw9N7wJVW92X859SPls Neb9XjUyy/EtYLyXL1nYoZmiaduHofuzO5lPpYDzEC3H2NAjAmo6yogMnKPNpvKj25Cs pk6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708539682; x=1709144482; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qYU6nCebrZao7rRGfPAJpKxrYnxAjdViLCf8sgh5PNA=; b=IJwU32Uf3qfd+IvBq1F28ZnmSZDvsEcC6mw8AxVukmg7qZWm6lrT6GbKgexDA/W384 2+rn7p7jUScvN5j+W0BEo/NOJjRVxJ+1kQeJ9c2NYvXk7vr1uvkI4+R7Q9bSoj1nCbdx IkItM9WudDnT6HqXZdvyb1JZj2nrKb/BuzuwZBixuowCY9ydwFCbVmlPCQDJP2aoUoic yQajj1I/4Hs/dzMPGDH6zsTNF0n4Uy6Xo4Gx4zWJqDU8jI9mCCaaAfg8rHdDs4w+TK+v tTGEYsHelmD1PQubaqRDohhLu8FYHlY9MGnIpguGNvCzVhMl3AFTSIzUAf1gt4LPvkQW F9tw== X-Gm-Message-State: AOJu0YzWPNhX1oBerqkIFPuKxgGADVC5rq9jkoN7F02o2Yp/zpHgnP5n qCjsef8ga3TQ45dIO0fQx7Fh4jeLiqU62o+p47Y3iY4amisGhdxUJtyWxmSE X-Google-Smtp-Source: AGHT+IHuQgqqx8QbVW6rJdY8A6OTYD9jWSvGOrewfmjtfSvKaf7JXBy3z7LPpc/SD/xl/3TLNN9UDQ== X-Received: by 2002:a17:90a:bb15:b0:299:dd2:47f with SMTP id u21-20020a17090abb1500b002990dd2047fmr14394930pjr.10.1708539681710; Wed, 21 Feb 2024 10:21:21 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Fan Ni Subject: [PATCH v4 09/10] hw/cxl/events: Add qmp interfaces to add/release dynamic capacity extents Date: Wed, 21 Feb 2024 10:16:02 -0800 Message-ID: <20240221182020.1086096-10-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240221182020.1086096-1-nifan.cxl@gmail.com> References: <20240221182020.1086096-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::1032; envelope-from=nifan.cxl@gmail.com; helo=mail-pj1-x1032.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1708539751327100007 Content-Type: text/plain; charset="utf-8" From: Fan Ni Since fabric manager emulation is not supported yet, the change implements the functions to add/release dynamic capacity extents as QMP interfaces. Note: we skips any FM issued extent release request if the exact extent does not exist in the extent list of the device. We will loose the restriction later once we have partial release support in the kernel. 1. Add dynamic capacity extents: For example, the command to add two continuous extents (each 128MiB long) to region 0 (starting at DPA offset 0) looks like below: { "execute": "qmp_capabilities" } { "execute": "cxl-add-dynamic-capacity", "arguments": { "path": "/machine/peripheral/cxl-dcd0", "region-id": 0, "extents": [ { "dpa": 0, "len": 134217728 }, { "dpa": 134217728, "len": 134217728 } ] } } 2. Release dynamic capacity extents: For example, the command to release an extent of size 128MiB from region 0 (DPA offset 128MiB) look like below: { "execute": "cxl-release-dynamic-capacity", "arguments": { "path": "/machine/peripheral/cxl-dcd0", "region-id": 0, "extents": [ { "dpa": 134217728, "len": 134217728 } ] } } Signed-off-by: Fan Ni --- hw/cxl/cxl-mailbox-utils.c | 26 +++- hw/mem/cxl_type3.c | 249 +++++++++++++++++++++++++++++++++++- hw/mem/cxl_type3_stubs.c | 14 ++ include/hw/cxl/cxl_device.h | 7 + include/hw/cxl/cxl_events.h | 18 +++ qapi/cxl.json | 61 ++++++++- 6 files changed, 366 insertions(+), 9 deletions(-) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index 65ed28f700..34c4ebbd12 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -1418,7 +1418,7 @@ static CXLRetCode cmd_dcd_get_dyn_cap_ext_list(const = struct cxl_cmd *cmd, * Check whether any bit between addr[nr, nr+size) is set, * return true if any bit is set, otherwise return false */ -static bool test_any_bits_set(const unsigned long *addr, int nr, int size) +bool test_any_bits_set(const unsigned long *addr, int nr, int size) { unsigned long res =3D find_next_bit(addr, size + nr, nr); =20 @@ -1456,7 +1456,7 @@ CXLDCDRegion *cxl_find_dc_region(CXLType3Dev *ct3d, u= int64_t dpa, uint64_t len) return NULL; } =20 -static void cxl_insert_extent_to_extent_list(CXLDCExtentList *list, +void cxl_insert_extent_to_extent_list(CXLDCExtentList *list, uint64_t dpa, uint64_t len, uint8_t *tag, @@ -1597,16 +1597,28 @@ static CXLRetCode cmd_dcd_add_dyn_cap_rsp(const str= uct cxl_cmd *cmd, } } =20 - /* - * TODO: we will add a pending extent list based on event log reco= rd - * and verify the input response; also, the "More" flag is not - * considered at the moment. - */ + QTAILQ_FOREACH(ent, &ct3d->dc.extents_pending_to_add, node) { + if (ent->start_dpa <=3D dpa && + dpa + len <=3D ent->start_dpa + ent->len) { + break; + } + } + if (!ent) { + return CXL_MBOX_INVALID_PA; + } + + cxl_remove_extent_from_extent_list(&ct3d->dc.extents_pending_to_ad= d, + ent); =20 cxl_insert_extent_to_extent_list(extent_list, dpa, len, NULL, 0); ct3d->dc.total_extent_count +=3D 1; } =20 + /* + * TODO: extents_pending_to_add needs to be cleared so the extents not + * accepted can be reclaimed base on spec r3.1: 8.2.9.9.9.3 + */ + return CXL_MBOX_SUCCESS; } =20 diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index f4edada303..b8c4273e99 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -662,6 +662,7 @@ static bool cxl_create_dc_regions(CXLType3Dev *ct3d, Er= ror **errp) ct3d->dc.total_capacity +=3D region->len; } QTAILQ_INIT(&ct3d->dc.extents); + QTAILQ_INIT(&ct3d->dc.extents_pending_to_add); =20 return true; } @@ -1424,7 +1425,8 @@ static int ct3d_qmp_cxl_event_log_enc(CxlEventLog log) return CXL_EVENT_TYPE_FAIL; case CXL_EVENT_LOG_FATAL: return CXL_EVENT_TYPE_FATAL; -/* DCD not yet supported */ + case CXL_EVENT_LOG_DYNCAP: + return CXL_EVENT_TYPE_DYNAMIC_CAP; default: return -EINVAL; } @@ -1675,6 +1677,251 @@ void qmp_cxl_inject_memory_module_event(const char = *path, CxlEventLog log, } } =20 +/* CXL r3.1 Table 8-50: Dynanic Capacity Event Record */ +static const QemuUUID dynamic_capacity_uuid =3D { + .data =3D UUID(0xca95afa7, 0xf183, 0x4018, 0x8c, 0x2f, + 0x95, 0x26, 0x8e, 0x10, 0x1a, 0x2a), +}; + +typedef enum CXLDCEventType { + DC_EVENT_ADD_CAPACITY =3D 0x0, + DC_EVENT_RELEASE_CAPACITY =3D 0x1, + DC_EVENT_FORCED_RELEASE_CAPACITY =3D 0x2, + DC_EVENT_REGION_CONFIG_UPDATED =3D 0x3, + DC_EVENT_ADD_CAPACITY_RSP =3D 0x4, + DC_EVENT_CAPACITY_RELEASED =3D 0x5, +} CXLDCEventType; + +/* + * Check whether the exact extent exists in the list + * Return value: the extent pointer in the list; else null + */ +static CXLDCExtent *cxl_dc_extent_exists(CXLDCExtentList *list, + CXLDCExtentRaw *ext) +{ + CXLDCExtent *ent; + + if (!ext || !list) { + return NULL; + } + + QTAILQ_FOREACH(ent, list, node) { + if (ent->start_dpa !=3D ext->start_dpa) { + continue; + } + + /*Found exact extent*/ + return ent->len =3D=3D ext->len ? ent : NULL; + } + + return NULL; +} + +/* + * The main function to process dynamic capacity event. Currently DC exten= ts + * add/release requests are processed. + */ +static void qmp_cxl_process_dynamic_capacity(const char *path, CxlEventLog= log, + CXLDCEventType type, uint16_t= hid, + uint8_t rid, + CXLDCExtentRecordList *record= s, + Error **errp) +{ + Object *obj; + CXLEventDynamicCapacity dCap =3D {}; + CXLEventRecordHdr *hdr =3D &dCap.hdr; + CXLType3Dev *dcd; + uint8_t flags =3D 1 << CXL_EVENT_TYPE_INFO; + uint32_t num_extents =3D 0; + CXLDCExtentRecordList *list; + g_autofree CXLDCExtentRaw *extents =3D NULL; + uint8_t enc_log; + uint64_t offset, len, block_size; + int i; + int rc; + g_autofree unsigned long *blk_bitmap =3D NULL; + + obj =3D object_resolve_path(path, NULL); + if (!obj) { + error_setg(errp, "Unable to resolve path"); + return; + } + if (!object_dynamic_cast(obj, TYPE_CXL_TYPE3)) { + error_setg(errp, "Path not point to a valid CXL type3 device"); + return; + } + + dcd =3D CXL_TYPE3(obj); + if (!dcd->dc.num_regions) { + error_setg(errp, "No dynamic capacity support from the device"); + return; + } + + rc =3D ct3d_qmp_cxl_event_log_enc(log); + if (rc < 0) { + error_setg(errp, "Unhandled error log type"); + return; + } + enc_log =3D rc; + + if (rid >=3D dcd->dc.num_regions) { + error_setg(errp, "region id is too large"); + return; + } + block_size =3D dcd->dc.regions[rid].block_size; + + /* Sanity check and count the extents */ + list =3D records; + while (list) { + offset =3D list->value->offset; + len =3D list->value->len; + + if (len =3D=3D 0) { + error_setg(errp, "extent with 0 length is not allowed"); + return; + } + + if (offset % block_size || len % block_size) { + error_setg(errp, "dpa or len is not aligned to region block si= ze"); + return; + } + + if (offset + len > dcd->dc.regions[rid].len) { + error_setg(errp, "extent range is beyond the region end"); + return; + } + + num_extents++; + list =3D list->next; + } + if (num_extents =3D=3D 0) { + error_setg(errp, "No extents found in the command"); + return; + } + + blk_bitmap =3D bitmap_new(dcd->dc.regions[rid].len / block_size); + + /* Create Extent list for event being passed to host */ + i =3D 0; + list =3D records; + extents =3D g_new0(CXLDCExtentRaw, num_extents); + while (list) { + CXLDCExtent *ent; + bool skip_extent =3D false; + + offset =3D list->value->offset; + len =3D list->value->len; + + extents[i].start_dpa =3D offset + dcd->dc.regions[rid].base; + extents[i].len =3D len; + memset(extents[i].tag, 0, 0x10); + extents[i].shared_seq =3D 0; + + if (type =3D=3D DC_EVENT_RELEASE_CAPACITY || + type =3D=3D DC_EVENT_FORCED_RELEASE_CAPACITY) { + /* + * if the extent is still pending to be added to the host, + * remove it from the pending extent list, so later when the a= dd + * response for the extent arrives, the device can reject the + * extent as it is not in the pending list. + */ + ent =3D cxl_dc_extent_exists(&dcd->dc.extents_pending_to_add, + &extents[i]); + if (ent) { + QTAILQ_REMOVE(&dcd->dc.extents_pending_to_add, ent, node); + g_free(ent); + skip_extent =3D true; + } else if (!cxl_dc_extent_exists(&dcd->dc.extents, &extents[i]= )) { + /* If the exact extent is not in the accepted list, skip */ + skip_extent =3D true; + } + } + + /* No duplicate or overlapped extents are allowed */ + if (test_any_bits_set(blk_bitmap, offset / block_size, + len / block_size)) { + error_setg(errp, "duplicate or overlapped extents are detected= "); + return; + } + bitmap_set(blk_bitmap, offset / block_size, len / block_size); + + list =3D list->next; + if (!skip_extent) { + i++; + } + } + num_extents =3D i; + + switch (type) { + case DC_EVENT_ADD_CAPACITY: + break; + default: + break; + } + /* + * CXL r3.1 section 8.2.9.2.1.6: Dynamic Capacity Event Record + * + * All Dynamic Capacity event records shall set the Event Record Sever= ity + * field in the Common Event Record Format to Informational Event. All + * Dynamic Capacity related events shall be logged in the Dynamic Capa= city + * Event Log. + */ + cxl_assign_event_header(hdr, &dynamic_capacity_uuid, flags, sizeof(dCa= p), + cxl_device_get_timestamp(&dcd->cxl_dstate)); + + dCap.type =3D type; + /* FIXME: for now, validaity flag is cleared */ + dCap.validity_flags =3D 0; + stw_le_p(&dCap.host_id, hid); + /* only valid for DC_REGION_CONFIG_UPDATED event */ + dCap.updated_region_id =3D 0; + /* + * FIXME: for now, "More" flag is cleared as there is only one extent = for + * each record + */ + dCap.flags =3D 0; + + /* + * For current implementation, each DC event record only associates wi= th + * one extent, so the "More" flag does not need to be set. + */ + for (i =3D 0; i < num_extents; i++) { + memcpy(&dCap.dynamic_capacity_extent, &extents[i], + sizeof(CXLDCExtentRaw)); + + if (type =3D=3D DC_EVENT_ADD_CAPACITY) { + cxl_insert_extent_to_extent_list(&dcd->dc.extents_pending_to_a= dd, + extents[i].start_dpa, + extents[i].len, + extents[i].tag, + extents[i].shared_seq); + } + + if (cxl_event_insert(&dcd->cxl_dstate, enc_log, + (CXLEventRecordRaw *)&dCap)) { + cxl_event_irq_assert(dcd); + } + } +} + +void qmp_cxl_add_dynamic_capacity(const char *path, uint8_t region_id, + CXLDCExtentRecordList *records, + Error **errp) +{ + qmp_cxl_process_dynamic_capacity(path, CXL_EVENT_LOG_DYNCAP, + DC_EVENT_ADD_CAPACITY, 0, + region_id, records, errp); +} + +void qmp_cxl_release_dynamic_capacity(const char *path, uint8_t region_id, + CXLDCExtentRecordList *records, + Error **errp) +{ + qmp_cxl_process_dynamic_capacity(path, CXL_EVENT_LOG_DYNCAP, + DC_EVENT_RELEASE_CAPACITY, 0, + region_id, records, errp); +} + static void ct3_class_init(ObjectClass *oc, void *data) { DeviceClass *dc =3D DEVICE_CLASS(oc); diff --git a/hw/mem/cxl_type3_stubs.c b/hw/mem/cxl_type3_stubs.c index 3e1851e32b..d913b11b4d 100644 --- a/hw/mem/cxl_type3_stubs.c +++ b/hw/mem/cxl_type3_stubs.c @@ -67,3 +67,17 @@ void qmp_cxl_inject_correctable_error(const char *path, = CxlCorErrorType type, { error_setg(errp, "CXL Type 3 support is not compiled in"); } + +void qmp_cxl_add_dynamic_capacity(const char *path, uint8_t region_id, + CXLDCExtentRecordList *records, + Error **errp) +{ + error_setg(errp, "CXL Type 3 support is not compiled in"); +} + +void qmp_cxl_release_dynamic_capacity(const char *path, uint8_t region_id, + CXLDCExtentRecordList *records, + Error **errp) +{ + error_setg(errp, "CXL Type 3 support is not compiled in"); +} diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index 6178416cbb..1d31164bd3 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -490,6 +490,7 @@ struct CXLType3Dev { AddressSpace host_dc_as; uint64_t total_capacity; /* 256M aligned */ CXLDCExtentList extents; + CXLDCExtentList extents_pending_to_add; =20 uint32_t total_extent_count; uint32_t ext_list_gen_seq; @@ -550,4 +551,10 @@ void cxl_set_poison_list_overflowed(CXLType3Dev *ct3d); =20 CXLDCDRegion *cxl_find_dc_region(CXLType3Dev *ct3d, uint64_t dpa, uint64_t= len); =20 +void cxl_insert_extent_to_extent_list(CXLDCExtentList *list, + uint64_t dpa, + uint64_t len, + uint8_t *tag, + uint16_t shared_seq); +bool test_any_bits_set(const unsigned long *addr, int nr, int size); #endif diff --git a/include/hw/cxl/cxl_events.h b/include/hw/cxl/cxl_events.h index 5170b8dbf8..38cadaa0f3 100644 --- a/include/hw/cxl/cxl_events.h +++ b/include/hw/cxl/cxl_events.h @@ -166,4 +166,22 @@ typedef struct CXLEventMemoryModule { uint8_t reserved[0x3d]; } QEMU_PACKED CXLEventMemoryModule; =20 +/* + * CXL r3.1 section Table 8-50: Dynamic Capacity Event Record + * All fields little endian. + */ +typedef struct CXLEventDynamicCapacity { + CXLEventRecordHdr hdr; + uint8_t type; + uint8_t validity_flags; + uint16_t host_id; + uint8_t updated_region_id; + uint8_t flags; + uint8_t reserved2[2]; + uint8_t dynamic_capacity_extent[0x28]; /* defined in cxl_device.h */ + uint8_t reserved[0x18]; + uint32_t extents_avail; + uint32_t tags_avail; +} QEMU_PACKED CXLEventDynamicCapacity; + #endif /* CXL_EVENTS_H */ diff --git a/qapi/cxl.json b/qapi/cxl.json index 8cc4c72fa9..19927629c7 100644 --- a/qapi/cxl.json +++ b/qapi/cxl.json @@ -19,13 +19,16 @@ # # @fatal: Fatal Event Log # +# @dyncap: Dynamic Capacity Event Log +# # Since: 8.1 ## { 'enum': 'CxlEventLog', 'data': ['informational', 'warning', 'failure', - 'fatal'] + 'fatal', + 'dyncap'] } =20 ## @@ -361,3 +364,59 @@ ## {'command': 'cxl-inject-correctable-error', 'data': {'path': 'str', 'type': 'CxlCorErrorType'}} + +## +# @CXLDCExtentRecord: +# +# Record of a single extent to add/release +# +# @offset: offset to the start of the region where the extent to be operat= ed +# @len: length of the extent +# +# Since: 9.0 +## +{ 'struct': 'CXLDCExtentRecord', + 'data': { + 'offset':'uint64', + 'len': 'uint64' + } +} + +## +# @cxl-add-dynamic-capacity: +# +# Command to start add dynamic capacity extents flow. The device will +# have to ackowledged the acceptance of the extents before they are usable. +# +# @path: CXL DCD canonical QOM path +# @region-id: id of the region where the extent to add/release +# @extents: Extents to add +# +# Since : 9.0 +## +{ 'command': 'cxl-add-dynamic-capacity', + 'data': { 'path': 'str', + 'region-id': 'uint8', + 'extents': [ 'CXLDCExtentRecord' ] + } +} + +## +# @cxl-release-dynamic-capacity: +# +# Command to start release dynamic capacity extents flow. The host will +# need to respond to indicate that it has released the capacity before it +# is made unavailable for read and write and can be re-added. +# +# @path: CXL DCD canonical QOM path +# @region-id: id of the region where the extent to add/release +# @extents: Extents to release +# +# Since : 9.0 +## +{ 'command': 'cxl-release-dynamic-capacity', + 'data': { 'path': 'str', + 'region-id': 'uint8', + 'extents': [ 'CXLDCExtentRecord' ] + } +} --=20 2.43.0 From nobody Tue Nov 26 08:37:50 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1708539780; cv=none; d=zohomail.com; s=zohoarc; b=T9T9UIxgC6hnCWZCEj2Mc7AK1ZSIlYUQqdO8e5VzK+Y2MBTMptLrXqVZrqJNSrA3OcMHS/BeQB/VAqc/lMk8HDHG00iCSd7Y9smW0JuF4Qx+6lNK6eRoSoNgOU589Kq9nLxtMGsjgzVLguoUtHK23kwYvItxoubvYInkz3RwIII= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708539780; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=lopfkfP/Wd4iBuj5Kqs/1syP004OyEXA8CH53rf8oR0=; b=ele1Nt8fZYxU6w0woR4dajAD4lc/uXHparXmldg9r9wcQcr9/Sc4v56K5H3eFSGvxbZsOutaRZIXgHjjYkDcuHxBOj1YKJEX5L/QFTkU3Vs67hOy6EtZfJKyYXiNfZAmh20f5dW9jiLg5hZPYITzS1oOo3aqgiEofz/8TpFH2kA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708539780333249.91870842957007; Wed, 21 Feb 2024 10:23:00 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rcrDl-00088H-2r; Wed, 21 Feb 2024 13:21:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rcrDj-000842-7c for qemu-devel@nongnu.org; Wed, 21 Feb 2024 13:21:27 -0500 Received: from mail-pg1-x52e.google.com ([2607:f8b0:4864:20::52e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rcrDg-0003Lu-Hg for qemu-devel@nongnu.org; Wed, 21 Feb 2024 13:21:26 -0500 Received: by mail-pg1-x52e.google.com with SMTP id 41be03b00d2f7-5c229dabbb6so4005150a12.0 for ; Wed, 21 Feb 2024 10:21:24 -0800 (PST) Received: from localhost.localdomain ([2601:641:300:14de:5692:7e41:13a2:69a]) by smtp.gmail.com with ESMTPSA id cs16-20020a17090af51000b002992f49922csm9979009pjb.25.2024.02.21.10.21.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Feb 2024 10:21:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708539683; x=1709144483; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lopfkfP/Wd4iBuj5Kqs/1syP004OyEXA8CH53rf8oR0=; b=Jd/5T/cFVaRqhykeOO9M5M0WYVUgsDT4OnavolpvLSdHNLVEmgFRODYUUKti3MlTPi e4HfEISIZVMeG0VvRK1n0MJwUT1tmQfZuKIw7aXvGfepNyafixhv8zBeNlGaKJPicRvB R3xnIpB5fuFQGP6TW8yir/1LQMQQOhKj8BY5hlxXF4LmdX0ed0e7hyHoDltaX0q5RuDp erpWYB+XoDbBpXSH0q9LvBUmw5qTQxg3Jvix17RlGgcc+lNgsaWubMHdfZHnCsu1T/pX 5O6mXoMZNhKJA3ZvfNOgs1pchqK5yZfOqKl7K3gMgLgJdVCkbxi01ivcWHADWwAWUX0F gX6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708539683; x=1709144483; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lopfkfP/Wd4iBuj5Kqs/1syP004OyEXA8CH53rf8oR0=; b=rTz2EedPhPZjkLXs1iKhJ/C2dnYtNA/ACIRtGimEEVJCBN2oPXwdQaK1QmRxrIeHfc tw7wRVCRN2Ry2J4WUKnIE/7oCuUa6eu0ZEfSEQkGVMCOE2Cg9rc+Q4cJr6Rlnp9WmF5d JmldfpETDkpS5YtbMJsSgwKb2asXO+aSnJTbkV0M6EgVuzuDG8rCqEl+TEqkVm0DuqJf /54KGW2zbOIJAFP7h1XRV0pCkuu0x721c9Yzfn5kPhJ4ZQHYAvi5DcBiv0S0YmQFPRWW 7pnJcNwPfX9+FXZaxLHzcmaOoZG9HI7TQDn/UPyYZdhWYm7JxsYIQmrLi6jxUFNYVuvd JALg== X-Gm-Message-State: AOJu0Yy5wVPwVd1EP0002XBTQ/KcKcI6H72BbTtTbmlIxyTpsjUvgI7A mUbLiA3AmXMyHBnIQBsccu5UUYhy9I7dUF7rNiaLoeaAcfORoDPyDJVoNDbt X-Google-Smtp-Source: AGHT+IGI8ZUHUmT+j5cQpkDxDF7pWt+w5yGxEq/geBFaBWFR/5DdTtSK+rbAurJexC4zeUi0z++sIQ== X-Received: by 2002:a17:90a:f3cc:b0:29a:3bb5:7b91 with SMTP id ha12-20020a17090af3cc00b0029a3bb57b91mr601189pjb.2.1708539682849; Wed, 21 Feb 2024 10:21:22 -0800 (PST) From: nifan.cxl@gmail.com To: qemu-devel@nongnu.org Cc: jonathan.cameron@huawei.com, linux-cxl@vger.kernel.org, gregory.price@memverge.com, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, nifan.cxl@gmail.com, jim.harris@samsung.com, Fan Ni Subject: [PATCH v4 10/10] hw/mem/cxl_type3: Add dpa range validation for accesses to DC regions Date: Wed, 21 Feb 2024 10:16:03 -0800 Message-ID: <20240221182020.1086096-11-nifan.cxl@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240221182020.1086096-1-nifan.cxl@gmail.com> References: <20240221182020.1086096-1-nifan.cxl@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::52e; envelope-from=nifan.cxl@gmail.com; helo=mail-pg1-x52e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1708539781441100001 Content-Type: text/plain; charset="utf-8" From: Fan Ni Not all dpa range in the DC regions is valid to access until an extent covering the range has been added. Add a bitmap for each region to record whether a DC block in the region has been backed by DC extent. For the bitmap, a bit in the bitmap represents a DC block. When a DC extent is added, all the bits of the blocks in the extent will be set, which will be cleared when the extent is released. Signed-off-by: Fan Ni --- hw/cxl/cxl-mailbox-utils.c | 3 ++ hw/mem/cxl_type3.c | 82 +++++++++++++++++++++++++++++++++++++ include/hw/cxl/cxl_device.h | 7 ++++ 3 files changed, 92 insertions(+) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index 34c4ebbd12..fd3be2f9cf 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -1676,17 +1676,20 @@ static CXLRetCode cmd_dcd_release_dyn_cap(const str= uct cxl_cmd *cmd, found =3D true; cxl_remove_extent_from_extent_list(extent_list, ent); ct3d->dc.total_extent_count -=3D 1; + ct3_clear_region_block_backed(ct3d, ent_start_dpa, ent_len= ); =20 if (len1) { cxl_insert_extent_to_extent_list(extent_list, ent_start_dpa, len1, NULL, 0); ct3d->dc.total_extent_count +=3D 1; + ct3_set_region_block_backed(ct3d, ent_start_dpa, len1); } if (len2) { cxl_insert_extent_to_extent_list(extent_list, dpa + le= n, len2, NULL, 0); ct3d->dc.total_extent_count +=3D 1; + ct3_set_region_block_backed(ct3d, dpa + len, len2); } break; /*Currently we reject the attempt to remove a superset*/ diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index b8c4273e99..a56906db25 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -660,6 +660,7 @@ static bool cxl_create_dc_regions(CXLType3Dev *ct3d, Er= ror **errp) =20 region_base +=3D region->len; ct3d->dc.total_capacity +=3D region->len; + region->blk_bitmap =3D bitmap_new(region->len / region->block_size= ); } QTAILQ_INIT(&ct3d->dc.extents); QTAILQ_INIT(&ct3d->dc.extents_pending_to_add); @@ -667,6 +668,17 @@ static bool cxl_create_dc_regions(CXLType3Dev *ct3d, E= rror **errp) return true; } =20 +static void cxl_destroy_dc_regions(CXLType3Dev *ct3d) +{ + int i; + struct CXLDCDRegion *region; + + for (i =3D 0; i < ct3d->dc.num_regions; i++) { + region =3D &ct3d->dc.regions[i]; + g_free(region->blk_bitmap); + } +} + static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp) { DeviceState *ds =3D DEVICE(ct3d); @@ -860,6 +872,7 @@ err_free_special_ops: g_free(regs->special_ops); err_address_space_free: if (ct3d->dc.host_dc) { + cxl_destroy_dc_regions(ct3d); address_space_destroy(&ct3d->dc.host_dc_as); } if (ct3d->hostpmem) { @@ -881,6 +894,7 @@ static void ct3_exit(PCIDevice *pci_dev) cxl_doe_cdat_release(cxl_cstate); g_free(regs->special_ops); if (ct3d->dc.host_dc) { + cxl_destroy_dc_regions(ct3d); address_space_destroy(&ct3d->dc.host_dc_as); } if (ct3d->hostpmem) { @@ -891,6 +905,70 @@ static void ct3_exit(PCIDevice *pci_dev) } } =20 +/* + * Mark the DPA range [dpa, dap + len) to be backed and accessible. This + * happens when a DC extent is added and accepted by the host. + */ +void ct3_set_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa, + uint64_t len) +{ + CXLDCDRegion *region; + + region =3D cxl_find_dc_region(ct3d, dpa, len); + if (!region) { + return; + } + + bitmap_set(region->blk_bitmap, (dpa - region->base) / region->block_si= ze, + len / region->block_size); +} + +/* + * Check whether the DPA range [dpa, dpa + len) is backed with DC extents. + * Used when validating read/write to dc regions + */ +bool ct3_test_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa, + uint64_t len) +{ + CXLDCDRegion *region; + uint64_t nbits; + long nr; + + region =3D cxl_find_dc_region(ct3d, dpa, len); + if (!region) { + return false; + } + + nr =3D (dpa - region->base) / region->block_size; + nbits =3D DIV_ROUND_UP(len, region->block_size); + /* + * if bits between [dpa, dpa + len) are all 1s, meaning the DPA range = is + * backed with DC extents, return true; else return false. + */ + return find_next_zero_bit(region->blk_bitmap, nr + nbits, nr) =3D=3D n= r + nbits; +} + +/* + * Mark the DPA range [dpa, dap + len) to be unbacked and inaccessible. Th= is + * happens when a dc extent is released by the host. + */ +void ct3_clear_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa, + uint64_t len) +{ + CXLDCDRegion *region; + uint64_t nbits; + long nr; + + region =3D cxl_find_dc_region(ct3d, dpa, len); + if (!region) { + return; + } + + nr =3D (dpa - region->base) / region->block_size; + nbits =3D len / region->block_size; + bitmap_clear(region->blk_bitmap, nr, nbits); +} + static bool cxl_type3_dpa(CXLType3Dev *ct3d, hwaddr host_addr, uint64_t *d= pa) { int hdm_inc =3D R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_L= O; @@ -996,6 +1074,10 @@ static int cxl_type3_hpa_to_as_and_dpa(CXLType3Dev *c= t3d, *as =3D &ct3d->hostpmem_as; *dpa_offset -=3D vmr_size; } else { + if (!ct3_test_region_block_backed(ct3d, *dpa_offset, size)) { + return -ENODEV; + } + *as =3D &ct3d->dc.host_dc_as; *dpa_offset -=3D (vmr_size + pmr_size); } diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index 1d31164bd3..10f0389b50 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -450,6 +450,7 @@ typedef struct CXLDCDRegion { uint64_t block_size; uint32_t dsmadhandle; uint8_t flags; + unsigned long *blk_bitmap; } CXLDCDRegion; =20 struct CXLType3Dev { @@ -557,4 +558,10 @@ void cxl_insert_extent_to_extent_list(CXLDCExtentList = *list, uint8_t *tag, uint16_t shared_seq); bool test_any_bits_set(const unsigned long *addr, int nr, int size); +void ct3_set_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa, + uint64_t len); +void ct3_clear_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa, + uint64_t len); +bool ct3_test_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa, + uint64_t len); #endif --=20 2.43.0