From nobody Sun Nov 24 13:29:38 2024 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B53E51EB9E4; Tue, 5 Nov 2024 18:38:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.11 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730831934; cv=none; b=baKBi7Lw61Zr2vYdOWR2LT+owV1t9b5jwYz1nCJ2PlNsZmIOBwccfkEixVzxOZudvsKaxoGVjlOwHO0nt0VOlynGcmvRqFvoTiLlqKi85XmpH+aN8lx985QFpNNhfTKuJ+9ptebHB84s3MLCeh6aODhvYVyoh9MlZEhHSqSzk/M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730831934; c=relaxed/simple; bh=mP/39uB6KFgSJCccrYcbJwHbhU4pRB9iVaDNiG2eUEc=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=K4y4G523jGxZKFyBCPCJNxkzbastaUa0AdHe2skS7LsnhTCGjM65/H/zIw01nG10p8IJXDibntLy89Q9uo6o8XU9j8PvnxspvrZQ/EPGmGO9L4GznYn44PjzawhJqqBCZXyGfoiUI3L6DVvm0snaALvgVBhcEHKuOQj1LroGHCg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=F3M+ngdi; arc=none smtp.client-ip=198.175.65.11 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="F3M+ngdi" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1730831933; x=1762367933; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=mP/39uB6KFgSJCccrYcbJwHbhU4pRB9iVaDNiG2eUEc=; b=F3M+ngdi+bD055BcJoIOq+OhwJor3XYDB/d1X4lkLl+BIde7fzlwkQdf h3qP5S5od+XMzpfWeRoFyQc5Z0TfUM538R4L4IH+717pYx2g1DnRTQ+f6 IvePfoCBue7SBncUOEZ94nuC+BBfWSdHX/E3dVRV1u1B0vPXMY8h3phg4 /IZIhZ3su0nButvXbHDKpDvbYwrUv9kdUSLbzfLnuar0JbByxm4sDuCPT OFBEAfxnavZVSDlBQlwsV6hrfhiogiLUAWy9AhzQuv4niomFHBqrY2eue RIeNcuzS4AvWW+NrJYgCHSTOPjRBvR9Hpa7YZMgmv1GbK6UKPP6dTOAIU w==; X-CSE-ConnectionGUID: vutdkf19SHyLo5BaPXhPMQ== X-CSE-MsgGUID: cEVOR7ZXR+C88QpRPaL6QQ== X-IronPort-AV: E=McAfee;i="6700,10204,11222"; a="41153185" X-IronPort-AV: E=Sophos;i="6.11,199,1725346800"; d="scan'208";a="41153185" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Nov 2024 10:38:53 -0800 X-CSE-ConnectionGUID: OosBPomdS7CVpK3DOnFmeQ== X-CSE-MsgGUID: f2dPB+tRThaXI0Ydd4Gr5Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,260,1725346800"; d="scan'208";a="84235677" Received: from spandruv-mobl4.amr.corp.intel.com (HELO localhost) ([10.125.109.247]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Nov 2024 10:38:50 -0800 From: ira.weiny@intel.com Date: Tue, 05 Nov 2024 12:38:31 -0600 Subject: [PATCH v6 09/27] cxl/core: Separate region mode from decoder mode Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20241105-dcd-type2-upstream-v6-9-85c7fa2140fe@intel.com> References: <20241105-dcd-type2-upstream-v6-0-85c7fa2140fe@intel.com> In-Reply-To: <20241105-dcd-type2-upstream-v6-0-85c7fa2140fe@intel.com> To: Dave Jiang , Fan Ni , Jonathan Cameron , Navneet Singh , Jonathan Corbet , Andrew Morton Cc: Dan Williams , Davidlohr Bueso , Alison Schofield , Vishal Verma , Ira Weiny , linux-cxl@vger.kernel.org, linux-doc@vger.kernel.org, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org, Jonathan Cameron , Li Ming X-Mailer: b4 0.15-dev-2a633 X-Developer-Signature: v=1; a=ed25519-sha256; t=1730831904; l=11182; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=VIK7W0grb8am/P99hC4l92sVeGLL2N2lFNNyt2pqocQ=; b=cyCZLYEFRYVWe4DoOYqu+oWpA06Fn4wORMMrERujTDw7WQDtTgdw18+HoNjCOI+hRvK0aZqK5 e0InbmTjp08DSglQ4NHm6JQJeIkHrMVp5uRIw/QLt82u2TRdBTMDnZI X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= From: Navneet Singh Until now region modes and decoder modes were equivalent in that both modes were either PMEM or RAM. The addition of Dynamic Capacity partitions defines up to 8 DC partitions per device. The region mode is thus no longer equivalent to the endpoint decoder mode. IOW the endpoint decoders may have modes of DC0-DC7 while the region mode is simply DC. Define a new region mode enumeration which applies to regions separate from the decoder mode. Adjust the code to process these modes independently. There is no equal to decoder mode dead in region modes. Avoid constructing regions with decoders which have been flagged as dead. Suggested-by: Jonathan Cameron Signed-off-by: Navneet Singh Reviewed-by: Jonathan Cameron Reviewed-by: Fan Ni Reviewed-by: Dave Jiang Reviewed-by: Li Ming Co-developed-by: Ira Weiny Signed-off-by: Ira Weiny --- drivers/cxl/core/cdat.c | 6 ++-- drivers/cxl/core/region.c | 77 ++++++++++++++++++++++++++++++++++---------= ---- drivers/cxl/cxl.h | 26 ++++++++++++++-- 3 files changed, 83 insertions(+), 26 deletions(-) diff --git a/drivers/cxl/core/cdat.c b/drivers/cxl/core/cdat.c index ef1621d40f0542e85b01f243f888cd0368111885..b5d30c5bf1e20725d13b4397a7b= a90662bcd8766 100644 --- a/drivers/cxl/core/cdat.c +++ b/drivers/cxl/core/cdat.c @@ -571,17 +571,17 @@ static bool dpa_perf_contains(struct cxl_dpa_perf *pe= rf, } =20 static struct cxl_dpa_perf *cxled_get_dpa_perf(struct cxl_endpoint_decoder= *cxled, - enum cxl_decoder_mode mode) + enum cxl_region_mode mode) { struct cxl_memdev *cxlmd =3D cxled_to_memdev(cxled); struct cxl_memdev_state *mds =3D to_cxl_memdev_state(cxlmd->cxlds); struct cxl_dpa_perf *perf; =20 switch (mode) { - case CXL_DECODER_RAM: + case CXL_REGION_RAM: perf =3D &mds->ram_perf; break; - case CXL_DECODER_PMEM: + case CXL_REGION_PMEM: perf =3D &mds->pmem_perf; break; default: diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 02437e716b7e04493bb7a2b7d14649a2414c1cb7..b3beab787faeb552850ac383947= 2319fcf8f2835 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -144,7 +144,7 @@ static ssize_t uuid_show(struct device *dev, struct dev= ice_attribute *attr, rc =3D down_read_interruptible(&cxl_region_rwsem); if (rc) return rc; - if (cxlr->mode !=3D CXL_DECODER_PMEM) + if (cxlr->mode !=3D CXL_REGION_PMEM) rc =3D sysfs_emit(buf, "\n"); else rc =3D sysfs_emit(buf, "%pUb\n", &p->uuid); @@ -457,7 +457,7 @@ static umode_t cxl_region_visible(struct kobject *kobj,= struct attribute *a, * Support tooling that expects to find a 'uuid' attribute for all * regions regardless of mode. */ - if (a =3D=3D &dev_attr_uuid.attr && cxlr->mode !=3D CXL_DECODER_PMEM) + if (a =3D=3D &dev_attr_uuid.attr && cxlr->mode !=3D CXL_REGION_PMEM) return 0444; return a->mode; } @@ -620,7 +620,7 @@ static ssize_t mode_show(struct device *dev, struct dev= ice_attribute *attr, { struct cxl_region *cxlr =3D to_cxl_region(dev); =20 - return sysfs_emit(buf, "%s\n", cxl_decoder_mode_name(cxlr->mode)); + return sysfs_emit(buf, "%s\n", cxl_region_mode_name(cxlr->mode)); } static DEVICE_ATTR_RO(mode); =20 @@ -646,7 +646,7 @@ static int alloc_hpa(struct cxl_region *cxlr, resource_= size_t size) =20 /* ways, granularity and uuid (if PMEM) need to be set before HPA */ if (!p->interleave_ways || !p->interleave_granularity || - (cxlr->mode =3D=3D CXL_DECODER_PMEM && uuid_is_null(&p->uuid))) + (cxlr->mode =3D=3D CXL_REGION_PMEM && uuid_is_null(&p->uuid))) return -ENXIO; =20 div64_u64_rem(size, (u64)SZ_256M * p->interleave_ways, &remainder); @@ -1863,6 +1863,17 @@ static int cxl_region_sort_targets(struct cxl_region= *cxlr) return rc; } =20 +static bool cxl_modes_compatible(enum cxl_region_mode rmode, + enum cxl_decoder_mode dmode) +{ + if (rmode =3D=3D CXL_REGION_RAM && dmode =3D=3D CXL_DECODER_RAM) + return true; + if (rmode =3D=3D CXL_REGION_PMEM && dmode =3D=3D CXL_DECODER_PMEM) + return true; + + return false; +} + static int cxl_region_attach(struct cxl_region *cxlr, struct cxl_endpoint_decoder *cxled, int pos) { @@ -1882,9 +1893,11 @@ static int cxl_region_attach(struct cxl_region *cxlr, return rc; } =20 - if (cxled->mode !=3D cxlr->mode) { - dev_dbg(&cxlr->dev, "%s region mode: %d mismatch: %d\n", - dev_name(&cxled->cxld.dev), cxlr->mode, cxled->mode); + if (!cxl_modes_compatible(cxlr->mode, cxled->mode)) { + dev_dbg(&cxlr->dev, "%s region mode: %s mismatch decoder: %s\n", + dev_name(&cxled->cxld.dev), + cxl_region_mode_name(cxlr->mode), + cxl_decoder_mode_name(cxled->mode)); return -EINVAL; } =20 @@ -2446,7 +2459,7 @@ static int cxl_region_calculate_adistance(struct noti= fier_block *nb, * devm_cxl_add_region - Adds a region to a decoder * @cxlrd: root decoder * @id: memregion id to create, or memregion_free() on failure - * @mode: mode for the endpoint decoders of this region + * @mode: mode of this region * @type: select whether this is an expander or accelerator (type-2 or typ= e-3) * * This is the second step of region initialization. Regions exist within = an @@ -2457,7 +2470,7 @@ static int cxl_region_calculate_adistance(struct noti= fier_block *nb, */ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxl= rd, int id, - enum cxl_decoder_mode mode, + enum cxl_region_mode mode, enum cxl_decoder_type type) { struct cxl_port *port =3D to_cxl_port(cxlrd->cxlsd.cxld.dev.parent); @@ -2511,16 +2524,17 @@ static ssize_t create_ram_region_show(struct device= *dev, } =20 static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd, - enum cxl_decoder_mode mode, int id) + enum cxl_region_mode mode, int id) { int rc; =20 switch (mode) { - case CXL_DECODER_RAM: - case CXL_DECODER_PMEM: + case CXL_REGION_RAM: + case CXL_REGION_PMEM: break; default: - dev_err(&cxlrd->cxlsd.cxld.dev, "unsupported mode %d\n", mode); + dev_err(&cxlrd->cxlsd.cxld.dev, "unsupported mode %s\n", + cxl_region_mode_name(mode)); return ERR_PTR(-EINVAL); } =20 @@ -2537,7 +2551,7 @@ static struct cxl_region *__create_region(struct cxl_= root_decoder *cxlrd, } =20 static ssize_t create_region_store(struct device *dev, const char *buf, - size_t len, enum cxl_decoder_mode mode) + size_t len, enum cxl_region_mode mode) { struct cxl_root_decoder *cxlrd =3D to_cxl_root_decoder(dev); struct cxl_region *cxlr; @@ -2558,7 +2572,7 @@ static ssize_t create_pmem_region_store(struct device= *dev, struct device_attribute *attr, const char *buf, size_t len) { - return create_region_store(dev, buf, len, CXL_DECODER_PMEM); + return create_region_store(dev, buf, len, CXL_REGION_PMEM); } DEVICE_ATTR_RW(create_pmem_region); =20 @@ -2566,7 +2580,7 @@ static ssize_t create_ram_region_store(struct device = *dev, struct device_attribute *attr, const char *buf, size_t len) { - return create_region_store(dev, buf, len, CXL_DECODER_RAM); + return create_region_store(dev, buf, len, CXL_REGION_RAM); } DEVICE_ATTR_RW(create_ram_region); =20 @@ -3209,6 +3223,22 @@ static int match_region_by_range(struct device *dev,= void *data) return rc; } =20 +static enum cxl_region_mode +cxl_decoder_to_region_mode(enum cxl_decoder_mode mode) +{ + switch (mode) { + case CXL_DECODER_NONE: + return CXL_REGION_NONE; + case CXL_DECODER_RAM: + return CXL_REGION_RAM; + case CXL_DECODER_PMEM: + return CXL_REGION_PMEM; + case CXL_DECODER_MIXED: + default: + return CXL_REGION_MIXED; + } +} + /* Establish an empty region covering the given HPA range */ static struct cxl_region *construct_region(struct cxl_root_decoder *cxlrd, struct cxl_endpoint_decoder *cxled) @@ -3217,12 +3247,17 @@ static struct cxl_region *construct_region(struct c= xl_root_decoder *cxlrd, struct cxl_port *port =3D cxlrd_to_port(cxlrd); struct range *hpa =3D &cxled->cxld.hpa_range; struct cxl_region_params *p; + enum cxl_region_mode mode; struct cxl_region *cxlr; struct resource *res; int rc; =20 + if (cxled->mode =3D=3D CXL_DECODER_DEAD) + return ERR_PTR(-EINVAL); + + mode =3D cxl_decoder_to_region_mode(cxled->mode); do { - cxlr =3D __create_region(cxlrd, cxled->mode, + cxlr =3D __create_region(cxlrd, mode, atomic_read(&cxlrd->region_id)); } while (IS_ERR(cxlr) && PTR_ERR(cxlr) =3D=3D -EBUSY); =20 @@ -3425,9 +3460,9 @@ static int cxl_region_probe(struct device *dev) return rc; =20 switch (cxlr->mode) { - case CXL_DECODER_PMEM: + case CXL_REGION_PMEM: return devm_cxl_add_pmem_region(cxlr); - case CXL_DECODER_RAM: + case CXL_REGION_RAM: /* * The region can not be manged by CXL if any portion of * it is already online as 'System RAM' @@ -3439,8 +3474,8 @@ static int cxl_region_probe(struct device *dev) return 0; return devm_cxl_add_dax_region(cxlr); default: - dev_dbg(&cxlr->dev, "unsupported region mode: %d\n", - cxlr->mode); + dev_dbg(&cxlr->dev, "unsupported region mode: %s\n", + cxl_region_mode_name(cxlr->mode)); return -ENXIO; } } diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 0d8b810a51f04de299e88ee8b29136bff11ed93e..5d74eb4ffab3ea2656c8e3c0563= b8d7b69d76232 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -388,6 +388,27 @@ static inline const char *cxl_decoder_mode_name(enum c= xl_decoder_mode mode) return "mixed"; } =20 +enum cxl_region_mode { + CXL_REGION_NONE, + CXL_REGION_RAM, + CXL_REGION_PMEM, + CXL_REGION_MIXED, +}; + +static inline const char *cxl_region_mode_name(enum cxl_region_mode mode) +{ + static const char * const names[] =3D { + [CXL_REGION_NONE] =3D "none", + [CXL_REGION_RAM] =3D "ram", + [CXL_REGION_PMEM] =3D "pmem", + [CXL_REGION_MIXED] =3D "mixed", + }; + + if (mode >=3D CXL_REGION_NONE && mode <=3D CXL_REGION_MIXED) + return names[mode]; + return "mixed"; +} + /* * Track whether this decoder is reserved for region autodiscovery, or * free for userspace provisioning. @@ -515,7 +536,8 @@ struct cxl_region_params { * struct cxl_region - CXL region * @dev: This region's device * @id: This region's id. Id is globally unique across all regions - * @mode: Endpoint decoder allocation / access mode + * @mode: Region mode which defines which endpoint decoder modes the regio= n is + * compatible with * @type: Endpoint decoder target type * @cxl_nvb: nvdimm bridge for coordinating @cxlr_pmem setup / shutdown * @cxlr_pmem: (for pmem regions) cached copy of the nvdimm bridge @@ -528,7 +550,7 @@ struct cxl_region_params { struct cxl_region { struct device dev; int id; - enum cxl_decoder_mode mode; + enum cxl_region_mode mode; enum cxl_decoder_type type; struct cxl_nvdimm_bridge *cxl_nvb; struct cxl_pmem_region *cxlr_pmem; --=20 2.47.0