From nobody Sun Feb 8 01:03:30 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC393246778; Thu, 4 Dec 2025 02:21:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.9 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764814884; cv=none; b=eOMM7HIQ3AefKxMwaK1HgbpSLRIjwc/KnwUoB4vJI4Ofg40+WkuEd7TbLdpWoWM0y+LDhrnzHv4u3aChMDXNBYn7WTd1zWZ5xzekFjD2MdnfX1buWgpltq/5HCttYo9PWwTbCisUSYxadaYoSqTeRPmAnY+MEWbeMArIOuPjrTE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764814884; c=relaxed/simple; bh=7jk1Q0Rh3W0bZWpLoVJ4ApBhUwh7oh0Mr6tQKuIbIkQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JkI3wZDYmfTgeUvNke5z0rHqPgHab+x647QzAvbVg2Mi05r2KoSjXKENVSo6KkoF77esiM6C+zrPSgCpF9eRZvrRens1UeUHvlt+wvJb8LEJe6lHHkPo6OUg3rLKl57h43/tYF7lp/HsNog7DIfelrYdHD8gGIpVymrNBe1QXnU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=OaBEzRuS; arc=none smtp.client-ip=192.198.163.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="OaBEzRuS" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1764814883; x=1796350883; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7jk1Q0Rh3W0bZWpLoVJ4ApBhUwh7oh0Mr6tQKuIbIkQ=; b=OaBEzRuSM7SMFO+oeMq6h7C0n5zfK49g244H/Wg3Zm2MWbGodgxaPhgs bi3AmmM2nmcSCtTdRsmdLwrUWrz+WJDxN4R/OmGhDnixy7N1bGBbhFsq9 xX0khp2gZcPoF5BqUlh9RKDa1edEJTjBIsj5UZEgxK5dOnxwUMPHJ0Dtw AM6IqmAZmbDOhRxQ2XFh/ejYk6P2vha79jgLVORNHkQNQ9PSDi/Oul5Tp OQyAz1sp0SvcJ0Mq+u7ZARjNJlvFfMPNHTEraTnuus21l7E5ERy5nui+B tlsBGdayNmR53dTQfDztVrNj3QjpKksOW0Gs1fm8LrMyKBqOFx0+Y3t4M A==; X-CSE-ConnectionGUID: tWYwfodyTKm1tuLgtrAYAg== X-CSE-MsgGUID: Pm62CcecSdKoybwwz4kj5w== X-IronPort-AV: E=McAfee;i="6800,10657,11631"; a="77508639" X-IronPort-AV: E=Sophos;i="6.20,247,1758610800"; d="scan'208";a="77508639" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Dec 2025 18:21:22 -0800 X-CSE-ConnectionGUID: cRt8lDl9SRiE+QQ7acHaBw== X-CSE-MsgGUID: TDJ1pXPHT9q4aZdghgVPtA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.20,247,1758610800"; d="scan'208";a="225802519" Received: from dwillia2-desk.jf.intel.com ([10.88.27.145]) by fmviesa001.fm.intel.com with ESMTP; 03 Dec 2025 18:21:12 -0800 From: Dan Williams To: dave.jiang@intel.com Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Smita.KoralahalliChannabasappa@amd.com, alison.schofield@intel.com, terry.bowman@amd.com, alejandro.lucero-palau@amd.com, linux-pci@vger.kernel.org, Jonathan.Cameron@huawei.com, Shiju Jose Subject: [PATCH 1/6] cxl/mem: Fix devm_cxl_memdev_edac_release() confusion Date: Wed, 3 Dec 2025 18:21:31 -0800 Message-ID: <20251204022136.2573521-2-dan.j.williams@intel.com> X-Mailer: git-send-email 2.51.1 In-Reply-To: <20251204022136.2573521-1-dan.j.williams@intel.com> References: <20251204022136.2573521-1-dan.j.williams@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" A device release method is only for undoing allocations on the path to preparing the device for device_add(). In contrast, devm allocations are post device_add(), are acquired during / after ->probe() and are released synchronous with ->remove(). So, a "devm" helper in a "release" method is a clear anti-pattern. Move this devm release action where it belongs, an action created at edac object creation time. Otherwise, this leaks resources until cxl_memdev_release() time which may be long after these xarray and error record caches have gone idle. Note, this also fixes up the type of @cxlmd->err_rec_array which needlessly dropped type-safety. Fixes: 0b5ccb0de1e2 ("cxl/edac: Support for finding memory operation attrib= utes from the current boot") Cc: Dave Jiang Cc: Jonathan Cameron Cc: Shiju Jose Cc: Alison Schofield Signed-off-by: Dan Williams Reviewed-by: Alison Schofield Reviewed-by: Ben Cheatham Reviewed-by: Dave Jiang Reviewed-by: Jonathan Cameron Reviewed-by: Shiju Jose Tested-by: Alejandro Lucero Tested-by: Shiju Jose --- drivers/cxl/cxlmem.h | 5 +-- drivers/cxl/core/edac.c | 64 ++++++++++++++++++++++----------------- drivers/cxl/core/memdev.c | 1 - 3 files changed, 38 insertions(+), 32 deletions(-) diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 434031a0c1f7..c12ab4fc9512 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -63,7 +63,7 @@ struct cxl_memdev { int depth; u8 scrub_cycle; int scrub_region_id; - void *err_rec_array; + struct cxl_mem_err_rec *err_rec_array; }; =20 static inline struct cxl_memdev *to_cxl_memdev(struct device *dev) @@ -877,7 +877,6 @@ int devm_cxl_memdev_edac_register(struct cxl_memdev *cx= lmd); int devm_cxl_region_edac_register(struct cxl_region *cxlr); int cxl_store_rec_gen_media(struct cxl_memdev *cxlmd, union cxl_event *evt= ); int cxl_store_rec_dram(struct cxl_memdev *cxlmd, union cxl_event *evt); -void devm_cxl_memdev_edac_release(struct cxl_memdev *cxlmd); #else static inline int devm_cxl_memdev_edac_register(struct cxl_memdev *cxlmd) { return 0; } @@ -889,8 +888,6 @@ static inline int cxl_store_rec_gen_media(struct cxl_me= mdev *cxlmd, static inline int cxl_store_rec_dram(struct cxl_memdev *cxlmd, union cxl_event *evt) { return 0; } -static inline void devm_cxl_memdev_edac_release(struct cxl_memdev *cxlmd) -{ return; } #endif =20 #ifdef CONFIG_CXL_SUSPEND diff --git a/drivers/cxl/core/edac.c b/drivers/cxl/core/edac.c index 79994ca9bc9f..81160260e26b 100644 --- a/drivers/cxl/core/edac.c +++ b/drivers/cxl/core/edac.c @@ -1988,6 +1988,40 @@ static int cxl_memdev_soft_ppr_init(struct cxl_memde= v *cxlmd, return 0; } =20 +static void err_rec_free(void *_cxlmd) +{ + struct cxl_memdev *cxlmd =3D _cxlmd; + struct cxl_mem_err_rec *array_rec =3D cxlmd->err_rec_array; + struct cxl_event_gen_media *rec_gen_media; + struct cxl_event_dram *rec_dram; + unsigned long index; + + cxlmd->err_rec_array =3D NULL; + xa_for_each(&array_rec->rec_dram, index, rec_dram) + kfree(rec_dram); + xa_destroy(&array_rec->rec_dram); + + xa_for_each(&array_rec->rec_gen_media, index, rec_gen_media) + kfree(rec_gen_media); + xa_destroy(&array_rec->rec_gen_media); + kfree(array_rec); +} + +static int devm_cxl_memdev_setup_err_rec(struct cxl_memdev *cxlmd) +{ + struct cxl_mem_err_rec *array_rec =3D + kzalloc(sizeof(*array_rec), GFP_KERNEL); + + if (!array_rec) + return -ENOMEM; + + xa_init(&array_rec->rec_gen_media); + xa_init(&array_rec->rec_dram); + cxlmd->err_rec_array =3D array_rec; + + return devm_add_action_or_reset(&cxlmd->dev, err_rec_free, cxlmd); +} + int devm_cxl_memdev_edac_register(struct cxl_memdev *cxlmd) { struct edac_dev_feature ras_features[CXL_NR_EDAC_DEV_FEATURES]; @@ -2038,15 +2072,9 @@ int devm_cxl_memdev_edac_register(struct cxl_memdev = *cxlmd) } =20 if (repair_inst) { - struct cxl_mem_err_rec *array_rec =3D - devm_kzalloc(&cxlmd->dev, sizeof(*array_rec), - GFP_KERNEL); - if (!array_rec) - return -ENOMEM; - - xa_init(&array_rec->rec_gen_media); - xa_init(&array_rec->rec_dram); - cxlmd->err_rec_array =3D array_rec; + rc =3D devm_cxl_memdev_setup_err_rec(cxlmd); + if (rc) + return rc; } } =20 @@ -2088,22 +2116,4 @@ int devm_cxl_region_edac_register(struct cxl_region = *cxlr) } EXPORT_SYMBOL_NS_GPL(devm_cxl_region_edac_register, "CXL"); =20 -void devm_cxl_memdev_edac_release(struct cxl_memdev *cxlmd) -{ - struct cxl_mem_err_rec *array_rec =3D cxlmd->err_rec_array; - struct cxl_event_gen_media *rec_gen_media; - struct cxl_event_dram *rec_dram; - unsigned long index; - - if (!IS_ENABLED(CONFIG_CXL_EDAC_MEM_REPAIR) || !array_rec) - return; - - xa_for_each(&array_rec->rec_dram, index, rec_dram) - kfree(rec_dram); - xa_destroy(&array_rec->rec_dram); =20 - xa_for_each(&array_rec->rec_gen_media, index, rec_gen_media) - kfree(rec_gen_media); - xa_destroy(&array_rec->rec_gen_media); -} -EXPORT_SYMBOL_NS_GPL(devm_cxl_memdev_edac_release, "CXL"); diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index e370d733e440..4dff7f44d908 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -27,7 +27,6 @@ static void cxl_memdev_release(struct device *dev) struct cxl_memdev *cxlmd =3D to_cxl_memdev(dev); =20 ida_free(&cxl_memdev_ida, cxlmd->id); - devm_cxl_memdev_edac_release(cxlmd); kfree(cxlmd); } =20 --=20 2.51.1 From nobody Sun Feb 8 01:03:30 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 823201E2614; Thu, 4 Dec 2025 02:21:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.9 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764814893; cv=none; b=XLL6QLd2gbViGa04uHIIComcd0ASC75guoZ4hamgLVG12a/k8Oi3RGraHE1OFkonChbuP49DO5bODqKui30F+pj8/iv9f6OIw8/Z6WMYR+W7fLcbdK3NnVXiOQjO4cvmYFWlz5AQgvmXLUZn2XhuJtxa5pQq0U047idaq+eqw7E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764814893; c=relaxed/simple; bh=cCBuanvLyvbYL8oaI560CQtmGCc8xwgJYs2W7O8C3fY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BA4thUXy6mtt+bmCrJyJkxg0wQV3kZnBT9zx/yFfZg4ygXwfGUAR3EoYdVs2ZIWxsgQW5ghXTPL8f7luj881/Btc8pjmeGixv9T7RG8uSSbuYuNkqEJyJ/RZmhG2/Hn2+qiqWY4/2va7DJov7Y6lMUyuYO6EOztTX3IvW7N39bA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=iLeFjaWk; arc=none smtp.client-ip=192.198.163.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="iLeFjaWk" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1764814891; x=1796350891; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=cCBuanvLyvbYL8oaI560CQtmGCc8xwgJYs2W7O8C3fY=; b=iLeFjaWkAQUJ+5e/7Zgnhi5/KQFEyBXx2iemIvT0vA7jJOj9vGw/Lxuf 5XhTUBfLsLILXaIO9/inlIDKu21gSyEZPvKnyU+McXpMFRhF0erZ8dliq dSRn77y/cSfjHPZHeqM4L/09F8mMrWxmX4xsoF/CRnEjpzZ+wincubUr3 ZOcfDCrEpY/3XMi6V2JkY/weRW0u+a7MdWWy3eqjJzntgehrCgfnSsX3V FLTyK0YGCHSVrs3XaLJSUAE/F1DUu8w49w8x2wN8C7llPUnkAn9bi6ihs EIazge441XQ0ADVgofMJQVEhq1Q7q/l8XDeXZuccl+kaDVHtUZ7WKLV0/ w==; X-CSE-ConnectionGUID: nHkGVQv1QuGi4NEbKxE1KQ== X-CSE-MsgGUID: YgA0e+cjSVG77hDCeuD9gQ== X-IronPort-AV: E=McAfee;i="6800,10657,11631"; a="77508645" X-IronPort-AV: E=Sophos;i="6.20,247,1758610800"; d="scan'208";a="77508645" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Dec 2025 18:21:31 -0800 X-CSE-ConnectionGUID: ywvjbMiaSGmXXQWlNW2cRg== X-CSE-MsgGUID: atRIVDw1TZyq8uW+Cg4UcQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.20,247,1758610800"; d="scan'208";a="225802537" Received: from dwillia2-desk.jf.intel.com ([10.88.27.145]) by fmviesa001.fm.intel.com with ESMTP; 03 Dec 2025 18:21:23 -0800 From: Dan Williams To: dave.jiang@intel.com Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Smita.KoralahalliChannabasappa@amd.com, alison.schofield@intel.com, terry.bowman@amd.com, alejandro.lucero-palau@amd.com, linux-pci@vger.kernel.org, Jonathan.Cameron@huawei.com, Alejandro Lucero Subject: [PATCH 2/6] cxl/mem: Arrange for always-synchronous memdev attach Date: Wed, 3 Dec 2025 18:21:32 -0800 Message-ID: <20251204022136.2573521-3-dan.j.williams@intel.com> X-Mailer: git-send-email 2.51.1 In-Reply-To: <20251204022136.2573521-1-dan.j.williams@intel.com> References: <20251204022136.2573521-1-dan.j.williams@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In preparation for CXL accelerator drivers that have a hard dependency on CXL capability initialization, arrange for cxl_mem_probe() to always run synchronous with the device_add() of cxl_memdev instances. I.e. cxl_mem_driver registration is always complete before the first memdev creation event. At present, cxl_pci does not care about the attach state of the cxl_memdev because all generic memory expansion functionality can be handled by the cxl_core. For accelerators, however, that driver needs to perform driver specific initialization if CXL is available, or execute a fallback to PCIe only operation. This synchronous attach guarantee is also needed for Soft Reserve Recovery, which is an effort that needs to assert that devices have had a chance to attach before making a go / no-go decision on proceeding with CXL subsystem initialization. By moving devm_cxl_add_memdev() to cxl_mem.ko it removes async module loading as one reason that a memdev may not be attached upon return from devm_cxl_add_memdev(). Cc: Smita Koralahalli Cc: Alejandro Lucero Signed-off-by: Dan Williams Reviewed-by: Alison Schofield Reviewed-by: Ben Cheatham Reviewed-by: Dave Jiang Reviewed-by: Jonathan Cameron Tested-by: Alejandro Lucero Tested-by: Alison Schofield --- drivers/cxl/Kconfig | 2 +- drivers/cxl/cxlmem.h | 2 ++ drivers/cxl/core/memdev.c | 10 +++++++--- drivers/cxl/mem.c | 17 +++++++++++++++++ 4 files changed, 27 insertions(+), 4 deletions(-) diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig index 48b7314afdb8..f1361ed6a0d4 100644 --- a/drivers/cxl/Kconfig +++ b/drivers/cxl/Kconfig @@ -22,6 +22,7 @@ if CXL_BUS config CXL_PCI tristate "PCI manageability" default CXL_BUS + select CXL_MEM help The CXL specification defines a "CXL memory device" sub-class in the PCI "memory controller" base class of devices. Device's identified by @@ -89,7 +90,6 @@ config CXL_PMEM =20 config CXL_MEM tristate "CXL: Memory Expansion" - depends on CXL_PCI default CXL_BUS help The CXL.mem protocol allows a device to act as a provider of "System diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index c12ab4fc9512..012e68acad34 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -95,6 +95,8 @@ static inline bool is_cxl_endpoint(struct cxl_port *port) return is_cxl_memdev(port->uport_dev); } =20 +struct cxl_memdev *__devm_cxl_add_memdev(struct device *host, + struct cxl_dev_state *cxlds); struct cxl_memdev *devm_cxl_add_memdev(struct device *host, struct cxl_dev_state *cxlds); int devm_cxl_sanitize_setup_notifier(struct device *host, diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index 4dff7f44d908..7a4153e1c6a7 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -1050,8 +1050,12 @@ static const struct file_operations cxl_memdev_fops = =3D { .llseek =3D noop_llseek, }; =20 -struct cxl_memdev *devm_cxl_add_memdev(struct device *host, - struct cxl_dev_state *cxlds) +/* + * Core helper for devm_cxl_add_memdev() that wants to both create a devic= e and + * assert to the caller that upon return cxl_mem::probe() has been invoked. + */ +struct cxl_memdev *__devm_cxl_add_memdev(struct device *host, + struct cxl_dev_state *cxlds) { struct cxl_memdev *cxlmd; struct device *dev; @@ -1093,7 +1097,7 @@ struct cxl_memdev *devm_cxl_add_memdev(struct device = *host, put_device(dev); return ERR_PTR(rc); } -EXPORT_SYMBOL_NS_GPL(devm_cxl_add_memdev, "CXL"); +EXPORT_SYMBOL_FOR_MODULES(__devm_cxl_add_memdev, "cxl_mem"); =20 static void sanitize_teardown_notifier(void *data) { diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 6e6777b7bafb..55883797ab2d 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -201,6 +201,22 @@ static int cxl_mem_probe(struct device *dev) return devm_add_action_or_reset(dev, enable_suspend, NULL); } =20 +/** + * devm_cxl_add_memdev - Add a CXL memory device + * @host: devres alloc/release context and parent for the memdev + * @cxlds: CXL device state to associate with the memdev + * + * Upon return the device will have had a chance to attach to the + * cxl_mem driver, but may fail if the CXL topology is not ready + * (hardware CXL link down, or software platform CXL root not attached) + */ +struct cxl_memdev *devm_cxl_add_memdev(struct device *host, + struct cxl_dev_state *cxlds) +{ + return __devm_cxl_add_memdev(host, cxlds); +} +EXPORT_SYMBOL_NS_GPL(devm_cxl_add_memdev, "CXL"); + static ssize_t trigger_poison_list_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t len) @@ -248,6 +264,7 @@ static struct cxl_driver cxl_mem_driver =3D { .probe =3D cxl_mem_probe, .id =3D CXL_DEVICE_MEMORY_EXPANDER, .drv =3D { + .probe_type =3D PROBE_FORCE_SYNCHRONOUS, .dev_groups =3D cxl_mem_groups, }, }; --=20 2.51.1 From nobody Sun Feb 8 01:03:30 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B26D0252906; Thu, 4 Dec 2025 02:21:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.9 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764814898; cv=none; b=poLsF+xSiXg67UxewmpYNLF8pWtiWka+nZd0Ya5569tNpuEa1XN/KOhubGnD9wg/hQPVkg1c52bfiaKRLDwphCRGS6UWugQD2zl7VeHmIJ8cFXPoIVktw3MdMBCC5iS/GnWvpwS4BnMhAjVIVDBRkrOm9W0zDbgn8+7P+Ukl3gE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764814898; c=relaxed/simple; bh=TMdhnnbYYKKRBDoibQUniojJzhk2lMbrhlYlNN3Aios=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LGICAaVT81z15U6Rzq0P5uKCfHROlzMFzzfPJW4iVRhxpXxyqf0vVcFT/KAQAVz/52egwfqBBbALW5C6Z57Hk2FK0Kj9/qdn03To9T/161LK9ZZoEWQxklrZYJ1cZiV5KSQMCRLQcYTBGMzNwQAAJlV13U3U9DdkWpbWK4OL6bI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=PO9vtsUx; arc=none smtp.client-ip=192.198.163.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="PO9vtsUx" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1764814897; x=1796350897; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=TMdhnnbYYKKRBDoibQUniojJzhk2lMbrhlYlNN3Aios=; b=PO9vtsUxRSmtoCyQkaq3B5eqWqTWZTdvRrwyo8nj2xJtmxND60zELUkQ Dt1rsuZLivxwlT2nV4LCPRE76ulwR6jJ+1YkD12xemYurogdoSTiFmp/d nqzAFtXL24g5U0kogB+5/AwTfhjx59b+XcmDQGoiBNSISyQV98YPZSNWx 9nzo/UAChNmOjpN4oZSAY5a27iDBpS7q62JJ1JcJ5g0tzUSODzLiPnERw lSdz70geapsg19/Fdvo3IfUa1VyHr2NkO2Q1oZl6fjvLBRpyynmJEEiAH 40q0fUPDp8S/bmyK1LMv7xFM4g4uXkeDuUHPuREvEvB9+b2zFladz4Daz g==; X-CSE-ConnectionGUID: Ehm4KwJhQlaBdpdsO2N79g== X-CSE-MsgGUID: 5vD9E65ES0u00mjtJTYD9g== X-IronPort-AV: E=McAfee;i="6800,10657,11631"; a="77508649" X-IronPort-AV: E=Sophos;i="6.20,247,1758610800"; d="scan'208";a="77508649" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Dec 2025 18:21:36 -0800 X-CSE-ConnectionGUID: hoS64trDTSCqdsctwkPxOA== X-CSE-MsgGUID: qeLRdQkyQIWBXocU1CSUuA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.20,247,1758610800"; d="scan'208";a="225802550" Received: from dwillia2-desk.jf.intel.com ([10.88.27.145]) by fmviesa001.fm.intel.com with ESMTP; 03 Dec 2025 18:21:31 -0800 From: Dan Williams To: dave.jiang@intel.com Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Smita.KoralahalliChannabasappa@amd.com, alison.schofield@intel.com, terry.bowman@amd.com, alejandro.lucero-palau@amd.com, linux-pci@vger.kernel.org, Jonathan.Cameron@huawei.com, Alejandro Lucero Subject: [PATCH 3/6] cxl/port: Arrange for always synchronous endpoint attach Date: Wed, 3 Dec 2025 18:21:33 -0800 Message-ID: <20251204022136.2573521-4-dan.j.williams@intel.com> X-Mailer: git-send-email 2.51.1 In-Reply-To: <20251204022136.2573521-1-dan.j.williams@intel.com> References: <20251204022136.2573521-1-dan.j.williams@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Make it so that upon return from devm_cxl_add_endpoint() that cxl_mem_probe() can assume that the endpoint has had a chance to complete cxl_port_probe(). I.e. cxl_port module loading has completed prior to device registration. Delete the MODULE_SOFTDEP() as it is not sufficient for this purpose, but a hard link-time dependency is reliable. Specifically MODULE_SOFTDEP() does not guarantee that the module loading has completed prior to the completion of the current module's init. Cc: Smita Koralahalli Cc: Alejandro Lucero Signed-off-by: Dan Williams Reviewed-by: Alison Schofield Reviewed-by: Ben Cheatham Reviewed-by: Dave Jiang Reviewed-by: Jonathan Cameron Tested-by: Alejandro Lucero Tested-by: Alison Schofield --- drivers/cxl/cxl.h | 2 ++ drivers/cxl/mem.c | 43 ------------------------------------------- drivers/cxl/port.c | 40 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 42 insertions(+), 43 deletions(-) diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index ba17fa86d249..c796c3db36e0 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -780,6 +780,8 @@ struct cxl_port *devm_cxl_add_port(struct device *host, struct cxl_dport *parent_dport); struct cxl_root *devm_cxl_add_root(struct device *host, const struct cxl_root_ops *ops); +int devm_cxl_add_endpoint(struct device *host, struct cxl_memdev *cxlmd, + struct cxl_dport *parent_dport); struct cxl_root *find_cxl_root(struct cxl_port *port); =20 DEFINE_FREE(put_cxl_root, struct cxl_root *, if (_T) put_device(&_T->port.= dev)) diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 55883797ab2d..d62931526fd4 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -45,44 +45,6 @@ static int cxl_mem_dpa_show(struct seq_file *file, void = *data) return 0; } =20 -static int devm_cxl_add_endpoint(struct device *host, struct cxl_memdev *c= xlmd, - struct cxl_dport *parent_dport) -{ - struct cxl_port *parent_port =3D parent_dport->port; - struct cxl_port *endpoint, *iter, *down; - int rc; - - /* - * Now that the path to the root is established record all the - * intervening ports in the chain. - */ - for (iter =3D parent_port, down =3D NULL; !is_cxl_root(iter); - down =3D iter, iter =3D to_cxl_port(iter->dev.parent)) { - struct cxl_ep *ep; - - ep =3D cxl_ep_load(iter, cxlmd); - ep->next =3D down; - } - - /* Note: endpoint port component registers are derived from @cxlds */ - endpoint =3D devm_cxl_add_port(host, &cxlmd->dev, CXL_RESOURCE_NONE, - parent_dport); - if (IS_ERR(endpoint)) - return PTR_ERR(endpoint); - - rc =3D cxl_endpoint_autoremove(cxlmd, endpoint); - if (rc) - return rc; - - if (!endpoint->dev.driver) { - dev_err(&cxlmd->dev, "%s failed probe\n", - dev_name(&endpoint->dev)); - return -ENXIO; - } - - return 0; -} - static int cxl_debugfs_poison_inject(void *data, u64 dpa) { struct cxl_memdev *cxlmd =3D data; @@ -275,8 +237,3 @@ MODULE_DESCRIPTION("CXL: Memory Expansion"); MODULE_LICENSE("GPL v2"); MODULE_IMPORT_NS("CXL"); MODULE_ALIAS_CXL(CXL_DEVICE_MEMORY_EXPANDER); -/* - * create_endpoint() wants to validate port driver attach immediately after - * endpoint registration. - */ -MODULE_SOFTDEP("pre: cxl_port"); diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 51c8f2f84717..7937e7e53797 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -156,10 +156,50 @@ static struct cxl_driver cxl_port_driver =3D { .probe =3D cxl_port_probe, .id =3D CXL_DEVICE_PORT, .drv =3D { + .probe_type =3D PROBE_FORCE_SYNCHRONOUS, .dev_groups =3D cxl_port_attribute_groups, }, }; =20 +int devm_cxl_add_endpoint(struct device *host, struct cxl_memdev *cxlmd, + struct cxl_dport *parent_dport) +{ + struct cxl_port *parent_port =3D parent_dport->port; + struct cxl_port *endpoint, *iter, *down; + int rc; + + /* + * Now that the path to the root is established record all the + * intervening ports in the chain. + */ + for (iter =3D parent_port, down =3D NULL; !is_cxl_root(iter); + down =3D iter, iter =3D to_cxl_port(iter->dev.parent)) { + struct cxl_ep *ep; + + ep =3D cxl_ep_load(iter, cxlmd); + ep->next =3D down; + } + + /* Note: endpoint port component registers are derived from @cxlds */ + endpoint =3D devm_cxl_add_port(host, &cxlmd->dev, CXL_RESOURCE_NONE, + parent_dport); + if (IS_ERR(endpoint)) + return PTR_ERR(endpoint); + + rc =3D cxl_endpoint_autoremove(cxlmd, endpoint); + if (rc) + return rc; + + if (!endpoint->dev.driver) { + dev_err(&cxlmd->dev, "%s failed probe\n", + dev_name(&endpoint->dev)); + return -ENXIO; + } + + return 0; +} +EXPORT_SYMBOL_FOR_MODULES(devm_cxl_add_endpoint, "cxl_mem"); + static int __init cxl_port_init(void) { return cxl_driver_register(&cxl_port_driver); --=20 2.51.1 From nobody Sun Feb 8 01:03:30 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E52B72580D1; Thu, 4 Dec 2025 02:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.9 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764814901; cv=none; b=mFu9KjAqhXSifoMfWoLs1RzFEHvZjytCSs0EssMTlwqJHGRBMlKP4Kxb37TiHsnryPSuvaNczvopSzhewdPpZI2pIL5t7rXW87HgIwTpa0lY9b5/j3q01SQu+ZaPTrtfE4SzDbjXwB185jRSt2XYqHG2lN5+m3MK8svJF/7OwsY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764814901; c=relaxed/simple; bh=xjyl30mUFsDDunvEK7Sr/1lpXabunxBNQdKAfCZagm8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OaSJg2irG6PpXLhcBNL0XWWDU1YkHYHWR6Sxy4LReMHfAQ7juI944xevkcz8Pa4axllt0Z23iga9qNr5Ju5s0hHWJWILrvYBlPM4ysn8Ph4FbDmeOiQFw0t3RrME5eAWXIXMXoJzbr1Yp/9hWWJIvI25XT7J5ZWRGtkqJ78Gw8Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=fgd4kThe; arc=none smtp.client-ip=192.198.163.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="fgd4kThe" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1764814900; x=1796350900; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xjyl30mUFsDDunvEK7Sr/1lpXabunxBNQdKAfCZagm8=; b=fgd4kTheE2dLbVafWAPZ/az4hxBCYaeH3hr5+LNvCe2eWitGjM5xc1AX phqlvvqHTiFiyNo9sIU8mEB3CLqkebTZbamsBAyX9fHG8zCTNIgFv0ISt pghWQCyeQXxd9M82MqFEfrHuDC5zlT2G3hw4dkahquF57MTirXIjTdJip E2iLI6WoASDVyyhbJqL0Kqec3SXNTpdPzUJtn+iThyXLYgwbIeJ6k3Kxg TFNtuzG8QjcezhB3i3med9sLN42KckJ8xUCpKp98u95D2JCYP5c6YTg8x IGGbbRHRCXRO8h9CCDh1fTQRoIuh72SNoMD9D3y0M1GmnUxdGIu2j42fo A==; X-CSE-ConnectionGUID: nMdSLmB1Q/qjHoBtMA2vWQ== X-CSE-MsgGUID: pbqZDoGaR0enXp/JrxkVLw== X-IronPort-AV: E=McAfee;i="6800,10657,11631"; a="77508656" X-IronPort-AV: E=Sophos;i="6.20,247,1758610800"; d="scan'208";a="77508656" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Dec 2025 18:21:40 -0800 X-CSE-ConnectionGUID: iCDuahsbSMakzL/Dympfjw== X-CSE-MsgGUID: xn27NTa0R92IAjlzuhjoIQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.20,247,1758610800"; d="scan'208";a="225802556" Received: from dwillia2-desk.jf.intel.com ([10.88.27.145]) by fmviesa001.fm.intel.com with ESMTP; 03 Dec 2025 18:21:36 -0800 From: Dan Williams To: dave.jiang@intel.com Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Smita.KoralahalliChannabasappa@amd.com, alison.schofield@intel.com, terry.bowman@amd.com, alejandro.lucero-palau@amd.com, linux-pci@vger.kernel.org, Jonathan.Cameron@huawei.com, Alejandro Lucero Subject: [PATCH 4/6] cxl/mem: Convert devm_cxl_add_memdev() to scope-based-cleanup Date: Wed, 3 Dec 2025 18:21:34 -0800 Message-ID: <20251204022136.2573521-5-dan.j.williams@intel.com> X-Mailer: git-send-email 2.51.1 In-Reply-To: <20251204022136.2573521-1-dan.j.williams@intel.com> References: <20251204022136.2573521-1-dan.j.williams@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In preparation for adding more setup steps, convert the current implementation to scope-based cleanup. The cxl_memdev_shutdown() is only required after cdev_device_add(). With that moved to a helper function it precludes the need to add scope-based-handler for that cleanup if devm_add_action_or_reset() fails. Cc: Smita Koralahalli Cc: Alejandro Lucero Signed-off-by: Dan Williams Reviewed-by: Alison Schofield Reviewed-by: Dave Jiang Tested-by: Alejandro Lucero --- drivers/cxl/core/memdev.c | 58 +++++++++++++++++++++++---------------- 1 file changed, 34 insertions(+), 24 deletions(-) diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index 7a4153e1c6a7..d0efc9ceda2a 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -1050,6 +1050,33 @@ static const struct file_operations cxl_memdev_fops = =3D { .llseek =3D noop_llseek, }; =20 +/* + * Activate ioctl operations, no cxl_memdev_rwsem manipulation needed as t= his is + * ordered with cdev_add() publishing the device. + */ +static int cxlmd_add(struct cxl_memdev *cxlmd, struct cxl_dev_state *cxlds) +{ + int rc; + + cxlmd->cxlds =3D cxlds; + cxlds->cxlmd =3D cxlmd; + + rc =3D cdev_device_add(&cxlmd->cdev, &cxlmd->dev); + if (rc) { + /* + * The cdev was briefly live, shutdown any ioctl operations that + * saw that state. + */ + cxl_memdev_shutdown(&cxlmd->dev); + return rc; + } + + return 0; +} + +DEFINE_FREE(put_cxlmd, struct cxl_memdev *, + if (!IS_ERR_OR_NULL(_T)) put_device(&_T->dev);) + /* * Core helper for devm_cxl_add_memdev() that wants to both create a devic= e and * assert to the caller that upon return cxl_mem::probe() has been invoked. @@ -1057,45 +1084,28 @@ static const struct file_operations cxl_memdev_fops= =3D { struct cxl_memdev *__devm_cxl_add_memdev(struct device *host, struct cxl_dev_state *cxlds) { - struct cxl_memdev *cxlmd; struct device *dev; - struct cdev *cdev; int rc; =20 - cxlmd =3D cxl_memdev_alloc(cxlds, &cxl_memdev_fops); + struct cxl_memdev *cxlmd __free(put_cxlmd) =3D + cxl_memdev_alloc(cxlds, &cxl_memdev_fops); if (IS_ERR(cxlmd)) return cxlmd; =20 dev =3D &cxlmd->dev; rc =3D dev_set_name(dev, "mem%d", cxlmd->id); if (rc) - goto err; - - /* - * Activate ioctl operations, no cxl_memdev_rwsem manipulation - * needed as this is ordered with cdev_add() publishing the device. - */ - cxlmd->cxlds =3D cxlds; - cxlds->cxlmd =3D cxlmd; + return ERR_PTR(rc); =20 - cdev =3D &cxlmd->cdev; - rc =3D cdev_device_add(cdev, dev); + rc =3D cxlmd_add(cxlmd, cxlds); if (rc) - goto err; + return ERR_PTR(rc); =20 - rc =3D devm_add_action_or_reset(host, cxl_memdev_unregister, cxlmd); + rc =3D devm_add_action_or_reset(host, cxl_memdev_unregister, + no_free_ptr(cxlmd)); if (rc) return ERR_PTR(rc); return cxlmd; - -err: - /* - * The cdev was briefly live, shutdown any ioctl operations that - * saw that state. - */ - cxl_memdev_shutdown(dev); - put_device(dev); - return ERR_PTR(rc); } EXPORT_SYMBOL_FOR_MODULES(__devm_cxl_add_memdev, "cxl_mem"); =20 --=20 2.51.1 From nobody Sun Feb 8 01:03:30 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ACB2018D636; Thu, 4 Dec 2025 02:21:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.9 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764814905; cv=none; b=OJiwWhvFTnJY7sSRdZgdSCOs4ZBYRGM+a3VXU1AkIWC5luBkBrSovhfO63Oavxmj2K5vSFDLoqkhvFfcoR8H5HnhFjebn/tbxUFKrcAXG/AR7kRL3BoBVbpkoJ8qaPjzb/0U/rjprFBJS0uhgO7CQ5d/nvNvOZmXx4t6Xg9CU2Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764814905; c=relaxed/simple; bh=wqbpmCjeg3EVye2mFzccYeKbLMXEYvrRQ0gh9sxI6HA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pg7kl37AB2Vs/HNc3rjfX2hrVmZ5liBnqpdyS9EV+bRxA7H7Po46vBazjVH3TzcRgbDRX691wEdlTOoFrjsfFdb9fyfAa+5sBKOcdgynvRqXLXm/Nxg/Xf2YSDl0NSnZ4RFZNuLy7wT6v0+u4XVxEyVrNA2R1A4CW1J24Nt2K0g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=MGPiy0M/; arc=none smtp.client-ip=192.198.163.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="MGPiy0M/" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1764814904; x=1796350904; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=wqbpmCjeg3EVye2mFzccYeKbLMXEYvrRQ0gh9sxI6HA=; b=MGPiy0M/M4Q0fzDdd7lPRPv9UqEbO241nbzzpL5x3AAOj2HLBlF7iUtF KGZaj4eClsI7VSAq+6miMOVOH4DmAy5GmGW91B69MNgE/F7781IeyrO7O 33dBW/XdfsHAej1SsBFJcofkw/+2jP5p1IgEUGaAzEBq33Lqclx4umCWg uCBTzljMjwUdZOyFeVg99kAOIx6Yb3CNqzrcjS3p6tNVh3Yvo+RN7Jpnx hGq+SZgfDklaJrnSpK0WJUDU693N1Ir5Gb1xMn600DwjgtxQ3SR6JiBrR /NTEzHE9pnNUCve1iCZ9JOGzOIACQ4BWoRR0u72ypw6oJXE3b1+kumWdr A==; X-CSE-ConnectionGUID: GVojQCzhQFGK0KwpNymcyg== X-CSE-MsgGUID: N3fVe7mxT8uHSrGcZzgnxQ== X-IronPort-AV: E=McAfee;i="6800,10657,11631"; a="77508661" X-IronPort-AV: E=Sophos;i="6.20,247,1758610800"; d="scan'208";a="77508661" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Dec 2025 18:21:43 -0800 X-CSE-ConnectionGUID: TCTCj6HfRnCIWeNcsGk+iQ== X-CSE-MsgGUID: e/+og3qTREqcp0W1i93A+Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.20,247,1758610800"; d="scan'208";a="225802564" Received: from dwillia2-desk.jf.intel.com ([10.88.27.145]) by fmviesa001.fm.intel.com with ESMTP; 03 Dec 2025 18:21:40 -0800 From: Dan Williams To: dave.jiang@intel.com Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Smita.KoralahalliChannabasappa@amd.com, alison.schofield@intel.com, terry.bowman@amd.com, alejandro.lucero-palau@amd.com, linux-pci@vger.kernel.org, Jonathan.Cameron@huawei.com, Alejandro Lucero Subject: [PATCH 5/6] cxl/mem: Drop @host argument to devm_cxl_add_memdev() Date: Wed, 3 Dec 2025 18:21:35 -0800 Message-ID: <20251204022136.2573521-6-dan.j.williams@intel.com> X-Mailer: git-send-email 2.51.1 In-Reply-To: <20251204022136.2573521-1-dan.j.williams@intel.com> References: <20251204022136.2573521-1-dan.j.williams@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In all cases the device that created the 'struct cxl_dev_state' instance is also the device to host the devm cleanup of devm_cxl_add_memdev(). This simplifies the function prototype, and limits a degree of freedom of the API. Cc: Smita Koralahalli Cc: Alejandro Lucero Signed-off-by: Dan Williams Reviewed-by: Alison Schofield Reviewed-by: Ben Cheatham Reviewed-by: Dave Jiang Reviewed-by: Jonathan Cameron Tested-by: Alejandro Lucero --- drivers/cxl/cxlmem.h | 6 ++---- drivers/cxl/core/memdev.c | 5 ++--- drivers/cxl/mem.c | 9 +++++---- drivers/cxl/pci.c | 2 +- tools/testing/cxl/test/mem.c | 2 +- 5 files changed, 11 insertions(+), 13 deletions(-) diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 012e68acad34..9db31c7993c4 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -95,10 +95,8 @@ static inline bool is_cxl_endpoint(struct cxl_port *port) return is_cxl_memdev(port->uport_dev); } =20 -struct cxl_memdev *__devm_cxl_add_memdev(struct device *host, - struct cxl_dev_state *cxlds); -struct cxl_memdev *devm_cxl_add_memdev(struct device *host, - struct cxl_dev_state *cxlds); +struct cxl_memdev *__devm_cxl_add_memdev(struct cxl_dev_state *cxlds); +struct cxl_memdev *devm_cxl_add_memdev(struct cxl_dev_state *cxlds); int devm_cxl_sanitize_setup_notifier(struct device *host, struct cxl_memdev *cxlmd); struct cxl_memdev_state; diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index d0efc9ceda2a..7d85ba25835a 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -1081,8 +1081,7 @@ DEFINE_FREE(put_cxlmd, struct cxl_memdev *, * Core helper for devm_cxl_add_memdev() that wants to both create a devic= e and * assert to the caller that upon return cxl_mem::probe() has been invoked. */ -struct cxl_memdev *__devm_cxl_add_memdev(struct device *host, - struct cxl_dev_state *cxlds) +struct cxl_memdev *__devm_cxl_add_memdev(struct cxl_dev_state *cxlds) { struct device *dev; int rc; @@ -1101,7 +1100,7 @@ struct cxl_memdev *__devm_cxl_add_memdev(struct devic= e *host, if (rc) return ERR_PTR(rc); =20 - rc =3D devm_add_action_or_reset(host, cxl_memdev_unregister, + rc =3D devm_add_action_or_reset(cxlds->dev, cxl_memdev_unregister, no_free_ptr(cxlmd)); if (rc) return ERR_PTR(rc); diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index d62931526fd4..677996c65272 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -165,17 +165,18 @@ static int cxl_mem_probe(struct device *dev) =20 /** * devm_cxl_add_memdev - Add a CXL memory device - * @host: devres alloc/release context and parent for the memdev * @cxlds: CXL device state to associate with the memdev * * Upon return the device will have had a chance to attach to the * cxl_mem driver, but may fail if the CXL topology is not ready * (hardware CXL link down, or software platform CXL root not attached) + * + * The parent of the resulting device and the devm context for allocations= is + * @cxlds->dev. */ -struct cxl_memdev *devm_cxl_add_memdev(struct device *host, - struct cxl_dev_state *cxlds) +struct cxl_memdev *devm_cxl_add_memdev(struct cxl_dev_state *cxlds) { - return __devm_cxl_add_memdev(host, cxlds); + return __devm_cxl_add_memdev(cxlds); } EXPORT_SYMBOL_NS_GPL(devm_cxl_add_memdev, "CXL"); =20 diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 0be4e508affe..1c6fc5334806 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -1006,7 +1006,7 @@ static int cxl_pci_probe(struct pci_dev *pdev, const = struct pci_device_id *id) if (rc) dev_dbg(&pdev->dev, "No CXL Features discovered\n"); =20 - cxlmd =3D devm_cxl_add_memdev(&pdev->dev, cxlds); + cxlmd =3D devm_cxl_add_memdev(cxlds); if (IS_ERR(cxlmd)) return PTR_ERR(cxlmd); =20 diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index 176dcde570cd..8a22b7601627 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -1767,7 +1767,7 @@ static int cxl_mock_mem_probe(struct platform_device = *pdev) =20 cxl_mock_add_event_logs(&mdata->mes); =20 - cxlmd =3D devm_cxl_add_memdev(&pdev->dev, cxlds); + cxlmd =3D devm_cxl_add_memdev(cxlds); if (IS_ERR(cxlmd)) return PTR_ERR(cxlmd); =20 --=20 2.51.1 From nobody Sun Feb 8 01:03:30 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 12445246778; Thu, 4 Dec 2025 02:21:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.9 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764814908; cv=none; b=SKMRqozuOFGH5LxjRQw1W0fd/UQINX4MyMDfmrIoycCFxbESsXVhwQ6dWXGx1YZp3uTQqcBBZ+EU0EgmvKoNQVLFRJTHeJnmeuGOeP+h7UmykmUSJIV7bogF4OQ0JpoKIZvrlTjLsj+6L3HxzJqHOr34OV/KKGOcTassfefUipw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764814908; c=relaxed/simple; bh=dhRnMWJVTTaMotmEjQQ0wdNyIvt6Le1YRaYPaaQCU5c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=srGbzfZyTq7JloyhVYnmINgDOuPkhYgHIsx72ekJlpMIUYrp1Tk2OCHQ0AT5qbGPyXAczORXG4WeGzB9pRoZMwTqd2NJqLYXaqUAzhtt26lv2nGDTFyIsXplpzeOuUju6NjYl7ZZARdpvTH13nTx0e18ikMJi9/hYLWZD7lxRsA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=XJ5OKXi9; arc=none smtp.client-ip=192.198.163.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="XJ5OKXi9" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1764814907; x=1796350907; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dhRnMWJVTTaMotmEjQQ0wdNyIvt6Le1YRaYPaaQCU5c=; b=XJ5OKXi9Hh0Xc0BVeXspa7QcoyGEFgsEUXx054m1rcQsHqpHvgHVwABw lIbT9PmsvnEBre8noyNrweWdBYd/t7JOnu7mBWiuGhlIUJa49lDMpk78b tmnyfoq5uKRq0iik/jtWq4SEIrJxjdFNiYyjvJ+79+0yJCRDubT0QEjQh zPlohkyzeprThnnDpuulEnZAJ4THDS6640BlLS/zT//7m1eQRL5A/LEDf am2rEzLPIQ7zgEvB5XWpe6Aqbflr9ZkkqpsBz91jbbryvreBOwyepij02 XghWmECgEDUYnI4Y5WJYrPCY0CjafN6Z+yf4MU+7lSas04+BWymX0ruiC Q==; X-CSE-ConnectionGUID: +JDBXtgRQS2WO4hOSycV4g== X-CSE-MsgGUID: chkHcNE6Tiu4rTaUV6Qnyg== X-IronPort-AV: E=McAfee;i="6800,10657,11631"; a="77508665" X-IronPort-AV: E=Sophos;i="6.20,247,1758610800"; d="scan'208";a="77508665" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Dec 2025 18:21:46 -0800 X-CSE-ConnectionGUID: oexLYP4lRJyiD2MrUBSLrg== X-CSE-MsgGUID: IKWK40WqQmiqkJUwCO65NQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.20,247,1758610800"; d="scan'208";a="225802573" Received: from dwillia2-desk.jf.intel.com ([10.88.27.145]) by fmviesa001.fm.intel.com with ESMTP; 03 Dec 2025 18:21:43 -0800 From: Dan Williams To: dave.jiang@intel.com Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Smita.KoralahalliChannabasappa@amd.com, alison.schofield@intel.com, terry.bowman@amd.com, alejandro.lucero-palau@amd.com, linux-pci@vger.kernel.org, Jonathan.Cameron@huawei.com, Alejandro Lucero Subject: [PATCH 6/6] cxl/mem: Introduce a memdev creation ->probe() operation Date: Wed, 3 Dec 2025 18:21:36 -0800 Message-ID: <20251204022136.2573521-7-dan.j.williams@intel.com> X-Mailer: git-send-email 2.51.1 In-Reply-To: <20251204022136.2573521-1-dan.j.williams@intel.com> References: <20251204022136.2573521-1-dan.j.williams@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Allow for a driver to pass a routine to be called in cxl_mem_probe() context. This ability is inspired by and mirrors the semantics of faux_device_create(). It allows for the caller to run CXL-topology attach-dependent logic on behalf of the caller. This capability is needed for CXL accelerator device drivers that need to make decisions about enabling CXL dependent functionality in the device, or falling back to PCIe-only operation. The probe callback runs after the port topology is successfully attached for the given memdev. Cc: Smita Koralahalli Cc: Alejandro Lucero Signed-off-by: Dan Williams Reviewed-by: Dave Jiang Tested-by: Alejandro Lucero --- drivers/cxl/cxlmem.h | 12 ++++++++-- drivers/cxl/core/memdev.c | 46 +++++++++++++++++++++++++++++------- drivers/cxl/mem.c | 12 ++++++++-- drivers/cxl/pci.c | 2 +- tools/testing/cxl/test/mem.c | 2 +- 5 files changed, 60 insertions(+), 14 deletions(-) diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 9db31c7993c4..ed759f88b21f 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -34,6 +34,10 @@ (FIELD_GET(CXLMDEV_RESET_NEEDED_MASK, status) !=3D \ CXLMDEV_RESET_NEEDED_NOT) =20 +struct cxl_memdev_ops { + int (*probe)(struct cxl_memdev *cxlmd); +}; + /** * struct cxl_memdev - CXL bus object representing a Type-3 Memory Device * @dev: driver core device object @@ -43,6 +47,7 @@ * @cxl_nvb: coordinate removal of @cxl_nvd if present * @cxl_nvd: optional bridge to an nvdimm if the device supports pmem * @endpoint: connection to the CXL port topology for this memory device + * @ops: incremental caller specific probe routine * @id: id number of this memdev instance. * @depth: endpoint port depth * @scrub_cycle: current scrub cycle set for this device @@ -59,6 +64,7 @@ struct cxl_memdev { struct cxl_nvdimm_bridge *cxl_nvb; struct cxl_nvdimm *cxl_nvd; struct cxl_port *endpoint; + const struct cxl_memdev_ops *ops; int id; int depth; u8 scrub_cycle; @@ -95,8 +101,10 @@ static inline bool is_cxl_endpoint(struct cxl_port *por= t) return is_cxl_memdev(port->uport_dev); } =20 -struct cxl_memdev *__devm_cxl_add_memdev(struct cxl_dev_state *cxlds); -struct cxl_memdev *devm_cxl_add_memdev(struct cxl_dev_state *cxlds); +struct cxl_memdev *__devm_cxl_add_memdev(struct cxl_dev_state *cxlds, + const struct cxl_memdev_ops *ops); +struct cxl_memdev *devm_cxl_add_memdev(struct cxl_dev_state *cxlds, + const struct cxl_memdev_ops *ops); int devm_cxl_sanitize_setup_notifier(struct device *host, struct cxl_memdev *cxlmd); struct cxl_memdev_state; diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index 7d85ba25835a..41f1964d8cbf 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -648,7 +648,8 @@ static void detach_memdev(struct work_struct *work) static struct lock_class_key cxl_memdev_key; =20 static struct cxl_memdev *cxl_memdev_alloc(struct cxl_dev_state *cxlds, - const struct file_operations *fops) + const struct file_operations *fops, + const struct cxl_memdev_ops *ops) { struct cxl_memdev *cxlmd; struct device *dev; @@ -664,6 +665,8 @@ static struct cxl_memdev *cxl_memdev_alloc(struct cxl_d= ev_state *cxlds, goto err; cxlmd->id =3D rc; cxlmd->depth =3D -1; + cxlmd->ops =3D ops; + cxlmd->endpoint =3D ERR_PTR(-ENXIO); =20 dev =3D &cxlmd->dev; device_initialize(dev); @@ -1077,17 +1080,48 @@ static int cxlmd_add(struct cxl_memdev *cxlmd, stru= ct cxl_dev_state *cxlds) DEFINE_FREE(put_cxlmd, struct cxl_memdev *, if (!IS_ERR_OR_NULL(_T)) put_device(&_T->dev);) =20 +static struct cxl_memdev *cxl_memdev_autoremove(struct cxl_memdev *cxlmd) +{ + struct cxl_memdev *ret =3D cxlmd; + int rc; + + /* + * If ops is provided fail if the driver is not attached upon + * return. The ->endpoint ERR_PTR may have a more precise error + * code to convey. Note that failure here could be the result of + * a race to teardown the CXL port topology. I.e. + * cxl_mem_probe() could have succeeded and then cxl_mem unbound + * before the lock is acquired. + */ + guard(device)(&cxlmd->dev); + if (cxlmd->ops && !cxlmd->dev.driver) { + ret =3D ERR_PTR(-ENXIO); + if (IS_ERR(cxlmd->endpoint)) + ret =3D ERR_CAST(cxlmd->endpoint); + cxl_memdev_unregister(cxlmd); + return ret; + } + + rc =3D devm_add_action_or_reset(cxlmd->cxlds->dev, cxl_memdev_unregister, + cxlmd); + if (rc) + return ERR_PTR(rc); + + return ret; +} + /* * Core helper for devm_cxl_add_memdev() that wants to both create a devic= e and * assert to the caller that upon return cxl_mem::probe() has been invoked. */ -struct cxl_memdev *__devm_cxl_add_memdev(struct cxl_dev_state *cxlds) +struct cxl_memdev *__devm_cxl_add_memdev(struct cxl_dev_state *cxlds, + const struct cxl_memdev_ops *ops) { struct device *dev; int rc; =20 struct cxl_memdev *cxlmd __free(put_cxlmd) =3D - cxl_memdev_alloc(cxlds, &cxl_memdev_fops); + cxl_memdev_alloc(cxlds, &cxl_memdev_fops, ops); if (IS_ERR(cxlmd)) return cxlmd; =20 @@ -1100,11 +1134,7 @@ struct cxl_memdev *__devm_cxl_add_memdev(struct cxl_= dev_state *cxlds) if (rc) return ERR_PTR(rc); =20 - rc =3D devm_add_action_or_reset(cxlds->dev, cxl_memdev_unregister, - no_free_ptr(cxlmd)); - if (rc) - return ERR_PTR(rc); - return cxlmd; + return cxl_memdev_autoremove(no_free_ptr(cxlmd)); } EXPORT_SYMBOL_FOR_MODULES(__devm_cxl_add_memdev, "cxl_mem"); =20 diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 677996c65272..d61f121c6f97 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -142,6 +142,12 @@ static int cxl_mem_probe(struct device *dev) return rc; } =20 + if (cxlmd->ops) { + rc =3D cxlmd->ops->probe(cxlmd); + if (rc) + return rc; + } + rc =3D devm_cxl_memdev_edac_register(cxlmd); if (rc) dev_dbg(dev, "CXL memdev EDAC registration failed rc=3D%d\n", rc); @@ -166,6 +172,7 @@ static int cxl_mem_probe(struct device *dev) /** * devm_cxl_add_memdev - Add a CXL memory device * @cxlds: CXL device state to associate with the memdev + * @ops: optional operations to run in cxl_mem::{probe,remove}() context * * Upon return the device will have had a chance to attach to the * cxl_mem driver, but may fail if the CXL topology is not ready @@ -174,9 +181,10 @@ static int cxl_mem_probe(struct device *dev) * The parent of the resulting device and the devm context for allocations= is * @cxlds->dev. */ -struct cxl_memdev *devm_cxl_add_memdev(struct cxl_dev_state *cxlds) +struct cxl_memdev *devm_cxl_add_memdev(struct cxl_dev_state *cxlds, + const struct cxl_memdev_ops *ops) { - return __devm_cxl_add_memdev(cxlds); + return __devm_cxl_add_memdev(cxlds, ops); } EXPORT_SYMBOL_NS_GPL(devm_cxl_add_memdev, "CXL"); =20 diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 1c6fc5334806..549368a9c868 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -1006,7 +1006,7 @@ static int cxl_pci_probe(struct pci_dev *pdev, const = struct pci_device_id *id) if (rc) dev_dbg(&pdev->dev, "No CXL Features discovered\n"); =20 - cxlmd =3D devm_cxl_add_memdev(cxlds); + cxlmd =3D devm_cxl_add_memdev(cxlds, NULL); if (IS_ERR(cxlmd)) return PTR_ERR(cxlmd); =20 diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index 8a22b7601627..cb87e8c0e63c 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -1767,7 +1767,7 @@ static int cxl_mock_mem_probe(struct platform_device = *pdev) =20 cxl_mock_add_event_logs(&mdata->mes); =20 - cxlmd =3D devm_cxl_add_memdev(cxlds); + cxlmd =3D devm_cxl_add_memdev(cxlds, NULL); if (IS_ERR(cxlmd)) return PTR_ERR(cxlmd); =20 --=20 2.51.1