[PATCH v3 0/5] dax/hmem, cxl: Coordinate Soft Reserved handling with CXL

Smita Koralahalli posted 5 patches 4 months, 1 week ago
arch/x86/kernel/e820.c    |   2 +-
drivers/cxl/acpi.c        |   2 +-
drivers/dax/Kconfig       |   2 +
drivers/dax/hmem/device.c |   4 +-
drivers/dax/hmem/hmem.c   | 128 ++++++++++++++++++++++++++++++++++----
include/linux/ioport.h    |  13 +++-
kernel/resource.c         |  92 +++++++++++++++++++++++----
7 files changed, 213 insertions(+), 30 deletions(-)
[PATCH v3 0/5] dax/hmem, cxl: Coordinate Soft Reserved handling with CXL
Posted by Smita Koralahalli 4 months, 1 week ago
This series aims to address long-standing conflicts between dax_hmem and
CXL when handling Soft Reserved memory ranges.

Reworked from Dan's patch:
https://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git/patch/?id=ab70c6227ee6165a562c215d9dcb4a1c55620d5d

Previous work:
https://lore.kernel.org/all/20250715180407.47426-1-Smita.KoralahalliChannabasappa@amd.com/

Link to v2:
https://lore.kernel.org/all/20250930042814.213912-1-Smita.KoralahalliChannabasappa@amd.com/

To note:
I have dropped 6/6th patch from v1 based on discussion with Zhijian. This
patch series doesn't cover the case of DEV_DAX_HMEM=y and CXL=m, which
results in DEV_DAX_CXL being disabled.

In that configuration, ownership of the soft-reserved ranges is handled by
HMEM instead of being managed by CXL.

/proc/iomem for this case looks like below:

850000000-284fffffff : CXL Window 0
  850000000-284fffffff : region3
    850000000-284fffffff : Soft Reserved
      850000000-284fffffff : dax0.0
        850000000-284fffffff : System RAM (kmem)
2850000000-484fffffff : CXL Window 1
  2850000000-484fffffff : region4
    2850000000-484fffffff : Soft Reserved
      2850000000-484fffffff : dax1.0
        2850000000-484fffffff : System RAM (kmem)
4850000000-684fffffff : CXL Window 2
  4850000000-684fffffff : region5
    4850000000-684fffffff : Soft Reserved
      4850000000-684fffffff : dax2.0
        4850000000-684fffffff : System RAM (kmem)

Link to the patch and discussions on this:
https://lore.kernel.org/all/20250822034202.26896-7-Smita.KoralahalliChannabasappa@amd.com/

I would appreciate input on how best to handle this scenario efficiently.

Applies to mainline master.

v3 updates:
 - Fixed two "From".

v2 updates:
 - Removed conditional check on CONFIG_EFI_SOFT_RESERVE as dax_hmem
   depends on CONFIG_EFI_SOFT_RESERVE. (Zhijian)
 - Added TODO note. (Zhijian)
 - Included region_intersects_soft_reserve() inside CONFIG_EFI_SOFT_RESERVE
   conditional check. (Zhijian)
 - insert_resource_late() -> insert_resource_expand_to_fit() and
   __insert_resource_expand_to_fit() replacement. (Boris)
 - Fixed Co-developed and Signed-off by. (Dan)
 - Combined 2/6 and 3/6 into a single patch. (Zhijian).
 - Skip local variable in remove_soft_reserved. (Jonathan)
 - Drop kfree with __free(). (Jonathan)
 - return 0 -> return dev_add_action_or_reset(host...) (Jonathan)
 - Dropped 6/6.
 - Reviewed-by tags (Dave, Jonathan)

Dan Williams (4):
  dax/hmem, e820, resource: Defer Soft Reserved registration until hmem
    is ready
  dax/hmem: Request cxl_acpi and cxl_pci before walking Soft Reserved
    ranges
  dax/hmem: Use DEV_DAX_CXL instead of CXL_REGION for deferral
  dax/hmem: Defer Soft Reserved overlap handling until CXL region
    assembly completes

Smita Koralahalli (1):
  dax/hmem: Reintroduce Soft Reserved ranges back into the iomem tree

 arch/x86/kernel/e820.c    |   2 +-
 drivers/cxl/acpi.c        |   2 +-
 drivers/dax/Kconfig       |   2 +
 drivers/dax/hmem/device.c |   4 +-
 drivers/dax/hmem/hmem.c   | 128 ++++++++++++++++++++++++++++++++++----
 include/linux/ioport.h    |  13 +++-
 kernel/resource.c         |  92 +++++++++++++++++++++++----
 7 files changed, 213 insertions(+), 30 deletions(-)

-- 
2.17.1
Re: [PATCH v3 0/5] dax/hmem, cxl: Coordinate Soft Reserved handling with CXL
Posted by Alison Schofield 4 months ago
On Tue, Sep 30, 2025 at 04:47:52AM +0000, Smita Koralahalli wrote:
> This series aims to address long-standing conflicts between dax_hmem and
> CXL when handling Soft Reserved memory ranges.

Hi Smita,

Thanks for the updates Smita!

About those "long-standing conflicts": In the next rev, can you resurrect,
or recreate the issues list that this set is addressing. It's been a
long and winding road with several handoffs (me included) and it'll help
keep the focus.

Hotplug works :)  Auto region comes up, we tear it down and can recreate it,
in place, because the soft reserved resource is gone (no longer occupying
the CXL Window and causing recreate to fail.)

!CONFIG_CXL_REGION works :) All resources go directly to DAX.

The scenario that is failing is handoff to DAX after region assembly
failure. (Dan reminded me to check that today.) That is mostly related
to Patch4, so I'll respond there.

--Alison
Re: [PATCH v3 0/5] dax/hmem, cxl: Coordinate Soft Reserved handling with CXL
Posted by Alison Schofield 4 months ago
On Mon, Oct 06, 2025 at 06:16:24PM -0700, Alison Schofield wrote:
> On Tue, Sep 30, 2025 at 04:47:52AM +0000, Smita Koralahalli wrote:
> > This series aims to address long-standing conflicts between dax_hmem and
> > CXL when handling Soft Reserved memory ranges.
> 
> Hi Smita,
> 
> Thanks for the updates Smita!
> 
> About those "long-standing conflicts": In the next rev, can you resurrect,
> or recreate the issues list that this set is addressing. It's been a
> long and winding road with several handoffs (me included) and it'll help
> keep the focus.
> 
> Hotplug works :)  Auto region comes up, we tear it down and can recreate it,
> in place, because the soft reserved resource is gone (no longer occupying
> the CXL Window and causing recreate to fail.)
> 
> !CONFIG_CXL_REGION works :) All resources go directly to DAX.
> 
> The scenario that is failing is handoff to DAX after region assembly
> failure. (Dan reminded me to check that today.) That is mostly related
> to Patch4, so I'll respond there.
> 
> --Alison

Hi Smita -

(after off-list chat w Smita about what is and is not included)

This CXL failover to DAX case is not implemented. In my response in Patch 4,
I cobbled something together that made it work in one test case. But to be
clear, there was some trickery in the CXL region driver to even do that.

One path forward is to update this set restating the issues it addresses, and
remove any code and comments that are tied to failing over to DAX after a
region assembly failure.

That leaves the issue Dan raised, "shutdown CXL in favor of vanilla DAX devices
as an emergency fallback for platform configuration quirks and bugs"[1], for a
future patch.

-- Alison

[1] The failover to DAX was last described in response to v5 of the 'prior' patchset.
https://lore.kernel.org/linux-cxl/20250715180407.47426-1-Smita.KoralahalliChannabasappa@amd.com/
https://lore.kernel.org/linux-cxl/687ffcc0ee1c8_137e6b100ed@dwillia2-xfh.jf.intel.com.notmuch/
https://lore.kernel.org/linux-cxl/68808fb4e4cbf_137e6b100cc@dwillia2-xfh.jf.intel.com.notmuch/

> 
>