[PATCH 0/6] dax/hmem, cxl: Coordinate Soft Reserved handling with CXL

Smita Koralahalli posted 6 patches 1 month, 1 week ago
There is a newer version of this series
arch/x86/kernel/e820.c    |   2 +-
drivers/cxl/core/region.c |   4 +-
drivers/dax/Kconfig       |   3 +
drivers/dax/hmem/device.c |   4 +-
drivers/dax/hmem/hmem.c   | 137 +++++++++++++++++++++++++++++++++++---
include/linux/ioport.h    |  24 +++++++
kernel/resource.c         |  73 +++++++++++++++++---
7 files changed, 222 insertions(+), 25 deletions(-)
[PATCH 0/6] dax/hmem, cxl: Coordinate Soft Reserved handling with CXL
Posted by Smita Koralahalli 1 month, 1 week ago
This series aims to address long-standing conflicts between dax_hmem and
CXL when handling Soft Reserved memory ranges.

I have considered adding support for DAX_CXL_MODE_REGISTER, but I do not
yet have a solid approach. Since this came up in discussion yesterday,
I am sending the current work and would appreciate inputs on how best to
handle the DAX_CXL_MODE_REGISTER case.

Reworked from Dan's patch:
https://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git/patch/?id=ab70c6227ee6165a562c215d9dcb4a1c55620d5d

Previous work:
https://lore.kernel.org/all/20250715180407.47426-1-Smita.KoralahalliChannabasappa@amd.com/

Smita Koralahalli (6):
  dax/hmem, e820, resource: Defer Soft Reserved registration until hmem
    is ready
  dax/hmem: Request cxl_acpi and cxl_pci before walking Soft Reserved
    ranges
  dax/hmem, cxl: Tighten dependencies on DEV_DAX_CXL and dax_hmem
  dax/hmem: Defer Soft Reserved overlap handling until CXL region
    assembly completes
  dax/hmem: Reintroduce Soft Reserved ranges back into the iomem tree
  cxl/region, dax/hmem: Guard CXL DAX region creation and tighten HMEM
    deps

 arch/x86/kernel/e820.c    |   2 +-
 drivers/cxl/core/region.c |   4 +-
 drivers/dax/Kconfig       |   3 +
 drivers/dax/hmem/device.c |   4 +-
 drivers/dax/hmem/hmem.c   | 137 +++++++++++++++++++++++++++++++++++---
 include/linux/ioport.h    |  24 +++++++
 kernel/resource.c         |  73 +++++++++++++++++---
 7 files changed, 222 insertions(+), 25 deletions(-)

-- 
2.17.1
Re: [PATCH 0/6] dax/hmem, cxl: Coordinate Soft Reserved handling with CXL
Posted by Alison Schofield 1 month, 1 week ago
On Fri, Aug 22, 2025 at 03:41:56AM +0000, Smita Koralahalli wrote:
> This series aims to address long-standing conflicts between dax_hmem and
> CXL when handling Soft Reserved memory ranges.

Hi Smita,

I was able to try this out today and it looks good. See one question
about the !CXL_REGION case below.

Test case of a hot replace a dax region worked as expected. It appeared
with no soft reserved and after tear down, the same region could be
rebuilt in place.

Test case with CONFIG_CXL_REGION=N looks good too, as in DAX consumed
the entire resource. Do we intend the Soft Reserved resource to remain
like this:
c080000000-17dbfffffff : CXL Window 0
  c080000000-c47fffffff : Soft Reserved
    c080000000-c47fffffff : dax2.0
      c080000000-c47fffffff : System RAM (kmem)

These other issues noted previously did not re-appear:
- kmem dax3.0: probe with driver kmem failed with error -16
- resource: Unaddressable device  [ ] conflicts with []

-- Alison

snip
Re: [PATCH 0/6] dax/hmem, cxl: Coordinate Soft Reserved handling with CXL
Posted by Koralahalli Channabasappa, Smita 1 month ago
Hi Alison,

On 8/26/2025 4:21 PM, Alison Schofield wrote:
> On Fri, Aug 22, 2025 at 03:41:56AM +0000, Smita Koralahalli wrote:
>> This series aims to address long-standing conflicts between dax_hmem and
>> CXL when handling Soft Reserved memory ranges.
> 
> Hi Smita,
> 
> I was able to try this out today and it looks good. See one question
> about the !CXL_REGION case below.
> 
> Test case of a hot replace a dax region worked as expected. It appeared
> with no soft reserved and after tear down, the same region could be
> rebuilt in place.
> 
> Test case with CONFIG_CXL_REGION=N looks good too, as in DAX consumed
> the entire resource. Do we intend the Soft Reserved resource to remain
> like this:
> c080000000-17dbfffffff : CXL Window 0
>    c080000000-c47fffffff : Soft Reserved
>      c080000000-c47fffffff : dax2.0
>        c080000000-c47fffffff : System RAM (kmem)

Yes, that is how it currently looks. Maybe we should also add a log 
message to make it clear that this dax is coming from dax_hmem and not 
dax_cxl?

Another thought I had is that if we hand off fully to CXL even with 
regions disabled, we could avoid showing the Soft Reserved layer 
entirely (along with the kmem and devdax under it). The question is 
whether that approach would be preferable, since in that case the memory 
would end up left unclaimed/unavailable to Linux. Would be good to get 
your perspective on this.

https://lore.kernel.org/all/a2e900b0-1b89-4e88-a6d4-8c0e6de50f52@amd.com/

Thanks
Smita

> 
> These other issues noted previously did not re-appear:
> - kmem dax3.0: probe with driver kmem failed with error -16
> - resource: Unaddressable device  [ ] conflicts with []
> 
> -- Alison
> 
> snip
>