[v2] Add CXL LSA 2.1 format support in nvdimm and cxl pmem

[PATCH V2 01/20] nvdimm/label: Introduce NDD_CXL_LABEL flag to set cxl label format

Posted by Neeraj Kumar 6 months, 2 weeks ago

Prior to LSA 2.1 version, LSA contain only namespace labels. LSA 2.1
introduced in CXL 2.0 Spec, which contain region label along with
namespace label.

NDD_LABELING flag is used for namespace. Introduced NDD_CXL_LABEL
flag for region label. Based on these flags nvdimm driver performs
operation on namespace label or region label.

NDD_CXL_LABEL will be utilized by cxl driver to enable LSA2.1 region
label support

Accordingly updated label index version

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
---
 drivers/nvdimm/dimm.c      |  1 +
 drivers/nvdimm/dimm_devs.c |  7 +++++++
 drivers/nvdimm/label.c     | 22 ++++++++++++++++++----
 drivers/nvdimm/nd.h        |  1 +
 include/linux/libnvdimm.h  |  3 +++
 5 files changed, 30 insertions(+), 4 deletions(-)

diff --git a/drivers/nvdimm/dimm.c b/drivers/nvdimm/dimm.c
index 91d9163ee303..8753b5cd91cc 100644
--- a/drivers/nvdimm/dimm.c
+++ b/drivers/nvdimm/dimm.c
@@ -62,6 +62,7 @@ static int nvdimm_probe(struct device *dev)
 	if (rc < 0)
 		dev_dbg(dev, "failed to unlock dimm: %d\n", rc);
 
+	ndd->cxl = nvdimm_check_cxl_label_format(ndd->dev);
 
 	/*
 	 * EACCES failures reading the namespace label-area-properties
diff --git a/drivers/nvdimm/dimm_devs.c b/drivers/nvdimm/dimm_devs.c
index 21498d461fde..6149770c1b27 100644
--- a/drivers/nvdimm/dimm_devs.c
+++ b/drivers/nvdimm/dimm_devs.c
@@ -18,6 +18,13 @@
 
 static DEFINE_IDA(dimm_ida);
 
+bool nvdimm_check_cxl_label_format(struct device *dev)
+{
+	struct nvdimm *nvdimm = to_nvdimm(dev);
+
+	return test_bit(NDD_CXL_LABEL, &nvdimm->flags);
+}
+
 /*
  * Retrieve bus and dimm handle and return if this bus supports
  * get_config_data commands
diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
index 04f4a049599a..7a011ee02d79 100644
--- a/drivers/nvdimm/label.c
+++ b/drivers/nvdimm/label.c
@@ -688,11 +688,25 @@ static int nd_label_write_index(struct nvdimm_drvdata *ndd, int index, u32 seq,
 		- (unsigned long) to_namespace_index(ndd, 0);
 	nsindex->labeloff = __cpu_to_le64(offset);
 	nsindex->nslot = __cpu_to_le32(nslot);
-	nsindex->major = __cpu_to_le16(1);
-	if (sizeof_namespace_label(ndd) < 256)
+
+	/* Set LSA Label Index Version */
+	if (ndd->cxl) {
+		/* CXL r3.2 Spec: Table 9-9 Label Index Block Layout */
+		nsindex->major = __cpu_to_le16(2);
 		nsindex->minor = __cpu_to_le16(1);
-	else
-		nsindex->minor = __cpu_to_le16(2);
+	} else {
+		nsindex->major = __cpu_to_le16(1);
+		/*
+		 * NVDIMM Namespace Specification
+		 * Table 2: Namespace Label Index Block Fields
+		 */
+		if (sizeof_namespace_label(ndd) < 256)
+			nsindex->minor = __cpu_to_le16(1);
+		else
+		 /* UEFI Specification 2.7: Label Index Block Definitions */
+			nsindex->minor = __cpu_to_le16(2);
+	}
+
 	nsindex->checksum = __cpu_to_le64(0);
 	if (flags & ND_NSINDEX_INIT) {
 		unsigned long *free = (unsigned long *) nsindex->free;
diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
index cc5c8f3f81e8..1cc06cc58d14 100644
--- a/drivers/nvdimm/nd.h
+++ b/drivers/nvdimm/nd.h
@@ -522,6 +522,7 @@ void nvdimm_set_labeling(struct device *dev);
 void nvdimm_set_locked(struct device *dev);
 void nvdimm_clear_locked(struct device *dev);
 int nvdimm_security_setup_events(struct device *dev);
+bool nvdimm_check_cxl_label_format(struct device *dev);
 #if IS_ENABLED(CONFIG_NVDIMM_KEYS)
 int nvdimm_security_unlock(struct device *dev);
 #else
diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h
index e772aae71843..0a55900842c8 100644
--- a/include/linux/libnvdimm.h
+++ b/include/linux/libnvdimm.h
@@ -44,6 +44,9 @@ enum {
 	/* dimm provider wants synchronous registration by __nvdimm_create() */
 	NDD_REGISTER_SYNC = 8,
 
+	/* dimm supports region labels (LSA Format 2.1) */
+	NDD_CXL_LABEL = 9,
+
 	/* need to set a limit somewhere, but yes, this is likely overkill */
 	ND_IOCTL_MAX_BUFLEN = SZ_4M,
 	ND_CMD_MAX_ELEM = 5,
-- 
2.34.1

Re: [PATCH V2 01/20] nvdimm/label: Introduce NDD_CXL_LABEL flag to set cxl label format

Posted by Jonathan Cameron 6 months ago

On Wed, 30 Jul 2025 17:41:50 +0530
Neeraj Kumar <s.neeraj@samsung.com> wrote:

> Prior to LSA 2.1 version, LSA contain only namespace labels. LSA 2.1
> introduced in CXL 2.0 Spec, which contain region label along with
> namespace label.
> 
> NDD_LABELING flag is used for namespace. Introduced NDD_CXL_LABEL
> flag for region label. Based on these flags nvdimm driver performs
> operation on namespace label or region label.
> 
> NDD_CXL_LABEL will be utilized by cxl driver to enable LSA2.1 region
> label support
> 
> Accordingly updated label index version
> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
Hi Neeraj,

A few comments inline.

> diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
> index 04f4a049599a..7a011ee02d79 100644
> --- a/drivers/nvdimm/label.c
> +++ b/drivers/nvdimm/label.c
> @@ -688,11 +688,25 @@ static int nd_label_write_index(struct nvdimm_drvdata *ndd, int index, u32 seq,
>  		- (unsigned long) to_namespace_index(ndd, 0);
>  	nsindex->labeloff = __cpu_to_le64(offset);
>  	nsindex->nslot = __cpu_to_le32(nslot);
> -	nsindex->major = __cpu_to_le16(1);
> -	if (sizeof_namespace_label(ndd) < 256)
> +
> +	/* Set LSA Label Index Version */
> +	if (ndd->cxl) {
> +		/* CXL r3.2 Spec: Table 9-9 Label Index Block Layout */
> +		nsindex->major = __cpu_to_le16(2);
>  		nsindex->minor = __cpu_to_le16(1);
> -	else
> -		nsindex->minor = __cpu_to_le16(2);
> +	} else {
> +		nsindex->major = __cpu_to_le16(1);
> +		/*
> +		 * NVDIMM Namespace Specification
> +		 * Table 2: Namespace Label Index Block Fields
> +		 */
> +		if (sizeof_namespace_label(ndd) < 256)
> +			nsindex->minor = __cpu_to_le16(1);
> +		else
> +		 /* UEFI Specification 2.7: Label Index Block Definitions */

Odd comment alignment. Either put it on the else so
		else /* UEFI 2.7: Label Index Block Defintions */

or indent it an extra tab

		else
			/* UEFI 2.7: Label Index Block Definitions */
			
> +			nsindex->minor = __cpu_to_le16(2);
> +	}
> +
>  	nsindex->checksum = __cpu_to_le64(0);
>  	if (flags & ND_NSINDEX_INIT) {
>  		unsigned long *free = (unsigned long *) nsindex->free;

> diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h
> index e772aae71843..0a55900842c8 100644
> --- a/include/linux/libnvdimm.h
> +++ b/include/linux/libnvdimm.h
> @@ -44,6 +44,9 @@ enum {
>  	/* dimm provider wants synchronous registration by __nvdimm_create() */
>  	NDD_REGISTER_SYNC = 8,
>  
> +	/* dimm supports region labels (LSA Format 2.1) */
> +	NDD_CXL_LABEL = 9,

This enum is 'curious'.  It combined flags from a bunch of different
flags fields and some stuff that are nothing to do with flags.

Anyhow, putting that aside I'd either rename it to something like
NDD_REGION_LABELING (similar to NDD_LABELING that is there for namespace labels
or just have it a meaning it is LSA Format 2.1 and drop the fact htat
also means region labels are supported.

Combination of a comment that talks about one thing and a definition name
that doesn't associate with it seems confusing to me.

Jonathan


> +
>  	/* need to set a limit somewhere, but yes, this is likely overkill */
>  	ND_IOCTL_MAX_BUFLEN = SZ_4M,
>  	ND_CMD_MAX_ELEM = 5,

Re: [PATCH V2 01/20] nvdimm/label: Introduce NDD_CXL_LABEL flag to set cxl label format

Posted by Neeraj Kumar 5 months, 1 week ago

On 13/08/25 02:12PM, Jonathan Cameron wrote:
>On Wed, 30 Jul 2025 17:41:50 +0530
>Neeraj Kumar <s.neeraj@samsung.com> wrote:
>
>> Prior to LSA 2.1 version, LSA contain only namespace labels. LSA 2.1
>> introduced in CXL 2.0 Spec, which contain region label along with
>> namespace label.
>>
>> NDD_LABELING flag is used for namespace. Introduced NDD_CXL_LABEL
>> flag for region label. Based on these flags nvdimm driver performs
>> operation on namespace label or region label.
>>
>> NDD_CXL_LABEL will be utilized by cxl driver to enable LSA2.1 region
>> label support
>>
>> Accordingly updated label index version
>>
>> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
>Hi Neeraj,
>
>A few comments inline.
>
>> diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
>> index 04f4a049599a..7a011ee02d79 100644
>> --- a/drivers/nvdimm/label.c
>> +++ b/drivers/nvdimm/label.c
>> @@ -688,11 +688,25 @@ static int nd_label_write_index(struct nvdimm_drvdata *ndd, int index, u32 seq,
>>  		- (unsigned long) to_namespace_index(ndd, 0);
>>  	nsindex->labeloff = __cpu_to_le64(offset);
>>  	nsindex->nslot = __cpu_to_le32(nslot);
>> -	nsindex->major = __cpu_to_le16(1);
>> -	if (sizeof_namespace_label(ndd) < 256)
>> +
>> +	/* Set LSA Label Index Version */
>> +	if (ndd->cxl) {
>> +		/* CXL r3.2 Spec: Table 9-9 Label Index Block Layout */
>> +		nsindex->major = __cpu_to_le16(2);
>>  		nsindex->minor = __cpu_to_le16(1);
>> -	else
>> -		nsindex->minor = __cpu_to_le16(2);
>> +	} else {
>> +		nsindex->major = __cpu_to_le16(1);
>> +		/*
>> +		 * NVDIMM Namespace Specification
>> +		 * Table 2: Namespace Label Index Block Fields
>> +		 */
>> +		if (sizeof_namespace_label(ndd) < 256)
>> +			nsindex->minor = __cpu_to_le16(1);
>> +		else
>> +		 /* UEFI Specification 2.7: Label Index Block Definitions */
>
>Odd comment alignment. Either put it on the else so
>		else /* UEFI 2.7: Label Index Block Defintions */
>
>or indent it an extra tab
>
>		else
>			/* UEFI 2.7: Label Index Block Definitions */
>			

Thanks Jonathan, I will fix it in next patch-set

>> +			nsindex->minor = __cpu_to_le16(2);
>> +	}
>> +
>>  	nsindex->checksum = __cpu_to_le64(0);
>>  	if (flags & ND_NSINDEX_INIT) {
>>  		unsigned long *free = (unsigned long *) nsindex->free;
>
>> diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h
>> index e772aae71843..0a55900842c8 100644
>> --- a/include/linux/libnvdimm.h
>> +++ b/include/linux/libnvdimm.h
>> @@ -44,6 +44,9 @@ enum {
>>  	/* dimm provider wants synchronous registration by __nvdimm_create() */
>>  	NDD_REGISTER_SYNC = 8,
>>
>> +	/* dimm supports region labels (LSA Format 2.1) */
>> +	NDD_CXL_LABEL = 9,
>
>This enum is 'curious'.  It combined flags from a bunch of different
>flags fields and some stuff that are nothing to do with flags.
>
>Anyhow, putting that aside I'd either rename it to something like
>NDD_REGION_LABELING (similar to NDD_LABELING that is there for namespace labels
>or just have it a meaning it is LSA Format 2.1 and drop the fact htat
>also means region labels are supported.
>
>Combination of a comment that talks about one thing and a definition name
>that doesn't associate with it seems confusing to me.
>
>Jonathan
>

Sure, I will rename it in next patch-set

Regards,
Neeraj

>
>> +
>>  	/* need to set a limit somewhere, but yes, this is likely overkill */
>>  	ND_IOCTL_MAX_BUFLEN = SZ_4M,
>>  	ND_CMD_MAX_ELEM = 5,
>

Re: [PATCH V2 01/20] nvdimm/label: Introduce NDD_CXL_LABEL flag to set cxl label format

Posted by Dave Jiang 5 months, 4 weeks ago


On 8/13/25 6:12 AM, Jonathan Cameron wrote:
> On Wed, 30 Jul 2025 17:41:50 +0530
> Neeraj Kumar <s.neeraj@samsung.com> wrote:
> 
>> Prior to LSA 2.1 version, LSA contain only namespace labels. LSA 2.1
>> introduced in CXL 2.0 Spec, which contain region label along with
>> namespace label.
>>
>> NDD_LABELING flag is used for namespace. Introduced NDD_CXL_LABEL
>> flag for region label. Based on these flags nvdimm driver performs
>> operation on namespace label or region label.
>>
>> NDD_CXL_LABEL will be utilized by cxl driver to enable LSA2.1 region
>> label support
>>
>> Accordingly updated label index version
>>
>> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> Hi Neeraj,
> 
> A few comments inline.
> 
>> diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
>> index 04f4a049599a..7a011ee02d79 100644
>> --- a/drivers/nvdimm/label.c
>> +++ b/drivers/nvdimm/label.c
>> @@ -688,11 +688,25 @@ static int nd_label_write_index(struct nvdimm_drvdata *ndd, int index, u32 seq,
>>  		- (unsigned long) to_namespace_index(ndd, 0);
>>  	nsindex->labeloff = __cpu_to_le64(offset);
>>  	nsindex->nslot = __cpu_to_le32(nslot);
>> -	nsindex->major = __cpu_to_le16(1);
>> -	if (sizeof_namespace_label(ndd) < 256)
>> +
>> +	/* Set LSA Label Index Version */
>> +	if (ndd->cxl) {
>> +		/* CXL r3.2 Spec: Table 9-9 Label Index Block Layout */
>> +		nsindex->major = __cpu_to_le16(2);
>>  		nsindex->minor = __cpu_to_le16(1);
>> -	else
>> -		nsindex->minor = __cpu_to_le16(2);
>> +	} else {
>> +		nsindex->major = __cpu_to_le16(1);
>> +		/*
>> +		 * NVDIMM Namespace Specification
>> +		 * Table 2: Namespace Label Index Block Fields
>> +		 */
>> +		if (sizeof_namespace_label(ndd) < 256)
>> +			nsindex->minor = __cpu_to_le16(1);
>> +		else
>> +		 /* UEFI Specification 2.7: Label Index Block Definitions */
> 
> Odd comment alignment. Either put it on the else so
> 		else /* UEFI 2.7: Label Index Block Defintions */
> 
> or indent it an extra tab
> 
> 		else
> 			/* UEFI 2.7: Label Index Block Definitions */
> 			
>> +			nsindex->minor = __cpu_to_le16(2);
>> +	}
>> +
>>  	nsindex->checksum = __cpu_to_le64(0);
>>  	if (flags & ND_NSINDEX_INIT) {
>>  		unsigned long *free = (unsigned long *) nsindex->free;
> 
>> diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h
>> index e772aae71843..0a55900842c8 100644
>> --- a/include/linux/libnvdimm.h
>> +++ b/include/linux/libnvdimm.h
>> @@ -44,6 +44,9 @@ enum {
>>  	/* dimm provider wants synchronous registration by __nvdimm_create() */
>>  	NDD_REGISTER_SYNC = 8,
>>  
>> +	/* dimm supports region labels (LSA Format 2.1) */
>> +	NDD_CXL_LABEL = 9,
> 
> This enum is 'curious'.  It combined flags from a bunch of different
> flags fields and some stuff that are nothing to do with flags.
> 
> Anyhow, putting that aside I'd either rename it to something like
> NDD_REGION_LABELING (similar to NDD_LABELING that is there for namespace labels
> or just have it a meaning it is LSA Format 2.1 and drop the fact htat
> also means region labels are supported.

I agree. I had a conversation with Dan about it where I mentioned calling it CXL to describe LSA 2.1 just doesn't seem quite right. He also offered up something like NDD_REGION_LABELING instead of NDD_CXL_LABEL. So +1 to this comment.

DJ 

> 
> Combination of a comment that talks about one thing and a definition name
> that doesn't associate with it seems confusing to me.
> 
> Jonathan
> 
> 
>> +
>>  	/* need to set a limit somewhere, but yes, this is likely overkill */
>>  	ND_IOCTL_MAX_BUFLEN = SZ_4M,
>>  	ND_CMD_MAX_ELEM = 5,
> 
>

Re: [PATCH V2 01/20] nvdimm/label: Introduce NDD_CXL_LABEL flag to set cxl label format

Posted by Neeraj Kumar 5 months, 1 week ago

On 15/08/25 11:06AM, Dave Jiang wrote:
>
>
>On 8/13/25 6:12 AM, Jonathan Cameron wrote:
>> On Wed, 30 Jul 2025 17:41:50 +0530
>> Neeraj Kumar <s.neeraj@samsung.com> wrote:
>>
>>> Prior to LSA 2.1 version, LSA contain only namespace labels. LSA 2.1
>>> introduced in CXL 2.0 Spec, which contain region label along with
>>> namespace label.
>>>
>>> NDD_LABELING flag is used for namespace. Introduced NDD_CXL_LABEL
>>> flag for region label. Based on these flags nvdimm driver performs
>>> operation on namespace label or region label.
>>>
>>> NDD_CXL_LABEL will be utilized by cxl driver to enable LSA2.1 region
>>> label support
>>>
>>> Accordingly updated label index version
>>>
>>> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
>> Hi Neeraj,
>>
>> A few comments inline.
>>
>>> diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
>>> index 04f4a049599a..7a011ee02d79 100644
>>> --- a/drivers/nvdimm/label.c
>>> +++ b/drivers/nvdimm/label.c
>>> @@ -688,11 +688,25 @@ static int nd_label_write_index(struct nvdimm_drvdata *ndd, int index, u32 seq,
>>>  		- (unsigned long) to_namespace_index(ndd, 0);
>>>  	nsindex->labeloff = __cpu_to_le64(offset);
>>>  	nsindex->nslot = __cpu_to_le32(nslot);
>>> -	nsindex->major = __cpu_to_le16(1);
>>> -	if (sizeof_namespace_label(ndd) < 256)
>>> +
>>> +	/* Set LSA Label Index Version */
>>> +	if (ndd->cxl) {
>>> +		/* CXL r3.2 Spec: Table 9-9 Label Index Block Layout */
>>> +		nsindex->major = __cpu_to_le16(2);
>>>  		nsindex->minor = __cpu_to_le16(1);
>>> -	else
>>> -		nsindex->minor = __cpu_to_le16(2);
>>> +	} else {
>>> +		nsindex->major = __cpu_to_le16(1);
>>> +		/*
>>> +		 * NVDIMM Namespace Specification
>>> +		 * Table 2: Namespace Label Index Block Fields
>>> +		 */
>>> +		if (sizeof_namespace_label(ndd) < 256)
>>> +			nsindex->minor = __cpu_to_le16(1);
>>> +		else
>>> +		 /* UEFI Specification 2.7: Label Index Block Definitions */
>>
>> Odd comment alignment. Either put it on the else so
>> 		else /* UEFI 2.7: Label Index Block Defintions */
>>
>> or indent it an extra tab
>>
>> 		else
>> 			/* UEFI 2.7: Label Index Block Definitions */
>> 			
>>> +			nsindex->minor = __cpu_to_le16(2);
>>> +	}
>>> +
>>>  	nsindex->checksum = __cpu_to_le64(0);
>>>  	if (flags & ND_NSINDEX_INIT) {
>>>  		unsigned long *free = (unsigned long *) nsindex->free;
>>
>>> diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h
>>> index e772aae71843..0a55900842c8 100644
>>> --- a/include/linux/libnvdimm.h
>>> +++ b/include/linux/libnvdimm.h
>>> @@ -44,6 +44,9 @@ enum {
>>>  	/* dimm provider wants synchronous registration by __nvdimm_create() */
>>>  	NDD_REGISTER_SYNC = 8,
>>>
>>> +	/* dimm supports region labels (LSA Format 2.1) */
>>> +	NDD_CXL_LABEL = 9,
>>
>> This enum is 'curious'.  It combined flags from a bunch of different
>> flags fields and some stuff that are nothing to do with flags.
>>
>> Anyhow, putting that aside I'd either rename it to something like
>> NDD_REGION_LABELING (similar to NDD_LABELING that is there for namespace labels
>> or just have it a meaning it is LSA Format 2.1 and drop the fact htat
>> also means region labels are supported.
>
>I agree. I had a conversation with Dan about it where I mentioned calling it CXL to describe LSA 2.1 just doesn't seem quite right. He also offered up something like NDD_REGION_LABELING instead of NDD_CXL_LABEL. So +1 to this comment.
>
>DJ

Sure Dave, I will rename it to NDD_REGION_LABELING in next patch-set

Regards,
Neeraj