CXL and AER drivers need the ability to identify CXL devices.
Add set_pcie_cxl() with logic checking for CXL Flexbus DVSEC presence. The
CXL Flexbus DVSEC presence is used because it is required for all the CXL
PCIe devices.[1]
Add boolean 'struct pci_dev::is_cxl' with the purpose to cache the CXL
Flexbus presence.
Add function pcie_is_cxl() to return 'struct pci_dev::is_cxl'.
[1] CXL 3.1 Spec, 8.1.1 PCIe Designated Vendor-Specific Extended
Capability (DVSEC) ID Assignment, Table 8-2
Signed-off-by: Terry Bowman <terry.bowman@amd.com>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
drivers/pci/probe.c | 10 ++++++++++
include/linux/pci.h | 6 ++++++
include/uapi/linux/pci_regs.h | 8 +++++++-
3 files changed, 23 insertions(+), 1 deletion(-)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 4b8693ec9e4c..5d3548648d5c 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -1691,6 +1691,14 @@ static void set_pcie_thunderbolt(struct pci_dev *dev)
dev->is_thunderbolt = 1;
}
+static void set_pcie_cxl(struct pci_dev *dev)
+{
+ u16 dvsec = pci_find_dvsec_capability(dev, PCI_VENDOR_ID_CXL,
+ PCI_DVSEC_CXL_FLEXBUS);
+ if (dvsec)
+ dev->is_cxl = 1;
+}
+
static void set_pcie_untrusted(struct pci_dev *dev)
{
struct pci_dev *parent = pci_upstream_bridge(dev);
@@ -2021,6 +2029,8 @@ int pci_setup_device(struct pci_dev *dev)
/* Need to have dev->cfg_size ready */
set_pcie_thunderbolt(dev);
+ set_pcie_cxl(dev);
+
set_pcie_untrusted(dev);
if (pci_is_pcie(dev))
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 05e68f35f392..79878243b681 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -453,6 +453,7 @@ struct pci_dev {
unsigned int is_hotplug_bridge:1;
unsigned int shpc_managed:1; /* SHPC owned by shpchp */
unsigned int is_thunderbolt:1; /* Thunderbolt controller */
+ unsigned int is_cxl:1; /* Compute Express Link (CXL) */
/*
* Devices marked being untrusted are the ones that can potentially
* execute DMA attacks and similar. They are typically connected
@@ -744,6 +745,11 @@ static inline bool pci_is_vga(struct pci_dev *pdev)
return false;
}
+static inline bool pcie_is_cxl(struct pci_dev *pci_dev)
+{
+ return pci_dev->is_cxl;
+}
+
#define for_each_pci_bridge(dev, bus) \
list_for_each_entry(dev, &bus->devices, bus_list) \
if (!pci_is_bridge(dev)) {} else
diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
index a3a3e942dedf..fb9d77c69d5f 100644
--- a/include/uapi/linux/pci_regs.h
+++ b/include/uapi/linux/pci_regs.h
@@ -1225,9 +1225,15 @@
/* Deprecated old name, replaced with PCI_DOE_DATA_OBJECT_DISC_RSP_3_TYPE */
#define PCI_DOE_DATA_OBJECT_DISC_RSP_3_PROTOCOL PCI_DOE_DATA_OBJECT_DISC_RSP_3_TYPE
-/* Compute Express Link (CXL r3.1, sec 8.1.5) */
+/* Compute Express Link (CXL r3.2, sec 8.1)
+ *
+ * Note that CXL DVSEC id 3 and 7 to be ignored when the CXL link state
+ * is "disconnected" (CXL r3.2, sec 9.12.3). Re-enumerate these
+ * registers on downstream link-up events.
+ */
#define PCI_DVSEC_CXL_PORT 3
#define PCI_DVSEC_CXL_PORT_CTL 0x0c
#define PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR 0x00000001
+#define PCI_DVSEC_CXL_FLEXBUS 7
#endif /* LINUX_PCI_REGS_H */
--
2.34.1
On 6/26/25 23:42, Terry Bowman wrote:
> CXL and AER drivers need the ability to identify CXL devices.
>
> Add set_pcie_cxl() with logic checking for CXL Flexbus DVSEC presence. The
> CXL Flexbus DVSEC presence is used because it is required for all the CXL
> PCIe devices.[1]
>
> Add boolean 'struct pci_dev::is_cxl' with the purpose to cache the CXL
> Flexbus presence.
>
> Add function pcie_is_cxl() to return 'struct pci_dev::is_cxl'.
>
> [1] CXL 3.1 Spec, 8.1.1 PCIe Designated Vendor-Specific Extended
> Capability (DVSEC) ID Assignment, Table 8-2
>
> Signed-off-by: Terry Bowman <terry.bowman@amd.com>
> Reviewed-by: Ira Weiny <ira.weiny@intel.com>
> Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
> Reviewed-by: Dave Jiang <dave.jiang@intel.com>
> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> ---
> drivers/pci/probe.c | 10 ++++++++++
> include/linux/pci.h | 6 ++++++
> include/uapi/linux/pci_regs.h | 8 +++++++-
> 3 files changed, 23 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
> index 4b8693ec9e4c..5d3548648d5c 100644
> --- a/drivers/pci/probe.c
> +++ b/drivers/pci/probe.c
> @@ -1691,6 +1691,14 @@ static void set_pcie_thunderbolt(struct pci_dev *dev)
> dev->is_thunderbolt = 1;
> }
>
> +static void set_pcie_cxl(struct pci_dev *dev)
> +{
> + u16 dvsec = pci_find_dvsec_capability(dev, PCI_VENDOR_ID_CXL,
> + PCI_DVSEC_CXL_FLEXBUS);
> + if (dvsec)
> + dev->is_cxl = 1;
Hi Terry,
Should not this check for the bits about port status like IO_Enabled?
Maybe I'm confused with the goal, but we can know a device is a cxl one
without specifically looking at the FLEXBUS capability presence or not.
What this capability can tell, and what I think it is more interesting,
is if the link was successfully trained as CXL.io. From problems we have
had while testing our type2 device, if the training for CXL.io fails,
the device will use pcie and this bit will be 0, same if the device can
be configured for using pcie only.
If this is what we really need to know, changing is_cxl to
is_cxl_enabled could be clearer.
I come here after Dan suggesting to use this functionality for ensuring
the CXL device functionality is on, and it would require to inspect the
status instead of just the capability being present. Maybe I'm confused
because I remember some patches from Robert Richter dealing with
checking link up for enabling downstream ports, but I think the goal
here is different.
> +}
> +
> static void set_pcie_untrusted(struct pci_dev *dev)
> {
> struct pci_dev *parent = pci_upstream_bridge(dev);
> @@ -2021,6 +2029,8 @@ int pci_setup_device(struct pci_dev *dev)
> /* Need to have dev->cfg_size ready */
> set_pcie_thunderbolt(dev);
>
> + set_pcie_cxl(dev);
> +
> set_pcie_untrusted(dev);
>
> if (pci_is_pcie(dev))
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index 05e68f35f392..79878243b681 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -453,6 +453,7 @@ struct pci_dev {
> unsigned int is_hotplug_bridge:1;
> unsigned int shpc_managed:1; /* SHPC owned by shpchp */
> unsigned int is_thunderbolt:1; /* Thunderbolt controller */
> + unsigned int is_cxl:1; /* Compute Express Link (CXL) */
> /*
> * Devices marked being untrusted are the ones that can potentially
> * execute DMA attacks and similar. They are typically connected
> @@ -744,6 +745,11 @@ static inline bool pci_is_vga(struct pci_dev *pdev)
> return false;
> }
>
> +static inline bool pcie_is_cxl(struct pci_dev *pci_dev)
> +{
> + return pci_dev->is_cxl;
> +}
> +
> #define for_each_pci_bridge(dev, bus) \
> list_for_each_entry(dev, &bus->devices, bus_list) \
> if (!pci_is_bridge(dev)) {} else
> diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
> index a3a3e942dedf..fb9d77c69d5f 100644
> --- a/include/uapi/linux/pci_regs.h
> +++ b/include/uapi/linux/pci_regs.h
> @@ -1225,9 +1225,15 @@
> /* Deprecated old name, replaced with PCI_DOE_DATA_OBJECT_DISC_RSP_3_TYPE */
> #define PCI_DOE_DATA_OBJECT_DISC_RSP_3_PROTOCOL PCI_DOE_DATA_OBJECT_DISC_RSP_3_TYPE
>
> -/* Compute Express Link (CXL r3.1, sec 8.1.5) */
> +/* Compute Express Link (CXL r3.2, sec 8.1)
> + *
> + * Note that CXL DVSEC id 3 and 7 to be ignored when the CXL link state
> + * is "disconnected" (CXL r3.2, sec 9.12.3). Re-enumerate these
> + * registers on downstream link-up events.
> + */
> #define PCI_DVSEC_CXL_PORT 3
> #define PCI_DVSEC_CXL_PORT_CTL 0x0c
> #define PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR 0x00000001
> +#define PCI_DVSEC_CXL_FLEXBUS 7
>
> #endif /* LINUX_PCI_REGS_H */
On 8/9/2025 5:56 AM, Alejandro Lucero Palau wrote:
>
> On 6/26/25 23:42, Terry Bowman wrote:
>> CXL and AER drivers need the ability to identify CXL devices.
>>
>> Add set_pcie_cxl() with logic checking for CXL Flexbus DVSEC presence. The
>> CXL Flexbus DVSEC presence is used because it is required for all the CXL
>> PCIe devices.[1]
>>
>> Add boolean 'struct pci_dev::is_cxl' with the purpose to cache the CXL
>> Flexbus presence.
>>
>> Add function pcie_is_cxl() to return 'struct pci_dev::is_cxl'.
>>
>> [1] CXL 3.1 Spec, 8.1.1 PCIe Designated Vendor-Specific Extended
>> Capability (DVSEC) ID Assignment, Table 8-2
>>
>> Signed-off-by: Terry Bowman <terry.bowman@amd.com>
>> Reviewed-by: Ira Weiny <ira.weiny@intel.com>
>> Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
>> Reviewed-by: Dave Jiang <dave.jiang@intel.com>
>> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
>> ---
>> drivers/pci/probe.c | 10 ++++++++++
>> include/linux/pci.h | 6 ++++++
>> include/uapi/linux/pci_regs.h | 8 +++++++-
>> 3 files changed, 23 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
>> index 4b8693ec9e4c..5d3548648d5c 100644
>> --- a/drivers/pci/probe.c
>> +++ b/drivers/pci/probe.c
>> @@ -1691,6 +1691,14 @@ static void set_pcie_thunderbolt(struct pci_dev *dev)
>> dev->is_thunderbolt = 1;
>> }
>>
>> +static void set_pcie_cxl(struct pci_dev *dev)
>> +{
>> + u16 dvsec = pci_find_dvsec_capability(dev, PCI_VENDOR_ID_CXL,
>> + PCI_DVSEC_CXL_FLEXBUS);
>> + if (dvsec)
>> + dev->is_cxl = 1;
>
>
> Hi Terry,
>
>
> Should not this check for the bits about port status like IO_Enabled?
>
>
> Maybe I'm confused with the goal, but we can know a device is a cxl one
> without specifically looking at the FLEXBUS capability presence or not.
> What this capability can tell, and what I think it is more interesting,
> is if the link was successfully trained as CXL.io. From problems we have
> had while testing our type2 device, if the training for CXL.io fails,
> the device will use pcie and this bit will be 0, same if the device can
> be configured for using pcie only.
>
>
> If this is what we really need to know, changing is_cxl to
> is_cxl_enabled could be clearer.
>
>
> I come here after Dan suggesting to use this functionality for ensuring
> the CXL device functionality is on, and it would require to inspect the
> status instead of just the capability being present. Maybe I'm confused
> because I remember some patches from Robert Richter dealing with
> checking link up for enabling downstream ports, but I think the goal
> here is different.
>
Hi Alejandro,
I agree in large part. We need to check for training complete and any change
in training needs to be reflected in is_cxl(). My understanding is this
will be be added later in a following patch series.
Dan, can you add your thoughts ?
-Terry
Bowman, Terry wrote: [..] > On 8/9/2025 5:56 AM, Alejandro Lucero Palau wrote: > > I come here after Dan suggesting to use this functionality for ensuring > > the CXL device functionality is on, and it would require to inspect the > > status instead of just the capability being present. Maybe I'm confused > > because I remember some patches from Robert Richter dealing with > > checking link up for enabling downstream ports, but I think the goal > > here is different. > > > > Hi Alejandro, > > I agree in large part. We need to check for training complete and any change > in training needs to be reflected in is_cxl(). My understanding is this > will be be added later in a following patch series. > > Dan, can you add your thoughts ? Training completion is implicit in the fact that the device can be targeted by configuration requests. DVSEC 7 absence means you know that CXL is disabled. It is true that DVSEC 7 presence is inconclusive for whether the device supports CXL.mem or CXL.cache. I have seen some implementations hide DVSEC 7, but that can not be relied upon for is_cxl() purposes. CXL.io support is not interesting. So is_cxl() should probably indicate (CXL.mem || CXL.cache) because the presence of those needs CXL subsystem infrastructure.
Terry Bowman wrote:
> CXL and AER drivers need the ability to identify CXL devices.
>
> Add set_pcie_cxl() with logic checking for CXL Flexbus DVSEC presence. The
> CXL Flexbus DVSEC presence is used because it is required for all the CXL
> PCIe devices.[1]
>
> Add boolean 'struct pci_dev::is_cxl' with the purpose to cache the CXL
> Flexbus presence.
>
> Add function pcie_is_cxl() to return 'struct pci_dev::is_cxl'.
>
> [1] CXL 3.1 Spec, 8.1.1 PCIe Designated Vendor-Specific Extended
> Capability (DVSEC) ID Assignment, Table 8-2
>
> Signed-off-by: Terry Bowman <terry.bowman@amd.com>
> Reviewed-by: Ira Weiny <ira.weiny@intel.com>
> Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
> Reviewed-by: Dave Jiang <dave.jiang@intel.com>
> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> ---
> drivers/pci/probe.c | 10 ++++++++++
> include/linux/pci.h | 6 ++++++
> include/uapi/linux/pci_regs.h | 8 +++++++-
> 3 files changed, 23 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
> index 4b8693ec9e4c..5d3548648d5c 100644
> --- a/drivers/pci/probe.c
> +++ b/drivers/pci/probe.c
> @@ -1691,6 +1691,14 @@ static void set_pcie_thunderbolt(struct pci_dev *dev)
> dev->is_thunderbolt = 1;
> }
>
> +static void set_pcie_cxl(struct pci_dev *dev)
> +{
> + u16 dvsec = pci_find_dvsec_capability(dev, PCI_VENDOR_ID_CXL,
> + PCI_DVSEC_CXL_FLEXBUS);
> + if (dvsec)
> + dev->is_cxl = 1;
> +}
> +
> static void set_pcie_untrusted(struct pci_dev *dev)
> {
> struct pci_dev *parent = pci_upstream_bridge(dev);
> @@ -2021,6 +2029,8 @@ int pci_setup_device(struct pci_dev *dev)
> /* Need to have dev->cfg_size ready */
> set_pcie_thunderbolt(dev);
>
> + set_pcie_cxl(dev);
Per the comment in the header below, in the case of upstream ports and
endpoints, this should walk to the parent downstream port and make sure
the cxl setting matches. I.e. with hotplug the downstream port may
transition from not-cxl to is-cxl. Update downstream-port parents at the
beginning of life of their CXL child-devices.
> +
> set_pcie_untrusted(dev);
>
> if (pci_is_pcie(dev))
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index 05e68f35f392..79878243b681 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -453,6 +453,7 @@ struct pci_dev {
> unsigned int is_hotplug_bridge:1;
> unsigned int shpc_managed:1; /* SHPC owned by shpchp */
> unsigned int is_thunderbolt:1; /* Thunderbolt controller */
> + unsigned int is_cxl:1; /* Compute Express Link (CXL) */
> /*
> * Devices marked being untrusted are the ones that can potentially
> * execute DMA attacks and similar. They are typically connected
> @@ -744,6 +745,11 @@ static inline bool pci_is_vga(struct pci_dev *pdev)
> return false;
> }
>
> +static inline bool pcie_is_cxl(struct pci_dev *pci_dev)
> +{
> + return pci_dev->is_cxl;
> +}
> +
> #define for_each_pci_bridge(dev, bus) \
> list_for_each_entry(dev, &bus->devices, bus_list) \
> if (!pci_is_bridge(dev)) {} else
> diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
> index a3a3e942dedf..fb9d77c69d5f 100644
> --- a/include/uapi/linux/pci_regs.h
> +++ b/include/uapi/linux/pci_regs.h
> @@ -1225,9 +1225,15 @@
> /* Deprecated old name, replaced with PCI_DOE_DATA_OBJECT_DISC_RSP_3_TYPE */
> #define PCI_DOE_DATA_OBJECT_DISC_RSP_3_PROTOCOL PCI_DOE_DATA_OBJECT_DISC_RSP_3_TYPE
>
> -/* Compute Express Link (CXL r3.1, sec 8.1.5) */
> +/* Compute Express Link (CXL r3.2, sec 8.1)
> + *
> + * Note that CXL DVSEC id 3 and 7 to be ignored when the CXL link state
> + * is "disconnected" (CXL r3.2, sec 9.12.3). Re-enumerate these
> + * registers on downstream link-up events.
> + */
> #define PCI_DVSEC_CXL_PORT 3
> #define PCI_DVSEC_CXL_PORT_CTL 0x0c
> #define PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR 0x00000001
> +#define PCI_DVSEC_CXL_FLEXBUS 7
Please, let us not end up with multiple definitions for the same thing
in multiple places. I note that Alejandro introduces include/cxl/pci.h
with some of the CXL DVSEC definitions from drivers/cxl/cxlpci.h.
Although not all of them, I think a precursor patch to move all of them
in this patch set is the way to go. I.e. drop all these PCI_DVSEC_CXL_*
versions in favor of their existing CXL_DVSEC_* versions.
© 2016 - 2025 Red Hat, Inc.