The number of ob/ib windows is determined through write-read loops
on registers in the core driver. Some glue drivers need to adjust
the number of ob/ib windows to meet specific requirements,such as
hardware limitations. This change allows the glue driver to adjust
the number of ob/ib windows to satisfy platform-specific constraints.
The glue driver may adjust the number of ob/ib windows, but the values
must stay within hardware limits.

Signed-off-by: Randolph Lin <randolph@andestech.com>
---
 drivers/pci/controller/dwc/pcie-designware.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index 89aad5a08928..56c1e45adc06 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -907,8 +907,16 @@ void dw_pcie_iatu_detect(struct dw_pcie *pci)
 		max = 0;
 	}
 
-	pci->num_ob_windows = ob;
-	pci->num_ib_windows = ib;
+	if (!pci->num_ob_windows)
+		pci->num_ob_windows = ob;
+	else if (pci->num_ob_windows > ob)
+		dev_err(pci->dev, "Adjusted ob windows exceed the limit\n");
+
+	if (!pci->num_ib_windows)
+		pci->num_ib_windows = ib;
+	else if (pci->num_ib_windows > ib)
+		dev_err(pci->dev, "Adjusted ib windows exceed the limit\n");
+
 	pci->region_align = 1 << fls(min);
 	pci->region_limit = (max << 32) | (SZ_4G - 1);
 
-- 
2.34.1

On Fri, Oct 03, 2025 at 10:35:23AM +0800, Randolph Lin wrote:
> The number of ob/ib windows is determined through write-read loops
> on registers in the core driver. Some glue drivers need to adjust
> the number of ob/ib windows to meet specific requirements,such as

Missing space after comma.


> hardware limitations. This change allows the glue driver to adjust
> the number of ob/ib windows to satisfy platform-specific constraints.
> The glue driver may adjust the number of ob/ib windows, but the values
> must stay within hardware limits.

Could we please get a better explaination than "satisfy platform-specific
constraints" ?

Your PCIe controller is synthesized with a certain number of {in,out}bound
windows, and I assume that dw_pcie_iatu_detect() correctly detects the number
of {in,out}bound windows, and initializes num_ob_windows/num_ib_windows
accordingly.

So, is the problem that because of some errata, you cannot use all the
{in,out}bound windows of the iATU?

Because it is hard to understand what kind of "hardware limit" that would
cause your SoC to not be able to use all the available {in,out}bound windows.

Because it is simply a mapping in the iATU (internal Address Translation Unit).

In fact, in many cases, e.g. the NVMe EPF driver, then number of {in,out}bound
windows is a major limiting factor of how many outstanding I/Os you can have,
so usually, you really want to be able to use the maximum that the hardware
supports.


TL;DR: to modify this common code, I think your reasoning has to be more
detailed.



Kind regards,
Niklas

Hi Niklas,

On Tue, Oct 14, 2025 at 11:43:53AM +0200, Niklas Cassel wrote:
> [EXTERNAL MAIL]
> 
> On Fri, Oct 03, 2025 at 10:35:23AM +0800, Randolph Lin wrote:
> > The number of ob/ib windows is determined through write-read loops
> > on registers in the core driver. Some glue drivers need to adjust
> > the number of ob/ib windows to meet specific requirements,such as
> 
> Missing space after comma.
> 
> 

Thanks a lot. I will fix it in the next patch.

> > hardware limitations. This change allows the glue driver to adjust
> > the number of ob/ib windows to satisfy platform-specific constraints.
> > The glue driver may adjust the number of ob/ib windows, but the values
> > must stay within hardware limits.
> 
> Could we please get a better explaination than "satisfy platform-specific
> constraints" ?
> 

Due to this SoC design, only iATU regions with mapped addresses within the
32-bits address range need to be programmed. However, this SoC has a design
limitation in which the maximum region size supported by a single iATU
entry is restricted to 4 GB, as it is based on a 32-bits address region.

For most EP devices, we can only define one entry in the "ranges" property
of the devicetree that maps an address within the 32-bit range,
as shown below:
	ranges = <0x02000000 0x0 0x10000000 0x0 0x10000000 0x0 0xf0000000>;

For EP devices that require 64-bits address mapping (e.g., GPUs), BAR
resources cannot be assigned.
To support such devices, an additional entry for 64-bits address mapping is
required, as shown below:
	ranges = <0x02000000 0x0 0x10000000 0x0 0x10000000 0x0 0xf0000000>,
		 <0x43000000 0x1 0x00000000 0x1 0x00000000 0x7 0x00000000>;

In the current common implementation, all ranges entries are programmed to
the iATU. However, the size of entry for 64-bit address mapping exceeds the
maximum region size that a single iATU entry can support. As a result, an
error is reported during iATU programming, showing that the size of 64-bit
address entry exceeds the region limit.

In this SoC design, 64-bit addresses are hard-wired and can skip iATU
programming. Thus, the driver needs to recount the "ranges" entries whose
size fits within the 4GB platform limit.

There are four scenarios:
32-bits address, size < 4GB: program to iATU
64-bits address, size < 4GB: program to iATU
32-bits address, size > 4GB: assuming this condition does not exist
64-bits address, size > 4GB: skip case

We will recount how many outbound windows will be programmed to the iATU; 
this is why we need to adjust the number of entries programmed to the iATU.

> Your PCIe controller is synthesized with a certain number of {in,out}bound
> windows, and I assume that dw_pcie_iatu_detect() correctly detects the number
> of {in,out}bound windows, and initializes num_ob_windows/num_ib_windows
> accordingly.
> 
> So, is the problem that because of some errata, you cannot use all the
> {in,out}bound windows of the iATU?
>

Similar to the erratum, all inbound and outbound windows remain functional,
as long as each iATU entry complies with the 4 GB size constraint.

> Because it is hard to understand what kind of "hardware limit" that would
> cause your SoC to not be able to use all the available {in,out}bound windows.
> 
> Because it is simply a mapping in the iATU (internal Address Translation Unit).
> 
> In fact, in many cases, e.g. the NVMe EPF driver, then number of {in,out}bound
> windows is a major limiting factor of how many outstanding I/Os you can have,
> so usually, you really want to be able to use the maximum that the hardware
> supports.
> 
> 
> TL;DR: to modify this common code, I think your reasoning has to be more
> detailed.
> 

I will include additional explanations along with the application scenarios of
this SoC, and refactor the commit message.

> 
> 
> Kind regards,
> Niklas

Sincerely,
Randolph

Hello Randolph,

On Thu, Oct 16, 2025 at 07:12:36PM +0800, Randolph Lin wrote:
> > 
> > Could we please get a better explaination than "satisfy platform-specific
> > constraints" ?
> > 
> 
> Due to this SoC design, only iATU regions with mapped addresses within the
> 32-bits address range need to be programmed. However, this SoC has a design
> limitation in which the maximum region size supported by a single iATU
> entry is restricted to 4 GB, as it is based on a 32-bits address region.
> 
> For most EP devices, we can only define one entry in the "ranges" property
> of the devicetree that maps an address within the 32-bit range,
> as shown below:
> 	ranges = <0x02000000 0x0 0x10000000 0x0 0x10000000 0x0 0xf0000000>;
> 
> For EP devices that require 64-bits address mapping (e.g., GPUs), BAR
> resources cannot be assigned.
> To support such devices, an additional entry for 64-bits address mapping is
> required, as shown below:
> 	ranges = <0x02000000 0x0 0x10000000 0x0 0x10000000 0x0 0xf0000000>,
> 		 <0x43000000 0x1 0x00000000 0x1 0x00000000 0x7 0x00000000>;
> 
> In the current common implementation, all ranges entries are programmed to
> the iATU. However, the size of entry for 64-bit address mapping exceeds the
> maximum region size that a single iATU entry can support. As a result, an
> error is reported during iATU programming, showing that the size of 64-bit
> address entry exceeds the region limit.

Note that each iATU can map up to IATU_LIMIT_ADDR_OFF_OUTBOUND_i +
IATU_UPPR_LIMIT_ADDR_OFF_OUTBOUND_i.

Some DWC controllers have this at 4G, others have this at 8G.

Samuel has submitted a patch to use multiple iATUs to support
a window size larger than the iATU limit of a single iATU:
https://lore.kernel.org/linux-pci/aPDObXsvMoz1OYso@ryzen/T/#m11c3d95215982411d0bbd36940e70122b70ae820

Perhaps this patch could be of use for you too?


Kind regards,
Niklas

Hello Niklas,

On Thu, Oct 16, 2025 at 01:54:35PM +0200, Niklas Cassel wrote:
> [EXTERNAL MAIL]
> 
> Hello Randolph,
> 
> On Thu, Oct 16, 2025 at 07:12:36PM +0800, Randolph Lin wrote:
> > >
> > > Could we please get a better explaination than "satisfy platform-specific
> > > constraints" ?
> > >
> >
> > Due to this SoC design, only iATU regions with mapped addresses within the
> > 32-bits address range need to be programmed. However, this SoC has a design
> > limitation in which the maximum region size supported by a single iATU
> > entry is restricted to 4 GB, as it is based on a 32-bits address region.
> >
> > For most EP devices, we can only define one entry in the "ranges" property
> > of the devicetree that maps an address within the 32-bit range,
> > as shown below:
> >       ranges = <0x02000000 0x0 0x10000000 0x0 0x10000000 0x0 0xf0000000>;
> >
> > For EP devices that require 64-bits address mapping (e.g., GPUs), BAR
> > resources cannot be assigned.
> > To support such devices, an additional entry for 64-bits address mapping is
> > required, as shown below:
> >       ranges = <0x02000000 0x0 0x10000000 0x0 0x10000000 0x0 0xf0000000>,
> >                <0x43000000 0x1 0x00000000 0x1 0x00000000 0x7 0x00000000>;
> >
> > In the current common implementation, all ranges entries are programmed to
> > the iATU. However, the size of entry for 64-bit address mapping exceeds the
> > maximum region size that a single iATU entry can support. As a result, an
> > error is reported during iATU programming, showing that the size of 64-bit
> > address entry exceeds the region limit.
> 
> Note that each iATU can map up to IATU_LIMIT_ADDR_OFF_OUTBOUND_i +
> IATU_UPPR_LIMIT_ADDR_OFF_OUTBOUND_i.
> 
> Some DWC controllers have this at 4G, others have this at 8G.
> 
> Samuel has submitted a patch to use multiple iATUs to support
> a window size larger than the iATU limit of a single iATU:
> https://lore.kernel.org/linux-pci/aPDObXsvMoz1OYso@ryzen/T/#m11c3d95215982411d0bbd36940e70122b70ae820
> 
> Perhaps this patch could be of use for you too?
>

Thank you for the information.
After applying Samuel’s patch, the code passes the basic functionality
tests. Therefore, the common code patch is no longer needed.

> 
> Kind regards,
> Niklas

Sincerely,
Randolph Lin