[v2] PCI: Update Resizable BAR Capability Register fields

[PATCH v2] PCI: Update Resizable BAR Capability Register fields

Posted by Zhiyuan Dai 10 months ago

PCI Express Base Spec r6.0 defines BAR size up to 8 EB (2^63 bytes),
but supporting anything bigger than 128TB requires changes to pci_rebar_get_possible_sizes()
to read the additional Capability bits from the Control register.

Signed-off-by: Zhiyuan Dai <daizhiyuan@phytium.com.cn>
---
 drivers/pci/pci.c             | 14 ++++++++++----
 include/uapi/linux/pci_regs.h |  3 ++-
 2 files changed, 12 insertions(+), 5 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 661f98c6c63a..8903deb2d891 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -3752,12 +3752,13 @@ static int pci_rebar_find_pos(struct pci_dev *pdev, int bar)
  * @bar: BAR to query
  *
  * Get the possible sizes of a resizable BAR as bitmask defined in the spec
- * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizable.
+ * (bit 0=1MB, bit 43=8EB). Returns 0 if BAR isn't resizable.
  */
-u32 pci_rebar_get_possible_sizes(struct pci_dev *pdev, int bar)
+u64 pci_rebar_get_possible_sizes(struct pci_dev *pdev, int bar)
 {
 	int pos;
-	u32 cap;
+	u64 cap;
+	u32 cap2;
 
 	pos = pci_rebar_find_pos(pdev, bar);
 	if (pos < 0)
@@ -3766,6 +3767,11 @@ u32 pci_rebar_get_possible_sizes(struct pci_dev *pdev, int bar)
 	pci_read_config_dword(pdev, pos + PCI_REBAR_CAP, &cap);
 	cap = FIELD_GET(PCI_REBAR_CAP_SIZES, cap);
 
+	pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &cap2);
+	cap2 = FIELD_GET(PCI_REBAR_CTRL_CAP_SIZES, cap2);
+
+	cap |= (cap2 << 32);
+
 	/* Sapphire RX 5600 XT Pulse has an invalid cap dword for BAR 0 */
 	if (pdev->vendor == PCI_VENDOR_ID_ATI && pdev->device == 0x731f &&
 	    bar == 0 && cap == 0x700)
@@ -3800,7 +3806,7 @@ int pci_rebar_get_current_size(struct pci_dev *pdev, int bar)
  * pci_rebar_set_size - set a new size for a BAR
  * @pdev: PCI device
  * @bar: BAR to set size to
- * @size: new size as defined in the spec (0=1MB, 19=512GB)
+ * @size: new size as defined in the spec (0=1MB, 43=8EB)
  *
  * Set the new size of a BAR as defined in the spec.
  * Returns zero if resizing was successful, error code otherwise.
diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
index 1601c7ed5fab..345f45264567 100644
--- a/include/uapi/linux/pci_regs.h
+++ b/include/uapi/linux/pci_regs.h
@@ -1013,13 +1013,14 @@
 
 /* Resizable BARs */
 #define PCI_REBAR_CAP		4	/* capability register */
-#define  PCI_REBAR_CAP_SIZES		0x00FFFFF0  /* supported BAR sizes */
+#define  PCI_REBAR_CAP_SIZES		0xFFFFFFF0  /* supported BAR sizes */
 #define PCI_REBAR_CTRL		8	/* control register */
 #define  PCI_REBAR_CTRL_BAR_IDX		0x00000007  /* BAR index */
 #define  PCI_REBAR_CTRL_NBAR_MASK	0x000000E0  /* # of resizable BARs */
 #define  PCI_REBAR_CTRL_NBAR_SHIFT	5	    /* shift for # of BARs */
 #define  PCI_REBAR_CTRL_BAR_SIZE	0x00001F00  /* BAR size */
 #define  PCI_REBAR_CTRL_BAR_SHIFT	8	    /* shift for BAR size */
+#define  PCI_REBAR_CTRL_CAP_SIZES	0xFFFF0000  /* supported BAR sizes */
 
 /* Dynamic Power Allocation */
 #define PCI_DPA_CAP		4	/* capability register */
-- 
2.43.0

Re: [PATCH v2] PCI: Update Resizable BAR Capability Register fields

Posted by Bjorn Helgaas 10 months ago

On Wed, Feb 19, 2025 at 10:27:12AM +0800, Zhiyuan Dai wrote:
> PCI Express Base Spec r6.0 defines BAR size up to 8 EB (2^63 bytes),
> but supporting anything bigger than 128TB requires changes to pci_rebar_get_possible_sizes()
> to read the additional Capability bits from the Control register.
> 
> Signed-off-by: Zhiyuan Dai <daizhiyuan@phytium.com.cn>
> ---
>  drivers/pci/pci.c             | 14 ++++++++++----
>  include/uapi/linux/pci_regs.h |  3 ++-
>  2 files changed, 12 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 661f98c6c63a..8903deb2d891 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -3752,12 +3752,13 @@ static int pci_rebar_find_pos(struct pci_dev *pdev, int bar)
>   * @bar: BAR to query
>   *
>   * Get the possible sizes of a resizable BAR as bitmask defined in the spec
> - * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizable.
> + * (bit 0=1MB, bit 43=8EB). Returns 0 if BAR isn't resizable.
>   */
> -u32 pci_rebar_get_possible_sizes(struct pci_dev *pdev, int bar)
> +u64 pci_rebar_get_possible_sizes(struct pci_dev *pdev, int bar)

Callers need to be updated so they're prepared for a u64 instead of a
u32.

If you don't actually need sizes bigger than 128TB right now, it's
fine to keep this as a u32, only add support up to 128TB, and leave
the >128TB support for later.

Bjorn

[PATCH v3] PCI: Update Resizable BAR Capability Register fields

Posted by Zhiyuan Dai 10 months ago

PCI Express Base Spec r6.0 defines BAR size up to 8 EB (2^63 bytes),
but supporting anything bigger than 128TB requires changes to pci_rebar_get_possible_sizes()
to read the additional Capability bits from the Control register.

If 8EB support is required, callers will need to be updated to handle u64 instead of u32.
For now, support is limited to 128TB, and support for sizes greater than 128TB can be
deferred to a later time.

Signed-off-by: Zhiyuan Dai <daizhiyuan@phytium.com.cn>
---
 drivers/pci/pci.c             | 4 ++--
 include/uapi/linux/pci_regs.h | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 661f98c6c63a..77b9ceefb4e1 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -3752,7 +3752,7 @@ static int pci_rebar_find_pos(struct pci_dev *pdev, int bar)
  * @bar: BAR to query
  *
  * Get the possible sizes of a resizable BAR as bitmask defined in the spec
- * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizable.
+ * (bit 0=1MB, bit 31=128TB). Returns 0 if BAR isn't resizable.
  */
 u32 pci_rebar_get_possible_sizes(struct pci_dev *pdev, int bar)
 {
@@ -3800,7 +3800,7 @@ int pci_rebar_get_current_size(struct pci_dev *pdev, int bar)
  * pci_rebar_set_size - set a new size for a BAR
  * @pdev: PCI device
  * @bar: BAR to set size to
- * @size: new size as defined in the spec (0=1MB, 19=512GB)
+ * @size: new size as defined in the spec (0=1MB, 31=128TB)
  *
  * Set the new size of a BAR as defined in the spec.
  * Returns zero if resizing was successful, error code otherwise.
diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
index 1601c7ed5fab..ce99d4f34ce5 100644
--- a/include/uapi/linux/pci_regs.h
+++ b/include/uapi/linux/pci_regs.h
@@ -1013,7 +1013,7 @@
 
 /* Resizable BARs */
 #define PCI_REBAR_CAP		4	/* capability register */
-#define  PCI_REBAR_CAP_SIZES		0x00FFFFF0  /* supported BAR sizes */
+#define  PCI_REBAR_CAP_SIZES		0xFFFFFFF0  /* supported BAR sizes */
 #define PCI_REBAR_CTRL		8	/* control register */
 #define  PCI_REBAR_CTRL_BAR_IDX		0x00000007  /* BAR index */
 #define  PCI_REBAR_CTRL_NBAR_MASK	0x000000E0  /* # of resizable BARs */
-- 
2.43.0

Re: [PATCH v3] PCI: Update Resizable BAR Capability Register fields

Posted by Bjorn Helgaas 9 months, 4 weeks ago

On Thu, Feb 20, 2025 at 09:25:46AM +0800, Zhiyuan Dai wrote:
> PCI Express Base Spec r6.0 defines BAR size up to 8 EB (2^63 bytes),
> but supporting anything bigger than 128TB requires changes to pci_rebar_get_possible_sizes()
> to read the additional Capability bits from the Control register.
> 
> If 8EB support is required, callers will need to be updated to handle u64 instead of u32.
> For now, support is limited to 128TB, and support for sizes greater than 128TB can be
> deferred to a later time.
> 
> Signed-off-by: Zhiyuan Dai <daizhiyuan@phytium.com.cn>

Applied to pci/resource for v6.15, thanks!

> ---
>  drivers/pci/pci.c             | 4 ++--
>  include/uapi/linux/pci_regs.h | 2 +-
>  2 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 661f98c6c63a..77b9ceefb4e1 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -3752,7 +3752,7 @@ static int pci_rebar_find_pos(struct pci_dev *pdev, int bar)
>   * @bar: BAR to query
>   *
>   * Get the possible sizes of a resizable BAR as bitmask defined in the spec
> - * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizable.
> + * (bit 0=1MB, bit 31=128TB). Returns 0 if BAR isn't resizable.
>   */
>  u32 pci_rebar_get_possible_sizes(struct pci_dev *pdev, int bar)
>  {
> @@ -3800,7 +3800,7 @@ int pci_rebar_get_current_size(struct pci_dev *pdev, int bar)
>   * pci_rebar_set_size - set a new size for a BAR
>   * @pdev: PCI device
>   * @bar: BAR to set size to
> - * @size: new size as defined in the spec (0=1MB, 19=512GB)
> + * @size: new size as defined in the spec (0=1MB, 31=128TB)
>   *
>   * Set the new size of a BAR as defined in the spec.
>   * Returns zero if resizing was successful, error code otherwise.
> diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
> index 1601c7ed5fab..ce99d4f34ce5 100644
> --- a/include/uapi/linux/pci_regs.h
> +++ b/include/uapi/linux/pci_regs.h
> @@ -1013,7 +1013,7 @@
>  
>  /* Resizable BARs */
>  #define PCI_REBAR_CAP		4	/* capability register */
> -#define  PCI_REBAR_CAP_SIZES		0x00FFFFF0  /* supported BAR sizes */
> +#define  PCI_REBAR_CAP_SIZES		0xFFFFFFF0  /* supported BAR sizes */
>  #define PCI_REBAR_CTRL		8	/* control register */
>  #define  PCI_REBAR_CTRL_BAR_IDX		0x00000007  /* BAR index */
>  #define  PCI_REBAR_CTRL_NBAR_MASK	0x000000E0  /* # of resizable BARs */
> -- 
> 2.43.0
>

Re: [PATCH v3] PCI: Update Resizable BAR Capability Register fields

Posted by Krzysztof Wilczyński 10 months ago

Hello,

> PCI Express Base Spec r6.0 defines BAR size up to 8 EB (2^63 bytes),
> but supporting anything bigger than 128TB requires changes to pci_rebar_get_possible_sizes()
> to read the additional Capability bits from the Control register.
> 
> If 8EB support is required, callers will need to be updated to handle u64 instead of u32.
> For now, support is limited to 128TB, and support for sizes greater than 128TB can be
> deferred to a later time.

I think this would be a v4?

If the reviews still stand, then we can correct the subject line while
applying the patch.  No need to send another e-mail.

	Krzysztof

[PATCH v3] PCI: Update Resizable BAR Capability Register fields

Posted by Zhiyuan Dai 10 months ago

PCI Express Base Spec r6.0 defines BAR size up to 8 EB (2^63 bytes),
but supporting anything bigger than 128TB requires changes to pci_rebar_get_possible_sizes()
to read the additional Capability bits from the Control register.

If 8EB support is required, callers will need to be updated to handle u64 instead of u32.
For now, support is limited to 128TB, and support for sizes greater than 128TB can be
deferred to a later time.

Signed-off-by: Zhiyuan Dai <daizhiyuan@phytium.com.cn>
---
 drivers/pci/pci.c             | 4 ++--
 include/uapi/linux/pci_regs.h | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 661f98c6c63a..77b9ceefb4e1 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -3752,7 +3752,7 @@ static int pci_rebar_find_pos(struct pci_dev *pdev, int bar)
  * @bar: BAR to query
  *
  * Get the possible sizes of a resizable BAR as bitmask defined in the spec
- * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizable.
+ * (bit 0=1MB, bit 31=128TB). Returns 0 if BAR isn't resizable.
  */
 u32 pci_rebar_get_possible_sizes(struct pci_dev *pdev, int bar)
 {
@@ -3800,7 +3800,7 @@ int pci_rebar_get_current_size(struct pci_dev *pdev, int bar)
  * pci_rebar_set_size - set a new size for a BAR
  * @pdev: PCI device
  * @bar: BAR to set size to
- * @size: new size as defined in the spec (0=1MB, 19=512GB)
+ * @size: new size as defined in the spec (0=1MB, 31=128TB)
  *
  * Set the new size of a BAR as defined in the spec.
  * Returns zero if resizing was successful, error code otherwise.
diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
index 1601c7ed5fab..ce99d4f34ce5 100644
--- a/include/uapi/linux/pci_regs.h
+++ b/include/uapi/linux/pci_regs.h
@@ -1013,7 +1013,7 @@
 
 /* Resizable BARs */
 #define PCI_REBAR_CAP		4	/* capability register */
-#define  PCI_REBAR_CAP_SIZES		0x00FFFFF0  /* supported BAR sizes */
+#define  PCI_REBAR_CAP_SIZES		0xFFFFFFF0  /* supported BAR sizes */
 #define PCI_REBAR_CTRL		8	/* control register */
 #define  PCI_REBAR_CTRL_BAR_IDX		0x00000007  /* BAR index */
 #define  PCI_REBAR_CTRL_NBAR_MASK	0x000000E0  /* # of resizable BARs */
-- 
2.43.0

Re: [PATCH v3] PCI: Update Resizable BAR Capability Register fields

Posted by Niklas Cassel 10 months ago

On Thu, Feb 20, 2025 at 09:30:34AM +0800, Zhiyuan Dai wrote:
> PCI Express Base Spec r6.0 defines BAR size up to 8 EB (2^63 bytes),
> but supporting anything bigger than 128TB requires changes to pci_rebar_get_possible_sizes()
> to read the additional Capability bits from the Control register.
> 
> If 8EB support is required, callers will need to be updated to handle u64 instead of u32.
> For now, support is limited to 128TB, and support for sizes greater than 128TB can be
> deferred to a later time.

Did you run ./scripts/checkpatch.pl on this?

I'm guessing that you will see:
Prefer a maximum 75 chars per line (possible unwrapped commit description?)

With that fixed:
Reviewed-by: Niklas Cassel <cassel@kernel.org>

> 
> Signed-off-by: Zhiyuan Dai <daizhiyuan@phytium.com.cn>
> ---
>  drivers/pci/pci.c             | 4 ++--
>  include/uapi/linux/pci_regs.h | 2 +-
>  2 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 661f98c6c63a..77b9ceefb4e1 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -3752,7 +3752,7 @@ static int pci_rebar_find_pos(struct pci_dev *pdev, int bar)
>   * @bar: BAR to query
>   *
>   * Get the possible sizes of a resizable BAR as bitmask defined in the spec
> - * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizable.
> + * (bit 0=1MB, bit 31=128TB). Returns 0 if BAR isn't resizable.
>   */
>  u32 pci_rebar_get_possible_sizes(struct pci_dev *pdev, int bar)
>  {
> @@ -3800,7 +3800,7 @@ int pci_rebar_get_current_size(struct pci_dev *pdev, int bar)
>   * pci_rebar_set_size - set a new size for a BAR
>   * @pdev: PCI device
>   * @bar: BAR to set size to
> - * @size: new size as defined in the spec (0=1MB, 19=512GB)
> + * @size: new size as defined in the spec (0=1MB, 31=128TB)
>   *
>   * Set the new size of a BAR as defined in the spec.
>   * Returns zero if resizing was successful, error code otherwise.
> diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
> index 1601c7ed5fab..ce99d4f34ce5 100644
> --- a/include/uapi/linux/pci_regs.h
> +++ b/include/uapi/linux/pci_regs.h
> @@ -1013,7 +1013,7 @@
>  
>  /* Resizable BARs */
>  #define PCI_REBAR_CAP		4	/* capability register */
> -#define  PCI_REBAR_CAP_SIZES		0x00FFFFF0  /* supported BAR sizes */
> +#define  PCI_REBAR_CAP_SIZES		0xFFFFFFF0  /* supported BAR sizes */
>  #define PCI_REBAR_CTRL		8	/* control register */
>  #define  PCI_REBAR_CTRL_BAR_IDX		0x00000007  /* BAR index */
>  #define  PCI_REBAR_CTRL_NBAR_MASK	0x000000E0  /* # of resizable BARs */
> -- 
> 2.43.0
>

[PATCH v3] PCI: Update Resizable BAR Capability Register fields

Posted by Zhiyuan Dai 10 months ago

PCI Express Base Spec r6.0 defines BAR size up to 8 EB (2^63 bytes),
but supporting anything bigger than 128TB requires changes to 
pci_rebar_get_possible_sizes() to read the additional Capability bits 
from the Control register.

If 8EB support is required, callers will need to be updated to handle u64 
instead of u32. For now, support is limited to 128TB, and support for 
sizes greater than 128TB can be deferred to a later time.

Signed-off-by: Zhiyuan Dai <daizhiyuan@phytium.com.cn>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Niklas Cassel <cassel@kernel.org>
---
 drivers/pci/pci.c             | 4 ++--
 include/uapi/linux/pci_regs.h | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 661f98c6c63a..77b9ceefb4e1 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -3752,7 +3752,7 @@ static int pci_rebar_find_pos(struct pci_dev *pdev, int bar)
  * @bar: BAR to query
  *
  * Get the possible sizes of a resizable BAR as bitmask defined in the spec
- * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizable.
+ * (bit 0=1MB, bit 31=128TB). Returns 0 if BAR isn't resizable.
  */
 u32 pci_rebar_get_possible_sizes(struct pci_dev *pdev, int bar)
 {
@@ -3800,7 +3800,7 @@ int pci_rebar_get_current_size(struct pci_dev *pdev, int bar)
  * pci_rebar_set_size - set a new size for a BAR
  * @pdev: PCI device
  * @bar: BAR to set size to
- * @size: new size as defined in the spec (0=1MB, 19=512GB)
+ * @size: new size as defined in the spec (0=1MB, 31=128TB)
  *
  * Set the new size of a BAR as defined in the spec.
  * Returns zero if resizing was successful, error code otherwise.
diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
index 1601c7ed5fab..ce99d4f34ce5 100644
--- a/include/uapi/linux/pci_regs.h
+++ b/include/uapi/linux/pci_regs.h
@@ -1013,7 +1013,7 @@
 
 /* Resizable BARs */
 #define PCI_REBAR_CAP		4	/* capability register */
-#define  PCI_REBAR_CAP_SIZES		0x00FFFFF0  /* supported BAR sizes */
+#define  PCI_REBAR_CAP_SIZES		0xFFFFFFF0  /* supported BAR sizes */
 #define PCI_REBAR_CTRL		8	/* control register */
 #define  PCI_REBAR_CTRL_BAR_IDX		0x00000007  /* BAR index */
 #define  PCI_REBAR_CTRL_NBAR_MASK	0x000000E0  /* # of resizable BARs */
-- 
2.43.0

Re: [PATCH v3] PCI: Update Resizable BAR Capability Register fields

Posted by Ilpo Järvinen 9 months, 2 weeks ago

On Fri, 21 Feb 2025, Zhiyuan Dai wrote:

> PCI Express Base Spec r6.0 defines BAR size up to 8 EB (2^63 bytes),
> but supporting anything bigger than 128TB requires changes to 
> pci_rebar_get_possible_sizes() to read the additional Capability bits 
> from the Control register.
> 
> If 8EB support is required, callers will need to be updated to handle u64 
> instead of u32. For now, support is limited to 128TB, and support for 
> sizes greater than 128TB can be deferred to a later time.
> 
> Signed-off-by: Zhiyuan Dai <daizhiyuan@phytium.com.cn>
> Reviewed-by: Christian König <christian.koenig@amd.com>
> Reviewed-by: Niklas Cassel <cassel@kernel.org>
> ---
>  drivers/pci/pci.c             | 4 ++--
>  include/uapi/linux/pci_regs.h | 2 +-
>  2 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 661f98c6c63a..77b9ceefb4e1 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -3752,7 +3752,7 @@ static int pci_rebar_find_pos(struct pci_dev *pdev, int bar)
>   * @bar: BAR to query
>   *
>   * Get the possible sizes of a resizable BAR as bitmask defined in the spec
> - * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizable.
> + * (bit 0=1MB, bit 31=128TB). Returns 0 if BAR isn't resizable.
>   */
>  u32 pci_rebar_get_possible_sizes(struct pci_dev *pdev, int bar)
>  {
> @@ -3800,7 +3800,7 @@ int pci_rebar_get_current_size(struct pci_dev *pdev, int bar)
>   * pci_rebar_set_size - set a new size for a BAR
>   * @pdev: PCI device
>   * @bar: BAR to set size to
> - * @size: new size as defined in the spec (0=1MB, 19=512GB)
> + * @size: new size as defined in the spec (0=1MB, 31=128TB)
>   *
>   * Set the new size of a BAR as defined in the spec.
>   * Returns zero if resizing was successful, error code otherwise.
> diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
> index 1601c7ed5fab..ce99d4f34ce5 100644
> --- a/include/uapi/linux/pci_regs.h
> +++ b/include/uapi/linux/pci_regs.h
> @@ -1013,7 +1013,7 @@
>  
>  /* Resizable BARs */
>  #define PCI_REBAR_CAP		4	/* capability register */
> -#define  PCI_REBAR_CAP_SIZES		0x00FFFFF0  /* supported BAR sizes */
> +#define  PCI_REBAR_CAP_SIZES		0xFFFFFFF0  /* supported BAR sizes */
>  #define PCI_REBAR_CTRL		8	/* control register */
>  #define  PCI_REBAR_CTRL_BAR_IDX		0x00000007  /* BAR index */
>  #define  PCI_REBAR_CTRL_NBAR_MASK	0x000000E0  /* # of resizable BARs */

pbus_size_mem() can only handle up to 8TB so its aligns array should be 
enlarged as well to support sizes up to 128TB.

-- 
 i.

[PATCH v4] PCI: Update Resizable BAR Capability Register fields

Posted by Zhiyuan Dai 9 months, 2 weeks ago

PCI Express Base Spec r6.0 defines BAR size up to 8 EB (2^63 bytes),
but supporting anything bigger than 128TB requires changes to
pci_rebar_get_possible_sizes() to read the additional Capability bits
from the Control register.

If 8EB support is required, callers will need to be updated to handle u64
instead of u32. For now, support is limited to 128TB, and support for
sizes greater than 128TB can be deferred to a later time.

Expand the alignment array of `pbus_size_mem` to support up to 128TB.

Signed-off-by: Zhiyuan Dai <daizhiyuan@phytium.com.cn>
Reviewed-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Niklas Cassel <cassel@kernel.org>
---
 drivers/pci/pci.c             | 4 ++--
 drivers/pci/setup-bus.c       | 2 +-
 include/uapi/linux/pci_regs.h | 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 661f98c6c63a..77b9ceefb4e1 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -3752,7 +3752,7 @@ static int pci_rebar_find_pos(struct pci_dev *pdev, int bar)
  * @bar: BAR to query
  *
  * Get the possible sizes of a resizable BAR as bitmask defined in the spec
- * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizable.
+ * (bit 0=1MB, bit 31=128TB). Returns 0 if BAR isn't resizable.
  */
 u32 pci_rebar_get_possible_sizes(struct pci_dev *pdev, int bar)
 {
@@ -3800,7 +3800,7 @@ int pci_rebar_get_current_size(struct pci_dev *pdev, int bar)
  * pci_rebar_set_size - set a new size for a BAR
  * @pdev: PCI device
  * @bar: BAR to set size to
- * @size: new size as defined in the spec (0=1MB, 19=512GB)
+ * @size: new size as defined in the spec (0=1MB, 31=128TB)
  *
  * Set the new size of a BAR as defined in the spec.
  * Returns zero if resizing was successful, error code otherwise.
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 5e00cecf1f1a..edb64a6b5585 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1059,7 +1059,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 {
 	struct pci_dev *dev;
 	resource_size_t min_align, win_align, align, size, size0, size1;
-	resource_size_t aligns[24]; /* Alignments from 1MB to 8TB */
+	resource_size_t aligns[28]; /* Alignments from 1MB to 128TB */
 	int order, max_order;
 	struct resource *b_res = find_bus_resource_of_type(bus,
 					mask | IORESOURCE_PREFETCH, type);
diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
index 1601c7ed5fab..ce99d4f34ce5 100644
--- a/include/uapi/linux/pci_regs.h
+++ b/include/uapi/linux/pci_regs.h
@@ -1013,7 +1013,7 @@
 
 /* Resizable BARs */
 #define PCI_REBAR_CAP		4	/* capability register */
-#define  PCI_REBAR_CAP_SIZES		0x00FFFFF0  /* supported BAR sizes */
+#define  PCI_REBAR_CAP_SIZES		0xFFFFFFF0  /* supported BAR sizes */
 #define PCI_REBAR_CTRL		8	/* control register */
 #define  PCI_REBAR_CTRL_BAR_IDX		0x00000007  /* BAR index */
 #define  PCI_REBAR_CTRL_NBAR_MASK	0x000000E0  /* # of resizable BARs */
-- 
2.43.0

Re: [PATCH v4] PCI: Update Resizable BAR Capability Register fields

Posted by Bjorn Helgaas 9 months, 2 weeks ago

On Fri, Mar 07, 2025 at 01:35:29PM +0800, Zhiyuan Dai wrote:
> PCI Express Base Spec r6.0 defines BAR size up to 8 EB (2^63 bytes),
> but supporting anything bigger than 128TB requires changes to
> pci_rebar_get_possible_sizes() to read the additional Capability bits
> from the Control register.
> 
> If 8EB support is required, callers will need to be updated to handle u64
> instead of u32. For now, support is limited to 128TB, and support for
> sizes greater than 128TB can be deferred to a later time.
> 
> Expand the alignment array of `pbus_size_mem` to support up to 128TB.
> 
> Signed-off-by: Zhiyuan Dai <daizhiyuan@phytium.com.cn>
> Reviewed-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> Reviewed-by: Christian König <christian.koenig@amd.com>
> Reviewed-by: Niklas Cassel <cassel@kernel.org>

Replaced the v3 patch that was already applied with this v4 patch,
thanks.

Please:

  - Include a changelog below the "---" marker to tell me what changed
    between v3 and v4.

  - Don't include Reviewed-by from people who haven't explicitly
    replied with that tag.  In this case, arguably you could retain
    those from Christian and Niklas, because they did give that tag
    for v3, and you only added the pbus_size_mem() change.

    But Ilpo only gave you a comment on v3, and did *not* supply his
    Reviewed-by.  You should never create a Reviewed-by tag in that
    event.

    I dropped all the Reviewed-by tags for now; happy to add them
    if/when the reviewers actually supply them.

> ---
>  drivers/pci/pci.c             | 4 ++--
>  drivers/pci/setup-bus.c       | 2 +-
>  include/uapi/linux/pci_regs.h | 2 +-
>  3 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 661f98c6c63a..77b9ceefb4e1 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -3752,7 +3752,7 @@ static int pci_rebar_find_pos(struct pci_dev *pdev, int bar)
>   * @bar: BAR to query
>   *
>   * Get the possible sizes of a resizable BAR as bitmask defined in the spec
> - * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizable.
> + * (bit 0=1MB, bit 31=128TB). Returns 0 if BAR isn't resizable.
>   */
>  u32 pci_rebar_get_possible_sizes(struct pci_dev *pdev, int bar)
>  {
> @@ -3800,7 +3800,7 @@ int pci_rebar_get_current_size(struct pci_dev *pdev, int bar)
>   * pci_rebar_set_size - set a new size for a BAR
>   * @pdev: PCI device
>   * @bar: BAR to set size to
> - * @size: new size as defined in the spec (0=1MB, 19=512GB)
> + * @size: new size as defined in the spec (0=1MB, 31=128TB)
>   *
>   * Set the new size of a BAR as defined in the spec.
>   * Returns zero if resizing was successful, error code otherwise.
> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> index 5e00cecf1f1a..edb64a6b5585 100644
> --- a/drivers/pci/setup-bus.c
> +++ b/drivers/pci/setup-bus.c
> @@ -1059,7 +1059,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
>  {
>  	struct pci_dev *dev;
>  	resource_size_t min_align, win_align, align, size, size0, size1;
> -	resource_size_t aligns[24]; /* Alignments from 1MB to 8TB */
> +	resource_size_t aligns[28]; /* Alignments from 1MB to 128TB */
>  	int order, max_order;
>  	struct resource *b_res = find_bus_resource_of_type(bus,
>  					mask | IORESOURCE_PREFETCH, type);
> diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
> index 1601c7ed5fab..ce99d4f34ce5 100644
> --- a/include/uapi/linux/pci_regs.h
> +++ b/include/uapi/linux/pci_regs.h
> @@ -1013,7 +1013,7 @@
>  
>  /* Resizable BARs */
>  #define PCI_REBAR_CAP		4	/* capability register */
> -#define  PCI_REBAR_CAP_SIZES		0x00FFFFF0  /* supported BAR sizes */
> +#define  PCI_REBAR_CAP_SIZES		0xFFFFFFF0  /* supported BAR sizes */
>  #define PCI_REBAR_CTRL		8	/* control register */
>  #define  PCI_REBAR_CTRL_BAR_IDX		0x00000007  /* BAR index */
>  #define  PCI_REBAR_CTRL_NBAR_MASK	0x000000E0  /* # of resizable BARs */
> -- 
> 2.43.0
>

Re: [PATCH v4] PCI: Update Resizable BAR Capability Register fields

Posted by Zhiyuan Dai 9 months, 1 week ago

Thank you very much for everyone's comments and guidance.

On Fri, 7 Mar 2025 11:32:45 -0600
Bjorn Helgaas <helgaas@kernel.org> wrote:

> On Fri, Mar 07, 2025 at 01:35:29PM +0800, Zhiyuan Dai wrote:
> > PCI Express Base Spec r6.0 defines BAR size up to 8 EB (2^63 bytes),
> > but supporting anything bigger than 128TB requires changes to
> > pci_rebar_get_possible_sizes() to read the additional Capability
> > bits from the Control register.
> > 
> > If 8EB support is required, callers will need to be updated to
> > handle u64 instead of u32. For now, support is limited to 128TB,
> > and support for sizes greater than 128TB can be deferred to a later
> > time.
> > 
> > Expand the alignment array of `pbus_size_mem` to support up to
> > 128TB.
> > 
> > Signed-off-by: Zhiyuan Dai <daizhiyuan@phytium.com.cn>
> > Reviewed-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> > Reviewed-by: Christian König <christian.koenig@amd.com>
> > Reviewed-by: Niklas Cassel <cassel@kernel.org>  
> 
> Replaced the v3 patch that was already applied with this v4 patch,
> thanks.
> 
> Please:
> 
>   - Include a changelog below the "---" marker to tell me what changed
>     between v3 and v4.
> 
>   - Don't include Reviewed-by from people who haven't explicitly
>     replied with that tag.  In this case, arguably you could retain
>     those from Christian and Niklas, because they did give that tag
>     for v3, and you only added the pbus_size_mem() change.
> 
>     But Ilpo only gave you a comment on v3, and did *not* supply his
>     Reviewed-by.  You should never create a Reviewed-by tag in that
>     event.
> 
>     I dropped all the Reviewed-by tags for now; happy to add them
>     if/when the reviewers actually supply them.
> 
> > ---
> >  drivers/pci/pci.c             | 4 ++--
> >  drivers/pci/setup-bus.c       | 2 +-
> >  include/uapi/linux/pci_regs.h | 2 +-
> >  3 files changed, 4 insertions(+), 4 deletions(-)
> > 
> > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> > index 661f98c6c63a..77b9ceefb4e1 100644
> > --- a/drivers/pci/pci.c
> > +++ b/drivers/pci/pci.c
> > @@ -3752,7 +3752,7 @@ static int pci_rebar_find_pos(struct pci_dev
> > *pdev, int bar)
> >   * @bar: BAR to query
> >   *
> >   * Get the possible sizes of a resizable BAR as bitmask defined in
> > the spec
> > - * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizable.
> > + * (bit 0=1MB, bit 31=128TB). Returns 0 if BAR isn't resizable.
> >   */
> >  u32 pci_rebar_get_possible_sizes(struct pci_dev *pdev, int bar)
> >  {
> > @@ -3800,7 +3800,7 @@ int pci_rebar_get_current_size(struct pci_dev
> > *pdev, int bar)
> >   * pci_rebar_set_size - set a new size for a BAR
> >   * @pdev: PCI device
> >   * @bar: BAR to set size to
> > - * @size: new size as defined in the spec (0=1MB, 19=512GB)
> > + * @size: new size as defined in the spec (0=1MB, 31=128TB)
> >   *
> >   * Set the new size of a BAR as defined in the spec.
> >   * Returns zero if resizing was successful, error code otherwise.
> > diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> > index 5e00cecf1f1a..edb64a6b5585 100644
> > --- a/drivers/pci/setup-bus.c
> > +++ b/drivers/pci/setup-bus.c
> > @@ -1059,7 +1059,7 @@ static int pbus_size_mem(struct pci_bus *bus,
> > unsigned long mask, {
> >  	struct pci_dev *dev;
> >  	resource_size_t min_align, win_align, align, size, size0,
> > size1;
> > -	resource_size_t aligns[24]; /* Alignments from 1MB to 8TB
> > */
> > +	resource_size_t aligns[28]; /* Alignments from 1MB to
> > 128TB */ int order, max_order;
> >  	struct resource *b_res = find_bus_resource_of_type(bus,
> >  					mask |
> > IORESOURCE_PREFETCH, type); diff --git
> > a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
> > index 1601c7ed5fab..ce99d4f34ce5 100644 ---
> > a/include/uapi/linux/pci_regs.h +++ b/include/uapi/linux/pci_regs.h
> > @@ -1013,7 +1013,7 @@
> >  
> >  /* Resizable BARs */
> >  #define PCI_REBAR_CAP		4	/* capability
> > register */ -#define  PCI_REBAR_CAP_SIZES		0x00FFFFF0
> >  /* supported BAR sizes */ +#define  PCI_REBAR_CAP_SIZES
> > 	0xFFFFFFF0  /* supported BAR sizes */ #define
> > PCI_REBAR_CTRL		8	/* control register */
> > #define  PCI_REBAR_CTRL_BAR_IDX		0x00000007  /* BAR
> > index */ #define  PCI_REBAR_CTRL_NBAR_MASK	0x000000E0  /* #
> > of resizable BARs */ -- 2.43.0
> >

Re: [PATCH v3] PCI: Update Resizable BAR Capability Register fields

Posted by Christian König 10 months ago

Am 20.02.25 um 02:30 schrieb Zhiyuan Dai:
> PCI Express Base Spec r6.0 defines BAR size up to 8 EB (2^63 bytes),
> but supporting anything bigger than 128TB requires changes to pci_rebar_get_possible_sizes()
> to read the additional Capability bits from the Control register.
>
> If 8EB support is required, callers will need to be updated to handle u64 instead of u32.
> For now, support is limited to 128TB, and support for sizes greater than 128TB can be
> deferred to a later time.
>
> Signed-off-by: Zhiyuan Dai <daizhiyuan@phytium.com.cn>

Reviewed-by: Christian König <christian.koenig@amd.com>

> ---
>  drivers/pci/pci.c             | 4 ++--
>  include/uapi/linux/pci_regs.h | 2 +-
>  2 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 661f98c6c63a..77b9ceefb4e1 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -3752,7 +3752,7 @@ static int pci_rebar_find_pos(struct pci_dev *pdev, int bar)
>   * @bar: BAR to query
>   *
>   * Get the possible sizes of a resizable BAR as bitmask defined in the spec
> - * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizable.
> + * (bit 0=1MB, bit 31=128TB). Returns 0 if BAR isn't resizable.
>   */
>  u32 pci_rebar_get_possible_sizes(struct pci_dev *pdev, int bar)
>  {
> @@ -3800,7 +3800,7 @@ int pci_rebar_get_current_size(struct pci_dev *pdev, int bar)
>   * pci_rebar_set_size - set a new size for a BAR
>   * @pdev: PCI device
>   * @bar: BAR to set size to
> - * @size: new size as defined in the spec (0=1MB, 19=512GB)
> + * @size: new size as defined in the spec (0=1MB, 31=128TB)
>   *
>   * Set the new size of a BAR as defined in the spec.
>   * Returns zero if resizing was successful, error code otherwise.
> diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
> index 1601c7ed5fab..ce99d4f34ce5 100644
> --- a/include/uapi/linux/pci_regs.h
> +++ b/include/uapi/linux/pci_regs.h
> @@ -1013,7 +1013,7 @@
>  
>  /* Resizable BARs */
>  #define PCI_REBAR_CAP		4	/* capability register */
> -#define  PCI_REBAR_CAP_SIZES		0x00FFFFF0  /* supported BAR sizes */
> +#define  PCI_REBAR_CAP_SIZES		0xFFFFFFF0  /* supported BAR sizes */
>  #define PCI_REBAR_CTRL		8	/* control register */
>  #define  PCI_REBAR_CTRL_BAR_IDX		0x00000007  /* BAR index */
>  #define  PCI_REBAR_CTRL_NBAR_MASK	0x000000E0  /* # of resizable BARs */