[v2] xen/arm: Increase DOM0_FDT_EXTRA_SIZE to support max reserved memory banks

[PATCH V2] xen/arm: Increase DOM0_FDT_EXTRA_SIZE to support max reserved memory banks

Posted by Oleksandr Tyshchenko 1 day, 5 hours ago

From: Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>

Xen fails to construct the hardware domain's device tree with
FDT_ERR_NOSPACE (-3) when the host memory map is highly fragmented
(e.g., numerous reserved memory regions) and the host DTB represents
RAM compactly (e.g., a single reg pair or just a few).

This occurs because DOM0_FDT_EXTRA_SIZE underestimates the space
required for an extra /memory node. While the host DTB might
represent RAM compactly, make_memory_node() aggregates all
reserved regions into a single reg property. With NR_MEM_BANKS (256)
and 64-bit address/size cells, this property can grow up to
4KB (256 * 16), easily exceeding the space originally occupied by
the host DTB's nodes plus the current padding, thereby overflowing
the allocated buffer.

Additionally, the SHM regions require space for discrete sub-nodes
under /reserved-memory node, as well as an appendage to the
main /memory node. Each of the up to NR_SHMEM_BANKS (32) regions
triggers the creation of a sub-node with properties (compatible,
reg, xen,id, and xen,offset). These runtime-generated sub-nodes
require approximately 142 bytes each, while the appendage consumes
an additional 16 bytes per region.

Fix this by increasing DOM0_FDT_EXTRA_SIZE to account for fragmented
reg properties (NR_MEM_BANKS * 16), the discrete SHM sub-nodes, and
the SHM appendage to the /memory node (NR_SHMEM_BANKS * (160 + 16)).
The SHM overhead is conditionally evaluated to avoid over-allocating
memory when CONFIG_STATIC_SHM=n.

Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
  V2:
   - update commit description
   - update in-code comment
   - update macro
---
---
 xen/arch/arm/domain_build.c | 18 ++++++++++++++----
 1 file changed, 14 insertions(+), 4 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 3cd251beed..07f331eac8 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -98,11 +98,21 @@ int __init parse_arch_dom0_param(const char *s, const char *e)
 #endif
 
 /*
- * Amount of extra space required to dom0's device tree.  No new nodes
- * are added (yet) but one terminating reserve map entry (16 bytes) is
- * added.
+ * Amount of extra space required to dom0's device tree. This covers
+ * nodes generated by Xen, which are not directly copied from the host DTB.
+ * It is calculated as:
+ *  - Space for /hypervisor node (128 bytes).
+ *  - The reserve map terminator (16 bytes).
+ *  - Space for a generated /memory node covering all possible reserved
+ *    memory regions (NR_MEM_BANKS * 16).
+ *  - Space for a generated /reserved-memory node with discrete SHM sub-nodes,
+ *    plus the appendage to the main /memory node (NR_SHMEM_BANKS * (160 + 16)
+ *    bytes). This overhead is dropped when CONFIG_STATIC_SHM is disabled.
  */
-#define DOM0_FDT_EXTRA_SIZE (128 + sizeof(struct fdt_reserve_entry))
+#define DOM0_FDT_EXTRA_SIZE (128 + sizeof(struct fdt_reserve_entry) + \
+                             (NR_MEM_BANKS * 16) +                    \
+                             (IS_ENABLED(CONFIG_STATIC_SHM) ?         \
+                              (NR_SHMEM_BANKS * (160 + 16)) : 0))
 
 unsigned int __init dom0_max_vcpus(void)
 {
-- 
2.34.1

Re: [PATCH V2] xen/arm: Increase DOM0_FDT_EXTRA_SIZE to support max reserved memory banks

Posted by Orzel, Michal 14 hours ago


On 31/03/2026 17:43, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>
> 
> Xen fails to construct the hardware domain's device tree with
> FDT_ERR_NOSPACE (-3) when the host memory map is highly fragmented
> (e.g., numerous reserved memory regions) and the host DTB represents
> RAM compactly (e.g., a single reg pair or just a few).
> 
> This occurs because DOM0_FDT_EXTRA_SIZE underestimates the space
> required for an extra /memory node. While the host DTB might
> represent RAM compactly, make_memory_node() aggregates all
> reserved regions into a single reg property. With NR_MEM_BANKS (256)
> and 64-bit address/size cells, this property can grow up to
> 4KB (256 * 16), easily exceeding the space originally occupied by
> the host DTB's nodes plus the current padding, thereby overflowing
> the allocated buffer.
> 
> Additionally, the SHM regions require space for discrete sub-nodes
> under /reserved-memory node, as well as an appendage to the
> main /memory node. Each of the up to NR_SHMEM_BANKS (32) regions
> triggers the creation of a sub-node with properties (compatible,
> reg, xen,id, and xen,offset). These runtime-generated sub-nodes
> require approximately 142 bytes each, while the appendage consumes
> an additional 16 bytes per region.
> 
> Fix this by increasing DOM0_FDT_EXTRA_SIZE to account for fragmented
> reg properties (NR_MEM_BANKS * 16), the discrete SHM sub-nodes, and
> the SHM appendage to the /memory node (NR_SHMEM_BANKS * (160 + 16)).
> The SHM overhead is conditionally evaluated to avoid over-allocating
> memory when CONFIG_STATIC_SHM=n.
> 
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> ---
>   V2:
>    - update commit description
>    - update in-code comment
>    - update macro
> ---
> ---
>  xen/arch/arm/domain_build.c | 18 ++++++++++++++----
>  1 file changed, 14 insertions(+), 4 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 3cd251beed..07f331eac8 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -98,11 +98,21 @@ int __init parse_arch_dom0_param(const char *s, const char *e)
>  #endif
>  
>  /*
> - * Amount of extra space required to dom0's device tree.  No new nodes
> - * are added (yet) but one terminating reserve map entry (16 bytes) is
> - * added.
> + * Amount of extra space required to dom0's device tree. This covers
> + * nodes generated by Xen, which are not directly copied from the host DTB.
> + * It is calculated as:
> + *  - Space for /hypervisor node (128 bytes).
> + *  - The reserve map terminator (16 bytes).
> + *  - Space for a generated /memory node covering all possible reserved
> + *    memory regions (NR_MEM_BANKS * 16).
> + *  - Space for a generated /reserved-memory node with discrete SHM sub-nodes,
> + *    plus the appendage to the main /memory node (NR_SHMEM_BANKS * (160 + 16)
> + *    bytes). This overhead is dropped when CONFIG_STATIC_SHM is disabled.
>   */
> -#define DOM0_FDT_EXTRA_SIZE (128 + sizeof(struct fdt_reserve_entry))
> +#define DOM0_FDT_EXTRA_SIZE (128 + sizeof(struct fdt_reserve_entry) + \
> +                             (NR_MEM_BANKS * 16) +                    \
NIT: alignment. I'll fix on commit

Reviewed-by: Michal Orzel <michal.orzel@amd.com>

~Michal

Re: [PATCH V2] xen/arm: Increase DOM0_FDT_EXTRA_SIZE to support max reserved memory banks

Posted by Orzel, Michal 13 hours ago


On 01/04/2026 08:41, Orzel, Michal wrote:
> 
> 
> On 31/03/2026 17:43, Oleksandr Tyshchenko wrote:
>> From: Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>
>>
>> Xen fails to construct the hardware domain's device tree with
>> FDT_ERR_NOSPACE (-3) when the host memory map is highly fragmented
>> (e.g., numerous reserved memory regions) and the host DTB represents
>> RAM compactly (e.g., a single reg pair or just a few).
>>
>> This occurs because DOM0_FDT_EXTRA_SIZE underestimates the space
>> required for an extra /memory node. While the host DTB might
>> represent RAM compactly, make_memory_node() aggregates all
>> reserved regions into a single reg property. With NR_MEM_BANKS (256)
>> and 64-bit address/size cells, this property can grow up to
>> 4KB (256 * 16), easily exceeding the space originally occupied by
>> the host DTB's nodes plus the current padding, thereby overflowing
>> the allocated buffer.
>>
>> Additionally, the SHM regions require space for discrete sub-nodes
>> under /reserved-memory node, as well as an appendage to the
>> main /memory node. Each of the up to NR_SHMEM_BANKS (32) regions
>> triggers the creation of a sub-node with properties (compatible,
>> reg, xen,id, and xen,offset). These runtime-generated sub-nodes
>> require approximately 142 bytes each, while the appendage consumes
>> an additional 16 bytes per region.
>>
>> Fix this by increasing DOM0_FDT_EXTRA_SIZE to account for fragmented
>> reg properties (NR_MEM_BANKS * 16), the discrete SHM sub-nodes, and
>> the SHM appendage to the /memory node (NR_SHMEM_BANKS * (160 + 16)).
>> The SHM overhead is conditionally evaluated to avoid over-allocating
>> memory when CONFIG_STATIC_SHM=n.
>>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>> ---
>>   V2:
>>    - update commit description
>>    - update in-code comment
>>    - update macro
>> ---
>> ---
>>  xen/arch/arm/domain_build.c | 18 ++++++++++++++----
>>  1 file changed, 14 insertions(+), 4 deletions(-)
>>
>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>> index 3cd251beed..07f331eac8 100644
>> --- a/xen/arch/arm/domain_build.c
>> +++ b/xen/arch/arm/domain_build.c
>> @@ -98,11 +98,21 @@ int __init parse_arch_dom0_param(const char *s, const char *e)
>>  #endif
>>  
>>  /*
>> - * Amount of extra space required to dom0's device tree.  No new nodes
>> - * are added (yet) but one terminating reserve map entry (16 bytes) is
>> - * added.
>> + * Amount of extra space required to dom0's device tree. This covers
>> + * nodes generated by Xen, which are not directly copied from the host DTB.
>> + * It is calculated as:
>> + *  - Space for /hypervisor node (128 bytes).
>> + *  - The reserve map terminator (16 bytes).
>> + *  - Space for a generated /memory node covering all possible reserved
>> + *    memory regions (NR_MEM_BANKS * 16).
>> + *  - Space for a generated /reserved-memory node with discrete SHM sub-nodes,
>> + *    plus the appendage to the main /memory node (NR_SHMEM_BANKS * (160 + 16)
>> + *    bytes). This overhead is dropped when CONFIG_STATIC_SHM is disabled.
>>   */
>> -#define DOM0_FDT_EXTRA_SIZE (128 + sizeof(struct fdt_reserve_entry))
>> +#define DOM0_FDT_EXTRA_SIZE (128 + sizeof(struct fdt_reserve_entry) + \
>> +                             (NR_MEM_BANKS * 16) +                    \
> NIT: alignment. I'll fix on commit
Never mind, the alignment is in fact correct.

~Michal

> 
> Reviewed-by: Michal Orzel <michal.orzel@amd.com>
> 
> ~Michal
> 
>