Add device tree based NUMA support to Arm64

[XEN RFC PATCH 36/40] xen/arm: keep guest still be NUMA unware

Posted by Wei Chen 4 years, 5 months ago

We have not wanted to make Xen guest be NUMA aware in this patch
series. So in this patch, Xen will skip NUMA distance matrix node
and skip the numa-node-id property in CPU node and memory node,
when Xen is creating guest device tree binary.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/arm/domain_build.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index cf341f349f..e62fa761bd 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -584,6 +584,10 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
                 continue;
         }
 
+        /* Guest is numa unaware in current stage */
+        if ( dt_property_name_is_equal(prop, "numa-node-id") )
+            continue;
+
         res = fdt_property(kinfo->fdt, prop->name, prop_data, prop_len);
 
         if ( res )
@@ -1454,6 +1458,8 @@ static int __init handle_node(struct domain *d, struct kernel_info *kinfo,
         DT_MATCH_TYPE("memory"),
         /* The memory mapped timer is not supported by Xen. */
         DT_MATCH_COMPATIBLE("arm,armv7-timer-mem"),
+        /* Numa info doesn't need to be exposed to Domain-0 */
+        DT_MATCH_COMPATIBLE("numa-distance-map-v1"),
         { /* sentinel */ },
     };
     static const struct dt_device_match timer_matches[] __initconst =
-- 
2.25.1

Re: [XEN RFC PATCH 36/40] xen/arm: keep guest still be NUMA unware

Posted by Julien Grall 4 years, 5 months ago

Hi Wei,

On 11/08/2021 11:24, Wei Chen wrote:
> We have not wanted to make Xen guest be NUMA aware in this patch
> series. 

The concept of patch series ceases to exist once we merge the code. So 
about how:

"The NUMA information provided in the host Device-Tree are only for Xen. 
For dom0, we want to hide them as they may be different (for now, dom0 
is still not aware of NUMA".

> So in this patch, Xen will skip NUMA distance matrix node
> and skip the numa-node-id property in CPU node and memory node,
> when Xen is creating guest device tree binary.

The CPU and memory nodes are recreated from scratch for the domain. So 
we already skip the property numa-node-id. However...

> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>   xen/arch/arm/domain_build.c | 6 ++++++
>   1 file changed, 6 insertions(+)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index cf341f349f..e62fa761bd 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -584,6 +584,10 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
>                   continue;
>           }
>   
> +        /* Guest is numa unaware in current stage */
> +        if ( dt_property_name_is_equal(prop, "numa-node-id") )
> +            continue;

... your code is doing more than skipping the property for the two nodes 
you mentionned. Can the property exists in other nodes?

> +
>           res = fdt_property(kinfo->fdt, prop->name, prop_data, prop_len);
>   
>           if ( res )
> @@ -1454,6 +1458,8 @@ static int __init handle_node(struct domain *d, struct kernel_info *kinfo,
>           DT_MATCH_TYPE("memory"),
>           /* The memory mapped timer is not supported by Xen. */
>           DT_MATCH_COMPATIBLE("arm,armv7-timer-mem"),
> +        /* Numa info doesn't need to be exposed to Domain-0 */
> +        DT_MATCH_COMPATIBLE("numa-distance-map-v1"),
>           { /* sentinel */ },
>       };
>       static const struct dt_device_match timer_matches[] __initconst =
> 

Cheers,

-- 
Julien Grall

RE: [XEN RFC PATCH 36/40] xen/arm: keep guest still be NUMA unware

Posted by Wei Chen 4 years, 5 months ago

Hi Julien,

> -----Original Message-----
> From: Julien Grall <julien@xen.org>
> Sent: 2021年8月27日 22:28
> To: Wei Chen <Wei.Chen@arm.com>; xen-devel@lists.xenproject.org;
> sstabellini@kernel.org; jbeulich@suse.com
> Cc: Bertrand Marquis <Bertrand.Marquis@arm.com>
> Subject: Re: [XEN RFC PATCH 36/40] xen/arm: keep guest still be NUMA
> unware
> 
> Hi Wei,
> 
> On 11/08/2021 11:24, Wei Chen wrote:
> > We have not wanted to make Xen guest be NUMA aware in this patch
> > series.
> 
> The concept of patch series ceases to exist once we merge the code. So
> about how:
> 
> "The NUMA information provided in the host Device-Tree are only for Xen.
> For dom0, we want to hide them as they may be different (for now, dom0
> is still not aware of NUMA".
> 

Thanks, I will do it.

> > So in this patch, Xen will skip NUMA distance matrix node
> > and skip the numa-node-id property in CPU node and memory node,
> > when Xen is creating guest device tree binary.
> 
> The CPU and memory nodes are recreated from scratch for the domain. So
> we already skip the property numa-node-id. However...
> 
> > Signed-off-by: Wei Chen <wei.chen@arm.com>
> > ---
> >   xen/arch/arm/domain_build.c | 6 ++++++
> >   1 file changed, 6 insertions(+)
> >
> > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > index cf341f349f..e62fa761bd 100644
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -584,6 +584,10 @@ static int __init write_properties(struct domain *d,
> struct kernel_info *kinfo,
> >                   continue;
> >           }
> >
> > +        /* Guest is numa unaware in current stage */
> > +        if ( dt_property_name_is_equal(prop, "numa-node-id") )
> > +            continue;
> 
> ... your code is doing more than skipping the property for the two nodes
> you mentionned. Can the property exists in other nodes?

Some devices like PCIe may have numa-node-id. In the future maybe more
device can have NUMA property.

> 
> > +
> >           res = fdt_property(kinfo->fdt, prop->name, prop_data,
> prop_len);
> >
> >           if ( res )
> > @@ -1454,6 +1458,8 @@ static int __init handle_node(struct domain *d,
> struct kernel_info *kinfo,
> >           DT_MATCH_TYPE("memory"),
> >           /* The memory mapped timer is not supported by Xen. */
> >           DT_MATCH_COMPATIBLE("arm,armv7-timer-mem"),
> > +        /* Numa info doesn't need to be exposed to Domain-0 */
> > +        DT_MATCH_COMPATIBLE("numa-distance-map-v1"),
> >           { /* sentinel */ },
> >       };
> >       static const struct dt_device_match timer_matches[] __initconst =
> >
> 
> Cheers,
> 
> --
> Julien Grall