[XEN RFC PATCH 17/40] xen/arm: Introduce DEVICE_TREE_NUMA Kconfig for arm64

Wei Chen posted 40 patches 4 years, 5 months ago
[XEN RFC PATCH 17/40] xen/arm: Introduce DEVICE_TREE_NUMA Kconfig for arm64
Posted by Wei Chen 4 years, 5 months ago
We need a Kconfig option to distinguish with ACPI based
NUMA. So we introduce the new Kconfig option:
DEVICE_TREE_NUMA in this patch for Arm64.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/arm/Kconfig | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index ecfa6822e4..678cc98ea3 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -33,6 +33,16 @@ config ACPI
 	  Advanced Configuration and Power Interface (ACPI) support for Xen is
 	  an alternative to device tree on ARM64.
 
+config DEVICE_TREE_NUMA
+	bool "NUMA (Non-Uniform Memory Access) Support (UNSUPPORTED)" if UNSUPPORTED
+	depends on ARM_64
+	select NUMA
+	---help---
+
+	  Non-Uniform Memory Access (NUMA) is a computer memory design used in
+	  multiprocessing, where the memory access time depends on the memory
+	  location relative to the processor.
+
 config GICV3
 	bool "GICv3 driver"
 	depends on ARM_64 && !NEW_VGIC
-- 
2.25.1


Re: [XEN RFC PATCH 17/40] xen/arm: Introduce DEVICE_TREE_NUMA Kconfig for arm64
Posted by Julien Grall 4 years, 5 months ago
Hi,

On 11/08/2021 11:24, Wei Chen wrote:
> We need a Kconfig option to distinguish with ACPI based
> NUMA. So we introduce the new Kconfig option:
> DEVICE_TREE_NUMA in this patch for Arm64.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>   xen/arch/arm/Kconfig | 10 ++++++++++
>   1 file changed, 10 insertions(+)
> 
> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
> index ecfa6822e4..678cc98ea3 100644
> --- a/xen/arch/arm/Kconfig
> +++ b/xen/arch/arm/Kconfig
> @@ -33,6 +33,16 @@ config ACPI
>   	  Advanced Configuration and Power Interface (ACPI) support for Xen is
>   	  an alternative to device tree on ARM64.
>   
> +config DEVICE_TREE_NUMA

The name suggests that NUMA should only be enabled for Device-Tree... 
But the description looks generic.

However, I think the user should only have the choice to say whether 
they want NUMA to be enabled or not. We should not give them the choice 
to enable/disable the parsing for DT/ACPI.

So we should have a generic config that will then select DT (and ACPI in 
the future).

> +	bool "NUMA (Non-Uniform Memory Access) Support (UNSUPPORTED)" if UNSUPPORTED
> +	depends on ARM_64
> +	select NUMA
> +	---help---
> +
> +	  Non-Uniform Memory Access (NUMA) is a computer memory design used in
> +	  multiprocessing, where the memory access time depends on the memory
> +	  location relative to the processor.
> +

Cheers,

-- 
Julien Grall

RE: [XEN RFC PATCH 17/40] xen/arm: Introduce DEVICE_TREE_NUMA Kconfig for arm64
Posted by Wei Chen 4 years, 5 months ago
Hi Julien,

> -----Original Message-----
> From: Julien Grall <julien@xen.org>
> Sent: 2021年8月19日 21:38
> To: Wei Chen <Wei.Chen@arm.com>; xen-devel@lists.xenproject.org;
> sstabellini@kernel.org; jbeulich@suse.com
> Cc: Bertrand Marquis <Bertrand.Marquis@arm.com>
> Subject: Re: [XEN RFC PATCH 17/40] xen/arm: Introduce DEVICE_TREE_NUMA
> Kconfig for arm64
> 
> Hi,
> 
> On 11/08/2021 11:24, Wei Chen wrote:
> > We need a Kconfig option to distinguish with ACPI based
> > NUMA. So we introduce the new Kconfig option:
> > DEVICE_TREE_NUMA in this patch for Arm64.
> >
> > Signed-off-by: Wei Chen <wei.chen@arm.com>
> > ---
> >   xen/arch/arm/Kconfig | 10 ++++++++++
> >   1 file changed, 10 insertions(+)
> >
> > diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
> > index ecfa6822e4..678cc98ea3 100644
> > --- a/xen/arch/arm/Kconfig
> > +++ b/xen/arch/arm/Kconfig
> > @@ -33,6 +33,16 @@ config ACPI
> >   	  Advanced Configuration and Power Interface (ACPI) support for Xen
> is
> >   	  an alternative to device tree on ARM64.
> >
> > +config DEVICE_TREE_NUMA
> 
> The name suggests that NUMA should only be enabled for Device-Tree...
> But the description looks generic.
> 
> However, I think the user should only have the choice to say whether
> they want NUMA to be enabled or not. We should not give them the choice
> to enable/disable the parsing for DT/ACPI.
> 
> So we should have a generic config that will then select DT (and ACPI in
> the future).
> 

How about we select DT_NUMA default on Arm64. And DT_NUMA select NUMA
like what we have done in patch#6 in x86? And remove the description?

If we make generic NUMA as a selectable option, and depends on
NUMA to select DT or ACPI NUMA. It seems to be quite different from
the existing logic?

> > +	bool "NUMA (Non-Uniform Memory Access) Support (UNSUPPORTED)" if
> UNSUPPORTED
> > +	depends on ARM_64
> > +	select NUMA
> > +	---help---
> > +
> > +	  Non-Uniform Memory Access (NUMA) is a computer memory design used
> in
> > +	  multiprocessing, where the memory access time depends on the
> memory
> > +	  location relative to the processor.
> > +
> 
> Cheers,
> 
> --
> Julien Grall
Re: [XEN RFC PATCH 17/40] xen/arm: Introduce DEVICE_TREE_NUMA Kconfig for arm64
Posted by Julien Grall 4 years, 5 months ago
On 20/08/2021 03:30, Wei Chen wrote:
> Hi Julien,

Hi Wei,

>> -----Original Message-----
>> From: Julien Grall <julien@xen.org>
>> Sent: 2021年8月19日 21:38
>> To: Wei Chen <Wei.Chen@arm.com>; xen-devel@lists.xenproject.org;
>> sstabellini@kernel.org; jbeulich@suse.com
>> Cc: Bertrand Marquis <Bertrand.Marquis@arm.com>
>> Subject: Re: [XEN RFC PATCH 17/40] xen/arm: Introduce DEVICE_TREE_NUMA
>> Kconfig for arm64
>>
>> Hi,
>>
>> On 11/08/2021 11:24, Wei Chen wrote:
>>> We need a Kconfig option to distinguish with ACPI based
>>> NUMA. So we introduce the new Kconfig option:
>>> DEVICE_TREE_NUMA in this patch for Arm64.
>>>
>>> Signed-off-by: Wei Chen <wei.chen@arm.com>
>>> ---
>>>    xen/arch/arm/Kconfig | 10 ++++++++++
>>>    1 file changed, 10 insertions(+)
>>>
>>> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
>>> index ecfa6822e4..678cc98ea3 100644
>>> --- a/xen/arch/arm/Kconfig
>>> +++ b/xen/arch/arm/Kconfig
>>> @@ -33,6 +33,16 @@ config ACPI
>>>    	  Advanced Configuration and Power Interface (ACPI) support for Xen
>> is
>>>    	  an alternative to device tree on ARM64.
>>>
>>> +config DEVICE_TREE_NUMA
>>
>> The name suggests that NUMA should only be enabled for Device-Tree...
>> But the description looks generic.
>>
>> However, I think the user should only have the choice to say whether
>> they want NUMA to be enabled or not. We should not give them the choice
>> to enable/disable the parsing for DT/ACPI.
>>
>> So we should have a generic config that will then select DT (and ACPI in
>> the future).
>>
> 
> How about we select DT_NUMA default on Arm64. And DT_NUMA select NUMA
> like what we have done in patch#6 in x86? And remove the description?
I would rather not make NUMA supported by default on Arm64. Instead, we 
should go throught the same process as other new features and gate it 
behind UNSUPPORTED until it is mature enough.

> 
> If we make generic NUMA as a selectable option, and depends on
> NUMA to select DT or ACPI NUMA. It seems to be quite different from
> the existing logic?

I am a bit confused. You added just logic to select NUMA from ACPI, 
right? So are you talking about a different logic?

Cheers,

-- 
Julien Grall

RE: [XEN RFC PATCH 17/40] xen/arm: Introduce DEVICE_TREE_NUMA Kconfig for arm64
Posted by Wei Chen 4 years, 5 months ago
Hi Julien,

> -----Original Message-----
> From: Julien Grall <julien@xen.org>
> Sent: 2021年8月20日 16:41
> To: Wei Chen <Wei.Chen@arm.com>; xen-devel@lists.xenproject.org;
> sstabellini@kernel.org; jbeulich@suse.com
> Cc: Bertrand Marquis <Bertrand.Marquis@arm.com>
> Subject: Re: [XEN RFC PATCH 17/40] xen/arm: Introduce DEVICE_TREE_NUMA
> Kconfig for arm64
> 
> On 20/08/2021 03:30, Wei Chen wrote:
> > Hi Julien,
> 
> Hi Wei,
> 
> >> -----Original Message-----
> >> From: Julien Grall <julien@xen.org>
> >> Sent: 2021年8月19日 21:38
> >> To: Wei Chen <Wei.Chen@arm.com>; xen-devel@lists.xenproject.org;
> >> sstabellini@kernel.org; jbeulich@suse.com
> >> Cc: Bertrand Marquis <Bertrand.Marquis@arm.com>
> >> Subject: Re: [XEN RFC PATCH 17/40] xen/arm: Introduce DEVICE_TREE_NUMA
> >> Kconfig for arm64
> >>
> >> Hi,
> >>
> >> On 11/08/2021 11:24, Wei Chen wrote:
> >>> We need a Kconfig option to distinguish with ACPI based
> >>> NUMA. So we introduce the new Kconfig option:
> >>> DEVICE_TREE_NUMA in this patch for Arm64.
> >>>
> >>> Signed-off-by: Wei Chen <wei.chen@arm.com>
> >>> ---
> >>>    xen/arch/arm/Kconfig | 10 ++++++++++
> >>>    1 file changed, 10 insertions(+)
> >>>
> >>> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
> >>> index ecfa6822e4..678cc98ea3 100644
> >>> --- a/xen/arch/arm/Kconfig
> >>> +++ b/xen/arch/arm/Kconfig
> >>> @@ -33,6 +33,16 @@ config ACPI
> >>>    	  Advanced Configuration and Power Interface (ACPI) support
> for Xen
> >> is
> >>>    	  an alternative to device tree on ARM64.
> >>>
> >>> +config DEVICE_TREE_NUMA
> >>
> >> The name suggests that NUMA should only be enabled for Device-Tree...
> >> But the description looks generic.
> >>
> >> However, I think the user should only have the choice to say whether
> >> they want NUMA to be enabled or not. We should not give them the choice
> >> to enable/disable the parsing for DT/ACPI.
> >>
> >> So we should have a generic config that will then select DT (and ACPI
> in
> >> the future).
> >>
> >
> > How about we select DT_NUMA default on Arm64. And DT_NUMA select NUMA
> > like what we have done in patch#6 in x86? And remove the description?
> I would rather not make NUMA supported by default on Arm64. Instead, we
> should go throught the same process as other new features and gate it
> behind UNSUPPORTED until it is mature enough.
> 

Ok. I agree with this.

> >
> > If we make generic NUMA as a selectable option, and depends on
> > NUMA to select DT or ACPI NUMA. It seems to be quite different from
> > the existing logic?
> 
> I am a bit confused. You added just logic to select NUMA from ACPI,
> right? So are you talking about a different logic?
> 

No, I didn't want a different one. I thought you wanted it that way.
Obviously, I mis-understanded your comments.

Can I understand your previous comments like following:
1. We should have a generic config that will then select DT and ACPI:
   Because we already have CONFIG_NUMA in common layer. So we need to
   add another one for Arm like CONFIG_ARM_NUMA?
   And in this option, we can select CONFIG_DEVICE_TREE_NUMA
   automatically if device tree is enabled. If CONFIG_ACPI
   is enabled, we will select CONFIG_ACPI_NUMA too (in the
   future)
   In Xen code, DT_NUMA and ACPI_NUMA code can co-exist, Xen
   will check the system ACPI support status to decide to use
   DT_NUMA or ACPI_NUMA?


> Cheers,
> 
> --
> Julien Grall
Re: [XEN RFC PATCH 17/40] xen/arm: Introduce DEVICE_TREE_NUMA Kconfig for arm64
Posted by Julien Grall 4 years, 5 months ago

On 20/08/2021 11:49, Wei Chen wrote:
> Hi Julien,

Hi Wei,

> 
>> -----Original Message-----
>> From: Julien Grall <julien@xen.org>
>> Sent: 2021年8月20日 16:41
>> To: Wei Chen <Wei.Chen@arm.com>; xen-devel@lists.xenproject.org;
>> sstabellini@kernel.org; jbeulich@suse.com
>> Cc: Bertrand Marquis <Bertrand.Marquis@arm.com>
>> Subject: Re: [XEN RFC PATCH 17/40] xen/arm: Introduce DEVICE_TREE_NUMA
>> Kconfig for arm64
>>
>> On 20/08/2021 03:30, Wei Chen wrote:
>>> Hi Julien,
>>
>> Hi Wei,
>>
>>>> -----Original Message-----
>>>> From: Julien Grall <julien@xen.org>
>>>> Sent: 2021年8月19日 21:38
>>>> To: Wei Chen <Wei.Chen@arm.com>; xen-devel@lists.xenproject.org;
>>>> sstabellini@kernel.org; jbeulich@suse.com
>>>> Cc: Bertrand Marquis <Bertrand.Marquis@arm.com>
>>>> Subject: Re: [XEN RFC PATCH 17/40] xen/arm: Introduce DEVICE_TREE_NUMA
>>>> Kconfig for arm64
>>>>
>>>> Hi,
>>>>
>>>> On 11/08/2021 11:24, Wei Chen wrote:
>>>>> We need a Kconfig option to distinguish with ACPI based
>>>>> NUMA. So we introduce the new Kconfig option:
>>>>> DEVICE_TREE_NUMA in this patch for Arm64.
>>>>>
>>>>> Signed-off-by: Wei Chen <wei.chen@arm.com>
>>>>> ---
>>>>>     xen/arch/arm/Kconfig | 10 ++++++++++
>>>>>     1 file changed, 10 insertions(+)
>>>>>
>>>>> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
>>>>> index ecfa6822e4..678cc98ea3 100644
>>>>> --- a/xen/arch/arm/Kconfig
>>>>> +++ b/xen/arch/arm/Kconfig
>>>>> @@ -33,6 +33,16 @@ config ACPI
>>>>>     	  Advanced Configuration and Power Interface (ACPI) support
>> for Xen
>>>> is
>>>>>     	  an alternative to device tree on ARM64.
>>>>>
>>>>> +config DEVICE_TREE_NUMA
>>>>
>>>> The name suggests that NUMA should only be enabled for Device-Tree...
>>>> But the description looks generic.
>>>>
>>>> However, I think the user should only have the choice to say whether
>>>> they want NUMA to be enabled or not. We should not give them the choice
>>>> to enable/disable the parsing for DT/ACPI.
>>>>
>>>> So we should have a generic config that will then select DT (and ACPI
>> in
>>>> the future).
>>>>
>>>
>>> How about we select DT_NUMA default on Arm64. And DT_NUMA select NUMA
>>> like what we have done in patch#6 in x86? And remove the description?
>> I would rather not make NUMA supported by default on Arm64. Instead, we
>> should go throught the same process as other new features and gate it
>> behind UNSUPPORTED until it is mature enough.
>>
> 
> Ok. I agree with this.
> 
>>>
>>> If we make generic NUMA as a selectable option, and depends on
>>> NUMA to select DT or ACPI NUMA. It seems to be quite different from
>>> the existing logic?
>>
>> I am a bit confused. You added just logic to select NUMA from ACPI,
>> right? So are you talking about a different logic?
>>
> 
> No, I didn't want a different one. I thought you wanted it that way.
> Obviously, I mis-understanded your comments.
> 
> Can I understand your previous comments like following:
> 1. We should have a generic config that will then select DT and ACPI:
>     Because we already have CONFIG_NUMA in common layer. So we need to
>     add another one for Arm like CONFIG_ARM_NUMA?

I think so.

>     And in this option, we can select CONFIG_DEVICE_TREE_NUMA
>     automatically if device tree is enabled. If CONFIG_ACPI
>     is enabled, we will select CONFIG_ACPI_NUMA too (in the
>     future)
>     In Xen code, DT_NUMA and ACPI_NUMA code can co-exist, Xen

Distributions should not have to build a different Xen for DT and ACPI. 
So it is more they *must* co-exist.

>     will check the system ACPI support status to decide to use
>     DT_NUMA or ACPI_NUMA?

Yes. A user should only have to say "I want to use NUMA". This is Xen to 
figure out whether we need to compile the support for DT and/or ACPI.

Once we have support for APCI, it doesn't make a lot of sense for the 
users to say "I want to compile with DT and ACPI but I only want NUMA 
when using DT".

Cheers,

-- 
Julien Grall

RE: [XEN RFC PATCH 17/40] xen/arm: Introduce DEVICE_TREE_NUMA Kconfig for arm64
Posted by Wei Chen 4 years, 5 months ago
Hi Julien,

> -----Original Message-----
> From: Julien Grall <julien@xen.org>
> Sent: 2021年8月20日 19:29
> To: Wei Chen <Wei.Chen@arm.com>; xen-devel@lists.xenproject.org;
> sstabellini@kernel.org
> Cc: Bertrand Marquis <Bertrand.Marquis@arm.com>
> Subject: Re: [XEN RFC PATCH 17/40] xen/arm: Introduce DEVICE_TREE_NUMA
> Kconfig for arm64
> 
> 
> 
> On 20/08/2021 11:49, Wei Chen wrote:
> > Hi Julien,
> 
> Hi Wei,
> 
> >
> >> -----Original Message-----
> >> From: Julien Grall <julien@xen.org>
> >> Sent: 2021年8月20日 16:41
> >> To: Wei Chen <Wei.Chen@arm.com>; xen-devel@lists.xenproject.org;
> >> sstabellini@kernel.org; jbeulich@suse.com
> >> Cc: Bertrand Marquis <Bertrand.Marquis@arm.com>
> >> Subject: Re: [XEN RFC PATCH 17/40] xen/arm: Introduce DEVICE_TREE_NUMA
> >> Kconfig for arm64
> >>
> >> On 20/08/2021 03:30, Wei Chen wrote:
> >>> Hi Julien,
> >>
> >> Hi Wei,
> >>
> >>>> -----Original Message-----
> >>>> From: Julien Grall <julien@xen.org>
> >>>> Sent: 2021年8月19日 21:38
> >>>> To: Wei Chen <Wei.Chen@arm.com>; xen-devel@lists.xenproject.org;
> >>>> sstabellini@kernel.org; jbeulich@suse.com
> >>>> Cc: Bertrand Marquis <Bertrand.Marquis@arm.com>
> >>>> Subject: Re: [XEN RFC PATCH 17/40] xen/arm: Introduce
> DEVICE_TREE_NUMA
> >>>> Kconfig for arm64
> >>>>
> >>>> Hi,
> >>>>
> >>>> On 11/08/2021 11:24, Wei Chen wrote:
> >>>>> We need a Kconfig option to distinguish with ACPI based
> >>>>> NUMA. So we introduce the new Kconfig option:
> >>>>> DEVICE_TREE_NUMA in this patch for Arm64.
> >>>>>
> >>>>> Signed-off-by: Wei Chen <wei.chen@arm.com>
> >>>>> ---
> >>>>>     xen/arch/arm/Kconfig | 10 ++++++++++
> >>>>>     1 file changed, 10 insertions(+)
> >>>>>
> >>>>> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
> >>>>> index ecfa6822e4..678cc98ea3 100644
> >>>>> --- a/xen/arch/arm/Kconfig
> >>>>> +++ b/xen/arch/arm/Kconfig
> >>>>> @@ -33,6 +33,16 @@ config ACPI
> >>>>>     	  Advanced Configuration and Power Interface (ACPI) support
> >> for Xen
> >>>> is
> >>>>>     	  an alternative to device tree on ARM64.
> >>>>>
> >>>>> +config DEVICE_TREE_NUMA
> >>>>
> >>>> The name suggests that NUMA should only be enabled for Device-Tree...
> >>>> But the description looks generic.
> >>>>
> >>>> However, I think the user should only have the choice to say whether
> >>>> they want NUMA to be enabled or not. We should not give them the
> choice
> >>>> to enable/disable the parsing for DT/ACPI.
> >>>>
> >>>> So we should have a generic config that will then select DT (and ACPI
> >> in
> >>>> the future).
> >>>>
> >>>
> >>> How about we select DT_NUMA default on Arm64. And DT_NUMA select NUMA
> >>> like what we have done in patch#6 in x86? And remove the description?
> >> I would rather not make NUMA supported by default on Arm64. Instead, we
> >> should go throught the same process as other new features and gate it
> >> behind UNSUPPORTED until it is mature enough.
> >>
> >
> > Ok. I agree with this.
> >
> >>>
> >>> If we make generic NUMA as a selectable option, and depends on
> >>> NUMA to select DT or ACPI NUMA. It seems to be quite different from
> >>> the existing logic?
> >>
> >> I am a bit confused. You added just logic to select NUMA from ACPI,
> >> right? So are you talking about a different logic?
> >>
> >
> > No, I didn't want a different one. I thought you wanted it that way.
> > Obviously, I mis-understanded your comments.
> >
> > Can I understand your previous comments like following:
> > 1. We should have a generic config that will then select DT and ACPI:
> >     Because we already have CONFIG_NUMA in common layer. So we need to
> >     add another one for Arm like CONFIG_ARM_NUMA?
> 
> I think so.
> 
> >     And in this option, we can select CONFIG_DEVICE_TREE_NUMA
> >     automatically if device tree is enabled. If CONFIG_ACPI
> >     is enabled, we will select CONFIG_ACPI_NUMA too (in the
> >     future)
> >     In Xen code, DT_NUMA and ACPI_NUMA code can co-exist, Xen
> 
> Distributions should not have to build a different Xen for DT and ACPI.
> So it is more they *must* co-exist.
> 
> >     will check the system ACPI support status to decide to use
> >     DT_NUMA or ACPI_NUMA?
> 
> Yes. A user should only have to say "I want to use NUMA". This is Xen to
> figure out whether we need to compile the support for DT and/or ACPI.
> 
> Once we have support for APCI, it doesn't make a lot of sense for the
> users to say "I want to compile with DT and ACPI but I only want NUMA
> when using DT".
> 

I am glad we are now in the same page. Ok, I will change the Kconfig
like this in next version.

> Cheers,
> 
> --
> Julien Grall