Hi Eric,

> -----Original Message-----
> From: Eric Auger <eauger@redhat.com>
> Sent: Thursday, December 12, 2024 8:42 AM
> To: eric.auger@redhat.com; Cornelia Huck <cohuck@redhat.com>;
> eric.auger.pro@gmail.com; qemu-devel@nongnu.org; qemu-
> arm@nongnu.org; kvmarm@lists.linux.dev; peter.maydell@linaro.org;
> richard.henderson@linaro.org; alex.bennee@linaro.org; maz@kernel.org;
> oliver.upton@linux.dev; sebott@redhat.com; Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com>; armbru@redhat.com;
> berrange@redhat.com; abologna@redhat.com; jdenemar@redhat.com
> Cc: shahuang@redhat.com; mark.rutland@arm.com; philmd@linaro.org;
> pbonzini@redhat.com
> Subject: Re: [PATCH RFCv2 00/20] kvm/arm: Introduce a customizable
> aarch64 KVM host model
> 
> Shameer,
> 
> On 12/12/24 09:12, Eric Auger wrote:
> > Connie,
> >
> > On 12/6/24 12:21, Cornelia Huck wrote:
> >> A respin/update on the aarch64 KVM cpu models. Also available at
> >> gitlab.com/cohuck/qemu arm-cpu-model-rfcv2
> >>
> >> Find Eric's original cover letter below, so that I do not need to
> >> repeat myself on the aspects that have not changed since RFCv1 :)
> >>
> >> Changes from RFCv1:
> >>
> >> Rebased on more recent QEMU (some adaptions in the register
> conversions
> >> of the first few patches.)
> >>
> >> Based on feedback, I have removed the "custom" cpu model; instead, I
> >> have added the new SYSREG_<REG>_<FIELD> properties to the "host"
> model.
> >> This works well if you want to tweak anything that does not correspond
> >> to the existing properties for the host model; however, if you e.g.
> >> wanted to tweak sve, you have two ways to do so -- we'd probably either
> >> want to check for conflicts, or just declare precedence. The kvm-specific
> >> props remain unchanged, as they are orthogonal to this configuration.
> >>
> >> The cpu model expansion for the "host" model now dumps the new
> SYSREG_
> >> properties in addition to the existing host model properties; this is a
> >> bit ugly, but I don't see a good way on how to split this up.
> >>
> >> Some more adaptions due to the removal of the "custom" model.
> >>
> >> Things *not* changed from RFCv1:
> >>
> >> SYSREG_ property naming (can be tweaked easily, once we are clear on
> what
> >> the interface should look like.)
> >>
> >> Sysreg generation scripts, and the generated files (I have not updated
> >> anything there.) I think generating the various definitions makes sense,
> >> as long as we double-check the generated files on each update (which
> would
> >> be something to trigger manually anyway.)
> >>
> >> What I would like us to reach some kind of consensus on:
> >>
> >> How to continue with the patches moving the ID registers from the isar
> >> struct into the idregs array. These are a bit of churn to drag along;
> >> if they make sense, maybe they can be picked independently of this
> series?
> >>
> >> Whether it make sense to continue with the approach of tweaking
> values in
> >> the ID registers in general. If we want to be able to migrate between
> cpus
> >> that do not differ wildly, we'll encounter differences that cannot be
> >> expressed via FEAT_xxx -- e.g. when comparing various AmpereAltra Max
> systems,
> >> they only differ in parts of CTR_EL0 -- which is not a feature register, but
> >> a writable register.
> > In v1 most of the commenters said they would prefer to see FEAT props
> > instead of IDREG field props. I think we shall try to go in this
> > direction anyway. As you pointed out there will be some cases where
> FEAT
> > won't be enough (CTR_EL0 is a good example). So I tend to think the end
> > solution will be a mix of FEAT and ID reg field props.
> >
> > Personally I would smoothly migrate what we can from ID reg field props
> > to FEAT props (maybe using prop aliases?), starting from the easiest 1-1
> > mappings and then adressing the FEAT that are more complex but are
> > explictly needed to enable the use cases we are interested in, at RedHat:
> > migration within Ampere AltraMax family, migration within NVidia Grace
> > family, migration within AmpereOne family and migration between
> Graviton3/4.
> >
> > We have no info about other's use cases. If some of you want to see some
> > other live migration combinations addressed, please raise your voice.
> In relation to [1] you seem to be also interested in the migration
> between heterogeneous systems with qemu.

Yes. That is correct.

> Do you think targeting migration within a cpu family is enough for your
> use cases. How different are the source and destination host on your
> cases. Do you thing feat props are relevant in your case or would you
> need lower granularity at idreg field levelto pass the migration?

I think, from the current requirement we have for migration, the source and
destination mostly can be handled by FEAT_XXX. But like Ampere, we do need
to manage the CTR_EL0 differences[1].

Also we do have differences in GIC support as well(AA64PFR0_EL1.GIC) which 
I am not sure how to manage with FEAT_XXX.

And we are checking with our Product team whether we need to support migration
from an old CPU type in which case we have to do a bit more analysis.

Thanks,
Shameer

1. https://lore.kernel.org/kvmarm/20241022073943.35764-1-shameerali.kolothum.thodi@huawei.com/



 
 

Hi Shameer,
On 12/12/24 14:09, Shameerali Kolothum Thodi wrote:
> Hi Eric,
> 
>> -----Original Message-----
>> From: Eric Auger <eauger@redhat.com>
>> Sent: Thursday, December 12, 2024 8:42 AM
>> To: eric.auger@redhat.com; Cornelia Huck <cohuck@redhat.com>;
>> eric.auger.pro@gmail.com; qemu-devel@nongnu.org; qemu-
>> arm@nongnu.org; kvmarm@lists.linux.dev; peter.maydell@linaro.org;
>> richard.henderson@linaro.org; alex.bennee@linaro.org; maz@kernel.org;
>> oliver.upton@linux.dev; sebott@redhat.com; Shameerali Kolothum Thodi
>> <shameerali.kolothum.thodi@huawei.com>; armbru@redhat.com;
>> berrange@redhat.com; abologna@redhat.com; jdenemar@redhat.com
>> Cc: shahuang@redhat.com; mark.rutland@arm.com; philmd@linaro.org;
>> pbonzini@redhat.com
>> Subject: Re: [PATCH RFCv2 00/20] kvm/arm: Introduce a customizable
>> aarch64 KVM host model
>>
>> Shameer,
>>
>> On 12/12/24 09:12, Eric Auger wrote:
>>> Connie,
>>>
>>> On 12/6/24 12:21, Cornelia Huck wrote:
>>>> A respin/update on the aarch64 KVM cpu models. Also available at
>>>> gitlab.com/cohuck/qemu arm-cpu-model-rfcv2
>>>>
>>>> Find Eric's original cover letter below, so that I do not need to
>>>> repeat myself on the aspects that have not changed since RFCv1 :)
>>>>
>>>> Changes from RFCv1:
>>>>
>>>> Rebased on more recent QEMU (some adaptions in the register
>> conversions
>>>> of the first few patches.)
>>>>
>>>> Based on feedback, I have removed the "custom" cpu model; instead, I
>>>> have added the new SYSREG_<REG>_<FIELD> properties to the "host"
>> model.
>>>> This works well if you want to tweak anything that does not correspond
>>>> to the existing properties for the host model; however, if you e.g.
>>>> wanted to tweak sve, you have two ways to do so -- we'd probably either
>>>> want to check for conflicts, or just declare precedence. The kvm-specific
>>>> props remain unchanged, as they are orthogonal to this configuration.
>>>>
>>>> The cpu model expansion for the "host" model now dumps the new
>> SYSREG_
>>>> properties in addition to the existing host model properties; this is a
>>>> bit ugly, but I don't see a good way on how to split this up.
>>>>
>>>> Some more adaptions due to the removal of the "custom" model.
>>>>
>>>> Things *not* changed from RFCv1:
>>>>
>>>> SYSREG_ property naming (can be tweaked easily, once we are clear on
>> what
>>>> the interface should look like.)
>>>>
>>>> Sysreg generation scripts, and the generated files (I have not updated
>>>> anything there.) I think generating the various definitions makes sense,
>>>> as long as we double-check the generated files on each update (which
>> would
>>>> be something to trigger manually anyway.)
>>>>
>>>> What I would like us to reach some kind of consensus on:
>>>>
>>>> How to continue with the patches moving the ID registers from the isar
>>>> struct into the idregs array. These are a bit of churn to drag along;
>>>> if they make sense, maybe they can be picked independently of this
>> series?
>>>>
>>>> Whether it make sense to continue with the approach of tweaking
>> values in
>>>> the ID registers in general. If we want to be able to migrate between
>> cpus
>>>> that do not differ wildly, we'll encounter differences that cannot be
>>>> expressed via FEAT_xxx -- e.g. when comparing various AmpereAltra Max
>> systems,
>>>> they only differ in parts of CTR_EL0 -- which is not a feature register, but
>>>> a writable register.
>>> In v1 most of the commenters said they would prefer to see FEAT props
>>> instead of IDREG field props. I think we shall try to go in this
>>> direction anyway. As you pointed out there will be some cases where
>> FEAT
>>> won't be enough (CTR_EL0 is a good example). So I tend to think the end
>>> solution will be a mix of FEAT and ID reg field props.
>>>
>>> Personally I would smoothly migrate what we can from ID reg field props
>>> to FEAT props (maybe using prop aliases?), starting from the easiest 1-1
>>> mappings and then adressing the FEAT that are more complex but are
>>> explictly needed to enable the use cases we are interested in, at RedHat:
>>> migration within Ampere AltraMax family, migration within NVidia Grace
>>> family, migration within AmpereOne family and migration between
>> Graviton3/4.
>>>
>>> We have no info about other's use cases. If some of you want to see some
>>> other live migration combinations addressed, please raise your voice.
>> In relation to [1] you seem to be also interested in the migration
>> between heterogeneous systems with qemu.
> 
> Yes. That is correct.
> 
>> Do you think targeting migration within a cpu family is enough for your
>> use cases. How different are the source and destination host on your
>> cases. Do you thing feat props are relevant in your case or would you
>> need lower granularity at idreg field levelto pass the migration?
> 
> I think, from the current requirement we have for migration, the source and
> destination mostly can be handled by FEAT_XXX. But like Ampere, we do need
> to manage the CTR_EL0 differences[1].
OK
> 
> Also we do have differences in GIC support as well(AA64PFR0_EL1.GIC) which 
> I am not sure how to manage with FEAT_XXX.
interesting. We need to further look at this one.
> 
> And we are checking with our Product team whether we need to support migration
> from an old CPU type in which case we have to do a bit more analysis.
Sure, please come back to us whenever you get more insights. It will
help us define the scope of this upstream work

Thanks!

Eric
> 
> Thanks,
> Shameer
> 
> 1. https://lore.kernel.org/kvmarm/20241022073943.35764-1-shameerali.kolothum.thodi@huawei.com/
> 
> 
> 
>  
>