FEAT_MOPS introduces new instructions, we require that these
instructions not execute as UNDEFINED when we identify that the feature
is supported. Additionally, the instructions may generate a Memory Copy
or Memory Set exception, we require that the exception be taken to EL1
in case EL2 is not able to correctly handle it.

Signed-off-by: Kristina Martsenko <kristina.martsenko@arm.com>
---
 Documentation/arm64/booting.rst | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/Documentation/arm64/booting.rst b/Documentation/arm64/booting.rst
index ffeccdd6bdac..f3913ee94c42 100644
--- a/Documentation/arm64/booting.rst
+++ b/Documentation/arm64/booting.rst
@@ -379,6 +379,14 @@ Before jumping into the kernel, the following conditions must be met:
 
     - SMCR_EL2.EZT0 (bit 30) must be initialised to 0b1.
 
+  For CPUs with Memory Copy and Memory Set instructions (FEAT_MOPS):
+
+  - If the kernel is entered at EL1 and EL2 is present:
+
+    - HCRX_EL2.MSCEn (bit 11) must be initialised to 0b1.
+
+    - HCRX_EL2.MCE2 (bit 10) must be initialised to 0b0.
+
 The requirements described above for CPU mode, caches, MMUs, architected
 timers, coherency and system registers apply to all CPUs.  All CPUs must
 enter the kernel in the same exception level.  Where the values documented
-- 
2.25.1

On Thu, Feb 16, 2023 at 04:00:06PM +0000, Kristina Martsenko wrote:
> +  For CPUs with Memory Copy and Memory Set instructions (FEAT_MOPS):
> +
> +  - If the kernel is entered at EL1 and EL2 is present:
> +
> +    - HCRX_EL2.MSCEn (bit 11) must be initialised to 0b1.
> +
> +    - HCRX_EL2.MCE2 (bit 10) must be initialised to 0b0.

Regarding MCE2, does EL1 actually care if EL2 wants to handle all the
memcpy/memset exceptions? There may even be a valid case to do this at
EL2 if you run a guest that uses these instructions but has no clue on
how to deal with the specific exception like WrongOption.

-- 
Catalin

On 17/03/2023 15:07, Catalin Marinas wrote:
> On Thu, Feb 16, 2023 at 04:00:06PM +0000, Kristina Martsenko wrote:
>> +  For CPUs with Memory Copy and Memory Set instructions (FEAT_MOPS):
>> +
>> +  - If the kernel is entered at EL1 and EL2 is present:
>> +
>> +    - HCRX_EL2.MSCEn (bit 11) must be initialised to 0b1.
>> +
>> +    - HCRX_EL2.MCE2 (bit 10) must be initialised to 0b0.
> 
> Regarding MCE2, does EL1 actually care if EL2 wants to handle all the
> memcpy/memset exceptions? 

No, EL1 does not need to handle the exceptions itself, but I don't see any
current use case for allowing EL2 to handle it.

If it was allowed, I think booting.txt would need to specify exactly what Linux
expects EL2 to do if MCE2 is set (eg, that EL2 handles the exception by
reformatting registers, modifying single step state, etc).

> There may even be a valid case to do this at
> EL2 if you run a guest that uses these instructions but has no clue on
> how to deal with the specific exception like WrongOption.

Not sure I follow - this series adds the exception handling, so how can a Linux
guest not know how to handle the exception?

Or do you mean that there may be times when EL1 can't take the exception but
EL2 may move it to another CPU, and so EL2 would need to handle the exception?
I'm not sure if Linux ever uses mops instructions at times like that. Note that
this series does not add support for mops in guests yet. I think booting.txt
can be updated when that support is added.

Thanks,
Kristina

On Fri, Mar 24, 2023 at 01:00:43AM +0000, Kristina Martsenko wrote:
> On 17/03/2023 15:07, Catalin Marinas wrote:
> > On Thu, Feb 16, 2023 at 04:00:06PM +0000, Kristina Martsenko wrote:
> >> +  For CPUs with Memory Copy and Memory Set instructions (FEAT_MOPS):
> >> +
> >> +  - If the kernel is entered at EL1 and EL2 is present:
> >> +
> >> +    - HCRX_EL2.MSCEn (bit 11) must be initialised to 0b1.
> >> +
> >> +    - HCRX_EL2.MCE2 (bit 10) must be initialised to 0b0.
> > 
> > Regarding MCE2, does EL1 actually care if EL2 wants to handle all the
> > memcpy/memset exceptions? 
> 
> No, EL1 does not need to handle the exceptions itself, but I don't see any
> current use case for allowing EL2 to handle it.
> 
> If it was allowed, I think booting.txt would need to specify exactly what Linux
> expects EL2 to do if MCE2 is set (eg, that EL2 handles the exception by
> reformatting registers, modifying single step state, etc).

What I meant is that an EL1 kernel shouldn't care if MCE2 is 0 or 1. We
could clarify that if set to 1, it is expected that the hypervisor
handles the memory copy/set exceptions accordingly.

> > There may even be a valid case to do this at
> > EL2 if you run a guest that uses these instructions but has no clue on
> > how to deal with the specific exception like WrongOption.
> 
> Not sure I follow - this series adds the exception handling, so how can a Linux
> guest not know how to handle the exception?

The guest may not always be Linux (e.g. some microkernel) and may not
expect the hardware implementation to change underneath.

> Or do you mean that there may be times when EL1 can't take the exception but
> EL2 may move it to another CPU, and so EL2 would need to handle the exception?

I haven't thought of this but it's actually a good point. Are there any
cases where Linux can't handle a memcpy exception? I guess we need to be
careful with taking such exception in an atomic context (e.g. no
rescheduling on the return path).

> I'm not sure if Linux ever uses mops instructions at times like that.

The compiler may generate a memcpy() call by simply assigning a
structure to another. So we can't control where those instructions are
used.

> Note that this series does not add support for mops in guests yet. 

You mean there's no KVM support. But Linux may be run under a different
hypervisor (e.g. Hyper-V) as a guest.

> I think booting.txt can be updated when that support is added.

In booting.txt, when you say the kernel entered at EL1, it implies that
it may be run as a guest under a random hypervisor.

So maybe we should detail the MCE2 requirement a bit, saying that it can
be either 0 or 1 but, for the latter, the hypervisor must handle the
corresponding exceptions.

-- 
Catalin

On 04/04/2023 11:50, Catalin Marinas wrote:
> On Fri, Mar 24, 2023 at 01:00:43AM +0000, Kristina Martsenko wrote:
>> On 17/03/2023 15:07, Catalin Marinas wrote:
>>> On Thu, Feb 16, 2023 at 04:00:06PM +0000, Kristina Martsenko wrote:
>>>> +  For CPUs with Memory Copy and Memory Set instructions (FEAT_MOPS):
>>>> +
>>>> +  - If the kernel is entered at EL1 and EL2 is present:
>>>> +
>>>> +    - HCRX_EL2.MSCEn (bit 11) must be initialised to 0b1.
>>>> +
>>>> +    - HCRX_EL2.MCE2 (bit 10) must be initialised to 0b0.
>>>
>>> Regarding MCE2, does EL1 actually care if EL2 wants to handle all the
>>> memcpy/memset exceptions? 
>>
>> Note that this series does not add support for mops in guests yet. 
> 
> You mean there's no KVM support. But Linux may be run under a different
> hypervisor (e.g. Hyper-V) as a guest.
> 
>> I think booting.txt can be updated when that support is added.
> 
> In booting.txt, when you say the kernel entered at EL1, it implies that
> it may be run as a guest under a random hypervisor.
> 
> So maybe we should detail the MCE2 requirement a bit, saying that it can
> be either 0 or 1 but, for the latter, the hypervisor must handle the
> corresponding exceptions.

That makes sense. I was going to add this, but then realized that MCE2 only
traps memcpy exceptions from EL1, not EL0, so it does not have any effect with
this series. So I think we can just drop the MCE2 requirement entirely for now
and specify the requirement later when we add memcpy instructions into the
kernel itself.

Thanks,
Kristina