Booting Fedora kernels with -cpu max hangs very early in boot. Disabling
the la57 CPUID bit fixes the problem. git bisect traced the regression to

  commit 213ff024a2f92020290296cb9dc29c2af3d4a221 (HEAD, refs/bisect/bad)
  Author: Lara Lazier <laramglazier@gmail.com>
  Date:   Wed Jul 21 17:26:50 2021 +0200

    target/i386: Added consistency checks for CR4

    All MBZ bits in CR4 must be zero. (APM2 15.5)
    Added reserved bitmask and added checks in both
    helper_vmrun and helper_write_crN.

    Signed-off-by: Lara Lazier <laramglazier@gmail.com>
    Message-Id: <20210721152651.14683-2-laramglazier@gmail.com>
    Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

In this commit CR4_RESERVED_MASK is missing CR4_LA57_MASK and
two others. Adding this lets Fedora kernels boot once again.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 target/i386/cpu.h | 1 +
 1 file changed, 1 insertion(+)

I don't know the implications of CR4_VMXE_MASK and CR4_SMXE_MASK
being missing in CR4_RESERVED_MASK. It didn't cause any ill effects
either way, but I added them on the assumption that CR4_RESERVED_MASK
was supposed to contain all define bits.

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 6c50d3ab4f..ce85f1a29d 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -257,6 +257,7 @@ typedef enum X86Seg {
                 | CR4_DE_MASK | CR4_PSE_MASK | CR4_PAE_MASK \
                 | CR4_MCE_MASK | CR4_PGE_MASK | CR4_PCE_MASK \
                 | CR4_OSFXSR_MASK | CR4_OSXMMEXCPT_MASK |CR4_UMIP_MASK \
+                | CR4_LA57_MASK | CR4_VMXE_MASK | CR4_SMXE_MASK \
                 | CR4_FSGSBASE_MASK | CR4_PCIDE_MASK | CR4_OSXSAVE_MASK \
                 | CR4_SMEP_MASK | CR4_SMAP_MASK | CR4_PKE_MASK | CR4_PKS_MASK))
 
-- 
2.31.1

On Tue, Aug 31, 2021 at 06:50:33PM +0100, Daniel P. Berrangé wrote:
> Booting Fedora kernels with -cpu max hangs very early in boot. Disabling
> the la57 CPUID bit fixes the problem. git bisect traced the regression to
> 
>   commit 213ff024a2f92020290296cb9dc29c2af3d4a221 (HEAD, refs/bisect/bad)
>   Author: Lara Lazier <laramglazier@gmail.com>
>   Date:   Wed Jul 21 17:26:50 2021 +0200
> 
>     target/i386: Added consistency checks for CR4
> 
>     All MBZ bits in CR4 must be zero. (APM2 15.5)
>     Added reserved bitmask and added checks in both
>     helper_vmrun and helper_write_crN.
> 
>     Signed-off-by: Lara Lazier <laramglazier@gmail.com>
>     Message-Id: <20210721152651.14683-2-laramglazier@gmail.com>
>     Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> 
> In this commit CR4_RESERVED_MASK is missing CR4_LA57_MASK and
> two others. Adding this lets Fedora kernels boot once again.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>  target/i386/cpu.h | 1 +
>  1 file changed, 1 insertion(+)
> 
> I don't know the implications of CR4_VMXE_MASK and CR4_SMXE_MASK
> being missing in CR4_RESERVED_MASK. It didn't cause any ill effects
> either way, but I added them on the assumption that CR4_RESERVED_MASK
> was supposed to contain all define bits.
> 
> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
> index 6c50d3ab4f..ce85f1a29d 100644
> --- a/target/i386/cpu.h
> +++ b/target/i386/cpu.h
> @@ -257,6 +257,7 @@ typedef enum X86Seg {
>                  | CR4_DE_MASK | CR4_PSE_MASK | CR4_PAE_MASK \
>                  | CR4_MCE_MASK | CR4_PGE_MASK | CR4_PCE_MASK \
>                  | CR4_OSFXSR_MASK | CR4_OSXMMEXCPT_MASK |CR4_UMIP_MASK \
> +                | CR4_LA57_MASK | CR4_VMXE_MASK | CR4_SMXE_MASK \
>                  | CR4_FSGSBASE_MASK | CR4_PCIDE_MASK | CR4_OSXSAVE_MASK \
>                  | CR4_SMEP_MASK | CR4_SMAP_MASK | CR4_PKE_MASK | CR4_PKS_MASK))

First thing to say is I tested this locally and it fixes the
problem seen in https://bugzilla.redhat.com/show_bug.cgi?id=1999700.
I will also add this patch to Fedora soon.  So:

Tested-by: Richard W.M. Jones <rjones@redhat.com>

But my question is, does this mean that every time a new CPU feature
appears we must remember to update this code?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW

On 31/08/21 19:57, Richard W.M. Jones wrote:
>> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
>> index 6c50d3ab4f..ce85f1a29d 100644
>> --- a/target/i386/cpu.h
>> +++ b/target/i386/cpu.h
>> @@ -257,6 +257,7 @@ typedef enum X86Seg {
>>                   | CR4_DE_MASK | CR4_PSE_MASK | CR4_PAE_MASK \
>>                   | CR4_MCE_MASK | CR4_PGE_MASK | CR4_PCE_MASK \
>>                   | CR4_OSFXSR_MASK | CR4_OSXMMEXCPT_MASK |CR4_UMIP_MASK \
>> +                | CR4_LA57_MASK | CR4_VMXE_MASK | CR4_SMXE_MASK \
>>                   | CR4_FSGSBASE_MASK | CR4_PCIDE_MASK | CR4_OSXSAVE_MASK \
>>                   | CR4_SMEP_MASK | CR4_SMAP_MASK | CR4_PKE_MASK | CR4_PKS_MASK))
> First thing to say is I tested this locally and it fixes the
> problem seen inhttps://bugzilla.redhat.com/show_bug.cgi?id=1999700.
> I will also add this patch to Fedora soon.  So:
> 
> Tested-by: Richard W.M. Jones<rjones@redhat.com>
> 
> But my question is, does this mean that every time a new CPU feature
> appears we must remember to update this code?

This is used only by TCG, which explains why VMXE/SMXE were not there; 
however, LA57 is missing indeed.

New features must be included both here and in cr4_reserved_bits, but 
only if TCG supports them, otherwise they can be left out.  Since 
VMXE/SMXE are not supported by TCG, they should be either added both 
here and in cr4_reserved_bits (keyed on env->features[FEAT_1_ECX] & 
CPUID_EXT_{VMX,SMX} respectively), or they should not be added to 
CR4_RESERVED_MASK either.  On the other hand LA57 is already handled by 
cr4_reserved_bits, so it is okay to just add it here.

Thanks,

Paolo