From nobody Sat Jun 20 01:51:50 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A965EC433F5 for ; Thu, 24 Mar 2022 13:47:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350579AbiCXNsr (ORCPT ); Thu, 24 Mar 2022 09:48:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60912 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350565AbiCXNsm (ORCPT ); Thu, 24 Mar 2022 09:48:42 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 86A528A332 for ; Thu, 24 Mar 2022 06:47:10 -0700 (PDT) Message-ID: <20220324134623.408932232@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1648129628; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=k7ZwKCPvoxI5xqioe9wDMoAwUBZDO2Yad89sOWDbHAQ=; b=h38pHEDTkRb7pTSbSvahMDQW2Hpc2pG2Q27FZF6AExDGIjfH7Dt+17wvVHrNpCf20fK9sN dsMIQ5o3IPSzDR7k4s7wJcwc8Z4dkxAj6ivJCutMKJzl4zCmisG21mk0GB0kb0iT35VA0Q Dp3GMo+ZplFg0AEuyDns2JQ4CegwQDF7qE7IpuzJI3abYiU0RtGGb2spLfXRItuynz/YA+ zIRFBXbzfzVz7J4WuLb0q4WHivDspydqoXuSWYDEl6lnCIYAv4J4iYN8zCGF8qn7rSCG0X v0KO9vY2ouWqvu6nrVy6ZkLJ8IKyzeYIyC8D9OZTpEbYqoBIbJiC90t3NhOvyA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1648129628; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=k7ZwKCPvoxI5xqioe9wDMoAwUBZDO2Yad89sOWDbHAQ=; b=g8OeO4Li57ezADORSDUAqGr9xaiPxB/7rIm12yvP/JK1c+Rk7rk7H122jiB75avmp3IQz5 Wjc7qXHWsbp06AAQ== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, "Chang S. Bae" , Paolo Bonzini Subject: [patch 1/7] x86/fpu: Remove redundant XCOMP_BV initialization References: <20220324134548.432837672@linutronix.de> MIME-Version: 1.0 Date: Thu, 24 Mar 2022 14:47:08 +0100 (CET) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" fpu_copy_uabi_to_guest_fpstate() initializes the XCOMP_BV field in the XSAVE header. That's a leftover from the old KVM FPU buffer handling code. Since d69c1382e1b7 ("x86/kvm: Convert FPU handling to a single swap buffer") KVM uses the FPU core allocation code, which initializes the XCOMP_BV field already. Signed-off-by: Thomas Gleixner Cc: Paolo Bonzini --- arch/x86/kernel/fpu/core.c | 3 --- 1 file changed, 3 deletions(-) --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -415,9 +415,6 @@ int fpu_copy_uabi_to_guest_fpstate(struc xpkru =3D get_xsave_addr(&kstate->regs.xsave, XFEATURE_PKRU); *vpkru =3D xpkru->pkru; } - - /* Ensure that XCOMP_BV is set up for XSAVES */ - xstate_init_xcomp_bv(&kstate->regs.xsave, kstate->xfeatures); return 0; } EXPORT_SYMBOL_GPL(fpu_copy_uabi_to_guest_fpstate); From nobody Sat Jun 20 01:51:50 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3E61C433EF for ; Thu, 24 Mar 2022 13:47:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350583AbiCXNsw (ORCPT ); Thu, 24 Mar 2022 09:48:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350563AbiCXNsn (ORCPT ); Thu, 24 Mar 2022 09:48:43 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 76F777CDD0 for ; Thu, 24 Mar 2022 06:47:11 -0700 (PDT) Message-ID: <20220324134623.465066249@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1648129630; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=dhbG1adkOE8R+sSmMcE/7esFfa52zfgf9U61mYJxyYw=; b=NEYa9909YrtsAWAufxNgT+hBwo+G/s5ZXsE6I1Cwsg2rFoHX5hBKycmuhL6/DeBbPAseoN PRieKodSwQx+EgOqy/GawS2nIumvDAX3cIoFG3vB3ZYdaDDVT2BFdvnQ3WW8ceRPOQNwCG jd0oiwJWwMUmFyiqmXwG43JG+M9eVYgow0z57GNErKOaccSgGE3hw3j3CwU3jAFMYJvZiT eHaDBCMAsOEIFYJLvy99VojwrJ6JVALz83Mo2+ZvKZonmAmI88/A7cI5NaVwhwlPv6w93d 1QiIFHWK18LOPNNS26tnB5dqb6KgxJom1a9XgWzzVCdEG9z6UrbREOFoY4CpGA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1648129630; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=dhbG1adkOE8R+sSmMcE/7esFfa52zfgf9U61mYJxyYw=; b=WI9lMIfXdT+CqFDMctIkehPGZ5k1Ul7EltWd8i3tpjN9ysbIrq2xv/Dp9ubeBjPHzH18sK wsLYlT7156aS4BBQ== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, "Chang S. Bae" Subject: [patch 2/7] x86/fpu: Remove unused supervisor only offsets References: <20220324134548.432837672@linutronix.de> MIME-Version: 1.0 Date: Thu, 24 Mar 2022 14:47:09 +0100 (CET) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" No users. Signed-off-by: Thomas Gleixner --- arch/x86/kernel/fpu/xstate.c | 30 ------------------------------ 1 file changed, 30 deletions(-) --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -83,8 +83,6 @@ static unsigned int xstate_sizes[XFEATUR { [ 0 ... XFEATURE_MAX - 1] =3D -1}; static unsigned int xstate_comp_offsets[XFEATURE_MAX] __ro_after_init =3D { [ 0 ... XFEATURE_MAX - 1] =3D -1}; -static unsigned int xstate_supervisor_only_offsets[XFEATURE_MAX] __ro_afte= r_init =3D - { [ 0 ... XFEATURE_MAX - 1] =3D -1}; =20 /* * Return whether the system supports a given xfeature. @@ -325,33 +323,6 @@ static void __init setup_xstate_comp_off } =20 /* - * Setup offsets of a supervisor-state-only XSAVES buffer: - * - * The offsets stored in xstate_comp_offsets[] only work for one specific - * value of the Requested Feature BitMap (RFBM). In cases where a differe= nt - * RFBM value is used, a different set of offsets is required. This set of - * offsets is for when RFBM=3Dxfeatures_mask_supervisor(). - */ -static void __init setup_supervisor_only_offsets(void) -{ - unsigned int next_offset; - int i; - - next_offset =3D FXSAVE_SIZE + XSAVE_HDR_SIZE; - - for_each_extended_xfeature(i, fpu_kernel_cfg.max_features) { - if (!xfeature_is_supervisor(i)) - continue; - - if (xfeature_is_aligned(i)) - next_offset =3D ALIGN(next_offset, 64); - - xstate_supervisor_only_offsets[i] =3D next_offset; - next_offset +=3D xstate_sizes[i]; - } -} - -/* * Print out xstate component offsets and sizes */ static void __init print_xstate_offset_size(void) @@ -951,7 +922,6 @@ void __init fpu__init_system_xstate(unsi =20 setup_init_fpu_buf(); setup_xstate_comp_offsets(); - setup_supervisor_only_offsets(); =20 /* * Paranoia check whether something in the setup modified the From nobody Sat Jun 20 01:51:50 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D41C7C433F5 for ; Thu, 24 Mar 2022 13:47:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350624AbiCXNs6 (ORCPT ); Thu, 24 Mar 2022 09:48:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32850 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350576AbiCXNsp (ORCPT ); Thu, 24 Mar 2022 09:48:45 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B73A18F9A9 for ; Thu, 24 Mar 2022 06:47:12 -0700 (PDT) Message-ID: <20220324134623.519411939@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1648129631; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=mkdnb+b1Zmv6jwoILxAAtMLYCIJRUKxiXjyexb3mvlA=; b=Mu+EzWTBgKIHpxaq1AnEAX1K+gbnmR0RU9b3Ffo+n1SdY6JvSJQgPBP8rWccsOQuy8mPgX g0FO+UT1RksBZfZisj3fKshuJ9U20clfDMDjHpumeHKWyhrWVy+MVFMS5fnEQEjeV3Xs6e 0fv5gBOywntqh5edJowL8NgGnuiBqFUSwV0FhmzopZnLF29QpwsX0eaayjp4+5Nw+5pRLR I/mZrwwznKTHmjGGwmQwtIQ3ZP3K+84/5PI40YRQACLhJRTw+PUtjkodBjiFAsGLxo49ai 0nPe8ytoY5aJ+I/h3cgjPA0g4hBjui9x+Dn29oBM6Z/kJRI6dkR4GLUtnq0J/A== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1648129631; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=mkdnb+b1Zmv6jwoILxAAtMLYCIJRUKxiXjyexb3mvlA=; b=uuDif5DOITAoan4/wx31IbVAYrNBc+4hI0H66CtiwKIL6dfp4uCJYb94WVpud9cvrkTTFx 8ExKi3x+XLs877AA== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, "Chang S. Bae" Subject: [patch 3/7] x86/fpu/xsave: Initialize offset/size cache early References: <20220324134548.432837672@linutronix.de> MIME-Version: 1.0 Date: Thu, 24 Mar 2022 14:47:11 +0100 (CET) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Reading XSTATE feature information from CPUID over and over does not make sense. The information has to be cached anyway, so it can be done early. This prepares for runtime calculation of XSTATE offsets and allows consolidation of the size calculation functions in a later step. Rename the function while at it as it does not setup any features. Signed-off-by: Thomas Gleixner --- arch/x86/kernel/fpu/xstate.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -180,7 +180,7 @@ static bool xfeature_enabled(enum xfeatu * Record the offsets and sizes of various xstates contained * in the XSAVE state memory layout. */ -static void __init setup_xstate_features(void) +static void __init setup_xstate_cache(void) { u32 eax, ebx, ecx, edx, i; /* start at the beginning of the "extended state" */ @@ -390,7 +390,6 @@ static void __init setup_init_fpu_buf(vo if (!boot_cpu_has(X86_FEATURE_XSAVE)) return; =20 - setup_xstate_features(); print_xstate_features(); =20 xstate_init_xcomp_bv(&init_fpstate.regs.xsave, fpu_kernel_cfg.max_feature= s); @@ -906,6 +905,10 @@ void __init fpu__init_system_xstate(unsi =20 /* Enable xstate instructions to be able to continue with initialization:= */ fpu__init_cpu_xstate(); + + /* Cache size, offset and flags for initialization */ + setup_xstate_cache(); + err =3D init_xstate_size(); if (err) goto out_disable; From nobody Sat Jun 20 01:51:50 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECFC1C433EF for ; Thu, 24 Mar 2022 13:47:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350631AbiCXNtC (ORCPT ); Thu, 24 Mar 2022 09:49:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32948 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350577AbiCXNsr (ORCPT ); Thu, 24 Mar 2022 09:48:47 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E18FA94D5 for ; Thu, 24 Mar 2022 06:47:13 -0700 (PDT) Message-ID: <20220324134623.573656209@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1648129632; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=FlS4rDJenRO6wkC0JNfcNEGDOUzk/9jo8hCEAjDstZo=; b=rmrVo/s1ODwYu0HnOGUT7eAg3WFoEuXeL0tUpdCsaePLqweClDRipWWresthvglwM0urGQ /XKN4m+KzDB1EnqlLc87acckS53pqVCciAf+lErK81GHY3SeSqdWyes5ffv2GFBi86279b 9hM3Sf2Gu1dp/fcXiugSNg0bvoIXRndxR61vHP/O5Zo9klpwG7rWOVazif3B4pg/WMleuy DEJK9hWDpSDWX9mNnqYvaZHZXFDbBedl9Z68POjrA+Bb7Wa4Do0z/NNXguMKyB0KR75Rso devgcxiixtZMR/bPJ8ozGoDJHAU+h8Aw8MZE91fONbKXu3X3rDWCifJ7b1KwvA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1648129632; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=FlS4rDJenRO6wkC0JNfcNEGDOUzk/9jo8hCEAjDstZo=; b=FAG7FSdVTHDktEb/RD0aHM9TV4wUNhx3uIeUaskfpWIEO9Dr1kmMt8K0RehEkkOzmV3Mp3 HFWbAl4VLHX4DnAA== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, "Chang S. Bae" Subject: [patch 4/7] x86/fpu: Cache xfeature flags from CPUID References: <20220324134548.432837672@linutronix.de> MIME-Version: 1.0 Date: Thu, 24 Mar 2022 14:47:12 +0100 (CET) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In preparation for runtime calculation of XSAVE offsets cache the feature flags for each XSTATE component during feature enumeration via CPUID(0xD). EDX has two relevant bits: 0 Supervisor component 1 Feature storage must be 64 byte aligned These bits are currently only evaluated during init, but the alignment bit must be cached to make runtime calculation of XSAVE offsets efficient. Cache the full EDX content and use it for the existing alignment and supervisor checks. Signed-off-by: Thomas Gleixner --- arch/x86/kernel/fpu/xstate.c | 49 +++++++++++---------------------------= ----- 1 file changed, 13 insertions(+), 36 deletions(-) --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -83,6 +83,10 @@ static unsigned int xstate_sizes[XFEATUR { [ 0 ... XFEATURE_MAX - 1] =3D -1}; static unsigned int xstate_comp_offsets[XFEATURE_MAX] __ro_after_init =3D { [ 0 ... XFEATURE_MAX - 1] =3D -1}; +static unsigned int xstate_flags[XFEATURE_MAX] __ro_after_init; + +#define XSTATE_FLAG_SUPERVISOR BIT(0) +#define XSTATE_FLAG_ALIGNED64 BIT(1) =20 /* * Return whether the system supports a given xfeature. @@ -122,17 +126,14 @@ int cpu_has_xfeatures(u64 xfeatures_need } EXPORT_SYMBOL_GPL(cpu_has_xfeatures); =20 -static bool xfeature_is_supervisor(int xfeature_nr) +static bool xfeature_is_aligned64(int xfeature_nr) { - /* - * Extended State Enumeration Sub-leaves (EAX =3D 0DH, ECX =3D n, n > 1) - * returns ECX[0] set to (1) for a supervisor state, and cleared (0) - * for a user state. - */ - u32 eax, ebx, ecx, edx; + return xstate_flags[xfeature_nr] & XSTATE_FLAG_ALIGNED64; +} =20 - cpuid_count(XSTATE_CPUID, xfeature_nr, &eax, &ebx, &ecx, &edx); - return ecx & 1; +static bool xfeature_is_supervisor(int xfeature_nr) +{ + return xstate_flags[xfeature_nr] & XSTATE_FLAG_SUPERVISOR; } =20 /* @@ -203,6 +204,7 @@ static void __init setup_xstate_cache(vo cpuid_count(XSTATE_CPUID, i, &eax, &ebx, &ecx, &edx); =20 xstate_sizes[i] =3D eax; + xstate_flags[i] =3D ecx; =20 /* * If an xfeature is supervisor state, the offset in EBX is @@ -262,31 +264,6 @@ static void __init print_xstate_features } while (0) =20 /* - * We could cache this like xstate_size[], but we only use - * it here, so it would be a waste of space. - */ -static int xfeature_is_aligned(int xfeature_nr) -{ - u32 eax, ebx, ecx, edx; - - CHECK_XFEATURE(xfeature_nr); - - if (!xfeature_enabled(xfeature_nr)) { - WARN_ONCE(1, "Checking alignment of disabled xfeature %d\n", - xfeature_nr); - return 0; - } - - cpuid_count(XSTATE_CPUID, xfeature_nr, &eax, &ebx, &ecx, &edx); - /* - * The value returned by ECX[1] indicates the alignment - * of state component 'i' when the compacted format - * of the extended region of an XSAVE area is used: - */ - return !!(ecx & 2); -} - -/* * This function sets up offsets and sizes of all extended states in * xsave area. This supports both standard format and compacted format * of the xsave area. @@ -314,7 +291,7 @@ static void __init setup_xstate_comp_off next_offset =3D FXSAVE_SIZE + XSAVE_HDR_SIZE; =20 for_each_extended_xfeature(i, fpu_kernel_cfg.max_features) { - if (xfeature_is_aligned(i)) + if (xfeature_is_aligned64(i)) next_offset =3D ALIGN(next_offset, 64); =20 xstate_comp_offsets[i] =3D next_offset; @@ -619,7 +596,7 @@ static unsigned int xstate_calculate_siz =20 for_each_extended_xfeature(i, xfeatures) { /* Align from the end of the previous feature */ - if (xfeature_is_aligned(i)) + if (xfeature_is_aligned64(i)) size =3D ALIGN(size, 64); /* * In compacted format the enabled features are packed, From nobody Sat Jun 20 01:51:50 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51340C433EF for ; Thu, 24 Mar 2022 13:47:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350654AbiCXNtJ (ORCPT ); Thu, 24 Mar 2022 09:49:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350581AbiCXNsr (ORCPT ); Thu, 24 Mar 2022 09:48:47 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2AE9FA94D9 for ; Thu, 24 Mar 2022 06:47:15 -0700 (PDT) Message-ID: <20220324134623.627636809@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1648129633; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=2XKntiiVPPNJELg0vo1ejxeBcdE2A4TaubX0naDHtcM=; b=k1lOsjZ5Vbc8wdLVXOWUDW0faS4+rF0twMNJrE+BJgZojVAwZ5wveH4HJjibRNzatZkHg+ A64IhA2LLtHKNf4o+Y2UTtAUW6rAv0G1cDXfoUZ5/SZVXpbUiEZTPGTQ7idAPFSn4f3H7I di2lZY4V210vsEJJFoEBKuYqyYfiMKyQrsCkZiWohTdWeM1Y3NSz4EV9H5yJ4cUPzSv2rq K24JQ+djdpSddUr1hJ9jWp1x2Fo28CMG2OdLyE+xrfmYWLZ5Qxk8XoefWA+HBr7oQLrLo4 /nLq6sQs0h5Tquw2yfL/56t0lmdYDGJCRzeza0ZSkDGHhAuv+axA0GjFYq+Teg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1648129633; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=2XKntiiVPPNJELg0vo1ejxeBcdE2A4TaubX0naDHtcM=; b=2qe0JnWfgLR6R0eLdNvXqk6aUmNjox+TzF4TGLX4CNdKIZWXTOW/Ggc8sIlAScRnkIW6jf iR7oDWVxyz4Pg4Dw== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, "Chang S. Bae" , Paolo Bonzini , Fenghua Yu , Tony Luck Subject: [patch 5/7] x86/fpu/xsave: Handle compacted offsets correctly with supervisor states References: <20220324134548.432837672@linutronix.de> MIME-Version: 1.0 Date: Thu, 24 Mar 2022 14:47:13 +0100 (CET) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" So far the cached fixed compacted offsets worked, but with (re)enabling of ENQCMD this does not longer work with KVM fpstate. KVM does not have supervisor features enabled for the guest FPU, which means that KVM has then a different XSAVE area layout than the host FPU state. This in turn breaks the copy from/to UABI functions when invoked for a guest state. Remove the precalculated compacted offsets and calculate the offset of each component at runtime based on the XCOMP_BV field in the XSAVE header. The runtime overhead is not interesting because these copy from/to UABI functions are not used in critical fast paths. KVM uses them to save and restore FPU state during migration. The host uses them for ptrace and for the slow path of 32bit signal handling. Fixes: 7c1ef59145f1 ("x86/cpufeatures: Re-enable ENQCMD") Signed-off-by: Thomas Gleixner Cc: Paolo Bonzini Cc: Fenghua Yu Cc: Tony Luck --- arch/x86/kernel/fpu/xstate.c | 86 ++++++++++++++++++++------------------= ----- 1 file changed, 41 insertions(+), 45 deletions(-) --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -81,8 +81,6 @@ static unsigned int xstate_offsets[XFEAT { [ 0 ... XFEATURE_MAX - 1] =3D -1}; static unsigned int xstate_sizes[XFEATURE_MAX] __ro_after_init =3D { [ 0 ... XFEATURE_MAX - 1] =3D -1}; -static unsigned int xstate_comp_offsets[XFEATURE_MAX] __ro_after_init =3D - { [ 0 ... XFEATURE_MAX - 1] =3D -1}; static unsigned int xstate_flags[XFEATURE_MAX] __ro_after_init; =20 #define XSTATE_FLAG_SUPERVISOR BIT(0) @@ -136,6 +134,33 @@ static bool xfeature_is_supervisor(int x return xstate_flags[xfeature_nr] & XSTATE_FLAG_SUPERVISOR; } =20 +static unsigned int xfeature_get_offset(u64 xcomp_bv, int xfeature) +{ + unsigned int offs, i; + + /* + * Non-compacted format and legacy features use the cached fixed + * offsets. + */ + if (!cpu_feature_enabled(X86_FEATURE_XSAVES) || xfeature <=3D XFEATURE_SS= E) + return xstate_offsets[xfeature]; + + /* + * Compacted format offsets depend on the actual content of the + * compacted xsave area which is determined by the xcomp_bv header + * field. + */ + offs =3D FXSAVE_SIZE + XSAVE_HDR_SIZE; + for_each_extended_xfeature(i, xcomp_bv) { + if (xfeature_is_aligned64(i)) + offs =3D ALIGN(offs, 64); + if (i =3D=3D xfeature) + break; + offs +=3D xstate_sizes[i]; + } + return offs; +} + /* * Enable the extended processor state save/restore feature. * Called once per CPU onlining. @@ -264,42 +289,6 @@ static void __init print_xstate_features } while (0) =20 /* - * This function sets up offsets and sizes of all extended states in - * xsave area. This supports both standard format and compacted format - * of the xsave area. - */ -static void __init setup_xstate_comp_offsets(void) -{ - unsigned int next_offset; - int i; - - /* - * The FP xstates and SSE xstates are legacy states. They are always - * in the fixed offsets in the xsave area in either compacted form - * or standard form. - */ - xstate_comp_offsets[XFEATURE_FP] =3D 0; - xstate_comp_offsets[XFEATURE_SSE] =3D offsetof(struct fxregs_state, - xmm_space); - - if (!cpu_feature_enabled(X86_FEATURE_XSAVES)) { - for_each_extended_xfeature(i, fpu_kernel_cfg.max_features) - xstate_comp_offsets[i] =3D xstate_offsets[i]; - return; - } - - next_offset =3D FXSAVE_SIZE + XSAVE_HDR_SIZE; - - for_each_extended_xfeature(i, fpu_kernel_cfg.max_features) { - if (xfeature_is_aligned64(i)) - next_offset =3D ALIGN(next_offset, 64); - - xstate_comp_offsets[i] =3D next_offset; - next_offset +=3D xstate_sizes[i]; - } -} - -/* * Print out xstate component offsets and sizes */ static void __init print_xstate_offset_size(void) @@ -308,7 +297,8 @@ static void __init print_xstate_offset_s =20 for_each_extended_xfeature(i, fpu_kernel_cfg.max_features) { pr_info("x86/fpu: xstate_offset[%d]: %4d, xstate_sizes[%d]: %4d\n", - i, xstate_comp_offsets[i], i, xstate_sizes[i]); + i, xfeature_get_offset(fpu_kernel_cfg.max_features, i), + i, xstate_sizes[i]); } } =20 @@ -901,7 +891,6 @@ void __init fpu__init_system_xstate(unsi fpu_user_cfg.max_features); =20 setup_init_fpu_buf(); - setup_xstate_comp_offsets(); =20 /* * Paranoia check whether something in the setup modified the @@ -956,13 +945,19 @@ void fpu__resume_cpu(void) */ static void *__raw_xsave_addr(struct xregs_state *xsave, int xfeature_nr) { - if (!xfeature_enabled(xfeature_nr)) { - WARN_ON_FPU(1); + u64 xcomp_bv =3D xsave->header.xcomp_bv; + + if (WARN_ON_ONCE(!xfeature_enabled(xfeature_nr))) return NULL; + + if (cpu_feature_enabled(X86_FEATURE_XSAVES)) { + if (WARN_ON_ONCE(!(xcomp_bv & BIT_ULL(xfeature_nr)))) + return NULL; } =20 - return (void *)xsave + xstate_comp_offsets[xfeature_nr]; + return (void *)xsave + xfeature_get_offset(xcomp_bv, xfeature_nr); } + /* * Given the xsave area and a state inside, this function returns the * address of the state. @@ -993,8 +988,9 @@ void *get_xsave_addr(struct xregs_state * We should not ever be requesting features that we * have not enabled. */ - WARN_ONCE(!(fpu_kernel_cfg.max_features & BIT_ULL(xfeature_nr)), - "get of unsupported state"); + if (WARN_ON_ONCE(!xfeature_enabled(xfeature_nr))) + return NULL; + /* * This assumes the last 'xsave*' instruction to * have requested that 'xfeature_nr' be saved. From nobody Sat Jun 20 01:51:50 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5679C433F5 for ; Thu, 24 Mar 2022 13:47:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350647AbiCXNtS (ORCPT ); Thu, 24 Mar 2022 09:49:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33072 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350584AbiCXNst (ORCPT ); Thu, 24 Mar 2022 09:48:49 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B4BB6A94DE for ; Thu, 24 Mar 2022 06:47:16 -0700 (PDT) Message-ID: <20220324134623.681768598@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1648129635; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=fRGoaQA9ELB/t/I5jAlbmPcCoZHznW4PsontUd5UTeo=; b=oE6v15/Qr061F27NHTDBHMCy1kFOhASSrrCQiStooXcTJAXQG+pFtUX5J9HJjCH8s5bBNk NNJliAJODG9mFvr4W0GMPdvQ0+ww9lLHcZoVXXvwII5i3wptTgjmq9RFekjZqBXWMlzNi8 85BQnggxNt+n7de4gQqe15rIjVTuZzIhtUeK/2bdz+66libTsBqZmgNsr369uqvmmg188m 3s+t5O7bTElghz2XMvbSpl3Y14qzHLbSndImAH/+oKFFXBiKP/f/N84S4epbZheqwHi78s ZUIiwN07yNrXZoMvMwU1WIm+k4fR0EGNX1kdS6dnM0Xemus0ugQLZ/5+bLkDTA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1648129635; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=fRGoaQA9ELB/t/I5jAlbmPcCoZHznW4PsontUd5UTeo=; b=jqP05jj4pbBR0I3vIXgYP0K9j+8+EXNgovD4tMBlSEyEp1YhEZrHTEh5mbzO4mhuaBzGz0 VyfZhs5N+ZWk4EAg== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, "Chang S. Bae" , Paolo Bonzini , Fenghua Yu , Tony Luck Subject: [patch 6/7] x86/fpu/xstate: Handle supervisor states in XSTATE permissions References: <20220324134548.432837672@linutronix.de> MIME-Version: 1.0 Date: Thu, 24 Mar 2022 14:47:14 +0100 (CET) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The size calculation in __xstate_request_perm() fails to take supervisor states into account because the permission bitmap is only relevant for user states. Up to 5.17 this does not matter because there are no supervisor states supported, but the (re)enabling of ENQCMD makes them available. Fixes: 7c1ef59145f1 ("x86/cpufeatures: Re-enable ENQCMD") Signed-off-by: Thomas Gleixner Cc: Paolo Bonzini Cc: Fenghua Yu Cc: Tony Luck --- arch/x86/kernel/fpu/xstate.c | 3 +++ 1 file changed, 3 insertions(+) --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -1566,6 +1566,9 @@ static int __xstate_request_perm(u64 per =20 /* Calculate the resulting kernel state size */ mask =3D permitted | requested; + /* Take supervisor states into account on the host */ + if (!guest) + mask |=3D xfeatures_mask_supervisor(); ksize =3D xstate_calculate_size(mask, compacted); =20 /* Calculate the resulting user state size */ From nobody Sat Jun 20 01:51:50 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9A74C433EF for ; Thu, 24 Mar 2022 13:47:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350617AbiCXNtO (ORCPT ); Thu, 24 Mar 2022 09:49:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33164 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350591AbiCXNsu (ORCPT ); Thu, 24 Mar 2022 09:48:50 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 06FFFA94D9 for ; Thu, 24 Mar 2022 06:47:17 -0700 (PDT) Message-ID: <20220324134623.736990161@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1648129636; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=R6A5qasfkCP1yilN3Hx1CqR1fLfcwGqb+MZxXGRxBLk=; b=BUwWnm++qyFfkfypVTGJcfty77bat2cZNnIS5b82BWBviYGp5UPe6KuR2MJQKx3uh37vid NbfmjvslxGzTO7nYXj65CWwXcR3evAWa68bKcYZk8YtmMmHicXI1E0LJd52CAEBiXA0quf 56OIUDl/GNxbWE6FDU+Crdpfa2ItQBvMlcSDdso2tffBEXChz6RjEt3/SKm/0eUaacTFY5 l/tmK903vuESeFxXdeth3esrhBXVo6uZyBEf1a+0gaPf+BrBWSlNoV6Cffxin31eODbS4O 7uv09KMl/8nIZNstLriQNPPlaFm1fFpNmnyNVYhVL0auICTe9KHBeQTtPDyYrw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1648129636; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=R6A5qasfkCP1yilN3Hx1CqR1fLfcwGqb+MZxXGRxBLk=; b=uhku3XA5lZTnxAOLGPuoRxWvdslUdVxMWjjkv45KcVkPta+muLG5Rnj8fRC4NE+q0r5PRy iq6KGsGndBtZeJCw== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, "Chang S. Bae" Subject: [patch 7/7] x86/fpu/xstate: Consolidate size calculations References: <20220324134548.432837672@linutronix.de> MIME-Version: 1.0 Date: Thu, 24 Mar 2022 14:47:16 +0100 (CET) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the offset calculation to do the size calculation which avoids yet another series of CPUID instructions for each invocation. Signed-off-by: Thomas Gleixner --- arch/x86/kernel/fpu/xstate.c | 46 ++++----------------------------------= ----- 1 file changed, 5 insertions(+), 41 deletions(-) --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -385,25 +385,6 @@ static void __init setup_init_fpu_buf(vo fxsave(&init_fpstate.regs.fxsave); } =20 -static int xfeature_uncompacted_offset(int xfeature_nr) -{ - u32 eax, ebx, ecx, edx; - - /* - * Only XSAVES supports supervisor states and it uses compacted - * format. Checking a supervisor state's uncompacted offset is - * an error. - */ - if (XFEATURE_MASK_SUPERVISOR_ALL & BIT_ULL(xfeature_nr)) { - WARN_ONCE(1, "No fixed offset for xstate %d\n", xfeature_nr); - return -1; - } - - CHECK_XFEATURE(xfeature_nr); - cpuid_count(XSTATE_CPUID, xfeature_nr, &eax, &ebx, &ecx, &edx); - return ebx; -} - int xfeature_size(int xfeature_nr) { u32 eax, ebx, ecx, edx; @@ -581,29 +562,12 @@ static bool __init check_xstate_against_ =20 static unsigned int xstate_calculate_size(u64 xfeatures, bool compacted) { - unsigned int size =3D FXSAVE_SIZE + XSAVE_HDR_SIZE; - int i; + unsigned int topmost =3D fls64(xfeatures) - 1; + unsigned int offset =3D xstate_offsets[topmost]; =20 - for_each_extended_xfeature(i, xfeatures) { - /* Align from the end of the previous feature */ - if (xfeature_is_aligned64(i)) - size =3D ALIGN(size, 64); - /* - * In compacted format the enabled features are packed, - * i.e. disabled features do not occupy space. - * - * In non-compacted format the offsets are fixed and - * disabled states still occupy space in the memory buffer. - */ - if (!compacted) - size =3D xfeature_uncompacted_offset(i); - /* - * Add the feature size even for non-compacted format - * to make the end result correct - */ - size +=3D xfeature_size(i); - } - return size; + if (compacted) + offset =3D xfeature_get_offset(xfeatures, topmost); + return offset + xstate_sizes[topmost]; } =20 /*