From nobody Sat Feb 7 11:31:01 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 41BF91DB34B for ; Wed, 9 Apr 2025 21:11:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744233102; cv=none; b=hXpn+0qSWJ1OVU5POPS2nBTUU0wxGTHSYeQnxf9aXJEruK/gfUrR8CKtL6pXd12gM03xoJ86BrAkZN9sq5mVMKmZX++iOgm5yDzwptGa4jirlgtdO1KvJTqm85Lm103ROJjPnb+XljG2V+ededstTa2NXM7n1Hd9OxCHu+A9a1M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744233102; c=relaxed/simple; bh=eJTlTLsy0my7U6cHhkhoHNY8hFx6eYTSa9v4rjr5Vpo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=c0CTZM4VhJUbFvuSs6KNtd3nEtEtMI/sSQMM50NgAgJpX3FMFj/Wj2J8LukFqNbL9ayUt4Bgx2uhAay1MfKYQPSVNuWh4kNE2H16VQJoT5cvQqurvyZvgtRmW4/xoKToy3xkt4F9Xo3LIjsRiIsOL0zKmDh5q874bPwJ5nNHSY0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HhyaIajH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HhyaIajH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 02712C4CEE9; Wed, 9 Apr 2025 21:11:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1744233101; bh=eJTlTLsy0my7U6cHhkhoHNY8hFx6eYTSa9v4rjr5Vpo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HhyaIajH5BuV1wZOhZ3G6JTvZf/ST0sLZe6Kx/+A87EgArWdvg1TVdu+D1CWUqb74 eWR9Y/mLReMZYWLoJv+Zf4mUgcJ5Uv9e9+01u/YlrSlwMEAVTZHjVbkLdstc6AvARV L961ZoB+EGrO6U/k8/BQS1WCxPcR8EEaWhLvTkMES2njDmE9D7lUbftQaaZNH4PLPI qjTKemDaLVJWnqg5nj/sX5sBenv1WfWDiRbh+CoUTo9yr2ZU7KnbtCVzJ9+dPKcUKb +wF3YdW/iVCXgzkvaLvPxYwAe8HZ93Xyq+L6ibjg7A33jiuZ+J1Q4X0MBrLm8VpT3S ICTQDydIfU2Mw== From: Ingo Molnar To: linux-kernel@vger.kernel.org Cc: Andy Lutomirski , Dave Hansen , Brian Gerst , Peter Zijlstra , Borislav Petkov , "H . Peter Anvin" , Linus Torvalds , Oleg Nesterov , Thomas Gleixner , "Chang S . Bae" , Ingo Molnar Subject: [PATCH 1/8] x86/fpu: Introduce the x86_task_fpu() helper method Date: Wed, 9 Apr 2025 23:11:20 +0200 Message-ID: <20250409211127.3544993-2-mingo@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250409211127.3544993-1-mingo@kernel.org> References: <20250409211127.3544993-1-mingo@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The per-task FPU context/save area is allocated right next to task_struct, currently in a variable-size array via task_struct::thread.fpu[], but we plan to fully hide it from the C type scope. Introduce the x86_task_fpu() accessor that gets to the FPU context pointer explicitly from the task pointer. Right now this is a simple (task)->thread.fpu wrapper. Signed-off-by: Ingo Molnar --- arch/x86/include/asm/processor.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/proces= sor.h index 5d2f7e5aff26..2f631e0adea3 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -523,6 +523,8 @@ struct thread_struct { */ }; =20 +#define x86_task_fpu(task) (&(task)->thread.fpu) + extern void fpu_thread_struct_whitelist(unsigned long *offset, unsigned lo= ng *size); =20 static inline void arch_thread_struct_whitelist(unsigned long *offset, --=20 2.45.2 From nobody Sat Feb 7 11:31:01 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A832C367 for ; Sat, 3 May 2025 14:39:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746283167; cv=none; b=bFU2NNNREou921kkSj8JASd778FAaPjQgnvLi4APfBZkmd54qL/cQfnifDUsEmPOrCGRGwt71MFf7YS3ZSrSmWak2AqL4tAm4Kbim4iPAcUYWY53yHRsnq7TTuJPCRHvIQ/8GYD6AimjZpDV4KVa5+18cSRqBBr/1SygYWQBRJ8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746283167; c=relaxed/simple; bh=rn7+o4EPtcG0oJ6Fi2Yb4BaDa4A0wgw01u7a4616hkU=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=YFpQ1BNiPY1Yo5Pz2jKLcQJB4Skcs/YKWg3P1o5j7ebO/z19wkzm+ADDiGa0ECxY86fYOS4MQ9Tq7HLPufHlDUJn/djrVGxXvAlCarPAJKzXHYp2rvtW8jWnDbhQRh+PTEMvzjViGcLj45rMd43LLTyLmvz+DnG66BIgDhYth6E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=GodSzr3k; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="GodSzr3k" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1746283164; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=BH81id2CJI4866z+gecRvcwrouD/J6fuMh6H8yNd8d4=; b=GodSzr3kAD7BYPRoYUC31MignLSP909TsEW2Y5VKhAt2QUd7IVgpg42J5+s4q3A/5HgLJA CIq1Q4rhia0HnL6x0VjuLtqPz5gWA8//CpHngiApN4NIcuQRlABBPaj3/eKMOnoZ9Kv8LQ Hw+DHedpVi1qacNqPQ+T0ghDVls05C8= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-247-ZtCzUT75PpS_CNbRNwHy4Q-1; Sat, 03 May 2025 10:39:19 -0400 X-MC-Unique: ZtCzUT75PpS_CNbRNwHy4Q-1 X-Mimecast-MFC-AGG-ID: ZtCzUT75PpS_CNbRNwHy4Q_1746283157 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C2E061800264; Sat, 3 May 2025 14:39:16 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.44.32.23]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id CDD0D30001A2; Sat, 3 May 2025 14:39:11 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Sat, 3 May 2025 16:38:36 +0200 (CEST) Date: Sat, 3 May 2025 16:38:30 +0200 From: Oleg Nesterov To: Ingo Molnar Cc: linux-kernel@vger.kernel.org, Andy Lutomirski , Dave Hansen , Brian Gerst , Peter Zijlstra , Borislav Petkov , "H . Peter Anvin" , Linus Torvalds , Thomas Gleixner , "Chang S . Bae" Subject: [PATCH tip/x86/fpu 1/6] x86/fpu: simplify the switch_fpu_prepare() + switch_fpu_finish() logic Message-ID: <20250503143830.GA8982@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20250409211127.3544993-1-mingo@kernel.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that switch_fpu_finish() doesn't load the FPU state, it makes more sense to fold it into switch_fpu_prepare() renamed to switch_fpu(), and more importantly, use the "prev_p" task as a target for TIF_NEED_FPU_LOAD. It doesn't make any sense to delay set_tsk_thread_flag(TIF_NEED_FPU_LOAD) until "prev_p" is scheduled again. There is no worry about the very first context switch, fpu_clone() must always set TIF_NEED_FPU_LOAD. Also, shift the test_tsk_thread_flag(TIF_NEED_FPU_LOAD) from the callers to switch_fpu(). Note that the "PF_KTHREAD | PF_USER_WORKER" check can be removed but this deserves a separate patch which can change more functions, say, kernel_fpu_begin_mask(). Signed-off-by: Oleg Nesterov --- arch/x86/include/asm/fpu/sched.h | 34 +++++++++----------------------- arch/x86/kernel/process_32.c | 5 +---- arch/x86/kernel/process_64.c | 5 +---- 3 files changed, 11 insertions(+), 33 deletions(-) diff --git a/arch/x86/include/asm/fpu/sched.h b/arch/x86/include/asm/fpu/sc= hed.h index 5fd12634bcc4..c060549c6c94 100644 --- a/arch/x86/include/asm/fpu/sched.h +++ b/arch/x86/include/asm/fpu/sched.h @@ -18,31 +18,25 @@ extern void fpu_flush_thread(void); /* * FPU state switching for scheduling. * - * This is a two-stage process: + * switch_fpu() saves the old state and sets TIF_NEED_FPU_LOAD if + * TIF_NEED_FPU_LOAD is not set. This is done within the context + * of the old process. * - * - switch_fpu_prepare() saves the old state. - * This is done within the context of the old process. - * - * - switch_fpu_finish() sets TIF_NEED_FPU_LOAD; the floating point state - * will get loaded on return to userspace, or when the kernel needs it. - * - * If TIF_NEED_FPU_LOAD is cleared then the CPU's FPU registers - * are saved in the current thread's FPU register state. - * - * If TIF_NEED_FPU_LOAD is set then CPU's FPU registers may not - * hold current()'s FPU registers. It is required to load the + * Once TIF_NEED_FPU_LOAD is set, it is required to load the * registers before returning to userland or using the content * otherwise. * * The FPU context is only stored/restored for a user task and * PF_KTHREAD is used to distinguish between kernel and user threads. */ -static inline void switch_fpu_prepare(struct task_struct *old, int cpu) +static inline void switch_fpu(struct task_struct *old, int cpu) { - if (cpu_feature_enabled(X86_FEATURE_FPU) && + if (!test_tsk_thread_flag(old, TIF_NEED_FPU_LOAD) && + cpu_feature_enabled(X86_FEATURE_FPU) && !(old->flags & (PF_KTHREAD | PF_USER_WORKER))) { struct fpu *old_fpu =3D x86_task_fpu(old); =20 + set_tsk_thread_flag(old, TIF_NEED_FPU_LOAD); save_fpregs_to_fpstate(old_fpu); /* * The save operation preserved register state, so the @@ -50,7 +44,7 @@ static inline void switch_fpu_prepare(struct task_struct = *old, int cpu) * current CPU number in @old_fpu, so the next return * to user space can avoid the FPU register restore * when is returns on the same CPU and still owns the - * context. + * context. See fpregs_restore_userregs(). */ old_fpu->last_cpu =3D cpu; =20 @@ -58,14 +52,4 @@ static inline void switch_fpu_prepare(struct task_struct= *old, int cpu) } } =20 -/* - * Delay loading of the complete FPU state until the return to userland. - * PKRU is handled separately. - */ -static inline void switch_fpu_finish(struct task_struct *new) -{ - if (cpu_feature_enabled(X86_FEATURE_FPU)) - set_tsk_thread_flag(new, TIF_NEED_FPU_LOAD); -} - #endif /* _ASM_X86_FPU_SCHED_H */ diff --git a/arch/x86/kernel/process_32.c b/arch/x86/kernel/process_32.c index 4636ef359973..9bd4fa694da5 100644 --- a/arch/x86/kernel/process_32.c +++ b/arch/x86/kernel/process_32.c @@ -160,8 +160,7 @@ __switch_to(struct task_struct *prev_p, struct task_str= uct *next_p) =20 /* never put a printk in __switch_to... printk() calls wake_up*() indirec= tly */ =20 - if (!test_tsk_thread_flag(prev_p, TIF_NEED_FPU_LOAD)) - switch_fpu_prepare(prev_p, cpu); + switch_fpu(prev_p, cpu); =20 /* * Save away %gs. No need to save %fs, as it was saved on the @@ -208,8 +207,6 @@ __switch_to(struct task_struct *prev_p, struct task_str= uct *next_p) =20 raw_cpu_write(current_task, next_p); =20 - switch_fpu_finish(next_p); - /* Load the Intel cache allocation PQR MSR. */ resctrl_sched_in(next_p); =20 diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index 7196ca7048be..d55310d3133c 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -616,8 +616,7 @@ __switch_to(struct task_struct *prev_p, struct task_str= uct *next_p) WARN_ON_ONCE(IS_ENABLED(CONFIG_DEBUG_ENTRY) && this_cpu_read(hardirq_stack_inuse)); =20 - if (!test_tsk_thread_flag(prev_p, TIF_NEED_FPU_LOAD)) - switch_fpu_prepare(prev_p, cpu); + switch_fpu(prev_p, cpu); =20 /* We must save %fs and %gs before load_TLS() because * %fs and %gs may be cleared by load_TLS(). @@ -671,8 +670,6 @@ __switch_to(struct task_struct *prev_p, struct task_str= uct *next_p) raw_cpu_write(current_task, next_p); raw_cpu_write(cpu_current_top_of_stack, task_top_of_stack(next_p)); =20 - switch_fpu_finish(next_p); - /* Reload sp0. */ update_task_stack(next_p); =20 --=20 2.25.1.362.g51ebf55 From nobody Sat Feb 7 11:31:01 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6E291215F5F for ; Wed, 9 Apr 2025 21:11:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744233105; cv=none; b=DbLW+YEu9F0xP7jLH1pLyqCsQDb3oogzJV7VoGANFapWkxEj88HuwggfCQH3hCHkPSRBp2i6BsBYqJ5ljnSaOrlcQBba+Db1E77m+D9uzvFpeYphwNrwQSOFSxIcY/0FozqqPE1XwBoFzu0kd3juCjI72ZMzu1eEMSlLfrhXG5o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744233105; c=relaxed/simple; bh=yNKC6xjCh/VHlgTUZucey/NA/hf/QQHH5vsP/s5EW/0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=S6G5PsoAbOoi9UnPn+iYuOzcjrMeDsj0ek+So8gmKr8S3xfk3QBVyRxjamZofl8chCjtLFg0Af7T9KqYzkV3bzbGHlfHdXRftNsvKNilvxaPdGSxRMXZinxLGbKmv6N6B4RWgMnP8j+Gf6XnHn6E9bQQ89HBG13X2ekqHw2Wcs0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=IVuA25C6; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="IVuA25C6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2B5BFC4CEE2; Wed, 9 Apr 2025 21:11:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1744233104; bh=yNKC6xjCh/VHlgTUZucey/NA/hf/QQHH5vsP/s5EW/0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IVuA25C6Xn4KU2T5RP/60NQiibjH6+O9ltZAfnIC/b/r8D87TNvYA3eSOvBMGkKPz /JsDPbAqtcmugYCUTwTFzN/QOMgK5mDlnlj6AjNWfSlTnzbbD2XSolevKOqpIaanrd SDH82ciOQvy8CdaRcrwuw9IrNssatWWGGCIBuq/Ukl6KJtuupf0YdtEt1RQ58r9J2j kKXDPQBPn94mXCxC38+7hvvO+quyvcgbp0cec98wLZYJCSiDhIKWQWE0Hv1deSzNWK +q8fvuaOTdsEEN8ZfoKagGg8XMhCzMHOK/oWtx1fM2MARIs58WeyO9/8ouYl/jOejc Fa3GEBbbQhkyQ== From: Ingo Molnar To: linux-kernel@vger.kernel.org Cc: Andy Lutomirski , Dave Hansen , Brian Gerst , Peter Zijlstra , Borislav Petkov , "H . Peter Anvin" , Linus Torvalds , Oleg Nesterov , Thomas Gleixner , "Chang S . Bae" , Ingo Molnar Subject: [PATCH 2/8] x86/fpu: Convert task_struct::thread.fpu accesses to use x86_task_fpu() Date: Wed, 9 Apr 2025 23:11:21 +0200 Message-ID: <20250409211127.3544993-3-mingo@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250409211127.3544993-1-mingo@kernel.org> References: <20250409211127.3544993-1-mingo@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This will make the removal of the task_struct::thread.fpu array easier. No change in functionality - code generated before and after this commit is identical on x86-defconfig: kepler:~/tip> diff -up vmlinux.before.asm vmlinux.after.asm kepler:~/tip> Signed-off-by: Ingo Molnar --- arch/x86/include/asm/fpu/sched.h | 2 +- arch/x86/kernel/fpu/context.h | 4 ++-- arch/x86/kernel/fpu/core.c | 30 +++++++++++++++--------------- arch/x86/kernel/fpu/init.c | 8 ++++---- arch/x86/kernel/fpu/regset.c | 22 +++++++++++----------- arch/x86/kernel/fpu/signal.c | 18 +++++++++--------- arch/x86/kernel/fpu/xstate.c | 22 +++++++++++----------- arch/x86/kernel/fpu/xstate.h | 6 +++--- arch/x86/kernel/process.c | 6 +++--- arch/x86/kernel/signal.c | 6 +++--- arch/x86/kernel/traps.c | 2 +- arch/x86/math-emu/fpu_aux.c | 2 +- arch/x86/math-emu/fpu_entry.c | 4 ++-- arch/x86/math-emu/fpu_system.h | 2 +- arch/x86/mm/extable.c | 2 +- 15 files changed, 68 insertions(+), 68 deletions(-) diff --git a/arch/x86/include/asm/fpu/sched.h b/arch/x86/include/asm/fpu/sc= hed.h index c485f1944c5f..1feaa68b7567 100644 --- a/arch/x86/include/asm/fpu/sched.h +++ b/arch/x86/include/asm/fpu/sched.h @@ -41,7 +41,7 @@ static inline void switch_fpu_prepare(struct task_struct = *old, int cpu) { if (cpu_feature_enabled(X86_FEATURE_FPU) && !(old->flags & (PF_KTHREAD | PF_USER_WORKER))) { - struct fpu *old_fpu =3D &old->thread.fpu; + struct fpu *old_fpu =3D x86_task_fpu(old); =20 save_fpregs_to_fpstate(old_fpu); /* diff --git a/arch/x86/kernel/fpu/context.h b/arch/x86/kernel/fpu/context.h index f6d856bd50bc..10d0a720659c 100644 --- a/arch/x86/kernel/fpu/context.h +++ b/arch/x86/kernel/fpu/context.h @@ -53,7 +53,7 @@ static inline void fpregs_activate(struct fpu *fpu) /* Internal helper for switch_fpu_return() and signal frame setup */ static inline void fpregs_restore_userregs(void) { - struct fpu *fpu =3D ¤t->thread.fpu; + struct fpu *fpu =3D x86_task_fpu(current); int cpu =3D smp_processor_id(); =20 if (WARN_ON_ONCE(current->flags & (PF_KTHREAD | PF_USER_WORKER))) @@ -67,7 +67,7 @@ static inline void fpregs_restore_userregs(void) * If PKRU is enabled, then the PKRU value is already * correct because it was either set in switch_to() or in * flush_thread(). So it is excluded because it might be - * not up to date in current->thread.fpu.xsave state. + * not up to date in current->thread.fpu->xsave state. * * XFD state is handled in restore_fpregs_from_fpstate(). */ diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index 91d6341f281f..dc6d7f93c446 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -211,7 +211,7 @@ static void fpu_init_guest_permissions(struct fpu_guest= *gfpu) return; =20 spin_lock_irq(¤t->sighand->siglock); - fpuperm =3D ¤t->group_leader->thread.fpu.guest_perm; + fpuperm =3D &x86_task_fpu(current->group_leader)->guest_perm; perm =3D fpuperm->__state_perm; =20 /* First fpstate allocation locks down permissions. */ @@ -323,7 +323,7 @@ EXPORT_SYMBOL_GPL(fpu_update_guest_xfd); */ void fpu_sync_guest_vmexit_xfd_state(void) { - struct fpstate *fps =3D current->thread.fpu.fpstate; + struct fpstate *fps =3D x86_task_fpu(current)->fpstate; =20 lockdep_assert_irqs_disabled(); if (fpu_state_size_dynamic()) { @@ -337,7 +337,7 @@ EXPORT_SYMBOL_GPL(fpu_sync_guest_vmexit_xfd_state); int fpu_swap_kvm_fpstate(struct fpu_guest *guest_fpu, bool enter_guest) { struct fpstate *guest_fps =3D guest_fpu->fpstate; - struct fpu *fpu =3D ¤t->thread.fpu; + struct fpu *fpu =3D x86_task_fpu(current); struct fpstate *cur_fps =3D fpu->fpstate; =20 fpregs_lock(); @@ -438,7 +438,7 @@ void kernel_fpu_begin_mask(unsigned int kfpu_mask) if (!(current->flags & (PF_KTHREAD | PF_USER_WORKER)) && !test_thread_flag(TIF_NEED_FPU_LOAD)) { set_thread_flag(TIF_NEED_FPU_LOAD); - save_fpregs_to_fpstate(¤t->thread.fpu); + save_fpregs_to_fpstate(x86_task_fpu(current)); } __cpu_invalidate_fpregs_state(); =20 @@ -467,7 +467,7 @@ EXPORT_SYMBOL_GPL(kernel_fpu_end); */ void fpu_sync_fpstate(struct fpu *fpu) { - WARN_ON_FPU(fpu !=3D ¤t->thread.fpu); + WARN_ON_FPU(fpu !=3D x86_task_fpu(current)); =20 fpregs_lock(); trace_x86_fpu_before_save(fpu); @@ -552,7 +552,7 @@ void fpstate_reset(struct fpu *fpu) static inline void fpu_inherit_perms(struct fpu *dst_fpu) { if (fpu_state_size_dynamic()) { - struct fpu *src_fpu =3D ¤t->group_leader->thread.fpu; + struct fpu *src_fpu =3D x86_task_fpu(current->group_leader); =20 spin_lock_irq(¤t->sighand->siglock); /* Fork also inherits the permissions of the parent */ @@ -572,7 +572,7 @@ static int update_fpu_shstk(struct task_struct *dst, un= signed long ssp) if (!ssp) return 0; =20 - xstate =3D get_xsave_addr(&dst->thread.fpu.fpstate->regs.xsave, + xstate =3D get_xsave_addr(&x86_task_fpu(dst)->fpstate->regs.xsave, XFEATURE_CET_USER); =20 /* @@ -593,8 +593,8 @@ static int update_fpu_shstk(struct task_struct *dst, un= signed long ssp) int fpu_clone(struct task_struct *dst, unsigned long clone_flags, bool min= imal, unsigned long ssp) { - struct fpu *src_fpu =3D ¤t->thread.fpu; - struct fpu *dst_fpu =3D &dst->thread.fpu; + struct fpu *src_fpu =3D x86_task_fpu(current); + struct fpu *dst_fpu =3D x86_task_fpu(dst); =20 /* The new task's FPU state cannot be valid in the hardware. */ dst_fpu->last_cpu =3D -1; @@ -686,7 +686,7 @@ void fpu__drop(struct fpu *fpu) { preempt_disable(); =20 - if (fpu =3D=3D ¤t->thread.fpu) { + if (fpu =3D=3D x86_task_fpu(current)) { /* Ignore delayed exceptions from user space */ asm volatile("1: fwait\n" "2:\n" @@ -720,7 +720,7 @@ static inline void restore_fpregs_from_init_fpstate(u64= features_mask) */ static void fpu_reset_fpregs(void) { - struct fpu *fpu =3D ¤t->thread.fpu; + struct fpu *fpu =3D x86_task_fpu(current); =20 fpregs_lock(); __fpu_invalidate_fpregs_state(fpu); @@ -749,7 +749,7 @@ static void fpu_reset_fpregs(void) */ void fpu__clear_user_states(struct fpu *fpu) { - WARN_ON_FPU(fpu !=3D ¤t->thread.fpu); + WARN_ON_FPU(fpu !=3D x86_task_fpu(current)); =20 fpregs_lock(); if (!cpu_feature_enabled(X86_FEATURE_FPU)) { @@ -782,7 +782,7 @@ void fpu__clear_user_states(struct fpu *fpu) =20 void fpu_flush_thread(void) { - fpstate_reset(¤t->thread.fpu); + fpstate_reset(x86_task_fpu(current)); fpu_reset_fpregs(); } /* @@ -823,7 +823,7 @@ void fpregs_lock_and_load(void) */ void fpregs_assert_state_consistent(void) { - struct fpu *fpu =3D ¤t->thread.fpu; + struct fpu *fpu =3D x86_task_fpu(current); =20 if (test_thread_flag(TIF_NEED_FPU_LOAD)) return; @@ -835,7 +835,7 @@ EXPORT_SYMBOL_GPL(fpregs_assert_state_consistent); =20 void fpregs_mark_activate(void) { - struct fpu *fpu =3D ¤t->thread.fpu; + struct fpu *fpu =3D x86_task_fpu(current); =20 fpregs_activate(fpu); fpu->last_cpu =3D smp_processor_id(); diff --git a/arch/x86/kernel/fpu/init.c b/arch/x86/kernel/fpu/init.c index 998a08f17e33..ad5cb2943d37 100644 --- a/arch/x86/kernel/fpu/init.c +++ b/arch/x86/kernel/fpu/init.c @@ -38,7 +38,7 @@ static void fpu__init_cpu_generic(void) /* Flush out any pending x87 state: */ #ifdef CONFIG_MATH_EMULATION if (!boot_cpu_has(X86_FEATURE_FPU)) - fpstate_init_soft(¤t->thread.fpu.fpstate->regs.soft); + fpstate_init_soft(&x86_task_fpu(current)->fpstate->regs.soft); else #endif asm volatile ("fninit"); @@ -154,7 +154,7 @@ static void __init fpu__init_task_struct_size(void) * Subtract off the static size of the register state. * It potentially has a bunch of padding. */ - task_size -=3D sizeof(current->thread.fpu.__fpstate.regs); + task_size -=3D sizeof(union fpregs_state); =20 /* * Add back the dynamically-calculated register state @@ -204,7 +204,7 @@ static void __init fpu__init_system_xstate_size_legacy(= void) fpu_kernel_cfg.default_size =3D size; fpu_user_cfg.max_size =3D size; fpu_user_cfg.default_size =3D size; - fpstate_reset(¤t->thread.fpu); + fpstate_reset(x86_task_fpu(current)); } =20 /* @@ -213,7 +213,7 @@ static void __init fpu__init_system_xstate_size_legacy(= void) */ void __init fpu__init_system(void) { - fpstate_reset(¤t->thread.fpu); + fpstate_reset(x86_task_fpu(current)); fpu__init_system_early_generic(); =20 /* diff --git a/arch/x86/kernel/fpu/regset.c b/arch/x86/kernel/fpu/regset.c index 887b0b8e21e3..0986c2200adc 100644 --- a/arch/x86/kernel/fpu/regset.c +++ b/arch/x86/kernel/fpu/regset.c @@ -45,7 +45,7 @@ int regset_xregset_fpregs_active(struct task_struct *targ= et, const struct user_r */ static void sync_fpstate(struct fpu *fpu) { - if (fpu =3D=3D ¤t->thread.fpu) + if (fpu =3D=3D x86_task_fpu(current)) fpu_sync_fpstate(fpu); } =20 @@ -63,7 +63,7 @@ static void fpu_force_restore(struct fpu *fpu) * Only stopped child tasks can be used to modify the FPU * state in the fpstate buffer: */ - WARN_ON_FPU(fpu =3D=3D ¤t->thread.fpu); + WARN_ON_FPU(fpu =3D=3D x86_task_fpu(current)); =20 __fpu_invalidate_fpregs_state(fpu); } @@ -71,7 +71,7 @@ static void fpu_force_restore(struct fpu *fpu) int xfpregs_get(struct task_struct *target, const struct user_regset *regs= et, struct membuf to) { - struct fpu *fpu =3D &target->thread.fpu; + struct fpu *fpu =3D x86_task_fpu(target); =20 if (!cpu_feature_enabled(X86_FEATURE_FXSR)) return -ENODEV; @@ -91,7 +91,7 @@ int xfpregs_set(struct task_struct *target, const struct = user_regset *regset, unsigned int pos, unsigned int count, const void *kbuf, const void __user *ubuf) { - struct fpu *fpu =3D &target->thread.fpu; + struct fpu *fpu =3D x86_task_fpu(target); struct fxregs_state newstate; int ret; =20 @@ -133,7 +133,7 @@ int xstateregs_get(struct task_struct *target, const st= ruct user_regset *regset, if (!cpu_feature_enabled(X86_FEATURE_XSAVE)) return -ENODEV; =20 - sync_fpstate(&target->thread.fpu); + sync_fpstate(x86_task_fpu(target)); =20 copy_xstate_to_uabi_buf(to, target, XSTATE_COPY_XSAVE); return 0; @@ -143,7 +143,7 @@ int xstateregs_set(struct task_struct *target, const st= ruct user_regset *regset, unsigned int pos, unsigned int count, const void *kbuf, const void __user *ubuf) { - struct fpu *fpu =3D &target->thread.fpu; + struct fpu *fpu =3D x86_task_fpu(target); struct xregs_state *tmpbuf =3D NULL; int ret; =20 @@ -187,7 +187,7 @@ int ssp_active(struct task_struct *target, const struct= user_regset *regset) int ssp_get(struct task_struct *target, const struct user_regset *regset, struct membuf to) { - struct fpu *fpu =3D &target->thread.fpu; + struct fpu *fpu =3D x86_task_fpu(target); struct cet_user_state *cetregs; =20 if (!cpu_feature_enabled(X86_FEATURE_USER_SHSTK) || @@ -214,7 +214,7 @@ int ssp_set(struct task_struct *target, const struct us= er_regset *regset, unsigned int pos, unsigned int count, const void *kbuf, const void __user *ubuf) { - struct fpu *fpu =3D &target->thread.fpu; + struct fpu *fpu =3D x86_task_fpu(target); struct xregs_state *xsave =3D &fpu->fpstate->regs.xsave; struct cet_user_state *cetregs; unsigned long user_ssp; @@ -368,7 +368,7 @@ static void __convert_from_fxsr(struct user_i387_ia32_s= truct *env, void convert_from_fxsr(struct user_i387_ia32_struct *env, struct task_struct *t= sk) { - __convert_from_fxsr(env, tsk, &tsk->thread.fpu.fpstate->regs.fxsave); + __convert_from_fxsr(env, tsk, &x86_task_fpu(tsk)->fpstate->regs.fxsave); } =20 void convert_to_fxsr(struct fxregs_state *fxsave, @@ -401,7 +401,7 @@ void convert_to_fxsr(struct fxregs_state *fxsave, int fpregs_get(struct task_struct *target, const struct user_regset *regse= t, struct membuf to) { - struct fpu *fpu =3D &target->thread.fpu; + struct fpu *fpu =3D x86_task_fpu(target); struct user_i387_ia32_struct env; struct fxregs_state fxsave, *fx; =20 @@ -433,7 +433,7 @@ int fpregs_set(struct task_struct *target, const struct= user_regset *regset, unsigned int pos, unsigned int count, const void *kbuf, const void __user *ubuf) { - struct fpu *fpu =3D &target->thread.fpu; + struct fpu *fpu =3D x86_task_fpu(target); struct user_i387_ia32_struct env; int ret; =20 diff --git a/arch/x86/kernel/fpu/signal.c b/arch/x86/kernel/fpu/signal.c index 6c69cb28b298..b8b4fa9c2d04 100644 --- a/arch/x86/kernel/fpu/signal.c +++ b/arch/x86/kernel/fpu/signal.c @@ -43,13 +43,13 @@ static inline bool check_xstate_in_sigframe(struct fxre= gs_state __user *fxbuf, * fpstate layout with out copying the extended state information * in the memory layout. */ - if (__get_user(magic2, (__u32 __user *)(fpstate + current->thread.fpu.fps= tate->user_size))) + if (__get_user(magic2, (__u32 __user *)(fpstate + x86_task_fpu(current)->= fpstate->user_size))) return false; =20 if (likely(magic2 =3D=3D FP_XSTATE_MAGIC2)) return true; setfx: - trace_x86_fpu_xstate_check_failed(¤t->thread.fpu); + trace_x86_fpu_xstate_check_failed(x86_task_fpu(current)); =20 /* Set the parameters for fx only state */ fx_sw->magic1 =3D 0; @@ -64,13 +64,13 @@ static inline bool check_xstate_in_sigframe(struct fxre= gs_state __user *fxbuf, static inline bool save_fsave_header(struct task_struct *tsk, void __user = *buf) { if (use_fxsr()) { - struct xregs_state *xsave =3D &tsk->thread.fpu.fpstate->regs.xsave; + struct xregs_state *xsave =3D &x86_task_fpu(tsk)->fpstate->regs.xsave; struct user_i387_ia32_struct env; struct _fpstate_32 __user *fp =3D buf; =20 fpregs_lock(); if (!test_thread_flag(TIF_NEED_FPU_LOAD)) - fxsave(&tsk->thread.fpu.fpstate->regs.fxsave); + fxsave(&x86_task_fpu(tsk)->fpstate->regs.fxsave); fpregs_unlock(); =20 convert_from_fxsr(&env, tsk); @@ -184,7 +184,7 @@ static inline int copy_fpregs_to_sigframe(struct xregs_= state __user *buf, u32 pk bool copy_fpstate_to_sigframe(void __user *buf, void __user *buf_fx, int s= ize, u32 pkru) { struct task_struct *tsk =3D current; - struct fpstate *fpstate =3D tsk->thread.fpu.fpstate; + struct fpstate *fpstate =3D x86_task_fpu(tsk)->fpstate; bool ia32_fxstate =3D (buf !=3D buf_fx); int ret; =20 @@ -272,7 +272,7 @@ static int __restore_fpregs_from_user(void __user *buf,= u64 ufeatures, */ static bool restore_fpregs_from_user(void __user *buf, u64 xrestore, bool = fx_only) { - struct fpu *fpu =3D ¤t->thread.fpu; + struct fpu *fpu =3D x86_task_fpu(current); int ret; =20 /* Restore enabled features only. */ @@ -332,7 +332,7 @@ static bool __fpu_restore_sig(void __user *buf, void __= user *buf_fx, bool ia32_fxstate) { struct task_struct *tsk =3D current; - struct fpu *fpu =3D &tsk->thread.fpu; + struct fpu *fpu =3D x86_task_fpu(tsk); struct user_i387_ia32_struct env; bool success, fx_only =3D false; union fpregs_state *fpregs; @@ -452,7 +452,7 @@ static inline unsigned int xstate_sigframe_size(struct = fpstate *fpstate) */ bool fpu__restore_sig(void __user *buf, int ia32_frame) { - struct fpu *fpu =3D ¤t->thread.fpu; + struct fpu *fpu =3D x86_task_fpu(current); void __user *buf_fx =3D buf; bool ia32_fxstate =3D false; bool success =3D false; @@ -499,7 +499,7 @@ unsigned long fpu__alloc_mathframe(unsigned long sp, int ia32_frame, unsigned long *buf_fx, unsigned long *size) { - unsigned long frame_size =3D xstate_sigframe_size(current->thread.fpu.fps= tate); + unsigned long frame_size =3D xstate_sigframe_size(x86_task_fpu(current)->= fpstate); =20 *buf_fx =3D sp =3D round_down(sp - frame_size, 64); if (ia32_frame && use_fxsr()) { diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c index 6a41d1610d8b..ab2588b8f17f 100644 --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -727,7 +727,7 @@ static void __init fpu__init_disable_system_xstate(unsi= gned int legacy_size) */ init_fpstate.xfd =3D 0; =20 - fpstate_reset(¤t->thread.fpu); + fpstate_reset(x86_task_fpu(current)); } =20 /* @@ -835,7 +835,7 @@ void __init fpu__init_system_xstate(unsigned int legacy= _size) goto out_disable; =20 /* Reset the state for the current task */ - fpstate_reset(¤t->thread.fpu); + fpstate_reset(x86_task_fpu(current)); =20 /* * Update info used for ptrace frames; use standard-format size and no @@ -909,7 +909,7 @@ void fpu__resume_cpu(void) } =20 if (fpu_state_size_dynamic()) - wrmsrl(MSR_IA32_XFD, current->thread.fpu.fpstate->xfd); + wrmsrl(MSR_IA32_XFD, x86_task_fpu(current)->fpstate->xfd); } =20 /* @@ -1191,8 +1191,8 @@ void __copy_xstate_to_uabi_buf(struct membuf to, stru= ct fpstate *fpstate, void copy_xstate_to_uabi_buf(struct membuf to, struct task_struct *tsk, enum xstate_copy_mode copy_mode) { - __copy_xstate_to_uabi_buf(to, tsk->thread.fpu.fpstate, - tsk->thread.fpu.fpstate->user_xfeatures, + __copy_xstate_to_uabi_buf(to, x86_task_fpu(tsk)->fpstate, + x86_task_fpu(tsk)->fpstate->user_xfeatures, tsk->thread.pkru, copy_mode); } =20 @@ -1332,7 +1332,7 @@ int copy_uabi_from_kernel_to_xstate(struct fpstate *f= pstate, const void *kbuf, u int copy_sigframe_from_user_to_xstate(struct task_struct *tsk, const void __user *ubuf) { - return copy_uabi_to_xstate(tsk->thread.fpu.fpstate, NULL, ubuf, &tsk->thr= ead.pkru); + return copy_uabi_to_xstate(x86_task_fpu(tsk)->fpstate, NULL, ubuf, &tsk->= thread.pkru); } =20 static bool validate_independent_components(u64 mask) @@ -1426,7 +1426,7 @@ static bool xstate_op_valid(struct fpstate *fpstate, = u64 mask, bool rstor) * The XFD MSR does not match fpstate->xfd. That's invalid when * the passed in fpstate is current's fpstate. */ - if (fpstate->xfd =3D=3D current->thread.fpu.fpstate->xfd) + if (fpstate->xfd =3D=3D x86_task_fpu(current)->fpstate->xfd) return false; =20 /* @@ -1503,7 +1503,7 @@ void fpstate_free(struct fpu *fpu) static int fpstate_realloc(u64 xfeatures, unsigned int ksize, unsigned int usize, struct fpu_guest *guest_fpu) { - struct fpu *fpu =3D ¤t->thread.fpu; + struct fpu *fpu =3D x86_task_fpu(current); struct fpstate *curfps, *newfps =3D NULL; unsigned int fpsize; bool in_use; @@ -1596,7 +1596,7 @@ static int __xstate_request_perm(u64 permitted, u64 r= equested, bool guest) * AVX512. */ bool compacted =3D cpu_feature_enabled(X86_FEATURE_XCOMPACTED); - struct fpu *fpu =3D ¤t->group_leader->thread.fpu; + struct fpu *fpu =3D x86_task_fpu(current->group_leader); struct fpu_state_perm *perm; unsigned int ksize, usize; u64 mask; @@ -1699,7 +1699,7 @@ int __xfd_enable_feature(u64 xfd_err, struct fpu_gues= t *guest_fpu) return -EPERM; } =20 - fpu =3D ¤t->group_leader->thread.fpu; + fpu =3D x86_task_fpu(current->group_leader); perm =3D guest_fpu ? &fpu->guest_perm : &fpu->perm; ksize =3D perm->__state_size; usize =3D perm->__user_state_size; @@ -1804,7 +1804,7 @@ long fpu_xstate_prctl(int option, unsigned long arg2) */ static void avx512_status(struct seq_file *m, struct task_struct *task) { - unsigned long timestamp =3D READ_ONCE(task->thread.fpu.avx512_timestamp); + unsigned long timestamp =3D READ_ONCE(x86_task_fpu(task)->avx512_timestam= p); long delta; =20 if (!timestamp) { diff --git a/arch/x86/kernel/fpu/xstate.h b/arch/x86/kernel/fpu/xstate.h index 0fd34f53f025..9a3a8ccf13bf 100644 --- a/arch/x86/kernel/fpu/xstate.h +++ b/arch/x86/kernel/fpu/xstate.h @@ -22,7 +22,7 @@ static inline void xstate_init_xcomp_bv(struct xregs_stat= e *xsave, u64 mask) =20 static inline u64 xstate_get_group_perm(bool guest) { - struct fpu *fpu =3D ¤t->group_leader->thread.fpu; + struct fpu *fpu =3D x86_task_fpu(current->group_leader); struct fpu_state_perm *perm; =20 /* Pairs with WRITE_ONCE() in xstate_request_perm() */ @@ -288,7 +288,7 @@ static inline int xsave_to_user_sigframe(struct xregs_s= tate __user *buf, u32 pkr * internally, e.g. PKRU. That's user space ABI and also required * to allow the signal handler to modify PKRU. */ - struct fpstate *fpstate =3D current->thread.fpu.fpstate; + struct fpstate *fpstate =3D x86_task_fpu(current)->fpstate; u64 mask =3D fpstate->user_xfeatures; u32 lmask; u32 hmask; @@ -322,7 +322,7 @@ static inline int xrstor_from_user_sigframe(struct xreg= s_state __user *buf, u64 u32 hmask =3D mask >> 32; int err; =20 - xfd_validate_state(current->thread.fpu.fpstate, mask, true); + xfd_validate_state(x86_task_fpu(current)->fpstate, mask, true); =20 stac(); XSTATE_OP(XRSTOR, xstate, lmask, hmask, err); diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c index 962c3ce39323..47694e391506 100644 --- a/arch/x86/kernel/process.c +++ b/arch/x86/kernel/process.c @@ -103,7 +103,7 @@ int arch_dup_task_struct(struct task_struct *dst, struc= t task_struct *src) dst->thread.vm86 =3D NULL; #endif /* Drop the copied pointer to current's fpstate */ - dst->thread.fpu.fpstate =3D NULL; + x86_task_fpu(dst)->fpstate =3D NULL; =20 return 0; } @@ -112,7 +112,7 @@ int arch_dup_task_struct(struct task_struct *dst, struc= t task_struct *src) void arch_release_task_struct(struct task_struct *tsk) { if (fpu_state_size_dynamic()) - fpstate_free(&tsk->thread.fpu); + fpstate_free(x86_task_fpu(tsk)); } #endif =20 @@ -122,7 +122,7 @@ void arch_release_task_struct(struct task_struct *tsk) void exit_thread(struct task_struct *tsk) { struct thread_struct *t =3D &tsk->thread; - struct fpu *fpu =3D &t->fpu; + struct fpu *fpu =3D x86_task_fpu(tsk); =20 if (test_thread_flag(TIF_IO_BITMAP)) io_bitmap_exit(tsk); diff --git a/arch/x86/kernel/signal.c b/arch/x86/kernel/signal.c index 5f441039b572..2404233336ab 100644 --- a/arch/x86/kernel/signal.c +++ b/arch/x86/kernel/signal.c @@ -255,7 +255,7 @@ static void handle_signal(struct ksignal *ksig, struct pt_regs *regs) { bool stepping, failed; - struct fpu *fpu =3D ¤t->thread.fpu; + struct fpu *fpu =3D x86_task_fpu(current); =20 if (v8086_mode(regs)) save_v86_state((struct kernel_vm86_regs *) regs, VM86_SIGNAL); @@ -423,14 +423,14 @@ bool sigaltstack_size_valid(size_t ss_size) if (!fpu_state_size_dynamic() && !strict_sigaltstack_size) return true; =20 - fsize +=3D current->group_leader->thread.fpu.perm.__user_state_size; + fsize +=3D x86_task_fpu(current->group_leader)->perm.__user_state_size; if (likely(ss_size > fsize)) return true; =20 if (strict_sigaltstack_size) return ss_size > fsize; =20 - mask =3D current->group_leader->thread.fpu.perm.__state_perm; + mask =3D x86_task_fpu(current->group_leader)->perm.__state_perm; if (mask & XFEATURE_MASK_USER_DYNAMIC) return ss_size > fsize; =20 diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 9f88b8a78e50..f48325dfaa01 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -1295,7 +1295,7 @@ DEFINE_IDTENTRY_RAW(exc_debug) static void math_error(struct pt_regs *regs, int trapnr) { struct task_struct *task =3D current; - struct fpu *fpu =3D &task->thread.fpu; + struct fpu *fpu =3D x86_task_fpu(task); int si_code; char *str =3D (trapnr =3D=3D X86_TRAP_MF) ? "fpu exception" : "simd exception"; diff --git a/arch/x86/math-emu/fpu_aux.c b/arch/x86/math-emu/fpu_aux.c index d62662bdd460..5f253ae406b6 100644 --- a/arch/x86/math-emu/fpu_aux.c +++ b/arch/x86/math-emu/fpu_aux.c @@ -53,7 +53,7 @@ void fpstate_init_soft(struct swregs_state *soft) =20 void finit(void) { - fpstate_init_soft(¤t->thread.fpu.fpstate->regs.soft); + fpstate_init_soft(&x86_task_fpu(current)->fpstate->regs.soft); } =20 /* diff --git a/arch/x86/math-emu/fpu_entry.c b/arch/x86/math-emu/fpu_entry.c index 91c52ead1226..5034df617740 100644 --- a/arch/x86/math-emu/fpu_entry.c +++ b/arch/x86/math-emu/fpu_entry.c @@ -641,7 +641,7 @@ int fpregs_soft_set(struct task_struct *target, unsigned int pos, unsigned int count, const void *kbuf, const void __user *ubuf) { - struct swregs_state *s387 =3D &target->thread.fpu.fpstate->regs.soft; + struct swregs_state *s387 =3D &x86_task_fpu(target)->fpstate->regs.soft; void *space =3D s387->st_space; int ret; int offset, other, i, tags, regnr, tag, newtop; @@ -692,7 +692,7 @@ int fpregs_soft_get(struct task_struct *target, const struct user_regset *regset, struct membuf to) { - struct swregs_state *s387 =3D &target->thread.fpu.fpstate->regs.soft; + struct swregs_state *s387 =3D &x86_task_fpu(target)->fpstate->regs.soft; const void *space =3D s387->st_space; int offset =3D (S387->ftop & 7) * 10, other =3D 80 - offset; =20 diff --git a/arch/x86/math-emu/fpu_system.h b/arch/x86/math-emu/fpu_system.h index eec3e4805c75..5e238e930fe3 100644 --- a/arch/x86/math-emu/fpu_system.h +++ b/arch/x86/math-emu/fpu_system.h @@ -73,7 +73,7 @@ static inline bool seg_writable(struct desc_struct *d) return (d->type & SEG_TYPE_EXECUTE_MASK) =3D=3D SEG_TYPE_WRITABLE; } =20 -#define I387 (¤t->thread.fpu.fpstate->regs) +#define I387 (&x86_task_fpu(current)->fpstate->regs) #define FPU_info (I387->soft.info) =20 #define FPU_CS (*(unsigned short *) &(FPU_info->regs->cs)) diff --git a/arch/x86/mm/extable.c b/arch/x86/mm/extable.c index 51986e8a9d35..bf8dab18be97 100644 --- a/arch/x86/mm/extable.c +++ b/arch/x86/mm/extable.c @@ -111,7 +111,7 @@ static bool ex_handler_sgx(const struct exception_table= _entry *fixup, =20 /* * Handler for when we fail to restore a task's FPU state. We should neve= r get - * here because the FPU state of a task using the FPU (task->thread.fpu.st= ate) + * here because the FPU state of a task using the FPU (struct fpu::fpstate) * should always be valid. However, past bugs have allowed userspace to s= et * reserved bits in the XSAVE area using PTRACE_SETREGSET or sys_rt_sigret= urn(). * These caused XRSTOR to fail when switching to the task, leaking the FPU --=20 2.45.2 From nobody Sat Feb 7 11:31:01 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E7AAF2798E6 for ; Sat, 3 May 2025 14:39:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746283171; cv=none; b=Gz207wGz3ML12GGfUQaoKsxJ+KE0Z6/6kNZyRp/znUzTb4nmn0FyCUkArlxD9B3C80ADw5fk1VK/Pa0p8dc5Pirc9pSMRAsTym+IHu3lgeYii6JlOnOIRJZwUoa9pT3QL2u0t0fcg1hXBkmxKWxhxdLNCQuk4ZDfbkpFGhAQAdg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746283171; c=relaxed/simple; bh=oZ79jSZRMT8Kkoom2xxvOFIs5yo4Zgkj4Xkn+LlOXxY=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=OsrfslGUiCxZCVVHiU/Bk5AVdJpJIWatCGnHGfan9O0CfKHN3V3DD2EQOy4nh8HJ3QWtm0/dX1lQMh6cJx7rYzDxBpY/+VTgpgQ1SlOzjM7dNdAqNR7xwm1baGbOHMt3z83tf+D5bJ1a9mMFXuC3qUrlNbidAJCmWE1eo3yxjW4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=e1E59XvA; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="e1E59XvA" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1746283168; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=faQPsERN80yWW4k1T0Fa5x0gtfy3HCm8A3emjTgbGw4=; b=e1E59XvALKlzlj+HpRVYzN/XploexUgLoFOYaHrTwpSPAZSmPsqK2cXAVwEVKPZITSMv7i zloM2thAM3UadfB3N1H80tY0lZuYtw0zKDFMJDDN9FeWolBfZsIZZMaLdtqkZmNhVUCafi jAEERvkalqY1e5cGI/c5iACKFwz6+Yw= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-529-y7McgesyMCCWTjWUutGUBg-1; Sat, 03 May 2025 10:39:25 -0400 X-MC-Unique: y7McgesyMCCWTjWUutGUBg-1 X-Mimecast-MFC-AGG-ID: y7McgesyMCCWTjWUutGUBg_1746283164 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2EFD0195608E; Sat, 3 May 2025 14:39:23 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.44.32.23]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id D05E0180036E; Sat, 3 May 2025 14:39:18 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Sat, 3 May 2025 16:38:42 +0200 (CEST) Date: Sat, 3 May 2025 16:38:37 +0200 From: Oleg Nesterov To: Ingo Molnar Cc: linux-kernel@vger.kernel.org, Andy Lutomirski , Dave Hansen , Brian Gerst , Peter Zijlstra , Borislav Petkov , "H . Peter Anvin" , Linus Torvalds , Thomas Gleixner , "Chang S . Bae" Subject: [PATCH tip/x86/fpu 2/6] x86/fpu: kill x86_init_fpu Message-ID: <20250503143837.GA8985@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20250409211127.3544993-1-mingo@kernel.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" It is not actually used after the commit 55bc30f2e34d ("x86/fpu: Remove the thread::fpu pointer"). Signed-off-by: Oleg Nesterov --- arch/x86/kernel/fpu/init.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/arch/x86/kernel/fpu/init.c b/arch/x86/kernel/fpu/init.c index 2d9b5e677559..6bb3e35c40e2 100644 --- a/arch/x86/kernel/fpu/init.c +++ b/arch/x86/kernel/fpu/init.c @@ -71,13 +71,9 @@ static bool __init fpu__probe_without_cpuid(void) return fsw =3D=3D 0 && (fcw & 0x103f) =3D=3D 0x003f; } =20 -static struct fpu x86_init_fpu __attribute__ ((aligned (64))) __read_mostl= y; - static void __init fpu__init_system_early_generic(void) { - fpstate_reset(&x86_init_fpu); set_thread_flag(TIF_NEED_FPU_LOAD); - x86_init_fpu.last_cpu =3D -1; =20 if (!boot_cpu_has(X86_FEATURE_CPUID) && !test_bit(X86_FEATURE_FPU, (unsigned long *)cpu_caps_cleared)) { --=20 2.25.1.362.g51ebf55 From nobody Sat Feb 7 11:31:01 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 548C1279795 for ; Sat, 3 May 2025 14:39:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746283179; cv=none; b=CvFy/BAA/HJCmvwDjMLKhIsn7Loa8Rig26WdgovSO58pa0wK6c/fbW7yAl59rM44vUPEqsVqNQKD7o51armX3ibPRP/S9fDdA1evvyn86JY3lJo9JjpS2EX5oTZPVo9w7Y2qb4SBnbzWe5eVSgqFO2JolknieZgUN7pK6Y4k/Xo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746283179; c=relaxed/simple; bh=bxlcEp3XLloySYmB0umuSp58rBdcgQI2Y7Ls+Su+gOU=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=ajgjj9gbqhYIXX2U6oKACZcaxT3dC35MhmcuKpl5AE02/M8qHfGCtqqgQaJclW4B3T6L6WT8ohh70jz9FKeHhaJDcPXLyDmgHW0kGfDzijQO/vOrh+B+ufp0w9XJI/+E2PQXMOLUo1b81DxIX+bm6Hj5r01+aTeMH0IZY22a+u0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Rxl3mDxw; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Rxl3mDxw" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1746283177; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=tIHKjKMdn85kvtz0zo6P6CjufT263uyReZrlNc9Pyh0=; b=Rxl3mDxw3LfLoQBHwzC13PiuPFSrkaoqlTe3sD3bpRK5nP9RsVds4EhvuR1+mJ4NhZImo+ dXF2IrHHd9G7DfOH7BQYLBZe8BiiiSs1gY78ubnERHLUsJGzIgNUOL2pKaGKrxW6PIMc3K g0ioU4hJlu0fQfyvS7yvt8BuIOqxcQk= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-605-mnEEVaHQPRW1X478oe6eMg-1; Sat, 03 May 2025 10:39:31 -0400 X-MC-Unique: mnEEVaHQPRW1X478oe6eMg-1 X-Mimecast-MFC-AGG-ID: mnEEVaHQPRW1X478oe6eMg_1746283170 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 827DC19560AB; Sat, 3 May 2025 14:39:29 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.44.32.23]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id 0FEB219560A3; Sat, 3 May 2025 14:39:24 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Sat, 3 May 2025 16:38:49 +0200 (CEST) Date: Sat, 3 May 2025 16:38:43 +0200 From: Oleg Nesterov To: Ingo Molnar Cc: linux-kernel@vger.kernel.org, Andy Lutomirski , Dave Hansen , Brian Gerst , Peter Zijlstra , Borislav Petkov , "H . Peter Anvin" , Linus Torvalds , Thomas Gleixner , "Chang S . Bae" Subject: [PATCH tip/x86/fpu 3/6] x86/fpu: kill DEFINE_EVENT(x86_fpu, x86_fpu_copy_src) Message-ID: <20250503143843.GA8989@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20250409211127.3544993-1-mingo@kernel.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" trace_x86_fpu_copy_src() has no users after the commit 22aafe3bcb67 ("x86/fpu: Remove init_task FPU state dependencies, add debugging warning for PF_KTHREAD tasks"). Signed-off-by: Oleg Nesterov --- arch/x86/include/asm/trace/fpu.h | 5 ----- 1 file changed, 5 deletions(-) diff --git a/arch/x86/include/asm/trace/fpu.h b/arch/x86/include/asm/trace/= fpu.h index 4645a6334063..0454d5e60e5d 100644 --- a/arch/x86/include/asm/trace/fpu.h +++ b/arch/x86/include/asm/trace/fpu.h @@ -74,11 +74,6 @@ DEFINE_EVENT(x86_fpu, x86_fpu_dropped, TP_ARGS(fpu) ); =20 -DEFINE_EVENT(x86_fpu, x86_fpu_copy_src, - TP_PROTO(struct fpu *fpu), - TP_ARGS(fpu) -); - DEFINE_EVENT(x86_fpu, x86_fpu_copy_dst, TP_PROTO(struct fpu *fpu), TP_ARGS(fpu) --=20 2.25.1.362.g51ebf55 From nobody Sat Feb 7 11:31:01 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F18F92147FE for ; Wed, 9 Apr 2025 21:11:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744233109; cv=none; b=snlFYlsx9woUgM4d2O1y/yEq4nMfMpgzx8dOqjt5Ds+J64TB2RanjjdGoeCiNnz+DQGn3ZiqkomVDFEdZTNN2Sm/elsWZZDTdvZQ/VVJTcxFAJ5ou8FnlziwzSU/rNjplTq68mMhll1OiE0R0h5HjDkE/r2lKBmxNyiJbzsmC1Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744233109; c=relaxed/simple; bh=HDFc8XxXpcfoEsZT3dQ6ISqPPlGcS2r3vI+88Ffx9xE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eItlXxz4/Av07rvt/5SLu0A0tO5KncQ+Wa/plxmMuG6GhbAG5iC25XMIV00zMhcjp4O7waAhe8T1aoxu1vTAoxJEX+4QREJPVG94zkU8MhlWNv/qE1yMkhM7tSx1Q8P/GZEEbaM+jgdGZlJhc3lYjRnKnN6Kzv7mz8R7jNQgXbg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Hl5BKyNG; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Hl5BKyNG" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5BA9CC4CEE9; Wed, 9 Apr 2025 21:11:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1744233108; bh=HDFc8XxXpcfoEsZT3dQ6ISqPPlGcS2r3vI+88Ffx9xE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Hl5BKyNGiw9/wcg2KlNeabgbcuA8c5E0PcOX3ce5h/nn08Io+lnRZPrl2l5hCagMg NxkIBt2tSF9nN5O+lK/QG/F6Ii+wYFPBmqwhpjWDKkVAwImKJoOUsyX8Dg7NMQUfIN AiuGxZ7QrM1arLIOYcC86XC55S6zg4RkIVDnAr+hRKoCDtvLZ5Ne8DmWx0Nr7OJ+xq XSd9Lm0cxADg4bjPHDQop36IvtUlpiy4Tewq+Ix9O7+BLcbb4AuD+U6B/kawcMOy9Z YRPiT9ei2TYeDJ+2pyQh+OUS4T810cQK2LQCNrcL9xinhrUd6vpab84Lm9veY5/6Xa EKw6MZFImeG8w== From: Ingo Molnar To: linux-kernel@vger.kernel.org Cc: Andy Lutomirski , Dave Hansen , Brian Gerst , Peter Zijlstra , Borislav Petkov , "H . Peter Anvin" , Linus Torvalds , Oleg Nesterov , Thomas Gleixner , "Chang S . Bae" , Ingo Molnar , Andy Lutomirski , Fenghua Yu , Dave Hansen , Uros Bizjak Subject: [PATCH 3/8] x86/fpu: Make task_struct::thread constant size Date: Wed, 9 Apr 2025 23:11:22 +0200 Message-ID: <20250409211127.3544993-4-mingo@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250409211127.3544993-1-mingo@kernel.org> References: <20250409211127.3544993-1-mingo@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Turn thread.fpu into a pointer. Since most FPU code internals work by passi= ng around the FPU pointer already, the code generation impact is small. This allows us to remove the old kludge of task_struct being variable size: struct task_struct { ... /* * New fields for task_struct should be added above here, so that * they are included in the randomized portion of task_struct. */ randomized_struct_fields_end /* CPU-specific state of this task: */ struct thread_struct thread; /* * WARNING: on x86, 'thread_struct' contains a variable-sized * structure. It *MUST* be at the end of 'task_struct'. * * Do not put anything below here! */ }; ... which creates a number of problems, such as requiring thread_struct to = be the last member of the struct - not allowing it to be struct-randomized, et= c. But the primary motivation is to allow the decoupling of task_struct from hardware details ( in particular), and to eventually allow the per-task infrastructure: DECLARE_PER_TASK(type, name); ... per_task(current, name) =3D val; ... which requires task_struct to be a constant size struct. The fpu_thread_struct_whitelist() quirk to hardened usercopy can be removed, now that the FPU structure is not embedded in the task struct anymore, which reduces text footprint a bit. Signed-off-by: Ingo Molnar Fixed-by: Oleg Nesterov Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Fenghua Yu Cc: H. Peter Anvin Cc: Linus Torvalds Cc: Dave Hansen Cc: Thomas Gleixner Cc: Uros Bizjak Link: https://lore.kernel.org/r/20240605083557.2051480-2-mingo@kernel.org --- arch/x86/include/asm/processor.h | 20 +++++++++----------- arch/x86/kernel/fpu/core.c | 23 ++++++++++++----------- arch/x86/kernel/fpu/init.c | 17 ++++++++++------- arch/x86/kernel/process.c | 2 +- include/linux/sched.h | 15 ++++----------- 5 files changed, 36 insertions(+), 41 deletions(-) diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/proces= sor.h index 2f631e0adea3..5ea7e5d2c4de 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -516,21 +516,19 @@ struct thread_struct { #endif =20 /* Floating point and extended processor state */ - struct fpu fpu; - /* - * WARNING: 'fpu' is dynamically-sized. It *MUST* be at - * the end. - */ + struct fpu *fpu; }; =20 -#define x86_task_fpu(task) (&(task)->thread.fpu) - -extern void fpu_thread_struct_whitelist(unsigned long *offset, unsigned lo= ng *size); +#define x86_task_fpu(task) ((task)->thread.fpu) =20 -static inline void arch_thread_struct_whitelist(unsigned long *offset, - unsigned long *size) +/* + * X86 doesn't need any embedded-FPU-struct quirks: + */ +static inline void +arch_thread_struct_whitelist(unsigned long *offset, unsigned long *size) { - fpu_thread_struct_whitelist(offset, size); + *offset =3D 0; + *size =3D 0; } =20 static inline void diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index dc6d7f93c446..853a738fdf2d 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -593,8 +593,19 @@ static int update_fpu_shstk(struct task_struct *dst, u= nsigned long ssp) int fpu_clone(struct task_struct *dst, unsigned long clone_flags, bool min= imal, unsigned long ssp) { + /* + * We allocate the new FPU structure right after the end of the task stru= ct. + * task allocation size already took this into account. + * + * This is safe because task_struct size is a multiple of cacheline size. + */ struct fpu *src_fpu =3D x86_task_fpu(current); - struct fpu *dst_fpu =3D x86_task_fpu(dst); + struct fpu *dst_fpu =3D (void *)dst + sizeof(*dst); + + BUILD_BUG_ON(sizeof(*dst) % SMP_CACHE_BYTES !=3D 0); + BUG_ON(!src_fpu); + + dst->thread.fpu =3D dst_fpu; =20 /* The new task's FPU state cannot be valid in the hardware. */ dst_fpu->last_cpu =3D -1; @@ -663,16 +674,6 @@ int fpu_clone(struct task_struct *dst, unsigned long c= lone_flags, bool minimal, return 0; } =20 -/* - * Whitelist the FPU register state embedded into task_struct for hardened - * usercopy. - */ -void fpu_thread_struct_whitelist(unsigned long *offset, unsigned long *siz= e) -{ - *offset =3D offsetof(struct thread_struct, fpu.__fpstate.regs); - *size =3D fpu_kernel_cfg.default_size; -} - /* * Drops current FPU state: deactivates the fpregs and * the fpstate. NOTE: it still leaves previous contents diff --git a/arch/x86/kernel/fpu/init.c b/arch/x86/kernel/fpu/init.c index ad5cb2943d37..848ea79886ba 100644 --- a/arch/x86/kernel/fpu/init.c +++ b/arch/x86/kernel/fpu/init.c @@ -71,8 +71,15 @@ static bool __init fpu__probe_without_cpuid(void) return fsw =3D=3D 0 && (fcw & 0x103f) =3D=3D 0x003f; } =20 +static struct fpu x86_init_fpu __attribute__ ((aligned (64))) __read_mostl= y; + static void __init fpu__init_system_early_generic(void) { + fpstate_reset(&x86_init_fpu); + current->thread.fpu =3D &x86_init_fpu; + set_thread_flag(TIF_NEED_FPU_LOAD); + x86_init_fpu.last_cpu =3D -1; + if (!boot_cpu_has(X86_FEATURE_CPUID) && !test_bit(X86_FEATURE_FPU, (unsigned long *)cpu_caps_cleared)) { if (fpu__probe_without_cpuid()) @@ -150,6 +157,8 @@ static void __init fpu__init_task_struct_size(void) { int task_size =3D sizeof(struct task_struct); =20 + task_size +=3D sizeof(struct fpu); + /* * Subtract off the static size of the register state. * It potentially has a bunch of padding. @@ -164,14 +173,9 @@ static void __init fpu__init_task_struct_size(void) =20 /* * We dynamically size 'struct fpu', so we require that - * it be at the end of 'thread_struct' and that - * 'thread_struct' be at the end of 'task_struct'. If - * you hit a compile error here, check the structure to - * see if something got added to the end. + * 'state' be at the end of 'it: */ CHECK_MEMBER_AT_END_OF(struct fpu, __fpstate); - CHECK_MEMBER_AT_END_OF(struct thread_struct, fpu); - CHECK_MEMBER_AT_END_OF(struct task_struct, thread); =20 arch_task_struct_size =3D task_size; } @@ -213,7 +217,6 @@ static void __init fpu__init_system_xstate_size_legacy(= void) */ void __init fpu__init_system(void) { - fpstate_reset(x86_task_fpu(current)); fpu__init_system_early_generic(); =20 /* diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c index 47694e391506..3ce4cce46f3f 100644 --- a/arch/x86/kernel/process.c +++ b/arch/x86/kernel/process.c @@ -103,7 +103,7 @@ int arch_dup_task_struct(struct task_struct *dst, struc= t task_struct *src) dst->thread.vm86 =3D NULL; #endif /* Drop the copied pointer to current's fpstate */ - x86_task_fpu(dst)->fpstate =3D NULL; + dst->thread.fpu =3D NULL; =20 return 0; } diff --git a/include/linux/sched.h b/include/linux/sched.h index f96ac1982893..4ecc0c6b1cb0 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1646,22 +1646,15 @@ struct task_struct { struct user_event_mm *user_event_mm; #endif =20 - /* - * New fields for task_struct should be added above here, so that - * they are included in the randomized portion of task_struct. - */ - randomized_struct_fields_end - /* CPU-specific state of this task: */ struct thread_struct thread; =20 /* - * WARNING: on x86, 'thread_struct' contains a variable-sized - * structure. It *MUST* be at the end of 'task_struct'. - * - * Do not put anything below here! + * New fields for task_struct should be added above here, so that + * they are included in the randomized portion of task_struct. */ -}; + randomized_struct_fields_end +} __attribute__ ((aligned (64))); =20 #define TASK_REPORT_IDLE (TASK_REPORT + 1) #define TASK_REPORT_MAX (TASK_REPORT_IDLE << 1) --=20 2.45.2 From nobody Sat Feb 7 11:31:01 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DFA25219312 for ; Wed, 9 Apr 2025 21:11:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744233113; cv=none; b=MhNTKy1FfuRs6p90n9nOLFenegjBp2s0cTph4mGQUHyWr+akyCYHQRjeGcCfhVjj+fnVfigycFA4gOeiEM4AEYvc9Tl0sMGm+44acWMsp2NcCkY+fI9fl0lNiZK3IG73bDacMb6QG2cIAXHAMs9dEODfcwgMbMk8oML0wpRMSXU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744233113; c=relaxed/simple; bh=gIsSH7NLDAYfvrqsitKKMX80cNwtsfaHUeTr2OobYTQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=I4JSzsG4hK/5llzxAFUFIU+FAs8SpJfCUfQtlxxdBpTBpI+Lyw3bbphBzCvLIJTMX+bwsmoSmzp8JA53e5tQwT0in4ramQnQYbo2qGiiu0d1aypxDamOYq5Lr/kp+jV3jhYVKzysfcw3jfbwQYBYa7H1AMW4CzTQcSSNYFVlKvo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=sjhWI0ad; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="sjhWI0ad" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4F538C4CEE2; Wed, 9 Apr 2025 21:11:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1744233112; bh=gIsSH7NLDAYfvrqsitKKMX80cNwtsfaHUeTr2OobYTQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sjhWI0adDOMsF+5uu3DN2H6nDT0foDq7caTYZCamXxDs/Xm7Slz8dQL/JXL/J0Dsy QX1WHRWFCFONJmL28FEmCJP1azG/nxZsdxveSb+NCxhYSIr0R9LQCV4aUkDSb1mNA0 pFQbdJWonjP3bGouCsoFXMwhAYDHAcAI/SbCBBA2MoJaJRayN8Xfv3JQF7HdOLYm6n tnoVjT5/1C46b7dEThRe4fbSH8Xa+3+emZMLaEW3fMov8E/46WdbCJn1GRVO6fKeXS 0oJcHpjTBFI810nhlCjZV6Jvp2lijrlkqSczIDr2DLQK9arQqX78aPOlQp+KqC9aNt ZHx0aQxRiS5Bg== From: Ingo Molnar To: linux-kernel@vger.kernel.org Cc: Andy Lutomirski , Dave Hansen , Brian Gerst , Peter Zijlstra , Borislav Petkov , "H . Peter Anvin" , Linus Torvalds , Oleg Nesterov , Thomas Gleixner , "Chang S . Bae" , Ingo Molnar , Andy Lutomirski , Fenghua Yu , Dave Hansen , Uros Bizjak Subject: [PATCH 4/8] x86/fpu: Remove the thread::fpu pointer Date: Wed, 9 Apr 2025 23:11:23 +0200 Message-ID: <20250409211127.3544993-5-mingo@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250409211127.3544993-1-mingo@kernel.org> References: <20250409211127.3544993-1-mingo@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" As suggested by Oleg, remove the thread::fpu pointer, as we can calculate it via x86_task_fpu() at compile-time. This improves code generation a bit: kepler:~/tip> size vmlinux.before vmlinux.after text data bss dec hex filename 26475405 10435342 1740804 38651551 24dc69f vmlinux.before 26475339 10959630 1216516 38651485 24dc65d vmlinux.after Suggested-by: Oleg Nesterov Signed-off-by: Ingo Molnar Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Fenghua Yu Cc: H. Peter Anvin Cc: Linus Torvalds Cc: Dave Hansen Cc: Thomas Gleixner Cc: Uros Bizjak Link: https://lore.kernel.org/r/20240605083557.2051480-3-mingo@kernel.org --- arch/x86/include/asm/processor.h | 5 +---- arch/x86/kernel/fpu/core.c | 4 +--- arch/x86/kernel/fpu/init.c | 1 - arch/x86/kernel/process.c | 2 -- arch/x86/kernel/vmlinux.lds.S | 4 ++++ 5 files changed, 6 insertions(+), 10 deletions(-) diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/proces= sor.h index 5ea7e5d2c4de..b7f7c9c83409 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -514,12 +514,9 @@ struct thread_struct { =20 struct thread_shstk shstk; #endif - - /* Floating point and extended processor state */ - struct fpu *fpu; }; =20 -#define x86_task_fpu(task) ((task)->thread.fpu) +#define x86_task_fpu(task) ((struct fpu *)((void *)(task) + sizeof(*(task)= ))) =20 /* * X86 doesn't need any embedded-FPU-struct quirks: diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index 853a738fdf2d..974b276ff0da 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -600,13 +600,11 @@ int fpu_clone(struct task_struct *dst, unsigned long = clone_flags, bool minimal, * This is safe because task_struct size is a multiple of cacheline size. */ struct fpu *src_fpu =3D x86_task_fpu(current); - struct fpu *dst_fpu =3D (void *)dst + sizeof(*dst); + struct fpu *dst_fpu =3D x86_task_fpu(dst); =20 BUILD_BUG_ON(sizeof(*dst) % SMP_CACHE_BYTES !=3D 0); BUG_ON(!src_fpu); =20 - dst->thread.fpu =3D dst_fpu; - /* The new task's FPU state cannot be valid in the hardware. */ dst_fpu->last_cpu =3D -1; =20 diff --git a/arch/x86/kernel/fpu/init.c b/arch/x86/kernel/fpu/init.c index 848ea79886ba..da41a1d2c40f 100644 --- a/arch/x86/kernel/fpu/init.c +++ b/arch/x86/kernel/fpu/init.c @@ -76,7 +76,6 @@ static struct fpu x86_init_fpu __attribute__ ((aligned (6= 4))) __read_mostly; static void __init fpu__init_system_early_generic(void) { fpstate_reset(&x86_init_fpu); - current->thread.fpu =3D &x86_init_fpu; set_thread_flag(TIF_NEED_FPU_LOAD); x86_init_fpu.last_cpu =3D -1; =20 diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c index 3ce4cce46f3f..88868a90459e 100644 --- a/arch/x86/kernel/process.c +++ b/arch/x86/kernel/process.c @@ -102,8 +102,6 @@ int arch_dup_task_struct(struct task_struct *dst, struc= t task_struct *src) #ifdef CONFIG_VM86 dst->thread.vm86 =3D NULL; #endif - /* Drop the copied pointer to current's fpstate */ - dst->thread.fpu =3D NULL; =20 return 0; } diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index ccdc45e5b759..d9ca2d1754da 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -181,6 +181,10 @@ SECTIONS /* equivalent to task_pt_regs(&init_task) */ __top_init_kernel_stack =3D __end_init_stack - TOP_OF_KERNEL_STACK_PADDI= NG - PTREGS_SIZE; =20 + __x86_init_fpu_begin =3D .; + . =3D __x86_init_fpu_begin + 128*PAGE_SIZE; + __x86_init_fpu_end =3D .; + #ifdef CONFIG_X86_32 /* 32 bit has nosave before _edata */ NOSAVE_DATA --=20 2.45.2 From nobody Sat Feb 7 11:31:01 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DBB1A2798E6 for ; Sat, 3 May 2025 14:39:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746283183; cv=none; b=h7aUGlGZGwdSBtC3xL3Pkmc2D0k5Kr0ipfg0kl9zVoAVOJ/26H1MX1zcYYEgGtI7jbk3aYyLduduS4SKhHogXRI/mebpwGd4kBsqyIUK4fnMH/98xR97CAX4e8SmiMRhr/5reY5me53dQcTI/LW3w3fcLRlX0TXYBvxL7SrewcE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746283183; c=relaxed/simple; bh=Y1Ih+B7V3riu0tf9b1r2d+GpHBiYz/f+IiY2sNEbobg=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=hjmtYWuRl60OhkSUJ48vI2IJMfGysPBGPzexS5bh8ld5od7RDBcI/rJsKwtnVUvGdGvFNpc7oFBX14a0OlX0o/MG0qzNHGOU7rfHL3Y3hoIl2bqguDf8P+pcsQsFiqvMMuiJ9/JM1dOIqHZeeeWZiIOL8/iB0Xys8QoIgyApmD0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=FXMVcxfQ; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="FXMVcxfQ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1746283180; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=j0kJYhfu8vF1mjLFoB0t+CIUCAM73kbVIq1ExPK13dQ=; b=FXMVcxfQDcUuirZJS9fYvlQazcsCvmS5hMbKA+TpKhA9JOhfvaJKn4NDWUf10ToBBnrLXg y+JgNJ4yAmo3IP+bTq3rmlpcEGqPYeT2QyryhZkJVFL6/OtXiDWZs9rWEFEpxBjy1BjLpr 1YB8kLEmiW/oYke5OpebrnZnI0DUdyg= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-687-yQZtQixpOzmbSMAQrDQVxA-1; Sat, 03 May 2025 10:39:37 -0400 X-MC-Unique: yQZtQixpOzmbSMAQrDQVxA-1 X-Mimecast-MFC-AGG-ID: yQZtQixpOzmbSMAQrDQVxA_1746283176 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9BB93180056F; Sat, 3 May 2025 14:39:35 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.44.32.23]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id 53F9D19560A3; Sat, 3 May 2025 14:39:31 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Sat, 3 May 2025 16:38:55 +0200 (CEST) Date: Sat, 3 May 2025 16:38:50 +0200 From: Oleg Nesterov To: Ingo Molnar Cc: linux-kernel@vger.kernel.org, Andy Lutomirski , Dave Hansen , Brian Gerst , Peter Zijlstra , Borislav Petkov , "H . Peter Anvin" , Linus Torvalds , Thomas Gleixner , "Chang S . Bae" Subject: [PATCH tip/x86/fpu 4/6] x86/fpu: arch_dup_task_struct: always use memcpy_and_pad() Message-ID: <20250503143850.GA8997@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20250409211127.3544993-1-mingo@kernel.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" It makes no sense to copy the bytes after sizeof(struct task_struct), FPU state will be initialized in fpu_clone(). A plain memcpy(dst, src, sizeof(struct task_struct)) should work too, but "_and_pad" looks more safe. Signed-off-by: Oleg Nesterov --- arch/x86/kernel/process.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c index 7a1bfb61d86f..d01adc028274 100644 --- a/arch/x86/kernel/process.c +++ b/arch/x86/kernel/process.c @@ -93,11 +93,9 @@ EXPORT_PER_CPU_SYMBOL_GPL(__tss_limit_invalid); */ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) { - /* init_task is not dynamically sized (incomplete FPU state) */ - if (unlikely(src =3D=3D &init_task)) - memcpy_and_pad(dst, arch_task_struct_size, src, sizeof(init_task), 0); - else - memcpy(dst, src, arch_task_struct_size); + /* fpu_clone() will initialize the "dst_fpu" memory */ + memcpy_and_pad(dst, arch_task_struct_size, src, + sizeof(struct task_struct), 0); =20 #ifdef CONFIG_VM86 dst->thread.vm86 =3D NULL; --=20 2.25.1.362.g51ebf55 From nobody Sat Feb 7 11:31:01 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8CD9268FF4 for ; Sat, 3 May 2025 14:39:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746283189; cv=none; b=ZNfL36+OlCMYfu0oErmwxdTjTLqf9ZxEyMwqza47dAy6MIliz7X8rreF+opw6/3iEuBMNZyOEDHwqnzfUKKgEyUqGhdosS1dB75M7Dn6yRwK8RFl6Yovkf4TT9Ed2DUHHPSovQIzdQobOYNNnt/sY9yev4OMT2FJ9eQCvhJImSQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746283189; c=relaxed/simple; bh=X2H9vXmKjO+gJWdzLddERjKxcXf8lPOxIeoMy8671J0=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=Dqoi/e6fE+57uBcJ9sGT+HA+VcP0J8/zTYa0CGT12oAmAC1+8rDrcIQWyr+wpIfjCVHNc9fUHhKtwVhYOwze0pRXji6VjDPbeelaYi6XjL610Ug6pjXxDA8bW5gRVUYJ99d0DkdvF4b2uGIST5QCNzZYDdEMvF9LCiUbHOp56FU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Ibhq5zxa; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Ibhq5zxa" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1746283185; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=+DmxfC8rGy/UAYF1Q6xSBguY0xVPIRYkk0B3H71yht8=; b=Ibhq5zxaMsbKX0ISG3LaiNhAAScZOvIo2MP8fssRgXgbpvPL7Icg+M66pfTAxTUAXmxIi+ jGj/fEPSXzOlMFaozgQqEzVIY55uUHtl2nVAqAfKQJcQabaqsR8SDxLOmJXUX35X+Q790l CDILmHJYLE14vd1qT6zooweCkrcr64U= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-43-Pck23KyaPW-2geQtqQnEvQ-1; Sat, 03 May 2025 10:39:43 -0400 X-MC-Unique: Pck23KyaPW-2geQtqQnEvQ-1 X-Mimecast-MFC-AGG-ID: Pck23KyaPW-2geQtqQnEvQ_1746283182 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C4A4219560AE; Sat, 3 May 2025 14:39:41 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.44.32.23]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id BA51C180036E; Sat, 3 May 2025 14:39:37 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Sat, 3 May 2025 16:39:01 +0200 (CEST) Date: Sat, 3 May 2025 16:38:56 +0200 From: Oleg Nesterov To: Ingo Molnar Cc: linux-kernel@vger.kernel.org, Andy Lutomirski , Dave Hansen , Brian Gerst , Peter Zijlstra , Borislav Petkov , "H . Peter Anvin" , Linus Torvalds , Thomas Gleixner , "Chang S . Bae" Subject: [PATCH tip/x86/fpu 5/6] x86/fpu: fpu__drop: check TIF_NEED_FPU_LOAD instead of PF_KTHREAD|PF_USER_WORKER Message-ID: <20250503143856.GA9009@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20250409211127.3544993-1-mingo@kernel.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" PF_KTHREAD|PF_USER_WORKER tasks should never clear TIF_NEED_FPU_LOAD, so the TIF_NEED_FPU_LOAD check should equally filter them out. And this way an exiting userspace task can avoid the unnecessary "fwait" if it does context_switch() at least once on its way to exit_thread(). Signed-off-by: Oleg Nesterov --- arch/x86/kernel/fpu/core.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index 8d674435f173..fa131299c7da 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -693,8 +693,7 @@ void fpu__drop(struct task_struct *tsk) { struct fpu *fpu; =20 - /* PF_KTHREAD tasks do not use the FPU context area: */ - if (tsk->flags & (PF_KTHREAD | PF_USER_WORKER)) + if (test_tsk_thread_flag(tsk, TIF_NEED_FPU_LOAD)) return; =20 fpu =3D x86_task_fpu(tsk); --=20 2.25.1.362.g51ebf55 From nobody Sat Feb 7 11:31:01 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 62D91219312 for ; Wed, 9 Apr 2025 21:11:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744233116; cv=none; b=t6YSdZXHLZ5t4QpKrQdsfzme7z4e9h8yzYPygbf0DlVwKpjDipQpscQxU/U4Cs4Y/VkiciLhHQ6byf+CAUk2p+11zYwPpfO+D1BY96k88EsJoPl12TFyVih7ahRYT1Xk8t3c0fsrk+IlVXQvnAAAVMoDRNyNHBwcByRFHEhPm74= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744233116; c=relaxed/simple; bh=mAxo/zY5+9TlvBWUkZPn3jnHud65Iaij3MTL0+7tdVU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EB6zBOQ/4g7+HVshAHWnOEx63W7ht1BR6DgPoTUl0SORjh+M/pncpEt8wmX2GYn2A8MsbturT0wuUgR+mGm7lawD61QM79VoN8HzplS/j2RsaDrGmhM1vBoU32DeUl5kYhf9VDik7vz2MLVO6surSvP9hqpxLG71pA+Re7Gm6gY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=dMqsDVd8; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="dMqsDVd8" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 42607C4CEE2; Wed, 9 Apr 2025 21:11:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1744233115; bh=mAxo/zY5+9TlvBWUkZPn3jnHud65Iaij3MTL0+7tdVU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dMqsDVd8Na/c/R2uoTk89VcUlyaE4cbo/DpS76fwE/C0je+BJYnNnX21p0VWAKWQO u5kcoxK7a9P3hy3b/YQysmftZDSkNYl+Ogux/kfiCXFAzzNk98y8Ame6h7umnVkaUM wC8fQMPOq8D/FyGWKEmwmF4iDanrVgjRqIy5bH3W0CsoZPqaYl5FNYPeBxK40WiUMW QbQdfeqL8OYcKuzEOI6sClTFxkqNBuBaT2mAkO5/4w6orIVpJ0xcEJwZd34cGeGXAc 99mOJ3XTy2X5BgATUHKkrfLcD4pMRNkpMN26NOo3MVX4tsTU7s1oq6IrrdxbrxJGuJ 4moCnAWTDITlw== From: Ingo Molnar To: linux-kernel@vger.kernel.org Cc: Andy Lutomirski , Dave Hansen , Brian Gerst , Peter Zijlstra , Borislav Petkov , "H . Peter Anvin" , Linus Torvalds , Oleg Nesterov , Thomas Gleixner , "Chang S . Bae" , Ingo Molnar Subject: [PATCH 5/8] x86/fpu: Push 'fpu' pointer calculation into the fpu__drop() call Date: Wed, 9 Apr 2025 23:11:24 +0200 Message-ID: <20250409211127.3544993-6-mingo@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250409211127.3544993-1-mingo@kernel.org> References: <20250409211127.3544993-1-mingo@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This encapsulates the fpu__drop() functionality better, and it will also enable other changes that want to check a task for PF_KTHREAD before calling x86_task_fpu(). Signed-off-by: Ingo Molnar --- arch/x86/include/asm/fpu/sched.h | 2 +- arch/x86/kernel/fpu/core.c | 4 +++- arch/x86/kernel/process.c | 3 +-- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/fpu/sched.h b/arch/x86/include/asm/fpu/sc= hed.h index 1feaa68b7567..5fd12634bcc4 100644 --- a/arch/x86/include/asm/fpu/sched.h +++ b/arch/x86/include/asm/fpu/sched.h @@ -10,7 +10,7 @@ #include =20 extern void save_fpregs_to_fpstate(struct fpu *fpu); -extern void fpu__drop(struct fpu *fpu); +extern void fpu__drop(struct task_struct *tsk); extern int fpu_clone(struct task_struct *dst, unsigned long clone_flags, = bool minimal, unsigned long shstk_addr); extern void fpu_flush_thread(void); diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index 974b276ff0da..e4c20908ee49 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -681,8 +681,10 @@ int fpu_clone(struct task_struct *dst, unsigned long c= lone_flags, bool minimal, * a state-restore is coming: either an explicit one, * or a reschedule. */ -void fpu__drop(struct fpu *fpu) +void fpu__drop(struct task_struct *tsk) { + struct fpu *fpu =3D x86_task_fpu(tsk); + preempt_disable(); =20 if (fpu =3D=3D x86_task_fpu(current)) { diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c index 88868a90459e..5fb502c97b08 100644 --- a/arch/x86/kernel/process.c +++ b/arch/x86/kernel/process.c @@ -120,7 +120,6 @@ void arch_release_task_struct(struct task_struct *tsk) void exit_thread(struct task_struct *tsk) { struct thread_struct *t =3D &tsk->thread; - struct fpu *fpu =3D x86_task_fpu(tsk); =20 if (test_thread_flag(TIF_IO_BITMAP)) io_bitmap_exit(tsk); @@ -128,7 +127,7 @@ void exit_thread(struct task_struct *tsk) free_vm86(t); =20 shstk_free(tsk); - fpu__drop(fpu); + fpu__drop(tsk); } =20 static int set_new_tls(struct task_struct *p, unsigned long tls) --=20 2.45.2 From nobody Sat Feb 7 11:31:01 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F831269D17 for ; Sat, 3 May 2025 14:39:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746283198; cv=none; b=qg2Xdb47Nz3l7+JRMNg40CrBRL+VLOlg4u1007KZpi8z5ATV1Tkowu2FFleYrtoj9WVBB4rVPbn94evOFGmxaQhp0aLC8wag4rukCwrbkq4pqdYStv01W/5b0ooW/qivod5n6Q9iBVMKU3CnxCdr7rzJurYrZTA6EzFwFhsqEwQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746283198; c=relaxed/simple; bh=b5JPJapqkRDIx21L118l9yq8gn3tc0L4YKehzhlqPqc=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=WBRNQn/syPSwpHfDPrhwuyOkQ9lcYdS3kVgHoXXtzheabQVGKvpPMo9O5j8IVl9QNh/vxttRfosWrSle064EsrlKAShHdr2VMGmFc0HW727hNZOJe+3elXYdWm2+pVHDcvKmz9vcvvp7vkz5GDUTq0BVJQ3gfifOiTfrqXdi2jg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=IGf6FCHs; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="IGf6FCHs" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1746283195; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=w53aMIvhRJSZKfkmYEz9W9UqbW6DfFdUeZrPEozRMig=; b=IGf6FCHs2oJ7NWnpi8fiL/zdDHkIKwip7yy2TNnHVCQPC39+yl940+LWJpckhh2Y3t3EPg q301duS24jOuGmX7m4lMQV0bEyNjYYbkqCody+QvKbi405e6kGcJIuTW0aNVActITmxAda q+d3IsKsZQkKXoG2/ocx07mDBMErDoo= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-511-2l0EyNRwNU-SxHCLzmEEgQ-1; Sat, 03 May 2025 10:39:49 -0400 X-MC-Unique: 2l0EyNRwNU-SxHCLzmEEgQ-1 X-Mimecast-MFC-AGG-ID: 2l0EyNRwNU-SxHCLzmEEgQ_1746283188 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0797A18001E0; Sat, 3 May 2025 14:39:48 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.44.32.23]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id C4FD41955F1B; Sat, 3 May 2025 14:39:43 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Sat, 3 May 2025 16:39:07 +0200 (CEST) Date: Sat, 3 May 2025 16:39:02 +0200 From: Oleg Nesterov To: Ingo Molnar Cc: linux-kernel@vger.kernel.org, Andy Lutomirski , Dave Hansen , Brian Gerst , Peter Zijlstra , Borislav Petkov , "H . Peter Anvin" , Linus Torvalds , Thomas Gleixner , "Chang S . Bae" Subject: [PATCH tip/x86/fpu 6/6] x86/fpu: shift fpregs_assert_state_consistent() from arch_exit_work() to its caller Message-ID: <20250503143902.GA9012@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20250409211127.3544993-1-mingo@kernel.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" If CONFIG_X86_DEBUG_FPU=3DY, arch_exit_to_user_mode_prepare() calls arch_exit_work() even if ti_work =3D=3D 0. There only reason is that we want to call fpregs_assert_state_consistent() if TIF_NEED_FPU_LOAD is not set. This looks confusing. arch_exit_to_user_mode_prepare() can just call fpregs_assert_state_consistent() unconditionally, it depends on CONFIG_X86_DEBUG_FPU and checks TIF_NEED_FPU_LOAD itself. Signed-off-by: Oleg Nesterov --- arch/x86/include/asm/entry-common.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/entry-common.h b/arch/x86/include/asm/ent= ry-common.h index 77d20555e04d..d535a97c7284 100644 --- a/arch/x86/include/asm/entry-common.h +++ b/arch/x86/include/asm/entry-common.h @@ -53,7 +53,6 @@ static inline void arch_exit_work(unsigned long ti_work) if (unlikely(ti_work & _TIF_IO_BITMAP)) tss_update_io_bitmap(); =20 - fpregs_assert_state_consistent(); if (unlikely(ti_work & _TIF_NEED_FPU_LOAD)) switch_fpu_return(); } @@ -61,7 +60,9 @@ static inline void arch_exit_work(unsigned long ti_work) static inline void arch_exit_to_user_mode_prepare(struct pt_regs *regs, unsigned long ti_work) { - if (IS_ENABLED(CONFIG_X86_DEBUG_FPU) || unlikely(ti_work)) + fpregs_assert_state_consistent(); + + if (unlikely(ti_work)) arch_exit_work(ti_work); =20 fred_update_rsp0(); --=20 2.25.1.362.g51ebf55 From nobody Sat Feb 7 11:31:01 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D6B921ABAB for ; Wed, 9 Apr 2025 21:11:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744233119; cv=none; b=Lo0WtCKX8xGp3MW3Xm4XZWalTGS7YFT+VrWEr9hSyI/KfoJ8cIbfOMalX1FdTmHErX8claagPuGP4qXLXLHHDr0UG8/t53BvTNw6pcOC2CRXFQb5IaJGxgwdBHf+S6yWxlJq/JDx9H7UpI4g2YPxcqZP7v+XjM3pPj6Vm+yxw6g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744233119; c=relaxed/simple; bh=GJ6oVkxqkplRcYkmLa6I7Q6iusu/lTsYw+WCUreyhGg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LDR0bHtQVAWsWQSj8KpmAL6qoj/zPq69Gd73dOCppPYsaoOIfuqcz2tzm3WgtanWfj0pQuXtCfPMBJKrVpGpZiTzzAOuNvlqVs/IotTbT1CU3PgCtAt1zY+NNyxK8QPtbH5/sUIC7lhER9ZtV8aXcvTGR0wb4TvJgfSycHSE0iI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Ff4nhkYq; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ff4nhkYq" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 67DD6C4CEE8; Wed, 9 Apr 2025 21:11:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1744233119; bh=GJ6oVkxqkplRcYkmLa6I7Q6iusu/lTsYw+WCUreyhGg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ff4nhkYq8MVOoI99+KA9vkpXZZIZnHRoC7/DDBRj8heysnRkNqcwpBt+X2A+CLGrj nUbzhjLMAquYUNrpqHsPGjarGqMo4fAFtsf8EUDZsr/4BPTz2Bhj/+5KnwVYhQqGaB SRFDOsvq/f3FX68CyZbEbcQkTmTqJdosASXIC5A4E3T8MJGez6SQEd7ZlXkJHcIuGH Bqx4ALqQelBuYzmvKshexxI61UHX9aW0ZREheEU6/u6cgz95fbEQ1iTLrFsSnZXBUX bcmdoWJweZ49z77gVdPa6hNZYAz7RDbG6gFuF+wUngkRRBX5XswaFSKREHqaiFsSLV gDBmesjqCqDnA== From: Ingo Molnar To: linux-kernel@vger.kernel.org Cc: Andy Lutomirski , Dave Hansen , Brian Gerst , Peter Zijlstra , Borislav Petkov , "H . Peter Anvin" , Linus Torvalds , Oleg Nesterov , Thomas Gleixner , "Chang S . Bae" , Ingo Molnar Subject: [PATCH 6/8] x86/fpu: Make sure x86_task_fpu() doesn't get called for PF_KTHREAD|PF_USER_WORKER tasks during exit Date: Wed, 9 Apr 2025 23:11:25 +0200 Message-ID: <20250409211127.3544993-7-mingo@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250409211127.3544993-1-mingo@kernel.org> References: <20250409211127.3544993-1-mingo@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" fpu__drop() and arch_release_task_struct() calls x86_task_fpu() unconditionally, while the FPU context area will not be present if it's the init task, and should not be in use when it's some other type of kthread. Return early for PF_KTHREAD or PF_USER_WORKER tasks. The debug warning in x86_task_fpu() will catch any kthreads attempting to use the FPU save area. Signed-off-by: Ingo Molnar --- arch/x86/kernel/fpu/core.c | 8 +++++++- arch/x86/kernel/process.c | 2 +- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index e4c20908ee49..4a2193892e5d 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -683,7 +683,13 @@ int fpu_clone(struct task_struct *dst, unsigned long c= lone_flags, bool minimal, */ void fpu__drop(struct task_struct *tsk) { - struct fpu *fpu =3D x86_task_fpu(tsk); + struct fpu *fpu; + + /* PF_KTHREAD tasks do not use the FPU context area: */ + if (tsk->flags & (PF_KTHREAD | PF_USER_WORKER)) + return; + + fpu =3D x86_task_fpu(tsk); =20 preempt_disable(); =20 diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c index 5fb502c97b08..de9df37c5d67 100644 --- a/arch/x86/kernel/process.c +++ b/arch/x86/kernel/process.c @@ -109,7 +109,7 @@ int arch_dup_task_struct(struct task_struct *dst, struc= t task_struct *src) #ifdef CONFIG_X86_64 void arch_release_task_struct(struct task_struct *tsk) { - if (fpu_state_size_dynamic()) + if (fpu_state_size_dynamic() && !(current->flags & (PF_KTHREAD | PF_USER_= WORKER))) fpstate_free(x86_task_fpu(tsk)); } #endif --=20 2.45.2 From nobody Sat Feb 7 11:31:01 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E3C92144B4 for ; Wed, 9 Apr 2025 21:12:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744233123; cv=none; b=C3paliFKD7b3l8rxPmeoOZvm6O0E4lQM82JHl1i2mlmkJqJfw1/WzPeC5jkV8x0JJNsTKcHgyDoFNPCVCrluxY9SJod9Snmy1mEPAL1I2810QgJF97df+vUJE06HwWhyE6ZUvtMiV4PY/WpPR2zz+u6mIkYLMSW+0rTaJEyZK4Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744233123; c=relaxed/simple; bh=+zdko4ltFn0osG/aK0C6R4EHXLADPKkqIWLi0Kot1ow=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=L27Sn5zDSYxx4osZqf3mBtmOTCbh6CZkpwYwzlYZJK6iM0+FuAo/gvG9YZaQO21SGQZ2SktuV9g2BQjGALfQfy4DtwXp32HbPKPr2WLO3fq03z076VtaV1HTY9HA7cSJzEEwAF27vp5hF7BN0M4Qm9aAoggVoIeADwFun1cadcw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=qqbDe4bv; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="qqbDe4bv" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 909B4C4CEEB; Wed, 9 Apr 2025 21:11:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1744233123; bh=+zdko4ltFn0osG/aK0C6R4EHXLADPKkqIWLi0Kot1ow=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qqbDe4bv38gERWdXxkT0tiD3thc/eflafvO6WizpzYbOl4O4UsVczsFPvaLtdmJT+ 80uyFckHh0MO6+ExXtIRJy3Y3ExGI/PqFBN62q3dufenk74JCWV5kZkjKJraCgTvQz bxi6FqwMjpn5+EZJcgVNr0SDhELNn3l25LeMUxJDPmh40QYd/u+xyMG0htHIse2xen l1m6uHaBRxCCkt/17dJk2K9ZxtpgfTrd0aM81ZLMGOj8c8GzbxqioMAzyAjhncG2mA jJ5Uo0AyeHUV/pAoCWDhRBwT3M78ZAShPPD0Ny/Fo8/IMR60q/eUsP/aa2IOQGbk5a pbqP0gAm6wnrA== From: Ingo Molnar To: linux-kernel@vger.kernel.org Cc: Andy Lutomirski , Dave Hansen , Brian Gerst , Peter Zijlstra , Borislav Petkov , "H . Peter Anvin" , Linus Torvalds , Oleg Nesterov , Thomas Gleixner , "Chang S . Bae" , Ingo Molnar , Andy Lutomirski , Fenghua Yu , Dave Hansen , Uros Bizjak Subject: [PATCH 7/8] x86/fpu: Remove init_task FPU state dependencies, add debugging warning for PF_KTHREAD tasks Date: Wed, 9 Apr 2025 23:11:26 +0200 Message-ID: <20250409211127.3544993-8-mingo@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250409211127.3544993-1-mingo@kernel.org> References: <20250409211127.3544993-1-mingo@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" init_task's FPU state initialization was a bit of a hack: __x86_init_fpu_begin =3D .; . =3D __x86_init_fpu_begin + 128*PAGE_SIZE; __x86_init_fpu_end =3D .; But the init task isn't supposed to be using the FPU context in any case, so remove the hack and add in some debug warnings. As Linus noted in the discussion, the init task (and other PF_KTHREAD tasks) *can* use the FPU via kernel_fpu_begin()/_end(), but they don't need the context area because their FPU use is not preemptible or reentrant, and they don't return to user-space. Signed-off-by: Ingo Molnar Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Fenghua Yu Cc: H. Peter Anvin Cc: Linus Torvalds Cc: Oleg Nesterov Cc: Dave Hansen Cc: Thomas Gleixner Cc: Uros Bizjak Link: https://lore.kernel.org/r/20240605083557.2051480-4-mingo@kernel.org --- arch/x86/include/asm/processor.h | 6 +++++- arch/x86/kernel/fpu/core.c | 15 +++++++++++---- arch/x86/kernel/fpu/init.c | 3 +-- arch/x86/kernel/fpu/xstate.c | 3 --- arch/x86/kernel/vmlinux.lds.S | 4 ---- 5 files changed, 17 insertions(+), 14 deletions(-) diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/proces= sor.h index b7f7c9c83409..eaa7214d6953 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -516,7 +516,11 @@ struct thread_struct { #endif }; =20 -#define x86_task_fpu(task) ((struct fpu *)((void *)(task) + sizeof(*(task)= ))) +#ifdef CONFIG_X86_DEBUG_FPU +extern struct fpu *x86_task_fpu(struct task_struct *task); +#else +# define x86_task_fpu(task) ((struct fpu *)((void *)(task) + sizeof(*(task= )))) +#endif =20 /* * X86 doesn't need any embedded-FPU-struct quirks: diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index 4a2193892e5d..4d1a205b7ce2 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -51,6 +51,16 @@ static DEFINE_PER_CPU(bool, in_kernel_fpu); */ DEFINE_PER_CPU(struct fpu *, fpu_fpregs_owner_ctx); =20 +#ifdef CONFIG_X86_DEBUG_FPU +struct fpu *x86_task_fpu(struct task_struct *task) +{ + if (WARN_ON_ONCE(task->flags & PF_KTHREAD)) + return NULL; + + return (void *)task + sizeof(*task); +} +#endif + /* * Can we use the FPU in kernel mode with the * whole "kernel_fpu_begin/end()" sequence? @@ -599,11 +609,9 @@ int fpu_clone(struct task_struct *dst, unsigned long c= lone_flags, bool minimal, * * This is safe because task_struct size is a multiple of cacheline size. */ - struct fpu *src_fpu =3D x86_task_fpu(current); - struct fpu *dst_fpu =3D x86_task_fpu(dst); + struct fpu *dst_fpu =3D (void *)dst + sizeof(*dst); =20 BUILD_BUG_ON(sizeof(*dst) % SMP_CACHE_BYTES !=3D 0); - BUG_ON(!src_fpu); =20 /* The new task's FPU state cannot be valid in the hardware. */ dst_fpu->last_cpu =3D -1; @@ -666,7 +674,6 @@ int fpu_clone(struct task_struct *dst, unsigned long cl= one_flags, bool minimal, if (update_fpu_shstk(dst, ssp)) return 1; =20 - trace_x86_fpu_copy_src(src_fpu); trace_x86_fpu_copy_dst(dst_fpu); =20 return 0; diff --git a/arch/x86/kernel/fpu/init.c b/arch/x86/kernel/fpu/init.c index da41a1d2c40f..16b6611634c3 100644 --- a/arch/x86/kernel/fpu/init.c +++ b/arch/x86/kernel/fpu/init.c @@ -38,7 +38,7 @@ static void fpu__init_cpu_generic(void) /* Flush out any pending x87 state: */ #ifdef CONFIG_MATH_EMULATION if (!boot_cpu_has(X86_FEATURE_FPU)) - fpstate_init_soft(&x86_task_fpu(current)->fpstate->regs.soft); + ; else #endif asm volatile ("fninit"); @@ -207,7 +207,6 @@ static void __init fpu__init_system_xstate_size_legacy(= void) fpu_kernel_cfg.default_size =3D size; fpu_user_cfg.max_size =3D size; fpu_user_cfg.default_size =3D size; - fpstate_reset(x86_task_fpu(current)); } =20 /* diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c index ab2588b8f17f..00e87afa876d 100644 --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -834,9 +834,6 @@ void __init fpu__init_system_xstate(unsigned int legacy= _size) if (err) goto out_disable; =20 - /* Reset the state for the current task */ - fpstate_reset(x86_task_fpu(current)); - /* * Update info used for ptrace frames; use standard-format size and no * supervisor xstates: diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index d9ca2d1754da..ccdc45e5b759 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -181,10 +181,6 @@ SECTIONS /* equivalent to task_pt_regs(&init_task) */ __top_init_kernel_stack =3D __end_init_stack - TOP_OF_KERNEL_STACK_PADDI= NG - PTREGS_SIZE; =20 - __x86_init_fpu_begin =3D .; - . =3D __x86_init_fpu_begin + 128*PAGE_SIZE; - __x86_init_fpu_end =3D .; - #ifdef CONFIG_X86_32 /* 32 bit has nosave before _edata */ NOSAVE_DATA --=20 2.45.2 From nobody Sat Feb 7 11:31:01 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C16482147E3 for ; Wed, 9 Apr 2025 21:12:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744233127; cv=none; b=iGfaRYD59fXWcDr99C7rzeEnS6AG+AwLQTLjcRWD8M4ES2q4xyYEDK1knYH2EvgKepCT4WATOnSI/FTT7wJZz1GXrwx1nYnxaXe8Fo4zWyEQSdR/n+GIIlGOkslqxRYcWRtG/THJgwRsCLxq0vO3RqBXR2u4HQauaZBNSX0auSI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744233127; c=relaxed/simple; bh=KyxwRkCpc0W3aXEsLcSVXCo0y6WnzNVBnEHVFD/JCvA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Cbs6ArWHrjqrDU0WhxoUrvzE6UVnFVYH0VSizFZzjD+WmLOoFuYKCUV5dAY1bzvtnUNlmhg1PPFt/jT4/tWLn8TN52O9qU/OUQWD3WU3eUiZBt9vO9gymrqUc1+DKVISFZnsFYfHah0IFoXJ0exrk9/4tv/KEvgmIpm+bkQZEnI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=H1KD3/g0; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="H1KD3/g0" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 84ED5C4CEE9; Wed, 9 Apr 2025 21:12:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1744233126; bh=KyxwRkCpc0W3aXEsLcSVXCo0y6WnzNVBnEHVFD/JCvA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=H1KD3/g04K6xRjD6iZt64mse9KSCnND6c2vS995TLgo1wPT4YQxMxWnuFo1ig7NWn G+0MWIkgRiOdwjyDDUdxuvOAtSABF4fhmYwf+LT/z0JS/+AVfSB4kV/yO+ZvJex1AP g81CtR/JiP8c8cPQjrSMoKDwxkfk3Og2RMw3x+XBKsZz8wK1Vqw0V9Vdfq2ZXVi5Kb lUSLmf0ebmhOvCEVnlRGuUj83USJ9Kr5uZIvj6cvq+J3P1rMf0RswndCjssUD014t+ CRiwHXcYHOoTp8ZgDczCVDvjiryvujJTC9evWnpj78bIQqdaLkoUXfGMMQhgYg17zY xwSY1GAgtKErg== From: Ingo Molnar To: linux-kernel@vger.kernel.org Cc: Andy Lutomirski , Dave Hansen , Brian Gerst , Peter Zijlstra , Borislav Petkov , "H . Peter Anvin" , Linus Torvalds , Oleg Nesterov , Thomas Gleixner , "Chang S . Bae" , Ingo Molnar Subject: [PATCH 8/8] x86/fpu: Use 'fpstate' variable names consistently Date: Wed, 9 Apr 2025 23:11:27 +0200 Message-ID: <20250409211127.3544993-9-mingo@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250409211127.3544993-1-mingo@kernel.org> References: <20250409211127.3544993-1-mingo@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" A few uses of 'fps' snuck in, which is rather confusing (to me) as it suggests frames-per-second. ;-) Rename them to the canonical 'fpstate' name. No change in functionality. Signed-off-by: Ingo Molnar --- arch/x86/include/asm/fpu/api.h | 2 +- arch/x86/kernel/fpu/core.c | 14 +++++++------- arch/x86/kernel/fpu/xstate.c | 4 ++-- 3 files changed, 10 insertions(+), 10 deletions(-) diff --git a/arch/x86/include/asm/fpu/api.h b/arch/x86/include/asm/fpu/api.h index f42de5f05e7e..8e6848f55dcd 100644 --- a/arch/x86/include/asm/fpu/api.h +++ b/arch/x86/include/asm/fpu/api.h @@ -136,7 +136,7 @@ static inline void fpstate_free(struct fpu *fpu) { } #endif =20 /* fpstate-related functions which are exported to KVM */ -extern void fpstate_clear_xstate_component(struct fpstate *fps, unsigned i= nt xfeature); +extern void fpstate_clear_xstate_component(struct fpstate *fpstate, unsign= ed int xfeature); =20 extern u64 xstate_get_guest_group_perm(void); =20 diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index 4d1a205b7ce2..d0a45f6492cb 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -273,16 +273,16 @@ EXPORT_SYMBOL_GPL(fpu_alloc_guest_fpstate); =20 void fpu_free_guest_fpstate(struct fpu_guest *gfpu) { - struct fpstate *fps =3D gfpu->fpstate; + struct fpstate *fpstate =3D gfpu->fpstate; =20 - if (!fps) + if (!fpstate) return; =20 - if (WARN_ON_ONCE(!fps->is_valloc || !fps->is_guest || fps->in_use)) + if (WARN_ON_ONCE(!fpstate->is_valloc || !fpstate->is_guest || fpstate->in= _use)) return; =20 gfpu->fpstate =3D NULL; - vfree(fps); + vfree(fpstate); } EXPORT_SYMBOL_GPL(fpu_free_guest_fpstate); =20 @@ -333,12 +333,12 @@ EXPORT_SYMBOL_GPL(fpu_update_guest_xfd); */ void fpu_sync_guest_vmexit_xfd_state(void) { - struct fpstate *fps =3D x86_task_fpu(current)->fpstate; + struct fpstate *fpstate =3D x86_task_fpu(current)->fpstate; =20 lockdep_assert_irqs_disabled(); if (fpu_state_size_dynamic()) { - rdmsrl(MSR_IA32_XFD, fps->xfd); - __this_cpu_write(xfd_state, fps->xfd); + rdmsrl(MSR_IA32_XFD, fpstate->xfd); + __this_cpu_write(xfd_state, fpstate->xfd); } } EXPORT_SYMBOL_GPL(fpu_sync_guest_vmexit_xfd_state); diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c index 00e87afa876d..79ad768647f8 100644 --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -1395,9 +1395,9 @@ void xrstors(struct xregs_state *xstate, u64 mask) } =20 #if IS_ENABLED(CONFIG_KVM) -void fpstate_clear_xstate_component(struct fpstate *fps, unsigned int xfea= ture) +void fpstate_clear_xstate_component(struct fpstate *fpstate, unsigned int = xfeature) { - void *addr =3D get_xsave_addr(&fps->regs.xsave, xfeature); + void *addr =3D get_xsave_addr(&fpstate->regs.xsave, xfeature); =20 if (addr) memset(addr, 0, xstate_sizes[xfeature]); --=20 2.45.2 From nobody Sat Feb 7 11:31:01 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 46D771A0BC5 for ; Tue, 22 Apr 2025 17:01:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745341314; cv=none; b=iwtvzUgSZOC222vOzJ1vZJHbI9bm6s9EPXAKY/vEjX+l6N1EtHalsLkpha0y7smNo/oZ5L0ULfKj12Bq603B5GOwtH37ssIhBRyyAsSAi8KtxHGJ229AqteuIuuPcStmoqHZ1RY0d2O1nVTyO1A7nTgfYdDlF2WrgmM6srOCPv4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745341314; c=relaxed/simple; bh=JO2KtBBUzB9kmwalO8oEasZFihssbjQC46p9OUxDhcM=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=agD+wcyNuQnQYNZwIFycSdWf/h1OBUutsRY+5xbQFzBq/KAiyrPIR70ZSSZb0Pymx9/nce3b7VaflwZjfb+wNud30bQYBdyzfyNJJ/S5cgIrEQVtnSimtRuzjGLX1ruWPVM+AALdxEhrFysZSj/HyQQH1udw+txuh8xrdi5/f2c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Ct1POQ3y; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Ct1POQ3y" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1745341312; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=szHYQ7G5Qg0pCM7SYXZ8+VGj8lOfryQyJTB2Zk1s5kU=; b=Ct1POQ3y8GAyuFTw4oNke+cMO2BpOeR1/MR8kTIyqKgDOQPHIQAtikkVqNMJGLOc+0adQu Ag49QPbx9WCSQcpyEf6O8oXcF63RwupKDqVOr+tfHcHU/2BemVABDc153TjmbHcJCH1cpX OmrvA5HpduElcnHvzLHpqbF5i01FaOs= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-304-ItTHBgBYM0-VyaQqx1ZLAQ-1; Tue, 22 Apr 2025 13:01:47 -0400 X-MC-Unique: ItTHBgBYM0-VyaQqx1ZLAQ-1 X-Mimecast-MFC-AGG-ID: ItTHBgBYM0-VyaQqx1ZLAQ_1745341305 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id CBA7919560A5; Tue, 22 Apr 2025 17:01:43 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.44.32.24]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id F40E2195608D; Tue, 22 Apr 2025 17:01:38 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Tue, 22 Apr 2025 19:01:06 +0200 (CEST) Date: Tue, 22 Apr 2025 19:01:00 +0200 From: Oleg Nesterov To: Ingo Molnar , Linus Torvalds Cc: linux-kernel@vger.kernel.org, Andy Lutomirski , Dave Hansen , Brian Gerst , Peter Zijlstra , Borislav Petkov , "H . Peter Anvin" , Thomas Gleixner , "Chang S . Bae" Subject: question about switch_fpu_prepare/switch_fpu_finish Message-ID: <20250422170059.GB1676@redhat.com> References: <20250409211127.3544993-1-mingo@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20250409211127.3544993-1-mingo@kernel.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" I must have missed something, but I can't understand this logic, it seems unnecessarily complicated today. 1. Now that switch_fpu_finish() doesn't load the FPU state, I think it can be folded into switch_fpu_prepare(). 2. But the main question is that I fail to understand why __switch_to() -> switch_fpu_finish() uses the "next" task to set TIF_NEED_FPU_LOAD. I think that set_tsk_thread_flag(prev_p, TIF_NEED_FPU_LOAD) makes more sense. Just in case, note that fpu_clone() sets TIF_NEED_FPU_LOAD, so we should not worry about the 1st __switch_to(next_p). IOW, can you explain why the (untested) patch below could be wrong? We can even remove the PF_KTHREAD check in switch_fpu_prepare(), kthreads should never clear TIF_NEED_FPU_LOAD... Oleg. --- diff --git a/arch/x86/include/asm/fpu/sched.h b/arch/x86/include/asm/fpu/sc= hed.h index 5fd12634bcc4..cdd60f434289 100644 --- a/arch/x86/include/asm/fpu/sched.h +++ b/arch/x86/include/asm/fpu/sched.h @@ -54,18 +54,10 @@ static inline void switch_fpu_prepare(struct task_struc= t *old, int cpu) */ old_fpu->last_cpu =3D cpu; =20 + set_tsk_thread_flag(old, TIF_NEED_FPU_LOAD); + trace_x86_fpu_regs_deactivated(old_fpu); } } =20 -/* - * Delay loading of the complete FPU state until the return to userland. - * PKRU is handled separately. - */ -static inline void switch_fpu_finish(struct task_struct *new) -{ - if (cpu_feature_enabled(X86_FEATURE_FPU)) - set_tsk_thread_flag(new, TIF_NEED_FPU_LOAD); -} - #endif /* _ASM_X86_FPU_SCHED_H */ diff --git a/arch/x86/kernel/process_32.c b/arch/x86/kernel/process_32.c index 4636ef359973..b398a6ef2923 100644 --- a/arch/x86/kernel/process_32.c +++ b/arch/x86/kernel/process_32.c @@ -208,8 +208,6 @@ __switch_to(struct task_struct *prev_p, struct task_str= uct *next_p) =20 raw_cpu_write(current_task, next_p); =20 - switch_fpu_finish(next_p); - /* Load the Intel cache allocation PQR MSR. */ resctrl_sched_in(next_p); =20 diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index 7196ca7048be..e8262e637ea4 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -671,8 +671,6 @@ __switch_to(struct task_struct *prev_p, struct task_str= uct *next_p) raw_cpu_write(current_task, next_p); raw_cpu_write(cpu_current_top_of_stack, task_top_of_stack(next_p)); =20 - switch_fpu_finish(next_p); - /* Reload sp0. */ update_task_stack(next_p);