From nobody Fri Dec 19 12:48:22 2025 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D7BC259CB4; Mon, 14 Apr 2025 07:34:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744616093; cv=none; b=rgH2d5rlkal+PC7GDcH77INNbg25KrcpLYXv/qTjy+CwJhAmGgLQddzrWMOfqtPZbg+/yfKGsAjWrNIGIqPGRNqAfUqjgHnfbRo6Yf7ukel9RX3s0taqNa5OH8d58UiiZOcdJ+f5q9/yNeLwWEkHEZCk700SEnFjuxQec5hF8JU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744616093; c=relaxed/simple; bh=Vu4ARyyxKFerMoLOXzMzx5Gc++3E4XOyvaqOX79pXPk=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=eO4cjWLFCbzts9bmsyPJ/upx3gC6gyzWV3VYds3lE6Z8ixV9KzB57rzXdBacQxZzOXry6Q1RkTI/1gKa/RvQ1tgEfYQ/82o2KC9eiiy5nQ08gseiaboH6Z0qNv78HRjXNqWVkS9brRDlwLGD9BRdhJFRSBEm9yV29W57W5XUIyE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=M7DXM+QG; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=t5Jcdpa2; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="M7DXM+QG"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="t5Jcdpa2" Date: Mon, 14 Apr 2025 07:34:48 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1744616089; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jyU+nWU4a7n//NboP9f4k47sjGjb+ctZ+kH6gaEuAD8=; b=M7DXM+QGlU5TwQDzazqn7zK4R5i52lWFtgkvIj/ytnaUXdr3L5deWqE4dhq4Sz3XZ63GfE h81S37RXLvdu5rinhwYKW0c2axSqezC4+NTbkTVBULdKc37i18W7cXx78c15gvyNiguQoW gfn8zLdgJrzWZZsTZpwh2AC2CliaPK8u9JZ5UKtmnJaMOBi0g9uxc0YeLmScIEnhDGhBQx djAUnjF3uknnR+SUIiw/EcoJZujiYR3MQzHvMf7Lh2wopmWFA/aibNIy8AUWhajcb5lrhS se90wDh/dbMDbxUfw2caR31dU2CoYwbgmMojWnTdc4x8VpGYzNHmCva89Aoe/A== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1744616089; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jyU+nWU4a7n//NboP9f4k47sjGjb+ctZ+kH6gaEuAD8=; b=t5Jcdpa20k6LLiZ/e/56ln/spWFb609MLJvhQTF7H+MR3J2TisXqdT4OHIh2Cv8YhzcG4Q N+Zidf7dq3ajvbDA== From: "tip-bot2 for Ingo Molnar" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: x86/merge] x86/fpu: Make task_struct::thread constant size Cc: Ingo Molnar , Andy Lutomirski , Brian Gerst , "Chang S. Bae" , "H. Peter Anvin" , Linus Torvalds , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20250409211127.3544993-4-mingo@kernel.org> References: <20250409211127.3544993-4-mingo@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <174461608898.31282.7663316545402138900.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the x86/merge branch of tip: Commit-ID: cb7ca40a3882360ce87191793449d48df0b29184 Gitweb: https://git.kernel.org/tip/cb7ca40a3882360ce87191793449d48df= 0b29184 Author: Ingo Molnar AuthorDate: Wed, 09 Apr 2025 23:11:22 +02:00 Committer: Ingo Molnar CommitterDate: Mon, 14 Apr 2025 08:18:29 +02:00 x86/fpu: Make task_struct::thread constant size Turn thread.fpu into a pointer. Since most FPU code internals work by passi= ng around the FPU pointer already, the code generation impact is small. This allows us to remove the old kludge of task_struct being variable size: struct task_struct { ... /* * New fields for task_struct should be added above here, so that * they are included in the randomized portion of task_struct. */ randomized_struct_fields_end /* CPU-specific state of this task: */ struct thread_struct thread; /* * WARNING: on x86, 'thread_struct' contains a variable-sized * structure. It *MUST* be at the end of 'task_struct'. * * Do not put anything below here! */ }; ... which creates a number of problems, such as requiring thread_struct to = be the last member of the struct - not allowing it to be struct-randomized, et= c. But the primary motivation is to allow the decoupling of task_struct from hardware details ( in particular), and to eventually allow the per-task infrastructure: DECLARE_PER_TASK(type, name); ... per_task(current, name) =3D val; ... which requires task_struct to be a constant size struct. The fpu_thread_struct_whitelist() quirk to hardened usercopy can be removed, now that the FPU structure is not embedded in the task struct anymore, which reduces text footprint a bit. Fixed-by: Oleg Nesterov Signed-off-by: Ingo Molnar Cc: Andy Lutomirski Cc: Brian Gerst Cc: Chang S. Bae Cc: H. Peter Anvin Cc: Linus Torvalds Link: https://lore.kernel.org/r/20250409211127.3544993-4-mingo@kernel.org --- arch/x86/include/asm/processor.h | 20 +++++++++----------- arch/x86/kernel/fpu/core.c | 23 ++++++++++++----------- arch/x86/kernel/fpu/init.c | 17 ++++++++++------- arch/x86/kernel/process.c | 2 +- include/linux/sched.h | 15 ++++----------- 5 files changed, 36 insertions(+), 41 deletions(-) diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/proces= sor.h index 2f631e0..5ea7e5d 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -516,21 +516,19 @@ struct thread_struct { #endif =20 /* Floating point and extended processor state */ - struct fpu fpu; - /* - * WARNING: 'fpu' is dynamically-sized. It *MUST* be at - * the end. - */ + struct fpu *fpu; }; =20 -#define x86_task_fpu(task) (&(task)->thread.fpu) - -extern void fpu_thread_struct_whitelist(unsigned long *offset, unsigned lo= ng *size); +#define x86_task_fpu(task) ((task)->thread.fpu) =20 -static inline void arch_thread_struct_whitelist(unsigned long *offset, - unsigned long *size) +/* + * X86 doesn't need any embedded-FPU-struct quirks: + */ +static inline void +arch_thread_struct_whitelist(unsigned long *offset, unsigned long *size) { - fpu_thread_struct_whitelist(offset, size); + *offset =3D 0; + *size =3D 0; } =20 static inline void diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index dc6d7f9..853a738 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -593,8 +593,19 @@ static int update_fpu_shstk(struct task_struct *dst, u= nsigned long ssp) int fpu_clone(struct task_struct *dst, unsigned long clone_flags, bool min= imal, unsigned long ssp) { + /* + * We allocate the new FPU structure right after the end of the task stru= ct. + * task allocation size already took this into account. + * + * This is safe because task_struct size is a multiple of cacheline size. + */ struct fpu *src_fpu =3D x86_task_fpu(current); - struct fpu *dst_fpu =3D x86_task_fpu(dst); + struct fpu *dst_fpu =3D (void *)dst + sizeof(*dst); + + BUILD_BUG_ON(sizeof(*dst) % SMP_CACHE_BYTES !=3D 0); + BUG_ON(!src_fpu); + + dst->thread.fpu =3D dst_fpu; =20 /* The new task's FPU state cannot be valid in the hardware. */ dst_fpu->last_cpu =3D -1; @@ -664,16 +675,6 @@ int fpu_clone(struct task_struct *dst, unsigned long c= lone_flags, bool minimal, } =20 /* - * Whitelist the FPU register state embedded into task_struct for hardened - * usercopy. - */ -void fpu_thread_struct_whitelist(unsigned long *offset, unsigned long *siz= e) -{ - *offset =3D offsetof(struct thread_struct, fpu.__fpstate.regs); - *size =3D fpu_kernel_cfg.default_size; -} - -/* * Drops current FPU state: deactivates the fpregs and * the fpstate. NOTE: it still leaves previous contents * in the fpregs in the eager-FPU case. diff --git a/arch/x86/kernel/fpu/init.c b/arch/x86/kernel/fpu/init.c index ad5cb29..848ea79 100644 --- a/arch/x86/kernel/fpu/init.c +++ b/arch/x86/kernel/fpu/init.c @@ -71,8 +71,15 @@ static bool __init fpu__probe_without_cpuid(void) return fsw =3D=3D 0 && (fcw & 0x103f) =3D=3D 0x003f; } =20 +static struct fpu x86_init_fpu __attribute__ ((aligned (64))) __read_mostl= y; + static void __init fpu__init_system_early_generic(void) { + fpstate_reset(&x86_init_fpu); + current->thread.fpu =3D &x86_init_fpu; + set_thread_flag(TIF_NEED_FPU_LOAD); + x86_init_fpu.last_cpu =3D -1; + if (!boot_cpu_has(X86_FEATURE_CPUID) && !test_bit(X86_FEATURE_FPU, (unsigned long *)cpu_caps_cleared)) { if (fpu__probe_without_cpuid()) @@ -150,6 +157,8 @@ static void __init fpu__init_task_struct_size(void) { int task_size =3D sizeof(struct task_struct); =20 + task_size +=3D sizeof(struct fpu); + /* * Subtract off the static size of the register state. * It potentially has a bunch of padding. @@ -164,14 +173,9 @@ static void __init fpu__init_task_struct_size(void) =20 /* * We dynamically size 'struct fpu', so we require that - * it be at the end of 'thread_struct' and that - * 'thread_struct' be at the end of 'task_struct'. If - * you hit a compile error here, check the structure to - * see if something got added to the end. + * 'state' be at the end of 'it: */ CHECK_MEMBER_AT_END_OF(struct fpu, __fpstate); - CHECK_MEMBER_AT_END_OF(struct thread_struct, fpu); - CHECK_MEMBER_AT_END_OF(struct task_struct, thread); =20 arch_task_struct_size =3D task_size; } @@ -213,7 +217,6 @@ static void __init fpu__init_system_xstate_size_legacy(= void) */ void __init fpu__init_system(void) { - fpstate_reset(x86_task_fpu(current)); fpu__init_system_early_generic(); =20 /* diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c index 47694e3..3ce4cce 100644 --- a/arch/x86/kernel/process.c +++ b/arch/x86/kernel/process.c @@ -103,7 +103,7 @@ int arch_dup_task_struct(struct task_struct *dst, struc= t task_struct *src) dst->thread.vm86 =3D NULL; #endif /* Drop the copied pointer to current's fpstate */ - x86_task_fpu(dst)->fpstate =3D NULL; + dst->thread.fpu =3D NULL; =20 return 0; } diff --git a/include/linux/sched.h b/include/linux/sched.h index f96ac19..4ecc0c6 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1646,22 +1646,15 @@ struct task_struct { struct user_event_mm *user_event_mm; #endif =20 - /* - * New fields for task_struct should be added above here, so that - * they are included in the randomized portion of task_struct. - */ - randomized_struct_fields_end - /* CPU-specific state of this task: */ struct thread_struct thread; =20 /* - * WARNING: on x86, 'thread_struct' contains a variable-sized - * structure. It *MUST* be at the end of 'task_struct'. - * - * Do not put anything below here! + * New fields for task_struct should be added above here, so that + * they are included in the randomized portion of task_struct. */ -}; + randomized_struct_fields_end +} __attribute__ ((aligned (64))); =20 #define TASK_REPORT_IDLE (TASK_REPORT + 1) #define TASK_REPORT_MAX (TASK_REPORT_IDLE << 1)