From nobody Sun Feb 8 12:31:16 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 939B620299F; Tue, 4 Mar 2025 10:36:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741084598; cv=none; b=YiAGMpETZnuN6MQ7FXzLMY8ZeHQyapRKOnz8hidb10EdsQ3tcdEqQiq92wlHLU2AR6VtqEMlCGxydN5E/QE/3Qlgy1XQwLdSjpsZ8+gbAYqdz/j/fEDbLSaxXuYY6FtEaLdYGoMN96q9bC1/5ZK5hogSPrUsgTKs1a0SRcdxI6Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741084598; c=relaxed/simple; bh=ugxXMVo90BimE6Ly46AJfhbruy3n3N1d5u4CNTBm/4Y=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=hdQIj7YiUNF8qORdW1qFq6hnmjDnO1PD7ltzflsYpJSjYpBU4l9Qin91cYAXr8vBqDsCOWzUEIYuMIj1gEq5fNpogFWRhH++iFeGg7S2APMD4Ql2DsnuQCE8oG0c6AESqz/yWM/tzIk2LKkhgLqyYcaUAzk3cs4fXj+yN7+B/AY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=j/Gj1O0F; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=T+dOIqbK; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="j/Gj1O0F"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="T+dOIqbK" Date: Tue, 04 Mar 2025 10:36:34 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1741084594; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/hH1U7nGIr3VrTZS5is9zSGBGEBq3gbta03iKmBqbzQ=; b=j/Gj1O0FyYokfyEDohShB4KZPblGq4n326FC9AZAQ5wKYhYg2YOPXqaIiEYM2Ts4B52/Bs SBjfI8VZ4tPuCSpPx5OKxEwz76t2L/b5p1dK42KZrQ6XyjMwkquWG0znzGWne7yOstgasc 0GfN4D3bwLFRfO69lvuGq2SgpXBiiDCHQ1LwYTrM0Y/1skuy0rZC7Lr61LbMS5SilVGoOY efdYfa8o8Y4pUG3PB6nOSGthfk/3m5T9uks26UAaXl/nsilJnbfC6mIbDRjnvQAG3cCIbw mXdeqLIb0cav8Heg6Vykk7AC+9rk9x2/Af2HH5gQcvYdWC+dkCOlUT7AVCwW5Q== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1741084594; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/hH1U7nGIr3VrTZS5is9zSGBGEBq3gbta03iKmBqbzQ=; b=T+dOIqbKj3GTD6i12y7B9pHZ680PQtbDYhsoUV+7+iY+ytfDkhjV2XiDDvBL+JwqtUT+47 s8WYbNXGki8FXRCw== From: "tip-bot2 for Brian Gerst" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: x86/core] x86/percpu: Move pcpu_hot to percpu hot section Cc: Brian Gerst , Ingo Molnar , Uros Bizjak , Linus Torvalds , Peter Zijlstra , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20250303165246.2175811-3-brgerst@gmail.com> References: <20250303165246.2175811-3-brgerst@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <174108459425.14745.1297994625067557710.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the x86/core branch of tip: Commit-ID: da682fa70c336042f3fff564c2795b2767556ec3 Gitweb: https://git.kernel.org/tip/da682fa70c336042f3fff564c2795b276= 7556ec3 Author: Brian Gerst AuthorDate: Mon, 03 Mar 2025 11:52:37 -05:00 Committer: Ingo Molnar CommitterDate: Tue, 04 Mar 2025 11:24:28 +01:00 x86/percpu: Move pcpu_hot to percpu hot section Also change the alignment of the percpu hot section: - PERCPU_SECTION(INTERNODE_CACHE_BYTES) + PERCPU_SECTION(L1_CACHE_BYTES) As vSMP will muck with INTERNODE_CACHE_BYTES that invalidates the too-large-section assert we do: ASSERT(__per_cpu_hot_end - __per_cpu_hot_start <=3D 64, "percpu cache hot= section too large") [ mingo: Added INTERNODE_CACHE_BYTES fix & explanation. ] Signed-off-by: Brian Gerst Signed-off-by: Ingo Molnar Acked-by: Uros Bizjak Cc: Linus Torvalds Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20250303165246.2175811-3-brgerst@gmail.com --- arch/x86/include/asm/current.h | 28 +++++++++++----------------- arch/x86/kernel/cpu/common.c | 2 +- arch/x86/kernel/vmlinux.lds.S | 5 ++++- 3 files changed, 16 insertions(+), 19 deletions(-) diff --git a/arch/x86/include/asm/current.h b/arch/x86/include/asm/current.h index bf59538..60bc66e 100644 --- a/arch/x86/include/asm/current.h +++ b/arch/x86/include/asm/current.h @@ -13,32 +13,26 @@ struct task_struct; =20 struct pcpu_hot { - union { - struct { - struct task_struct *current_task; - int preempt_count; - int cpu_number; + struct task_struct *current_task; + int preempt_count; + int cpu_number; #ifdef CONFIG_MITIGATION_CALL_DEPTH_TRACKING - u64 call_depth; + u64 call_depth; #endif - unsigned long top_of_stack; - void *hardirq_stack_ptr; - u16 softirq_pending; + unsigned long top_of_stack; + void *hardirq_stack_ptr; + u16 softirq_pending; #ifdef CONFIG_X86_64 - bool hardirq_stack_inuse; + bool hardirq_stack_inuse; #else - void *softirq_stack_ptr; + void *softirq_stack_ptr; #endif - }; - u8 pad[64]; - }; }; -static_assert(sizeof(struct pcpu_hot) =3D=3D 64); =20 -DECLARE_PER_CPU_ALIGNED(struct pcpu_hot, pcpu_hot); +DECLARE_PER_CPU_CACHE_HOT(struct pcpu_hot, pcpu_hot); =20 /* const-qualified alias to pcpu_hot, aliased by linker. */ -DECLARE_PER_CPU_ALIGNED(const struct pcpu_hot __percpu_seg_override, +DECLARE_PER_CPU_CACHE_HOT(const struct pcpu_hot __percpu_seg_override, const_pcpu_hot); =20 static __always_inline struct task_struct *get_current(void) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 88a6707..f00870b 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -2064,7 +2064,7 @@ static __init int setup_setcpuid(char *arg) } __setup("setcpuid=3D", setup_setcpuid); =20 -DEFINE_PER_CPU_ALIGNED(struct pcpu_hot, pcpu_hot) =3D { +DEFINE_PER_CPU_CACHE_HOT(struct pcpu_hot, pcpu_hot) =3D { .current_task =3D &init_task, .preempt_count =3D INIT_PREEMPT_COUNT, .top_of_stack =3D TOP_OF_INIT_STACK, diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 1769a71..0ef9870 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -187,6 +187,8 @@ SECTIONS =20 PAGE_ALIGNED_DATA(PAGE_SIZE) =20 + CACHE_HOT_DATA(L1_CACHE_BYTES) + CACHELINE_ALIGNED_DATA(L1_CACHE_BYTES) =20 DATA_DATA @@ -327,7 +329,8 @@ SECTIONS EXIT_DATA } =20 - PERCPU_SECTION(INTERNODE_CACHE_BYTES) + PERCPU_SECTION(L1_CACHE_BYTES) + ASSERT(__per_cpu_hot_end - __per_cpu_hot_start <=3D 64, "percpu cache hot= section too large") =20 RUNTIME_CONST_VARIABLES RUNTIME_CONST(ptr, USER_PTR_MAX)