From nobody Fri Jul 24 21:53:26 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3984837EFFC; Thu, 23 Jul 2026 10:32:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784802746; cv=none; b=ATBz9E+KxnrQoADUO/siDbeybcrB6dN37NuaK0iej9uB159WR1qjW7p9lGe4Qun2KIRMd2HNuq7g2q8Yu8B+iqKwsWq+gsTPx6/JqU1Z9Gb5qko/8o85vhiamtBoCHBSkXigE5KO69hl0nHORopRWtcDp/DTWBqs7jrr7m65V70= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784802746; c=relaxed/simple; bh=6SmcNdk2gI0ghzr8K+h1/A/0azzoxZCNZZETa1De+dw=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=HJeGZrmGxn7Zejg21Ne4tOp9XxeZl7m++n6S0p+JW5RTvhcHC2umOEOttGHGLn2uNJWX3HfEMP6+q4XVT+78QPil4etQHtcqzN2TKpuvD7v8aP84WvOTk5oJRy1uZKGUMw6z3FMuL5NzHFWZhaKAAsvodxgvfRdiL42ARhhfgJI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=4Ndlx4C6; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=8jh2BzAk; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="4Ndlx4C6"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="8jh2BzAk" Date: Thu, 23 Jul 2026 10:32:21 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1784802743; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CmmMyeXm4Z/TiBSCjI1EpyC4jgtGCup334A19TfUFf8=; b=4Ndlx4C6G+0wwvB0bUXP0HVUuY4oa4Apx+znsq7YsAuNTqhMq90P1/aZ2F0+c3UE3VJ/ho qxOx7nJ7sDng/6/Ipul2fDWsdwDBCiMCRYExMSJw4tXA95ujrKo1PVs7OpC37IeEWJeNQ3 fgtYlP0lPAv0SxXNlp9ZgHKInbM3nB64D3hc/KS7OGRN3H6OxKwMhvSb0axCTSecJn7NfW 2MAolW3/5hT0qn449v3fuF84aAPX7YUfpYRBJd0+x2R0ULupUFDNblAm/ZQW7DYF1TTPbO 4KwnbIpClrddIYYzmtZWMfnMj+bX/TyUoDtOKnacJOiE5zRDc5ZOf74h6lRFJw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1784802743; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CmmMyeXm4Z/TiBSCjI1EpyC4jgtGCup334A19TfUFf8=; b=8jh2BzAkwj3+ZyfDcvQZFmpA75+1FRyzDKJkzLZJh+43A6bSh0sXFsBk8CMFljE+mEUfYl E/9fWJupZ5dAZICQ== From: "tip-bot2 for Chuyi Zhou" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: x86/mm] x86/mm: Re-enable preemption before flush_tlb_multi() Cc: Chuyi Zhou , Thomas Gleixner , "Paul E. McKenney" , Sebastian Andrzej Siewior , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20260709122933.4021501-15-zhouchuyi@bytedance.com> References: <20260709122933.4021501-15-zhouchuyi@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <178480274158.2943223.9063459835821318530.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the x86/mm branch of tip: Commit-ID: a5a162fe1ae130e3d2ceefef3f43afe3773c1d56 Gitweb: https://git.kernel.org/tip/a5a162fe1ae130e3d2ceefef3f43afe37= 73c1d56 Author: Chuyi Zhou AuthorDate: Thu, 09 Jul 2026 20:29:33 +08:00 Committer: Thomas Gleixner CommitterDate: Thu, 23 Jul 2026 12:30:31 +02:00 x86/mm: Re-enable preemption before flush_tlb_multi() flush_tlb_mm_range() and arch_tlbbatch_flush() pin the current CPU while they decide whether the flush can be handled locally or must be sent to remote CPUs. The CPU pinning is needed for the current CPU number and for the local TLB flush path, which reads per-CPU TLB state. The caller does not need to remain pinned while waiting for a remote TLB flush to complete. After the remote-flush path has been selected, flush_tlb_info is caller-private stack storage, so the caller no longer has to stay on the same CPU to protect a shared per-CPU flush_tlb_info object. flush_tlb_multi() may also route through x86 PV backends. Those backends must protect their own CPU-local scratch state instead of relying on the caller to stay pinned. Hyper-V already does this by disabling interrupts while using hyperv_pcpu_input_arg, and Xen's multicall path brackets its per-CPU multicall buffer with xen_mc_batch() and xen_mc_issue(). kvm_flush_tlb_multi() also disables preemption while using __pv_cpu_mask. Remote TLB flushes may synchronously wait for many CPUs, and the wait can take tens of milliseconds when remote CPUs have interrupts disabled or when many CPUs are involved. Keeping preemption disabled for that whole wait unnecessarily increases scheduling latency on the initiating CPU. Drop the CPU pinning before calling flush_tlb_multi() in the remote paths of flush_tlb_mm_range() and arch_tlbbatch_flush(). Keep the local paths inside the pinned section because they still access this CPU's TLB state. Signed-off-by: Chuyi Zhou Signed-off-by: Thomas Gleixner Tested-by: Paul E. McKenney Reviewed-by: Sebastian Andrzej Siewior Link: https://patch.msgid.link/20260709122933.4021501-15-zhouchuyi@bytedanc= e.com --- arch/x86/mm/tlb.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index b464a73..e5a0c6e 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -1403,6 +1403,7 @@ void flush_tlb_mm_range(struct mm_struct *mm, unsigne= d long start, bool freed_tables) { struct flush_tlb_info info; + bool remote_flush =3D false; int cpu =3D get_cpu(); u64 new_tlb_gen; =20 @@ -1419,9 +1420,7 @@ void flush_tlb_mm_range(struct mm_struct *mm, unsigne= d long start, if (mm_global_asid(mm)) { broadcast_tlb_flush(&info); } else if (cpumask_any_but(mm_cpumask(mm), cpu) < nr_cpu_ids) { - info.trim_cpumask =3D should_trim_cpumask(mm); - flush_tlb_multi(mm_cpumask(mm), &info); - consider_global_asid(mm); + remote_flush =3D true; } else if (mm =3D=3D this_cpu_read(cpu_tlbstate.loaded_mm)) { lockdep_assert_irqs_enabled(); local_irq_disable(); @@ -1430,6 +1429,13 @@ void flush_tlb_mm_range(struct mm_struct *mm, unsign= ed long start, } =20 put_cpu(); + + if (remote_flush) { + info.trim_cpumask =3D should_trim_cpumask(mm); + flush_tlb_multi(mm_cpumask(mm), &info); + consider_global_asid(mm); + } + mmu_notifier_arch_invalidate_secondary_tlbs(mm, start, end); } =20 @@ -1676,7 +1682,7 @@ EXPORT_SYMBOL_FOR_KVM(__flush_tlb_all); void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch) { struct flush_tlb_info info; - + bool remote_flush =3D false; int cpu =3D get_cpu(); =20 init_flush_tlb_info(&info, NULL, 0, TLB_FLUSH_ALL, 0, false, @@ -1690,7 +1696,7 @@ void arch_tlbbatch_flush(struct arch_tlbflush_unmap_b= atch *batch) invlpgb_flush_all_nonglobals(); batch->unmapped_pages =3D false; } else if (cpumask_any_but(&batch->cpumask, cpu) < nr_cpu_ids) { - flush_tlb_multi(&batch->cpumask, &info); + remote_flush =3D true; } else if (cpumask_test_cpu(cpu, &batch->cpumask)) { lockdep_assert_irqs_enabled(); local_irq_disable(); @@ -1698,9 +1704,12 @@ void arch_tlbbatch_flush(struct arch_tlbflush_unmap_= batch *batch) local_irq_enable(); } =20 - cpumask_clear(&batch->cpumask); - put_cpu(); + + if (remote_flush) + flush_tlb_multi(&batch->cpumask, &info); + + cpumask_clear(&batch->cpumask); } =20 /*