From nobody Sat Feb 7 20:44:03 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 15B722046B7 for ; Wed, 2 Apr 2025 09:46:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743587163; cv=none; b=Cg8VoFXYVMGD7z1UKmmhbDLA+NlSHRJ7DOm6Sga1anVUfOUcfUS4Zhe0zePc/sRfDMAXsdGLU2dEpxMRPaC3o9CDpmHsgxLF87m+4itUe553OAPgsFKqJUsjAQ4H8Agi6OdasIn75y7WnFXpMXIvObsvWT9nhF1Q3Odw3SMvGbg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743587163; c=relaxed/simple; bh=98a0dpTp/y6SCUe1WeenUGwrztXboJxBLAkYY1lT0f0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LUtWMeAwDuW+rGnEkaGvBbGuj89FrMzkrOc73GrwcQxELv0+KUczfCXzBkz+nWeJEtp5PljJX7I/35XaOPY34Yt8pLWC2XS1HZv1hrU/gcS7TQtDIEDLvNmL2A4PM1lZpzBs2swwJiOFD6mFtsTo110jRo8q1V0i9nv9K/ukEXg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=JQduRNhN; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="JQduRNhN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 94894C4CEDD; Wed, 2 Apr 2025 09:46:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1743587162; bh=98a0dpTp/y6SCUe1WeenUGwrztXboJxBLAkYY1lT0f0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JQduRNhNUj7TSO2NdvsC9ApY+JLG3ZljOjrzOz1bOaJRpkPZWgxJ4L35uO82AHf+E BIcIR/C/1fWIHNhILGhqO3tdxjT9asLNdVqMYxHN4q6x4WWvS7ArYORBbQUzn8wYyd ERm/VD7OyS7D9L8jmJlx4aYp8c2guaUj31uKiES9RVywwr9EDMqsKk8/t4qATbqAsS nz4g2WbnP7MmW6SSQTUcDMNEXLh9/2rjYh2fW7WnvHlyvH9YdfBAbqENtTaL+x/bRL +TryIrBvMBLtz7tkI4kITvucrc0Y53gptK6rbnz48l2HpyyJ3hxEhT5w4s6fd9EFo9 Fxb1lDVPpyEMA== From: Ingo Molnar To: linux-kernel@vger.kernel.org Cc: Andy Lutomirski , Rik van Riel , "H . Peter Anvin" , Peter Zijlstra , Linus Torvalds , Andrew Morton , Ingo Molnar Subject: [PATCH 1/7] x86/mm: Add 'mm' argument to unuse_temporary_mm() Date: Wed, 2 Apr 2025 11:45:34 +0200 Message-ID: <20250402094540.3586683-2-mingo@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250402094540.3586683-1-mingo@kernel.org> References: <20250402094540.3586683-1-mingo@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra In commit 209954cbc7d0 ("x86/mm/tlb: Update mm_cpumask lazily") unuse_temporary_mm() grew the assumption that it gets used on poking_mm exclusively. While this is currently true, lets not hard code this assumption. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Cc: Andy Lutomirski Cc: Rik van Riel Cc: "H. Peter Anvin" Cc: Linus Torvalds Link: https://lore.kernel.org/r/20241119163035.322525475@infradead.org --- arch/x86/kernel/alternative.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 5b1a6252a4b9..cfffcb80f564 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -2161,14 +2161,14 @@ static inline struct mm_struct *use_temporary_mm(st= ruct mm_struct *temp_mm) __ro_after_init struct mm_struct *text_poke_mm; __ro_after_init unsigned long text_poke_mm_addr; =20 -static inline void unuse_temporary_mm(struct mm_struct *prev_mm) +static inline void unuse_temporary_mm(struct mm_struct *mm, struct mm_stru= ct *prev_mm) { lockdep_assert_irqs_disabled(); =20 switch_mm_irqs_off(NULL, prev_mm, current); =20 /* Clear the cpumask, to indicate no TLB flushing is needed anywhere */ - cpumask_clear_cpu(raw_smp_processor_id(), mm_cpumask(text_poke_mm)); + cpumask_clear_cpu(raw_smp_processor_id(), mm_cpumask(mm)); =20 /* * Restore the breakpoints if they were disabled before the temporary mm @@ -2275,7 +2275,7 @@ static void *__text_poke(text_poke_f func, void *addr= , const void *src, size_t l * instruction that already allows the core to see the updated version. * Xen-PV is assumed to serialize execution in a similar manner. */ - unuse_temporary_mm(prev_mm); + unuse_temporary_mm(text_poke_mm, prev_mm); =20 /* * Flushing the TLB might involve IPIs, which would require enabled --=20 2.45.2 From nobody Sat Feb 7 20:44:03 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2CCC0236A61 for ; Wed, 2 Apr 2025 09:46:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743587165; cv=none; b=jgG+ALaQjG5KR4w4jHG8Z6+2zYhQncLZwOfkzhvvKl8kMSv1SwqwoPD7UFPjd1u0XJyG9BFXYMZpknKA2Ifg1SA5GdaaswpKGWKjcfSHQo2cFEqlXgS3fnvIopeDQImSoisj8ySzVwdTP5/0rK0jNgu4qM8tkil+xlXPqo92Ieg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743587165; c=relaxed/simple; bh=757HnKsVV0oCVcNBFomeoHSBU+IWDVTrWkUMqXu+k1E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MoU7QjSwHgeVUwzDwTmRDasVCp3aZfe86QoN1b+1D/AxNERaYEwjDD1zWknfHdnZiTThnUzxJGUv85t3DTkvG0ggn+UdwFPlSbNb6xGYzGhR4/eFN1Lv/Lzr0iQv+dwhn2yXx99SkupvHu6P0Ag/1T3yxt+cYil7aIItdSqg2vU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=piXOuXaD; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="piXOuXaD" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 15BD7C4CEEA; Wed, 2 Apr 2025 09:46:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1743587165; bh=757HnKsVV0oCVcNBFomeoHSBU+IWDVTrWkUMqXu+k1E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=piXOuXaDYIXtNUuLOnLxvwt3/874ttIDI+lHYvdLCxwFDPGMst7GCIqcVhB6VCcgi fJL5UgSp/G/FmbosSmj7Ca2VbXfkxlrpzbL/p2dalLOanBuJJifsbhR+/THQVF0agw yNJJsJsMx8vtGZEQFqmJxQ+EOiGRo0thEImlWR1CSsbkum+qZqmMJgAOCrEqSqoSa1 jwAvVD+/3ZPmw4EWhCASyc/LOgALV/vUDw9UCoepmhtZG45HRRjiK0GFS1xRxiRm7T 31ktE77eUEKv4A/zis4s5VGDNSB+JnUpRj11M9vmigPPV3YkSNdUvPPXeJZpIOoqMH A9cgZ7fJZbyjg== From: Ingo Molnar To: linux-kernel@vger.kernel.org Cc: Andy Lutomirski , Rik van Riel , "H . Peter Anvin" , Peter Zijlstra , Linus Torvalds , Andrew Morton , Ingo Molnar Subject: [PATCH 2/7] x86/events, x86/insn-eval: Remove incorrect current->active_mm references Date: Wed, 2 Apr 2025 11:45:35 +0200 Message-ID: <20250402094540.3586683-3-mingo@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250402094540.3586683-1-mingo@kernel.org> References: <20250402094540.3586683-1-mingo@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Andy Lutomirski When decoding an instruction or handling a perf event that references an LDT segment, if we don't have a valid user context, trying to access the LDT by any means other than SLDT is racy. Certainly, using current->active_mm is wrong, as active_mm can point to a real user mm when CR3 and LDTR no longer reference that mm. Clean up the code. If nmi_uaccess_okay() says we don't have a valid context, just fail. Otherwise use current->mm. Signed-off-by: Andy Lutomirski Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Cc: Rik van Riel Cc: "H. Peter Anvin" Cc: Linus Torvalds Link: https://lore.kernel.org/r/20241119163035.433533770@infradead.org --- arch/x86/events/core.c | 9 ++++++++- arch/x86/lib/insn-eval.c | 13 ++++++++++--- 2 files changed, 18 insertions(+), 4 deletions(-) diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c index 6866cc5acb0b..95118b52b606 100644 --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -2803,8 +2803,15 @@ static unsigned long get_segment_base(unsigned int s= egment) #ifdef CONFIG_MODIFY_LDT_SYSCALL struct ldt_struct *ldt; =20 + /* + * If we're not in a valid context with a real (not just lazy) + * user mm, then don't even try. + */ + if (!nmi_uaccess_okay()) + return 0; + /* IRQs are off, so this synchronizes with smp_store_release */ - ldt =3D READ_ONCE(current->active_mm->context.ldt); + ldt =3D smp_load_acquire(¤t->mm->context.ldt); if (!ldt || idx >=3D ldt->nr_entries) return 0; =20 diff --git a/arch/x86/lib/insn-eval.c b/arch/x86/lib/insn-eval.c index 98631c0e7a11..f786401ac15d 100644 --- a/arch/x86/lib/insn-eval.c +++ b/arch/x86/lib/insn-eval.c @@ -631,14 +631,21 @@ static bool get_desc(struct desc_struct *out, unsigne= d short sel) /* Bits [15:3] contain the index of the desired entry. */ sel >>=3D 3; =20 - mutex_lock(¤t->active_mm->context.lock); - ldt =3D current->active_mm->context.ldt; + /* + * If we're not in a valid context with a real (not just lazy) + * user mm, then don't even try. + */ + if (!nmi_uaccess_okay()) + return false; + + mutex_lock(¤t->mm->context.lock); + ldt =3D current->mm->context.ldt; if (ldt && sel < ldt->nr_entries) { *out =3D ldt->entries[sel]; success =3D true; } =20 - mutex_unlock(¤t->active_mm->context.lock); + mutex_unlock(¤t->mm->context.lock); =20 return success; } --=20 2.45.2 From nobody Sat Feb 7 20:44:03 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0BA42237179 for ; Wed, 2 Apr 2025 09:46:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743587168; cv=none; b=OsJsufSMZrvhnYHWGq1ZStFs1KrpLrueDYv3c/YY3gKDavTvEdeyQaNaWeuH+EO3776yNzkN8Pec/O8MhDqmNM3qC5MZqbwGy/C0HuNEL8/vMkwFPtrZlLGw9aHD8OCvAf1WlKGoWSssjzmToFfTv3uEOnFpeoovKvkALMbvn/0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743587168; c=relaxed/simple; bh=NW9l22mXyPj/Fc8NtQZqNbq47+xPnMqEPj1Ls6OWb8M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kt+14zHC3yoD+f6LcYsN1d/wKcJZYSjx3z1fPKKbJjdHZTP5YBc4MKKTy09iSfyABVgv3qNkXF1Nu/pxDT4k11GHwSFerjY675RG6jSmmJvcz0Hjc6hUbLbBO9dq68dP61iCNuPoA++nWi3tuy+0n+Jczn6y9PcjvfDuCpnOMxk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Gj35kfmH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Gj35kfmH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 84835C4CEDD; Wed, 2 Apr 2025 09:46:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1743587167; bh=NW9l22mXyPj/Fc8NtQZqNbq47+xPnMqEPj1Ls6OWb8M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Gj35kfmHhqGaNY68yK9mQG3BJ0qiJaHMyIl23+hxL7Wwc2Ju4oTarfz1KYZmD9Vm3 4Isi3orA2NULC5EJ+WK7OStyCo5pdlk6u0GWSvWfgUL4VPOucTAcrnQ3bHZfEhnEa7 6y1gNdUjkBQGr2v9Ga9DwMpIA7y8sbZKUkKiCTx4KT1JUZ0iY7Pow2DkqEqGlN6cBq 0T3nI7SrJFGlbM7xd4L/sk0wDWepCbfAr445Niy7i8OXCzeEt6AnuUzlfGuTLvRzQ6 boUFjHxmrpBSWq9oEH9HxjFT0wY532dgY/jys0wjIezqJvb/dBd7KMCPudGEgxAXiu LxbfYxBe97Jqg== From: Ingo Molnar To: linux-kernel@vger.kernel.org Cc: Andy Lutomirski , Rik van Riel , "H . Peter Anvin" , Peter Zijlstra , Linus Torvalds , Andrew Morton , Ingo Molnar Subject: [PATCH 3/7] x86/mm: Make use_/unuse_temporary_mm() non-static Date: Wed, 2 Apr 2025 11:45:36 +0200 Message-ID: <20250402094540.3586683-4-mingo@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250402094540.3586683-1-mingo@kernel.org> References: <20250402094540.3586683-1-mingo@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Andy Lutomirski This prepares them for use outside of the alternative machinery. The code is unchanged. Signed-off-by: Andy Lutomirski Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Cc: Rik van Riel Cc: "H. Peter Anvin" Cc: Linus Torvalds Link: https://lore.kernel.org/r/20241119163035.533822339@infradead.org --- arch/x86/include/asm/mmu_context.h | 3 ++ arch/x86/kernel/alternative.c | 64 ----------------------------------= ---- arch/x86/mm/tlb.c | 64 ++++++++++++++++++++++++++++++++++= ++++ 3 files changed, 67 insertions(+), 64 deletions(-) diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_= context.h index 2398058b6e83..b103e1709a67 100644 --- a/arch/x86/include/asm/mmu_context.h +++ b/arch/x86/include/asm/mmu_context.h @@ -272,4 +272,7 @@ unsigned long __get_current_cr3_fast(void); =20 #include =20 +extern struct mm_struct *use_temporary_mm(struct mm_struct *temp_mm); +extern void unuse_temporary_mm(struct mm_struct *mm, struct mm_struct *pre= v_mm); + #endif /* _ASM_X86_MMU_CONTEXT_H */ diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index cfffcb80f564..25abadaf8751 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -2111,73 +2111,9 @@ void __init_or_module text_poke_early(void *addr, co= nst void *opcode, } } =20 -/* - * Using a temporary mm allows to set temporary mappings that are not acce= ssible - * by other CPUs. Such mappings are needed to perform sensitive memory wri= tes - * that override the kernel memory protections (e.g., W^X), without exposi= ng the - * temporary page-table mappings that are required for these write operati= ons to - * other CPUs. Using a temporary mm also allows to avoid TLB shootdowns wh= en the - * mapping is torn down. - * - * Context: The temporary mm needs to be used exclusively by a single core= . To - * harden security IRQs must be disabled while the temporary mm is - * loaded, thereby preventing interrupt handler bugs from overrid= ing - * the kernel memory protection. - */ -static inline struct mm_struct *use_temporary_mm(struct mm_struct *temp_mm) -{ - struct mm_struct *prev_mm; - - lockdep_assert_irqs_disabled(); - - /* - * Make sure not to be in TLB lazy mode, as otherwise we'll end up - * with a stale address space WITHOUT being in lazy mode after - * restoring the previous mm. - */ - if (this_cpu_read(cpu_tlbstate_shared.is_lazy)) - leave_mm(); - - prev_mm =3D this_cpu_read(cpu_tlbstate.loaded_mm); - switch_mm_irqs_off(NULL, temp_mm, current); - - /* - * If breakpoints are enabled, disable them while the temporary mm is - * used. Userspace might set up watchpoints on addresses that are used - * in the temporary mm, which would lead to wrong signals being sent or - * crashes. - * - * Note that breakpoints are not disabled selectively, which also causes - * kernel breakpoints (e.g., perf's) to be disabled. This might be - * undesirable, but still seems reasonable as the code that runs in the - * temporary mm should be short. - */ - if (hw_breakpoint_active()) - hw_breakpoint_disable(); - - return prev_mm; -} - __ro_after_init struct mm_struct *text_poke_mm; __ro_after_init unsigned long text_poke_mm_addr; =20 -static inline void unuse_temporary_mm(struct mm_struct *mm, struct mm_stru= ct *prev_mm) -{ - lockdep_assert_irqs_disabled(); - - switch_mm_irqs_off(NULL, prev_mm, current); - - /* Clear the cpumask, to indicate no TLB flushing is needed anywhere */ - cpumask_clear_cpu(raw_smp_processor_id(), mm_cpumask(mm)); - - /* - * Restore the breakpoints if they were disabled before the temporary mm - * was loaded. - */ - if (hw_breakpoint_active()) - hw_breakpoint_restore(); -} - static void text_poke_memcpy(void *dst, const void *src, size_t len) { memcpy(dst, src, len); diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 0925768d00cb..06a1ad39be74 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -972,6 +972,70 @@ void enter_lazy_tlb(struct mm_struct *mm, struct task_= struct *tsk) this_cpu_write(cpu_tlbstate_shared.is_lazy, true); } =20 +/* + * Using a temporary mm allows to set temporary mappings that are not acce= ssible + * by other CPUs. Such mappings are needed to perform sensitive memory wri= tes + * that override the kernel memory protections (e.g., W^X), without exposi= ng the + * temporary page-table mappings that are required for these write operati= ons to + * other CPUs. Using a temporary mm also allows to avoid TLB shootdowns wh= en the + * mapping is torn down. + * + * Context: The temporary mm needs to be used exclusively by a single core= . To + * harden security IRQs must be disabled while the temporary mm is + * loaded, thereby preventing interrupt handler bugs from overrid= ing + * the kernel memory protection. + */ +struct mm_struct *use_temporary_mm(struct mm_struct *temp_mm) +{ + struct mm_struct *prev_mm; + + lockdep_assert_irqs_disabled(); + + /* + * Make sure not to be in TLB lazy mode, as otherwise we'll end up + * with a stale address space WITHOUT being in lazy mode after + * restoring the previous mm. + */ + if (this_cpu_read(cpu_tlbstate_shared.is_lazy)) + leave_mm(); + + prev_mm =3D this_cpu_read(cpu_tlbstate.loaded_mm); + switch_mm_irqs_off(NULL, temp_mm, current); + + /* + * If breakpoints are enabled, disable them while the temporary mm is + * used. Userspace might set up watchpoints on addresses that are used + * in the temporary mm, which would lead to wrong signals being sent or + * crashes. + * + * Note that breakpoints are not disabled selectively, which also causes + * kernel breakpoints (e.g., perf's) to be disabled. This might be + * undesirable, but still seems reasonable as the code that runs in the + * temporary mm should be short. + */ + if (hw_breakpoint_active()) + hw_breakpoint_disable(); + + return prev_mm; +} + +void unuse_temporary_mm(struct mm_struct *mm, struct mm_struct *prev_mm) +{ + lockdep_assert_irqs_disabled(); + + switch_mm_irqs_off(NULL, prev_mm, current); + + /* Clear the cpumask, to indicate no TLB flushing is needed anywhere */ + cpumask_clear_cpu(raw_smp_processor_id(), mm_cpumask(mm)); + + /* + * Restore the breakpoints if they were disabled before the temporary mm + * was loaded. + */ + if (hw_breakpoint_active()) + hw_breakpoint_restore(); +} + /* * Call this when reinitializing a CPU. It fixes the following potential * problems: --=20 2.45.2 From nobody Sat Feb 7 20:44:03 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 80443236451 for ; Wed, 2 Apr 2025 09:46:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743587170; cv=none; b=o/dm8bsT4I1B9jsuB7IzzB9hF2xz7ayqzftTrcxpSk2WZGqtAqCYwbhz3JdDn6KDkgIBETW8aXFTWqQRUkkv8+AZqanMiR70Q1j1/V7osFsua7ML9qL6XNdYPlx9MFh2M2zbEd/b0aA3YeUG8PtiRTvdEDpfddcAN5LAhE1acDw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743587170; c=relaxed/simple; bh=UbElSkoimXQH4iUna5xDzRePb/1/vdvyt0zK786CFo0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tZD1UZKkzu/Zcvs7IXD/pOelNIDrO7S7RzHXLSLFD2CiLye21DQq85Wr5zUkSfaVnyyScNFW2KoIKPhMQX1hiSwMXEX6zbkrPYSviajB1v3zhOtRsS6lkJbHATgWljWZZ+WFfjL2WLfxYSVZQEszmk4c71ulaTEr0i4bK5Hwxw4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=mo+/bj6n; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="mo+/bj6n" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 00D43C4CEEA; Wed, 2 Apr 2025 09:46:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1743587170; bh=UbElSkoimXQH4iUna5xDzRePb/1/vdvyt0zK786CFo0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mo+/bj6nPuhgxDlIcgZZJEC+ckYy9CRWg51vk7//cnJ26TTJ2rPjlsFY58XA/rvGs 92ziJaMmQk4+d4Wg8cf95pDVqEgnFyAz0REf4WOKBhMwOXUdWdTJ06HHHSdMG7pA4w zwVHU7twTCCCYh746oqeJhMCJjfOaCLlumdCulSvw/w+xj2pV2wnX/IpUmuW+h/pk8 rGHS17v/8p3BcKKktP+QsKgtx0NdyYB36Fxn/yXRsdh69igxKprwk8Tl9AWJFHLoUp 9ekA0m0ovkYoOiGfG7lksoUr5iD3iRkBzfnMogZVL0zgAA+6uzqtrvkO8BAg4IDTwr eY4txXnG1v1fw== From: Ingo Molnar To: linux-kernel@vger.kernel.org Cc: Andy Lutomirski , Rik van Riel , "H . Peter Anvin" , Peter Zijlstra , Linus Torvalds , Andrew Morton , Ingo Molnar Subject: [PATCH 4/7] x86/mm: Remove 'mm' argument from unuse_temporary_mm() again Date: Wed, 2 Apr 2025 11:45:37 +0200 Message-ID: <20250402094540.3586683-5-mingo@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250402094540.3586683-1-mingo@kernel.org> References: <20250402094540.3586683-1-mingo@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra Now that unuse_temporary_mm() lives in tlb.c it can access cpu_tlbstate.loaded_mm. [ mingo: Merged it on top of x86/alternatives ] Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Cc: Andy Lutomirski Cc: Rik van Riel Cc: "H. Peter Anvin" Cc: Linus Torvalds Link: https://lore.kernel.org/r/20241119163035.648739178@infradead.org --- arch/x86/include/asm/mmu_context.h | 2 +- arch/x86/kernel/alternative.c | 2 +- arch/x86/mm/tlb.c | 8 ++++---- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_= context.h index b103e1709a67..988c11792634 100644 --- a/arch/x86/include/asm/mmu_context.h +++ b/arch/x86/include/asm/mmu_context.h @@ -273,6 +273,6 @@ unsigned long __get_current_cr3_fast(void); #include =20 extern struct mm_struct *use_temporary_mm(struct mm_struct *temp_mm); -extern void unuse_temporary_mm(struct mm_struct *mm, struct mm_struct *pre= v_mm); +extern void unuse_temporary_mm(struct mm_struct *prev_mm); =20 #endif /* _ASM_X86_MMU_CONTEXT_H */ diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 25abadaf8751..964a2eb0071a 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -2211,7 +2211,7 @@ static void *__text_poke(text_poke_f func, void *addr= , const void *src, size_t l * instruction that already allows the core to see the updated version. * Xen-PV is assumed to serialize execution in a similar manner. */ - unuse_temporary_mm(text_poke_mm, prev_mm); + unuse_temporary_mm(prev_mm); =20 /* * Flushing the TLB might involve IPIs, which would require enabled diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 06a1ad39be74..e672508ca158 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -1019,14 +1019,14 @@ struct mm_struct *use_temporary_mm(struct mm_struct= *temp_mm) return prev_mm; } =20 -void unuse_temporary_mm(struct mm_struct *mm, struct mm_struct *prev_mm) +void unuse_temporary_mm(struct mm_struct *prev_mm) { lockdep_assert_irqs_disabled(); =20 - switch_mm_irqs_off(NULL, prev_mm, current); - /* Clear the cpumask, to indicate no TLB flushing is needed anywhere */ - cpumask_clear_cpu(raw_smp_processor_id(), mm_cpumask(mm)); + cpumask_clear_cpu(smp_processor_id(), mm_cpumask(this_cpu_read(cpu_tlbsta= te.loaded_mm))); + + switch_mm_irqs_off(NULL, prev_mm, current); =20 /* * Restore the breakpoints if they were disabled before the temporary mm --=20 2.45.2 From nobody Sat Feb 7 20:44:03 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4605D237708 for ; Wed, 2 Apr 2025 09:46:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743587173; cv=none; b=YTSC4i5JjoA1gyJymy0Z8uqt4OUX1IdvnzFgCQplOslaPW7zduAVGmq2kOtclZ25lS+zFLtP6HXSnsaaMRQCk+xPOtuMJkjJyOMpj3SeVVHAPH1Gjv7vwkNt+edrINYQFBMlCZwiZeHpfx4QKECAekpdyBx+SDRzA8FCT7VAinM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743587173; c=relaxed/simple; bh=LzxZ5sTf9IdjgunuzNLwa+CdQ18YCsx3f6grbp1Qr2I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IJDF2AGpDHBsRThDMHMcRTSjVy14qs7BxFKTsfG00wp1JzFFaTB4FQWnTYiKp72iM80tlzTaBTzXRx5h7iA4wxQ1Dq46z9QWOJYxK9O0gMQCickEU4E4d9dodfmw1TerIDqdErVnvSPX0eN/rv863UrpCjaBcmtUn1xpcPkZB2o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=aOXJYfP8; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="aOXJYfP8" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 73293C4CEE9; Wed, 2 Apr 2025 09:46:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1743587172; bh=LzxZ5sTf9IdjgunuzNLwa+CdQ18YCsx3f6grbp1Qr2I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aOXJYfP8mVe3bJEhjjipQ1pTDIQx/L43ogVwl1d/uImHcukCS7i38pXw7Bq7GhIz5 /wem1Ax0i+drhDJuhrIKxWDecr3BUgb8EIB5A80syJWHtxeYoqmEu0AxF6KT41i94R 2dZlcuWELRQeH4yjR5RP744p0LKoHn3r+38DorfbyU+vBf7tL8KzOTSdggVkEwB8Yp HfrNADwFQPlWxKbkKvaWL0p/OgUY5n/tqM2rYaOx3tnBUt4YtXjbiluk0UOtroy3EG Jh0EaPO0988W+N2uqEHqj0CbeB7+gTmsfqSal6zcDm1QXjMFfqW68SLWDPdSf304Zb WYf+HHxCNW6pg== From: Ingo Molnar To: linux-kernel@vger.kernel.org Cc: Andy Lutomirski , Rik van Riel , "H . Peter Anvin" , Peter Zijlstra , Linus Torvalds , Andrew Morton , Ingo Molnar , Ard Biesheuvel Subject: [PATCH 5/7] x86/mm: Allow temporary MMs when IRQs are on Date: Wed, 2 Apr 2025 11:45:38 +0200 Message-ID: <20250402094540.3586683-6-mingo@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250402094540.3586683-1-mingo@kernel.org> References: <20250402094540.3586683-1-mingo@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Andy Lutomirski EFI runtime services should use temporary MMs, but EFI runtime services want IRQs on. Preemption must still be disabled in a temporary MM context. At some point, the entirely temporary MM mechanism should be moved out of arch code. Signed-off-by: Andy Lutomirski Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Cc: Rik van Riel Cc: "H. Peter Anvin" Cc: Linus Torvalds Cc: Ard Biesheuvel Link: https://lore.kernel.org/r/20241119163035.758732080@infradead.org --- arch/x86/mm/tlb.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index e672508ca158..8e4818ce04a5 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -978,18 +978,23 @@ void enter_lazy_tlb(struct mm_struct *mm, struct task= _struct *tsk) * that override the kernel memory protections (e.g., W^X), without exposi= ng the * temporary page-table mappings that are required for these write operati= ons to * other CPUs. Using a temporary mm also allows to avoid TLB shootdowns wh= en the - * mapping is torn down. + * mapping is torn down. Temporary mms can also be used for EFI runtime s= ervice + * calls or similar functionality. * - * Context: The temporary mm needs to be used exclusively by a single core= . To - * harden security IRQs must be disabled while the temporary mm is - * loaded, thereby preventing interrupt handler bugs from overrid= ing - * the kernel memory protection. + * It is illegal to schedule while using a temporary mm -- the context swi= tch + * code is unaware of the temporary mm and does not know how to context sw= itch. + * Use a real (non-temporary) mm in a kernel thread if you need to sleep. + * + * Note: For sensitive memory writes, the temporary mm needs to be used + * exclusively by a single core, and IRQs should be disabled while t= he + * temporary mm is loaded, thereby preventing interrupt handler bugs= from + * overriding the kernel memory protection. */ struct mm_struct *use_temporary_mm(struct mm_struct *temp_mm) { struct mm_struct *prev_mm; =20 - lockdep_assert_irqs_disabled(); + lockdep_assert_preemption_disabled(); =20 /* * Make sure not to be in TLB lazy mode, as otherwise we'll end up @@ -1021,7 +1026,7 @@ struct mm_struct *use_temporary_mm(struct mm_struct *= temp_mm) =20 void unuse_temporary_mm(struct mm_struct *prev_mm) { - lockdep_assert_irqs_disabled(); + lockdep_assert_preemption_disabled(); =20 /* Clear the cpumask, to indicate no TLB flushing is needed anywhere */ cpumask_clear_cpu(smp_processor_id(), mm_cpumask(this_cpu_read(cpu_tlbsta= te.loaded_mm))); --=20 2.45.2 From nobody Sat Feb 7 20:44:03 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3B2CC23814A for ; Wed, 2 Apr 2025 09:46:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743587175; cv=none; b=LrZmv+ym8T4zofXRgmjgswwXx8w/8iziqvr5Qd4o+9qLhSIOM0GIEDqUcKutgCYRt3MIL0ZZs3AEi3aSEhdf/vjNmENqmI/dxvPxF3F1rE9W/DXLeR5p0SB5mJwWGGQaGbgNkCZlAjmbYf+PLt3BIxQUkIYWW/P3eneGXEY2iog= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743587175; c=relaxed/simple; bh=UM46Z3LywVA8SgF6vCR4JHx+WH4GLGG/jRvAXsdQuKI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hXDQCwQMORqJcwkyXUCeokhAqfw1LF45g4Vk784ObB6EKEdf8symQVB2h49Hm66jUX4NAga63/vQw51omrT7jgahSTpIt+DoTjs3Ghb3Add4R8H0OCsFg9XB910EDNgZ3C6tqbmtpi/IV0FKelZheApv+KHCrtWqHBiW2EQE4+Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kB7v57eZ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kB7v57eZ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 207D6C4CEE8; Wed, 2 Apr 2025 09:46:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1743587175; bh=UM46Z3LywVA8SgF6vCR4JHx+WH4GLGG/jRvAXsdQuKI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kB7v57eZ5YAJXipGlLqcskI5To4WGavXLSdxv96UgnMcU78H3tEM3ydAAfwNhJq17 eTaoRKwA5V7PAli6MdHnXcLIAf8RAic6cnFc1FafsIhCc3Qoliy45sy0ETxmcgAyhB J0rJrhFmHIPInDk1IOt6sQcHfBhVL2IRma4m7KQ6GOL3nJLzHwE6RkBG2BcltHAtCv HorHOpnnl51htcdw5UeCKv0nh4O6Y4nt1dt3tGlurABUev9emJPqK89uQKRhIvZ6xE cDx+1oCFO1ide/j2SOt4isZioUSrQHh+RpoxLaQvo9EJBrlcIqsyZ7NPQ0KLMT6y9C UaHSjGYFvoUGg== From: Ingo Molnar To: linux-kernel@vger.kernel.org Cc: Andy Lutomirski , Rik van Riel , "H . Peter Anvin" , Peter Zijlstra , Linus Torvalds , Andrew Morton , Ingo Molnar Subject: [PATCH 6/7] x86/efi: Make efi_enter/leave_mm() use the use_/unuse_temporary_mm() machinery Date: Wed, 2 Apr 2025 11:45:39 +0200 Message-ID: <20250402094540.3586683-7-mingo@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250402094540.3586683-1-mingo@kernel.org> References: <20250402094540.3586683-1-mingo@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Andy Lutomirski This should be considerably more robust. It's also necessary for optimized for_each_possible_lazymm_cpu() on x86 -- without this patch, EFI calls in lazy context would remove the lazy mm from mm_cpumask(). [ mingo: Merged it on top of x86/alternatives ] Signed-off-by: Andy Lutomirski Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Cc: Rik van Riel Cc: "H. Peter Anvin" Cc: Linus Torvalds Link: https://lore.kernel.org/r/20241119163035.877939834@infradead.org --- arch/x86/platform/efi/efi_64.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c index ac57259a432b..a5d3496d32a5 100644 --- a/arch/x86/platform/efi/efi_64.c +++ b/arch/x86/platform/efi/efi_64.c @@ -434,15 +434,12 @@ void __init efi_dump_pagetable(void) */ static void efi_enter_mm(void) { - efi_prev_mm =3D current->active_mm; - current->active_mm =3D &efi_mm; - switch_mm(efi_prev_mm, &efi_mm, NULL); + efi_prev_mm =3D use_temporary_mm(&efi_mm); } =20 static void efi_leave_mm(void) { - current->active_mm =3D efi_prev_mm; - switch_mm(&efi_mm, efi_prev_mm, NULL); + unuse_temporary_mm(efi_prev_mm); } =20 void arch_efi_call_virt_setup(void) --=20 2.45.2 From nobody Sat Feb 7 20:44:03 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1AE7F23814A for ; Wed, 2 Apr 2025 09:46:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743587179; cv=none; b=tYZ7lAQmgU9K+PTCp0v/AaDmFdMluwr13HI956jL3frSQ18UPi306zmcNTS7M2OKVcvh6Te/+cF+bSkwSbEwf4vPTD69j+Ymnksk7EokuacJ/72pJJKLWUYpuR52oEgm6Ec9FdMn5Mc2R9xr3Y80e/+sZxJ/hyT1YlrsrvrODAc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743587179; c=relaxed/simple; bh=wwJkv9MVllRwnbE94gJpES5xAIsuRO9xRorGgHss54k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sVOGIdf2WfiH3/e6E3pVqcYsgMBppwL7+koq3wVWIi8z+69JPbEUTybfbEtAUSN3n/shas+Eh6YkwuLOTH3Zt0GZuJALF1DcftOo0R/juSPJgLcdPCoBgTRux4k+ZqcRA7OWZ7YFbw+51O4oncXvRvAkWGdA2guws2rAv9i9jgU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Pl1s3OTh; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Pl1s3OTh" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 92D88C4CEED; Wed, 2 Apr 2025 09:46:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1743587177; bh=wwJkv9MVllRwnbE94gJpES5xAIsuRO9xRorGgHss54k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Pl1s3OThcAMerSvAI3QBhdHsE3gmdzUE7h8Ied84vV7yCXxiPnTIvqvM+upZ9V05+ 7WIzyhq7PjUP7xQ6eQv+dUCD75sF9CK93JqT+7aafDC/EK2j1OptiD9jlYAp2a40fN jhl+6Dq9KjZGpjNsznmLWewVBqxL83bj3ugQvC/ChnPbpT6FF5neUeESvSnOqaOKsg /H9W2jW7kSx2qGA+plcMxMa7LDgLwppEzEx5K4H4ZMilXnzF2JTNC/J0p/oh0/i2+b 951hVktRS5e1BBpMgxdBdgF7nMNTfpRyHHsoWOHhIqIybIh9+Jt+qvvTYZ0qlAy3bN ndKe22kNTOCbw== From: Ingo Molnar To: linux-kernel@vger.kernel.org Cc: Andy Lutomirski , Rik van Riel , "H . Peter Anvin" , Peter Zijlstra , Linus Torvalds , Andrew Morton , Ingo Molnar Subject: [PATCH 7/7] x86/mm: Opt-in to IRQs-off activate_mm() Date: Wed, 2 Apr 2025 11:45:40 +0200 Message-ID: <20250402094540.3586683-8-mingo@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250402094540.3586683-1-mingo@kernel.org> References: <20250402094540.3586683-1-mingo@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Andy Lutomirski We gain nothing by having the core code enable IRQs right before calling activate_mm() only for us to turn them right back off again in switch_mm(). This will save a few cycles, so execve() should be blazingly fast with this patch applied! Signed-off-by: Andy Lutomirski Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Cc: Rik van Riel Cc: "H. Peter Anvin" Cc: Linus Torvalds Link: https://lore.kernel.org/r/20241119163035.985203915@infradead.org --- arch/x86/Kconfig | 1 + arch/x86/include/asm/mmu_context.h | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 98bd4935280c..6b90d93fc40e 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -149,6 +149,7 @@ config X86 select ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP if X86_64 select ARCH_WANTS_THP_SWAP if X86_64 select ARCH_HAS_PARANOID_L1D_FLUSH + select ARCH_WANT_IRQS_OFF_ACTIVATE_MM select BUILDTIME_TABLE_SORT select CLKEVT_I8253 select CLOCKSOURCE_WATCHDOG diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_= context.h index 988c11792634..c511f8584ae4 100644 --- a/arch/x86/include/asm/mmu_context.h +++ b/arch/x86/include/asm/mmu_context.h @@ -190,7 +190,7 @@ extern void switch_mm_irqs_off(struct mm_struct *prev, = struct mm_struct *next, #define activate_mm(prev, next) \ do { \ paravirt_enter_mmap(next); \ - switch_mm((prev), (next), NULL); \ + switch_mm_irqs_off((prev), (next), NULL); \ } while (0); =20 #ifdef CONFIG_X86_32 --=20 2.45.2