From nobody Tue Feb 10 04:03:45 2026 Received: from mail-lj1-f202.google.com (mail-lj1-f202.google.com [209.85.208.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C1B8628EC98 for ; Fri, 10 Jan 2025 18:47:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736534880; cv=none; b=BFwR4DMgPIsN2lxtGgBFhEeJVSs7GoGNM/1CQNVvSICTTItkFVqq+Eg5DXF73g743u3BMTWJ18pcBsaNELOVABnu5MWQROniMViE4JTdKEdHppVVfpB60mneBkDomLQuzAtgi6aZVKBcJ7eBGQJbFhnygGL2Le7sMBPUcKVjzkM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736534880; c=relaxed/simple; bh=Ua0Eznh6vBIqJokkBKNt1xiE4aA9cS4fZDm05XjqWbo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=t/KesuaVqs1Nd9Y3kzhTx8W9+oJJRZ0Zl5o/COKCgmbugaSkG4TXauRxB7OfO5G3hYgqCV243SOMM7HAmTaaIKyMvXAc0uvppQRPfKsuvEtVQxCRREJIV/39zGuYgQtIrI/74AKTAC/5r2hxuAbo8cdkeeMQP12domh6p/GJkiY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--jackmanb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=tGbAiYp/; arc=none smtp.client-ip=209.85.208.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--jackmanb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="tGbAiYp/" Received: by mail-lj1-f202.google.com with SMTP id 38308e7fff4ca-30221227ec8so11912231fa.0 for ; Fri, 10 Jan 2025 10:47:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736534877; x=1737139677; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=YGrdWVEeU4Ptbj/O5LlMCmxq+nbCGI8+4U1HKBQ0dew=; b=tGbAiYp/Wc/NelJqh79wmkYKI76IX8RHXxu5Aek8eu21SPuJxvs/dkNbnJbgf+Afci OggRngdvMXMhSQKVbZynI+rHCC1kfIThAUKFubaBcO9xqhaioKBJsxKExSjRZpCZ0stQ HMDB57xUFVmh/mN8olSncam/Z+7NsmIZOM/Z7c+NkJN4jUpMznkuBgFo4404EgKlymT5 Fmyh63OwmZrkJ9/uk1JGgRNXDQfcM3cmplFaVJlftqujVnyeXYg6v3nFRKHPikyeVItu 3KLylfyHWbxPxeQLNiNgRHvDG3T+Ti8UYTKDEMPgIQhAb+zYUfSP9LNlHqSxZWAw69sC PQaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736534877; x=1737139677; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=YGrdWVEeU4Ptbj/O5LlMCmxq+nbCGI8+4U1HKBQ0dew=; b=Oe5VKNCgqaOPS2WYDv2o13/f9Cds0oTC4JeZov22hWd9nHUJ9zd18RVFfj2oo+J5pq xObI0ot66soLyWgov9vCZAUEzUOkyqPucF8mdt/Q7S2Vb+Fjq384XU89b4rRiHxIawEP UmjzgRUIxD/MTGTwbaOFZdHKmmvxPiqEsf+WgXn860dNzIPCZLB07TcF99FwxfLNZLRd qY4c4gWuyhacXgMPgNGQDtltpXToP0ScWbJT8Vt7XqFVpxcB9R65oRw+NVSzuJKNep9X Hwb3yOb7oZ3zXNHGk/IP/Rn4h17lFJ5df7TpBQ+30S1kHDueDR1wbdI3gU9sPop3Pr4M la9Q== X-Forwarded-Encrypted: i=1; AJvYcCXyV+mIFOTjKx7g6vNnhbH5o9IQMXPrpS3v9y/yYo9ohK8hVvov2qwjLvnoZaRwoE+RWR5V2kJsWudSg3I=@vger.kernel.org X-Gm-Message-State: AOJu0YxubS9/HRkRgwFcZuM05IjrFtj2xWmsGu4xDlM5tfUXEG6Pj+1X z3OHQuorDZ2ZM2dUE3WDZNpTcnoFWhh9xMhXLITM9SdRt+TGzL5YPMvH1iD6krHEOgmqVapgG5t tB6Thzdz+fA== X-Google-Smtp-Source: AGHT+IEgsXbDoolH8oLFMalWBy0djT9uJM60zMy+Lo0ErgHawFZLEGOSZUiE5P0d6bxzOZZfpK4Vh92Jd6K16A== X-Received: from wmrn43.prod.google.com ([2002:a05:600c:502b:b0:434:a9bd:e68c]) (user=jackmanb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4f81:b0:434:f1d5:1453 with SMTP id 5b1f17b1804b1-436e2531ec8mr120237105e9.0.1736534506703; Fri, 10 Jan 2025 10:41:46 -0800 (PST) Date: Fri, 10 Jan 2025 18:40:53 +0000 In-Reply-To: <20250110-asi-rfc-v2-v2-0-8419288bc805@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250110-asi-rfc-v2-v2-0-8419288bc805@google.com> X-Mailer: b4 0.15-dev Message-ID: <20250110-asi-rfc-v2-v2-27-8419288bc805@google.com> Subject: [PATCH RFC v2 27/29] mm: asi: Add some mitigations on address space transitions From: Brendan Jackman To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Richard Henderson , Matt Turner , Vineet Gupta , Russell King , Catalin Marinas , Will Deacon , Guo Ren , Brian Cain , Huacai Chen , WANG Xuerui , Geert Uytterhoeven , Michal Simek , Thomas Bogendoerfer , Dinh Nguyen , Jonas Bonn , Stefan Kristiansson , Stafford Horne , "James E.J. Bottomley" , Helge Deller , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Naveen N Rao , Madhavan Srinivasan , Paul Walmsley , Palmer Dabbelt , Albert Ou , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Yoshinori Sato , Rich Felker , John Paul Adrian Glaubitz , "David S. Miller" , Andreas Larsson , Richard Weinberger , Anton Ivanov , Johannes Berg , Chris Zankel , Max Filippov , Arnd Bergmann , Andrew Morton , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Uladzislau Rezki , Christoph Hellwig , Masami Hiramatsu , Mathieu Desnoyers , Mike Rapoport , Arnaldo Carvalho de Melo , Namhyung Kim , Mark Rutland , Alexander Shishkin , Jiri Olsa , Ian Rogers , Adrian Hunter , Dennis Zhou , Tejun Heo , Christoph Lameter , Sean Christopherson , Paolo Bonzini , Ard Biesheuvel , Josh Poimboeuf , Pawan Gupta Cc: x86@kernel.org, linux-kernel@vger.kernel.org, linux-alpha@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, loongarch@lists.linux.dev, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, linux-openrisc@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, linux-um@lists.infradead.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, kvm@vger.kernel.org, linux-efi@vger.kernel.org, Brendan Jackman Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Here we ASI actually starts becoming a real exploit mitigation, On CPUs with L1TF, flush L1D when the ASI data taints say so. On all CPUs, do some general branch predictor clearing whenever the control taints say so. This policy is very much just a starting point for discussion. Primarily it's a vague gesture at the fact that there is leeway in how ASI is used: it can be used to target CPU-specific issues (as is the case for L1TF here), or it can be used as a fairly broad mitigation (asi_maybe_flush_control() mitigates several known Spectre-style attacks and very likely also some unknown ones). Signed-off-by: Brendan Jackman --- arch/x86/include/asm/nospec-branch.h | 2 ++ arch/x86/kvm/vmx/vmx.c | 1 + arch/x86/lib/l1tf.c | 2 ++ arch/x86/lib/retpoline.S | 10 ++++++++++ arch/x86/mm/asi.c | 29 +++++++++++++++++++++-------- 5 files changed, 36 insertions(+), 8 deletions(-) diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/no= spec-branch.h index 96b410b1d4e841eb02f53a4691ee794ceee4ad2c..4582fb1fb42f6fd226534012d96= 9ed13085e943a 100644 --- a/arch/x86/include/asm/nospec-branch.h +++ b/arch/x86/include/asm/nospec-branch.h @@ -614,6 +614,8 @@ static __always_inline void mds_idle_clear_cpu_buffers(= void) mds_clear_cpu_buffers(); } =20 +extern void fill_return_buffer(void); + #endif /* __ASSEMBLY__ */ =20 #endif /* _ASM_X86_NOSPEC_BRANCH_H_ */ diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index b1a02f27b3abce0ef6ac448b66bef2c653a52eef..a532783caaea97291cd92a2e2ca= c617f74f76c7e 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6635,6 +6635,7 @@ int vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t= exit_fastpath) return ret; } =20 +/* Must be reentrant, for use by vmx_post_asi_enter. */ static noinstr void vmx_l1d_flush(struct kvm_vcpu *vcpu) { /* diff --git a/arch/x86/lib/l1tf.c b/arch/x86/lib/l1tf.c index c474f18ae331c8dfa7a029c457dd3cf75bebf808..ffe1c3d0ef43ff8f1781f2e446a= ed041f4ce3179 100644 --- a/arch/x86/lib/l1tf.c +++ b/arch/x86/lib/l1tf.c @@ -46,6 +46,8 @@ EXPORT_SYMBOL(l1tf_flush_setup); * - may or may not work on other CPUs. * * Don't call unless l1tf_flush_setup() has returned successfully. + * + * Must be reentrant, for use by ASI. */ noinstr void l1tf_flush(void) { diff --git a/arch/x86/lib/retpoline.S b/arch/x86/lib/retpoline.S index 391059b2c6fbc4a571f0582c7c4654147a930cef..6d126fff6bf839889086fe21464= d8af07316d7e5 100644 --- a/arch/x86/lib/retpoline.S +++ b/arch/x86/lib/retpoline.S @@ -396,3 +396,13 @@ SYM_CODE_END(__x86_return_thunk) EXPORT_SYMBOL(__x86_return_thunk) =20 #endif /* CONFIG_MITIGATION_RETHUNK */ + +.pushsection .noinstr.text, "ax" +SYM_CODE_START(fill_return_buffer) + UNWIND_HINT_FUNC + ENDBR + __FILL_RETURN_BUFFER(%_ASM_AX,RSB_CLEAR_LOOPS) + RET +SYM_CODE_END(fill_return_buffer) +__EXPORT_THUNK(fill_return_buffer) +.popsection diff --git a/arch/x86/mm/asi.c b/arch/x86/mm/asi.c index 1e9dc568e79e8686a4dbf47f765f2c2535d025ec..f10f6614b26148e5ba423d8a44f= 640674573ee40 100644 --- a/arch/x86/mm/asi.c +++ b/arch/x86/mm/asi.c @@ -10,6 +10,7 @@ =20 #include #include +#include #include #include #include @@ -38,6 +39,8 @@ struct asi __asi_global_nonsensitive =3D { .mm =3D &init_mm, }; =20 +static bool do_l1tf_flush __ro_after_init; + static inline bool asi_class_id_valid(enum asi_class_id class_id) { return class_id >=3D 0 && class_id < ASI_MAX_NUM_CLASSES; @@ -361,6 +364,15 @@ static int __init asi_global_init(void) asi_clone_pgd(asi_global_nonsensitive_pgd, init_mm.pgd, VMEMMAP_START + (1UL << PGDIR_SHIFT)); =20 + if (boot_cpu_has_bug(X86_BUG_L1TF)) { + int err =3D l1tf_flush_setup(); + + if (err) + pr_warn("Failed to setup L1TF flushing for ASI (%pe)", ERR_PTR(err)); + else + do_l1tf_flush =3D true; + } + #ifdef CONFIG_PM_SLEEP register_syscore_ops(&asi_syscore_ops); #endif @@ -512,10 +524,12 @@ static __always_inline void maybe_flush_control(struc= t asi *next_asi) if (!taints) return; =20 - /* - * This is where we'll do the actual dirty work of clearing uarch state. - * For now we just pretend, clear the taints. - */ + /* Clear normal indirect branch predictions, if we haven't */ + if (cpu_feature_enabled(X86_FEATURE_IBPB)) + __wrmsr(MSR_IA32_PRED_CMD, PRED_CMD_IBPB, 0); + + fill_return_buffer(); + this_cpu_and(asi_taints, ~ASI_TAINTS_CONTROL_MASK); } =20 @@ -536,10 +550,9 @@ static __always_inline void maybe_flush_data(struct as= i *next_asi) if (!taints) return; =20 - /* - * This is where we'll do the actual dirty work of clearing uarch state. - * For now we just pretend, clear the taints. - */ + if (do_l1tf_flush) + l1tf_flush(); + this_cpu_and(asi_taints, ~ASI_TAINTS_DATA_MASK); } =20 --=20 2.47.1.613.gc27f4b7a9f-goog