From nobody Thu Dec 18 22:34:57 2025 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 104F626618E; Wed, 19 Mar 2025 11:04:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742382252; cv=none; b=A3vapmyU+b0LRnC1ARS7Nd+2QlNe3/K4vrR/IxgMxQVjyLTggeBlDiN6i5FgKb23BK1d3sPt+kCfft+ZIBFUFzC/KETevuj+7bMeFcmKWrzlylB0yBSOuuclaIxs4WrN1AE1c/ALLzkjL58NJwXHusKhGI/f3EJ4BjDjPtyGmXM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742382252; c=relaxed/simple; bh=+F1SnxCy1xPnHVzOpwFTvOC/sxI35RnlfG4fVkIfZpQ=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=BbuscbLSPBT3ySXBX1EjOaho5zepVFI77S4tvLymxB5fEEf7QXqdCZuP2BPDuV6h9KWJbPZMLY4Nxl1NtFb06CTTcnldcWu5v85+9f2JqBMnsOS6W0e4oWN6NgWWEa6Na22LFwU/rPDPMnr7Kb5ndvwHoqo74lbNGpaqyfY1L9c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=t+stdHtz; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=vlU8Js0j; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="t+stdHtz"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="vlU8Js0j" Date: Wed, 19 Mar 2025 11:04:07 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1742382248; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LtW+L1acHU0/FKowYZEMG91Y0sOEAYwRFtcNdiP5aac=; b=t+stdHtznnyDQ4y/hHOITHik8McXUa5RyMnj1yaKz64IjIz9Bmb03kDbatD7dBh+MDS943 7Nrbf8HEezVjBo6zoJPyMTCWtIPP0HK/AV/5QicE/p1rQEPQOkk2D3ToyIx++7cL3lD+XG g2x1iPfZBxWBI8NPaHDG5aj+tBUksT2VOW8txPVgL8zvtgm+xJEN/7ieQEwjuc6wNjH3GK CI6grK8HJhmkYy3TUBvhplmCJN7DMsiAVQVwhJ0XcSm1BqQLRv+Mx8S6qTASEarrhE7HEw PbQ/UEPxCCBWCEbn4tBExzUsPbeg17QkLypQykzobdX1oSpI7eWfYUdQ1t6NSg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1742382248; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LtW+L1acHU0/FKowYZEMG91Y0sOEAYwRFtcNdiP5aac=; b=vlU8Js0jXWBxdeGVFuVNiazvdiYkFJWY+h2Vo2gol414TG4g1FWk+8m1C9HEx5vZgomMWA NJ9uBLncZjoEtyDg== From: "tip-bot2 for Rik van Riel" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: x86/core] x86/mm: Add INVLPGB feature and Kconfig entry Cc: Rik van Riel , "Borislav Petkov (AMD)" , Ingo Molnar , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20250226030129.530345-3-riel@surriel.com> References: <20250226030129.530345-3-riel@surriel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <174238224752.14745.16434743884389140761.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the x86/core branch of tip: Commit-ID: 767ae437a32d644786c0779d0d54492ff9cbe574 Gitweb: https://git.kernel.org/tip/767ae437a32d644786c0779d0d54492ff= 9cbe574 Author: Rik van Riel AuthorDate: Wed, 19 Mar 2025 11:08:26 +01:00 Committer: Ingo Molnar CommitterDate: Wed, 19 Mar 2025 11:08:52 +01:00 x86/mm: Add INVLPGB feature and Kconfig entry In addition, the CPU advertises the maximum number of pages that can be shot down with one INVLPGB instruction in CPUID. Save that information for later use. [ bp: use cpu_has(), typos, massage. ] Signed-off-by: Rik van Riel Signed-off-by: Borislav Petkov (AMD) Signed-off-by: Ingo Molnar Link: https://lore.kernel.org/r/20250226030129.530345-3-riel@surriel.com --- arch/x86/Kconfig.cpu | 4 ++++ arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/disabled-features.h | 8 +++++++- arch/x86/include/asm/tlbflush.h | 3 +++ arch/x86/kernel/cpu/amd.c | 6 ++++++ 5 files changed, 21 insertions(+), 1 deletion(-) diff --git a/arch/x86/Kconfig.cpu b/arch/x86/Kconfig.cpu index f8b3296..753b876 100644 --- a/arch/x86/Kconfig.cpu +++ b/arch/x86/Kconfig.cpu @@ -334,6 +334,10 @@ menuconfig PROCESSOR_SELECT This lets you choose what x86 vendor support code your kernel will include. =20 +config BROADCAST_TLB_FLUSH + def_bool y + depends on CPU_SUP_AMD && 64BIT + config CPU_SUP_INTEL default y bool "Support Intel processors" if PROCESSOR_SELECT diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpuf= eatures.h index 508c0da..8770dc1 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -338,6 +338,7 @@ #define X86_FEATURE_CLZERO (13*32+ 0) /* "clzero" CLZERO instruction */ #define X86_FEATURE_IRPERF (13*32+ 1) /* "irperf" Instructions Retired Co= unt */ #define X86_FEATURE_XSAVEERPTR (13*32+ 2) /* "xsaveerptr" Always save/res= tore FP error pointers */ +#define X86_FEATURE_INVLPGB (13*32+ 3) /* INVLPGB and TLBSYNC instruction= s supported */ #define X86_FEATURE_RDPRU (13*32+ 4) /* "rdpru" Read processor register a= t user level */ #define X86_FEATURE_WBNOINVD (13*32+ 9) /* "wbnoinvd" WBNOINVD instructio= n */ #define X86_FEATURE_AMD_IBPB (13*32+12) /* Indirect Branch Prediction Bar= rier */ diff --git a/arch/x86/include/asm/disabled-features.h b/arch/x86/include/as= m/disabled-features.h index c492bdc..be8c388 100644 --- a/arch/x86/include/asm/disabled-features.h +++ b/arch/x86/include/asm/disabled-features.h @@ -129,6 +129,12 @@ #define DISABLE_SEV_SNP (1 << (X86_FEATURE_SEV_SNP & 31)) #endif =20 +#ifdef CONFIG_BROADCAST_TLB_FLUSH +#define DISABLE_INVLPGB 0 +#else +#define DISABLE_INVLPGB (1 << (X86_FEATURE_INVLPGB & 31)) +#endif + /* * Make sure to add features to the correct mask */ @@ -146,7 +152,7 @@ #define DISABLED_MASK11 (DISABLE_RETPOLINE|DISABLE_RETHUNK|DISABLE_UNRET| \ DISABLE_CALL_DEPTH_TRACKING|DISABLE_USER_SHSTK) #define DISABLED_MASK12 (DISABLE_FRED|DISABLE_LAM) -#define DISABLED_MASK13 0 +#define DISABLED_MASK13 (DISABLE_INVLPGB) #define DISABLED_MASK14 0 #define DISABLED_MASK15 0 #define DISABLED_MASK16 (DISABLE_PKU|DISABLE_OSPKE|DISABLE_LA57|DISABLE_UM= IP| \ diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflus= h.h index 3da6451..855c13d 100644 --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -183,6 +183,9 @@ static inline void cr4_init_shadow(void) extern unsigned long mmu_cr4_features; extern u32 *trampoline_cr4_features; =20 +/* How many pages can be invalidated with one INVLPGB. */ +extern u16 invlpgb_count_max; + extern void initialize_tlbstate_and_flush(void); =20 /* diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c index 3157664..351c030 100644 --- a/arch/x86/kernel/cpu/amd.c +++ b/arch/x86/kernel/cpu/amd.c @@ -29,6 +29,8 @@ =20 #include "cpu.h" =20 +u16 invlpgb_count_max __ro_after_init; + static inline int rdmsrl_amd_safe(unsigned msr, unsigned long long *p) { u32 gprs[8] =3D { 0 }; @@ -1139,6 +1141,10 @@ static void cpu_detect_tlb_amd(struct cpuinfo_x86 *c) tlb_lli_2m =3D eax & mask; =20 tlb_lli_4m =3D tlb_lli_2m >> 1; + + /* Max number of pages INVLPGB can invalidate in one shot */ + if (cpu_has(c, X86_FEATURE_INVLPGB)) + invlpgb_count_max =3D (cpuid_edx(0x80000008) & 0xffff) + 1; } =20 static const struct cpu_dev amd_cpu_dev =3D {