[v3] x86/mm: Disable broadcast TLB flush when PCID is disabled

[PATCH v3] x86/mm: Disable broadcast TLB flush when PCID is disabled

Posted by Tom Lendacky 4 days, 4 hours ago

Booting with "nopcid" clears X86_FEATURE_PCID and keeps CR4.PCIDE from
being set to one. On AMD CPUs that support INVLPGB, broadcast TLB
flushing remains enabled.

There are two checks that decide whether the global ASID code runs,
mm_global_asid() and consider_global_asid(), that key off of the
X86_FEATURE_INVLPGB feature. Once an mm becomes active on more than
three CPUs, consider_global_asid() assigns it a global ASID, after which
flush_tlb_mm_range() takes the broadcast_tlb_flush() path using a non-zero
PCID. Issuing an INVLPGB with a non-zero PCID while CR4.PCIDE is not set
results in a #GP:

 Oops: general protection fault, kernel NULL pointer dereference 0x1: 0000 [#1] SMP NOPTI
 CPU: 158 UID: 0 PID: 3119 Comm: snap Not tainted 7.1.0-rc3 #1 PREEMPT(full)
 Hardware name: ...
 RIP: 0010:broadcast_tlb_flush+0x9d/0x210
 Code: ... 89 da 48 83 c8 07 <0f> 01 fe eb 08 cc cc cc ...
 RSP: 0000:ffa0000031217cb0 EFLAGS: 00010202
 RAX: 00007f8ee8540007 RBX: 0000000008070000 RCX: 0000000000000000
 RDX: 0000000000070000 RSI: 0000000000000001 RDI: 00007f8ee8540000
 RBP: ff110001189a0700 R08: 00007f8ee8541000 R09: 000000000000000c
 R10: 000000000000000c R11: 0000000000070000 R12: ff1100c0433aec80
 R13: 0000000000000000 R14: 0000000000000008 R15: 0000000000000007
 FS:  00007f8ed3fff6c0(0000) GS:ff1100c0b7b8a000(0000) knlGS:0000000000000000
 CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 CR2: 00007f8ee8540058 CR3: 00080001433af000 CR4: 0000000000751ef0
 PKRU: 55555554
 Call Trace:
  <TASK>
  flush_tlb_mm_range+0xd8/0x220
  ptep_clear_flush+0x56/0x60
  wp_page_copy+0x293/0x6c0
  ? _raw_spin_unlock+0x15/0x30
  __handle_mm_fault+0x512/0x6d0
  handle_mm_fault+0x145/0x2a0
  do_user_addr_fault+0x187/0x6f0
  exc_page_fault+0x6a/0x180
  asm_exc_page_fault+0x22/0x30
 RIP: 0033:0x55c553ebc639
 Code: ...
 RSP: 002b:00007f8ed3ffeaa8 EFLAGS: 00010246
 RAX: 00007f8ee8540058 RBX: 000055c554ff77b8 RCX: 00007f8ee853c1c8
 RDX: 0000000000000000 RSI: 000000c000062b20 RDI: 00007f8ee8540040
 RBP: 00007f8ed3ffeb00 R08: 0000000000006040 R09: 0000000000000005
 R10: 0000000000000004 R11: 00000000000001c0 R12: 0000000000000001
 R13: 0000005990100204 R14: 000000c000624fc0 R15: 3fffffffffffffff
  </TASK>
 Modules linked in: ...
 ---[ end trace 0000000000000000 ]---

All processors that support broadcast TLB invalidation also have PCID
support, so it is only the "nopcid" scenario that is of concern. In
this situation just disable the broadcast TLB support using the CPUID
dependency support by making X86_FEATURE_INVLPGB dependent on
X86_FEATURE_PCID.

Fixes: 4afeb0ed1753 ("x86/mm: Enable broadcast TLB invalidation for multi-threaded processes")
Assisted-by: Claude:claude-opus-4.7
Suggested-by: Dave Hansen <dave.hansen@intel.com>
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>

---

Changes from v2:
- Update commit message. (Dave)

Changes from v1:
- Disable broadcast TLB support when PCID support is not available
  instead of adding PCID support checks to mm_global_asid() and
  consider_global_asid(). (Dave)
---
 arch/x86/kernel/cpu/cpuid-deps.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/x86/kernel/cpu/cpuid-deps.c b/arch/x86/kernel/cpu/cpuid-deps.c
index 146f6f8b0650..99801e844b30 100644
--- a/arch/x86/kernel/cpu/cpuid-deps.c
+++ b/arch/x86/kernel/cpu/cpuid-deps.c
@@ -92,6 +92,7 @@ static const struct cpuid_dep cpuid_deps[] = {
 	{ X86_FEATURE_FRED,			X86_FEATURE_LKGS      },
 	{ X86_FEATURE_SPEC_CTRL_SSBD,		X86_FEATURE_SPEC_CTRL },
 	{ X86_FEATURE_LASS,			X86_FEATURE_SMAP      },
+	{ X86_FEATURE_INVLPGB,			X86_FEATURE_PCID      },
 	{}
 };
 

base-commit: 2fb123664e61027830a6752a8e9235e7f7712504
-- 
2.51.1

Re: [PATCH v3] x86/mm: Disable broadcast TLB flush when PCID is disabled

Posted by Tom Lendacky 4 days, 3 hours ago

On 5/20/26 12:00, Tom Lendacky wrote:
> Booting with "nopcid" clears X86_FEATURE_PCID and keeps CR4.PCIDE from
> being set to one. On AMD CPUs that support INVLPGB, broadcast TLB
> flushing remains enabled.
> 
> There are two checks that decide whether the global ASID code runs,
> mm_global_asid() and consider_global_asid(), that key off of the
> X86_FEATURE_INVLPGB feature. Once an mm becomes active on more than
> three CPUs, consider_global_asid() assigns it a global ASID, after which
> flush_tlb_mm_range() takes the broadcast_tlb_flush() path using a non-zero
> PCID. Issuing an INVLPGB with a non-zero PCID while CR4.PCIDE is not set
> results in a #GP:
> 
>  Oops: general protection fault, kernel NULL pointer dereference 0x1: 0000 [#1] SMP NOPTI
>  CPU: 158 UID: 0 PID: 3119 Comm: snap Not tainted 7.1.0-rc3 #1 PREEMPT(full)
>  Hardware name: ...
>  RIP: 0010:broadcast_tlb_flush+0x9d/0x210
>  Code: ... 89 da 48 83 c8 07 <0f> 01 fe eb 08 cc cc cc ...
>  RSP: 0000:ffa0000031217cb0 EFLAGS: 00010202
>  RAX: 00007f8ee8540007 RBX: 0000000008070000 RCX: 0000000000000000
>  RDX: 0000000000070000 RSI: 0000000000000001 RDI: 00007f8ee8540000
>  RBP: ff110001189a0700 R08: 00007f8ee8541000 R09: 000000000000000c
>  R10: 000000000000000c R11: 0000000000070000 R12: ff1100c0433aec80
>  R13: 0000000000000000 R14: 0000000000000008 R15: 0000000000000007
>  FS:  00007f8ed3fff6c0(0000) GS:ff1100c0b7b8a000(0000) knlGS:0000000000000000
>  CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>  CR2: 00007f8ee8540058 CR3: 00080001433af000 CR4: 0000000000751ef0
>  PKRU: 55555554
>  Call Trace:
>   <TASK>
>   flush_tlb_mm_range+0xd8/0x220
>   ptep_clear_flush+0x56/0x60
>   wp_page_copy+0x293/0x6c0
>   ? _raw_spin_unlock+0x15/0x30
>   __handle_mm_fault+0x512/0x6d0
>   handle_mm_fault+0x145/0x2a0
>   do_user_addr_fault+0x187/0x6f0
>   exc_page_fault+0x6a/0x180
>   asm_exc_page_fault+0x22/0x30
>  RIP: 0033:0x55c553ebc639
>  Code: ...
>  RSP: 002b:00007f8ed3ffeaa8 EFLAGS: 00010246
>  RAX: 00007f8ee8540058 RBX: 000055c554ff77b8 RCX: 00007f8ee853c1c8
>  RDX: 0000000000000000 RSI: 000000c000062b20 RDI: 00007f8ee8540040
>  RBP: 00007f8ed3ffeb00 R08: 0000000000006040 R09: 0000000000000005
>  R10: 0000000000000004 R11: 00000000000001c0 R12: 0000000000000001
>  R13: 0000005990100204 R14: 000000c000624fc0 R15: 3fffffffffffffff
>   </TASK>
>  Modules linked in: ...
>  ---[ end trace 0000000000000000 ]---
> 
> All processors that support broadcast TLB invalidation also have PCID
> support, so it is only the "nopcid" scenario that is of concern. In
> this situation just disable the broadcast TLB support using the CPUID
> dependency support by making X86_FEATURE_INVLPGB dependent on
> X86_FEATURE_PCID.
> 
> Fixes: 4afeb0ed1753 ("x86/mm: Enable broadcast TLB invalidation for multi-threaded processes")
> Assisted-by: Claude:claude-opus-4.7
> Suggested-by: Dave Hansen <dave.hansen@intel.com>
> Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>

Sorry, I missed Rik's Acked-by: to the v2 patch.

Thanks,
Tom

> 
> ---
> 
> Changes from v2:
> - Update commit message. (Dave)
> 
> Changes from v1:
> - Disable broadcast TLB support when PCID support is not available
>   instead of adding PCID support checks to mm_global_asid() and
>   consider_global_asid(). (Dave)
> ---
>  arch/x86/kernel/cpu/cpuid-deps.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/arch/x86/kernel/cpu/cpuid-deps.c b/arch/x86/kernel/cpu/cpuid-deps.c
> index 146f6f8b0650..99801e844b30 100644
> --- a/arch/x86/kernel/cpu/cpuid-deps.c
> +++ b/arch/x86/kernel/cpu/cpuid-deps.c
> @@ -92,6 +92,7 @@ static const struct cpuid_dep cpuid_deps[] = {
>  	{ X86_FEATURE_FRED,			X86_FEATURE_LKGS      },
>  	{ X86_FEATURE_SPEC_CTRL_SSBD,		X86_FEATURE_SPEC_CTRL },
>  	{ X86_FEATURE_LASS,			X86_FEATURE_SMAP      },
> +	{ X86_FEATURE_INVLPGB,			X86_FEATURE_PCID      },
>  	{}
>  };
>  
> 
> base-commit: 2fb123664e61027830a6752a8e9235e7f7712504