From nobody Sun Dec 14 12:32:37 2025 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2CD74271A85 for ; Thu, 22 May 2025 23:37:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747957068; cv=none; b=ZeMA2A54YGGwUhP6GLegS3Wn34a+JJrcSgIAwzjw165J5XtqMTbHuxouXVTqYOEJ7L2LgxETPULOVgOoPbNOj8gIV4HW1RWz+G6O+HNwQ1Ai8nyY246g0Q1+rpM/oHTOp6mwq5wUAy+S5Rq6qalAGxFKjb7AhB20Gj8rWsIP6mk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747957068; c=relaxed/simple; bh=eNvNDKoeNT6JOGIaV2NQv10Yua3xIPhgPTUOOfxorRI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=dVPvq9u4MdnYGbtBc0qJwCypINCnqGb8kQZ+hTe3TGbNWN+IS1HM9/2MiZAHr9qcwQkIA8bXuCzpTqJ45RJ/n7QGkDbQpJt7TIvIxTc+5H+CwHrIhWqXnLtgfy4u0rIZouUYoiwCmvBwkGrSckOCJVj0H5GIjxEdBmMomuTUHBY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=p2QRDkdP; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="p2QRDkdP" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-b26e33ae9d5so8903484a12.1 for ; Thu, 22 May 2025 16:37:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747957066; x=1748561866; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=Yvw3d371CPvGh9bouzujsmuFtM9ubEnaNp4RJbDym00=; b=p2QRDkdPxTFxtS2IDSU9ohvjBa3+sgYoKfNbDIrC/8+OpDPLJHkl+Fda2CaXLicTWg wA/yATTXCMV2eAQ5zS5UAEWHxEaqNgrUnG9gV1J/cM6+JV/FDUYmbgcceCgYO9ulJ4OC usNe8O48DT5VWyplw9vxjKwdot9kuwaFCsravzXqbisb77loAN3DDc6FrOS+qOIS+ARW 6fHx7zI2IGRj1YVvSLeeDz1J/k9SmbuJ8xTPATlyl24T9zzMgxCukm9nAUOT91BUNPgS wjGbuuo3yW0XAbqFT1TyjQY44ljuETgrMyNqzrUElR2Csh3BydRgJBAO8aY8S4z9Ms9O yIfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747957066; x=1748561866; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=Yvw3d371CPvGh9bouzujsmuFtM9ubEnaNp4RJbDym00=; b=OBDZEZYQmDwOLRKSZwNsW28DSwjvDWyV23UkvWVi5CECxBh5Kh0Ai8yh8CjZfO+ckX I7BrBek0Jgz0SCh20BLmzC9U7YwkMsjKCgao1UXtkSir2f9rwfDdb9b34QIpWb9lUUdI HHyFvtl5VOZ/fcewLQACFfBkKsJ1T4yw9o+UnAI0CvLki+Eeoflpy6vIXGrABDq2Lp0q TyZjgdWP1Oye03l7K2viLYl7S5PgazOuPNuMmMQBRymH/1HeHTI2IiUTFBowy9rxb6Fy 7e6GhDoxa7wKe3YkmWU6FQKDKckEvMz+0pIuxT6C5Bar+nijalJXQ245Br/CaAmWvGrf 94OQ== X-Gm-Message-State: AOJu0YwbF9hylmpQvSHErjwiN2RGnrUrZkIbE30FGUU2kOAjegBZaT0M gfXLepX7GC3J45x28b5awQewEXAK1f7GtJCXdqWbTSNNJfRNjzfWmVqZVsE4Nl43YJJbzr2IXzI P+GglYw== X-Google-Smtp-Source: AGHT+IGuDScws7kpg/friq+b634gqXx+BwD3tk0suRuGvu3JjtUiXgJphwYoBlYj4PDUcCLFI+H5bjTOQbU= X-Received: from pjbsp5.prod.google.com ([2002:a17:90b:52c5:b0:2ff:5516:6add]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3d44:b0:30e:7d09:2a7 with SMTP id 98e67ed59e1d1-30e83107ed7mr42352428a91.14.1747957066514; Thu, 22 May 2025 16:37:46 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 22 May 2025 16:37:27 -0700 In-Reply-To: <20250522233733.3176144-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250522233733.3176144-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1151.ga128411c76-goog Message-ID: <20250522233733.3176144-4-seanjc@google.com> Subject: [PATCH v3 3/8] x86, lib: Add WBNOINVD helper functions From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, Kevin Loughlin , Tom Lendacky , Kai Huang , Ingo Molnar , Zheyun Shen , Mingwei Zhang , Francesco Lavra Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kevin Loughlin In line with WBINVD usage, add WBONINVD helper functions. Explicitly fall back to WBINVD (via alternative()) if WBNOINVD isn't supported even though the instruction itself is backwards compatible (WBNOINVD is WBINVD with an ignored REP prefix), so that disabling X86_FEATURE_WBNOINVD behaves as one would expect, e.g. in case there's a hardware issue that affects WBNOINVD. Opportunsitically add comments explaining the architectural behavior of WBINVD and WBNOINVD, and to provide hints and pointers to uarch-specific behavior. Note, alternative() ensures compatibility with early boot code as needed. Signed-off-by: Kevin Loughlin Reviewed-by: Tom Lendacky Reviewed-by: Kai Huang Acked-by: Ingo Molnar Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson --- arch/x86/include/asm/smp.h | 6 ++++++ arch/x86/include/asm/special_insns.h | 32 +++++++++++++++++++++++++++- arch/x86/lib/cache-smp.c | 11 ++++++++++ 3 files changed, 48 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/smp.h b/arch/x86/include/asm/smp.h index 028f126018c9..e08f1ae25401 100644 --- a/arch/x86/include/asm/smp.h +++ b/arch/x86/include/asm/smp.h @@ -113,6 +113,7 @@ void native_play_dead(void); void play_dead_common(void); void wbinvd_on_cpu(int cpu); void wbinvd_on_all_cpus(void); +void wbnoinvd_on_all_cpus(void); =20 void smp_kick_mwait_play_dead(void); void __noreturn mwait_play_dead(unsigned int eax_hint); @@ -153,6 +154,11 @@ static inline void wbinvd_on_all_cpus(void) wbinvd(); } =20 +static inline void wbnoinvd_on_all_cpus(void) +{ + wbnoinvd(); +} + static inline struct cpumask *cpu_llc_shared_mask(int cpu) { return (struct cpumask *)cpumask_of(0); diff --git a/arch/x86/include/asm/special_insns.h b/arch/x86/include/asm/sp= ecial_insns.h index 6266d6b9e0b8..f2240c4ac0ea 100644 --- a/arch/x86/include/asm/special_insns.h +++ b/arch/x86/include/asm/special_insns.h @@ -115,9 +115,39 @@ static inline void wrpkru(u32 pkru) } #endif =20 +/* + * Write back all modified lines in all levels of cache associated with th= is + * logical processor to main memory, and then invalidate all caches. Depe= nding + * on the micro-architecture, WBINVD (and WBNOINVD below) may or may not a= ffect + * lower level caches associated with another logical processor that share= s any + * level of this processor=E2=80=99s cache hierarchy. + * + * Note, AMD CPUs enumerate the behavior or WB{NO}{INVD} with respect to o= ther + * logical, non-originating processors in CPUID 0x8000001D.EAX[N:0]. + */ static __always_inline void wbinvd(void) { - asm volatile("wbinvd": : :"memory"); + asm volatile("wbinvd" : : : "memory"); +} + +/* Instruction encoding provided for binutils backwards compatibility. */ +#define ASM_WBNOINVD _ASM_BYTES(0xf3,0x0f,0x09) + +/* + * Write back all modified lines in all levels of cache associated with th= is + * logical processor to main memory, but do NOT explicitly invalidate cach= es, + * i.e. leave all/most cache lines in the hierarchy in non-modified state. + */ +static __always_inline void wbnoinvd(void) +{ + /* + * Explicitly encode WBINVD if X86_FEATURE_WBNOINVD is unavailable even + * though WBNOINVD is backwards compatible (it's simply WBINVD with an + * ignored REP prefix), to guarantee that WBNOINVD isn't used if it + * needs to be avoided for any reason. For all supported usage in the + * kernel, WBINVD is functionally a superset of WBNOINVD. + */ + alternative("wbinvd", ASM_WBNOINVD, X86_FEATURE_WBNOINVD); } =20 static inline unsigned long __read_cr4(void) diff --git a/arch/x86/lib/cache-smp.c b/arch/x86/lib/cache-smp.c index 079c3f3cd32c..1789db5d8825 100644 --- a/arch/x86/lib/cache-smp.c +++ b/arch/x86/lib/cache-smp.c @@ -19,3 +19,14 @@ void wbinvd_on_all_cpus(void) on_each_cpu(__wbinvd, NULL, 1); } EXPORT_SYMBOL(wbinvd_on_all_cpus); + +static void __wbnoinvd(void *dummy) +{ + wbnoinvd(); +} + +void wbnoinvd_on_all_cpus(void) +{ + on_each_cpu(__wbnoinvd, NULL, 1); +} +EXPORT_SYMBOL(wbnoinvd_on_all_cpus); --=20 2.49.0.1151.ga128411c76-goog