From nobody Sun Dec 14 06:40:54 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 66AE12494F9 for ; Thu, 22 May 2025 23:37:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747957065; cv=none; b=o7COW25DnuH0EDdSebNNgZMDlRo2UlodQ0/lUZuS1SMQXY6FMUEEUT6EqMrPPwZ1EInQ6dlz4ZeecJQ2xx0h1ozW1RMpBay+LgyGDdOeC9TRAG1rTevU+5DnFjn03BI1/VCifZe43cafhctDdZIrwSh8dvqruivDNnJ5NEJ6mUQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747957065; c=relaxed/simple; bh=7HznakEWAX3QxiOzowEE528i6qDkLEzzGNrvSoNsN9c=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=F/2rPVTXRLKi0XWnUqWz0R3nXfpYPaq47siOEqqf6fMfKxJr6EUsBqJogGHGAnzUfb3guWwAm/JHtn/42XzSJh0d9YaQwEjluPl6Vgoe9LwrRFrNzu4aWggos99JhQTbzzeaI5ORTL+Y12gwl33tovkx1mbwZEzq8/RupwfY7Vo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=SkXKFSYA; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="SkXKFSYA" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-30ebf91d150so6962338a91.0 for ; Thu, 22 May 2025 16:37:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747957063; x=1748561863; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=uPxUrn14vuCfCMnKXPuJKpCi+RnH/dlPI+aSAjTgJIQ=; b=SkXKFSYApdsPjWpIPz5B9XNLdQ+eWKcQiy0NvTTO3iNZqGZGZopPabFBd65HjlaWYB LOUSA3Uy0CijsNzYm9CcHH10rWoaK4Ge9UckmZndNmFjBBC39AU27wXZq0pdKrssiBuO MRWCydZQFtPJVR5CHBsIwGdOBRVAo5ZOezsoTed+IKQ3oEjBfzIi0iHgCjrC4mS3jdXn eFlJmq+UsK/oxPEMiQh9C6TQds++hzaLO3vYiDw4JZ7EpPlYFjen2Ohre3l2CGsUO0Pd B/0l+RIg+qaqijPZ79zzJ0EO8uAcirnw4CoLiN1Ms120Uo627BKikUL0tT4c4/fkth76 E1qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747957064; x=1748561864; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=uPxUrn14vuCfCMnKXPuJKpCi+RnH/dlPI+aSAjTgJIQ=; b=B66IwaScjcvbEvSzriLFgJIgZC2mK2RSUDku4CFVXw4pzNdvQf+ZU2yhT/t4GHI28v dvbVVXlmhz65f34RKwxbN9uavlF3RC3zWU2DmpCkW3j6sVxvf39uLpw2xBqH/RJXaSJr 7HlKxSB1YiQFgOvQjkpsSYLlXFXUvPbPz5ipNAr8F58JWbX2xfSCFOn7TRLKbRFzj36t Ufb2i12Q+rUoI0RJzUrTWMHetWeISNW5oVMFo2ngnxzqrdI1CJxidI0aB/50DrIveCSL Kdj8POfwxgO4vETA0rFDe3rHGB792DjXByNk7mdXwvodmRJmk0tZqDd8bN7t3chwbCm1 cXMg== X-Gm-Message-State: AOJu0YxDrbRbf9dvTgqX5z8no7W08ajLBtXXT85gzfzUGnCqbFpi22Gv XtSmG7rIHIov0ndB36hIYD1zwqEqqu58fvGagnbFnFb32vpIuDkBsbRo7m1Pz8D4bJrPRcHp2+T 4Xg6PIw== X-Google-Smtp-Source: AGHT+IGI69KyPY1At0pbvO9taKFOli2nLHw8tUa/zK4co4eocqDHMTFbGsbKlwZf9ojX+OImaEWTLGTboac= X-Received: from pjbsw15.prod.google.com ([2002:a17:90b:2c8f:b0:301:4260:4d23]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2c8c:b0:2fe:8c22:48b0 with SMTP id 98e67ed59e1d1-30e7d555a3bmr38071711a91.15.1747957063457; Thu, 22 May 2025 16:37:43 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 22 May 2025 16:37:25 -0700 In-Reply-To: <20250522233733.3176144-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250522233733.3176144-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1151.ga128411c76-goog Message-ID: <20250522233733.3176144-2-seanjc@google.com> Subject: [PATCH v3 1/8] drm/gpu: Remove dead checks on wbinvd_on_all_cpus()'s return value From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, Kevin Loughlin , Tom Lendacky , Kai Huang , Ingo Molnar , Zheyun Shen , Mingwei Zhang , Francesco Lavra Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove the checks and associated pr_err() on wbinvd_on_all_cpus() failure, as the helper has unconditionally returned 0/success since commit caa759323c73 ("smp: Remove smp_call_function() and on_each_cpu() return values"). Signed-off-by: Sean Christopherson Acked-by: Simona Vetter --- drivers/gpu/drm/drm_cache.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/drm/drm_cache.c b/drivers/gpu/drm/drm_cache.c index 7051c9c909c2..ea1d2d5d2c66 100644 --- a/drivers/gpu/drm/drm_cache.c +++ b/drivers/gpu/drm/drm_cache.c @@ -93,8 +93,7 @@ drm_clflush_pages(struct page *pages[], unsigned long num= _pages) return; } =20 - if (wbinvd_on_all_cpus()) - pr_err("Timed out waiting for cache flush\n"); + wbinvd_on_all_cpus(); =20 #elif defined(__powerpc__) unsigned long i; @@ -139,8 +138,7 @@ drm_clflush_sg(struct sg_table *st) return; } =20 - if (wbinvd_on_all_cpus()) - pr_err("Timed out waiting for cache flush\n"); + wbinvd_on_all_cpus(); #else WARN_ONCE(1, "Architecture has no drm_cache.c support\n"); #endif @@ -172,8 +170,7 @@ drm_clflush_virt_range(void *addr, unsigned long length) return; } =20 - if (wbinvd_on_all_cpus()) - pr_err("Timed out waiting for cache flush\n"); + wbinvd_on_all_cpus(); #else WARN_ONCE(1, "Architecture has no drm_cache.c support\n"); #endif --=20 2.49.0.1151.ga128411c76-goog From nobody Sun Dec 14 06:40:54 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 70408271A74 for ; Thu, 22 May 2025 23:37:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747957067; cv=none; b=ZHf4MQQSlJtvOk2o/JyEnFmQoTRw0eiBN41NzeUIRxRyvesPoXw05otSzX2LMgcUV++lOPa3TQFXaB0taJGUJHNXJwNheBUU4+iX+4mqJoNIFGI9ObqP3iNMRYbcBih4y+npaNbgIwWzO6grFPWueZieH18gxmWyvgMD6olKLZE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747957067; c=relaxed/simple; bh=Cf+rTiT37xWtJra2of8I2bzpA8nA/Gdxil8f28LKGj8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=E+mVhOkkA9jnZtIjG2W3ltfbRMD07ePMPGhP/eVmfdJzbU/LjZ/zEHH5lzzqjZOSnLkrkWI7aSFvB0sLB2lYQd424mPEMQZSC8IKYKPDBK2vGC9slCfjFvWQotMooVBJZpbwmaU3MO4h9rUzGs6lRgaV545NNxwfWE30q41ERQM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=k24CJp0P; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="k24CJp0P" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-30e9b2e7a34so4903326a91.0 for ; Thu, 22 May 2025 16:37:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747957065; x=1748561865; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=bZZENoThSq2WPUG3cA9ljTtD8JyVh5aqU2sA4XXQXGM=; b=k24CJp0PVCjvkhOg15+EM8LEYadnBZg4C6kkBB2UqoyY/k+6yZ6KJQdmnfk0Hwi2cY 9AUQAuhkEpRCcmmbuVpgSJdqb1VR00ciLK85kQ5qLvTNRj4PjCOKaKdW585KeHz/ogxt CQm5W+xxVgnOMMkFmYPJRWZ42/RJ4G9eZXnu+mlPFMe/HfSFC0iSKI3FO/upHnRxhZEy ouPITCUHD42WF/O7Ia4cJjeDFqLJeM6X3CR2KBw0+T/dzc+skLPAyoPtHCCSeHSSLQ0a Ta4T18RBYOSIE5Hz63/qU9uvOFto+5q8mMSaB3n+3BLi3ASChk2hIXFFAnNGcPhNsC1m Vcjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747957065; x=1748561865; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=bZZENoThSq2WPUG3cA9ljTtD8JyVh5aqU2sA4XXQXGM=; b=T7Hz8lMJ/2I/Cj3gbuT6xHFjjCPIoAMHaKWa1TwxG9GDjKdTLzARXnKyPjF5Sbreut ey9vEQOaB3GsT9R/1MoRP9cXS/ComuHSg1ShThhiCyNbVIL7iKNPWUtSI7F8OFlu6sMh bM+EDUCLtCQ6cRd/Rejhnh+wxyGmQIZLuNMW8YWRiUJsYcaShjUxkuBJU82LQEP3P7Nn DlrXxsrY3rxsitYeD/W/HmWk2Ki5z1CTqKNVcNQV8CQWSI0OzdPVgM5m85oaOjRAUXJy m1ItFixz0TodZtgTl8g5vT8tjvmxD4G+m3fwWjq5BLkDE+MWMb45IAZZ3yL/WVSMeNOV v6uA== X-Gm-Message-State: AOJu0Yy5JSBdep2TbIULOKW850XZsHuR91zFoNNAXZzLjSltNqu7xaJJ b6wtf+pGa+NrBVgeybPXx7VCXZlnVCn65Z41eUcUb6L3R5B6Bcp+48MRxRVMgoeyVSjDjK6q1bR xbrrqbg== X-Google-Smtp-Source: AGHT+IFExGJrdJE3NdFkvdN8xCuUi0nMv/3Dpb1wQbh8E9OJ+IjqmD6ZkqC1r2VP7vreaPIjC/XPdl8z9gU= X-Received: from pjbok18.prod.google.com ([2002:a17:90b:1d52:b0:30e:7f04:f467]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:d44e:b0:2ee:d63f:d8f with SMTP id 98e67ed59e1d1-310e96c9598mr1434238a91.13.1747957065045; Thu, 22 May 2025 16:37:45 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 22 May 2025 16:37:26 -0700 In-Reply-To: <20250522233733.3176144-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250522233733.3176144-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1151.ga128411c76-goog Message-ID: <20250522233733.3176144-3-seanjc@google.com> Subject: [PATCH v3 2/8] x86, lib: Drop the unused return value from wbinvd_on_all_cpus() From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, Kevin Loughlin , Tom Lendacky , Kai Huang , Ingo Molnar , Zheyun Shen , Mingwei Zhang , Francesco Lavra Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Drop wbinvd_on_all_cpus()'s return value; both the "real" version and the stub always return '0', and none of the callers check the return. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/smp.h | 5 ++--- arch/x86/lib/cache-smp.c | 3 +-- 2 files changed, 3 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/smp.h b/arch/x86/include/asm/smp.h index 0c1c68039d6f..028f126018c9 100644 --- a/arch/x86/include/asm/smp.h +++ b/arch/x86/include/asm/smp.h @@ -112,7 +112,7 @@ void __noreturn hlt_play_dead(void); void native_play_dead(void); void play_dead_common(void); void wbinvd_on_cpu(int cpu); -int wbinvd_on_all_cpus(void); +void wbinvd_on_all_cpus(void); =20 void smp_kick_mwait_play_dead(void); void __noreturn mwait_play_dead(unsigned int eax_hint); @@ -148,10 +148,9 @@ static inline struct cpumask *cpu_l2c_shared_mask(int = cpu) =20 #else /* !CONFIG_SMP */ #define wbinvd_on_cpu(cpu) wbinvd() -static inline int wbinvd_on_all_cpus(void) +static inline void wbinvd_on_all_cpus(void) { wbinvd(); - return 0; } =20 static inline struct cpumask *cpu_llc_shared_mask(int cpu) diff --git a/arch/x86/lib/cache-smp.c b/arch/x86/lib/cache-smp.c index 7af743bd3b13..079c3f3cd32c 100644 --- a/arch/x86/lib/cache-smp.c +++ b/arch/x86/lib/cache-smp.c @@ -14,9 +14,8 @@ void wbinvd_on_cpu(int cpu) } EXPORT_SYMBOL(wbinvd_on_cpu); =20 -int wbinvd_on_all_cpus(void) +void wbinvd_on_all_cpus(void) { on_each_cpu(__wbinvd, NULL, 1); - return 0; } EXPORT_SYMBOL(wbinvd_on_all_cpus); --=20 2.49.0.1151.ga128411c76-goog From nobody Sun Dec 14 06:40:54 2025 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2CD74271A85 for ; Thu, 22 May 2025 23:37:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747957068; cv=none; b=ZeMA2A54YGGwUhP6GLegS3Wn34a+JJrcSgIAwzjw165J5XtqMTbHuxouXVTqYOEJ7L2LgxETPULOVgOoPbNOj8gIV4HW1RWz+G6O+HNwQ1Ai8nyY246g0Q1+rpM/oHTOp6mwq5wUAy+S5Rq6qalAGxFKjb7AhB20Gj8rWsIP6mk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747957068; c=relaxed/simple; bh=eNvNDKoeNT6JOGIaV2NQv10Yua3xIPhgPTUOOfxorRI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=dVPvq9u4MdnYGbtBc0qJwCypINCnqGb8kQZ+hTe3TGbNWN+IS1HM9/2MiZAHr9qcwQkIA8bXuCzpTqJ45RJ/n7QGkDbQpJt7TIvIxTc+5H+CwHrIhWqXnLtgfy4u0rIZouUYoiwCmvBwkGrSckOCJVj0H5GIjxEdBmMomuTUHBY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=p2QRDkdP; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="p2QRDkdP" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-b26e33ae9d5so8903484a12.1 for ; Thu, 22 May 2025 16:37:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747957066; x=1748561866; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=Yvw3d371CPvGh9bouzujsmuFtM9ubEnaNp4RJbDym00=; b=p2QRDkdPxTFxtS2IDSU9ohvjBa3+sgYoKfNbDIrC/8+OpDPLJHkl+Fda2CaXLicTWg wA/yATTXCMV2eAQ5zS5UAEWHxEaqNgrUnG9gV1J/cM6+JV/FDUYmbgcceCgYO9ulJ4OC usNe8O48DT5VWyplw9vxjKwdot9kuwaFCsravzXqbisb77loAN3DDc6FrOS+qOIS+ARW 6fHx7zI2IGRj1YVvSLeeDz1J/k9SmbuJ8xTPATlyl24T9zzMgxCukm9nAUOT91BUNPgS wjGbuuo3yW0XAbqFT1TyjQY44ljuETgrMyNqzrUElR2Csh3BydRgJBAO8aY8S4z9Ms9O yIfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747957066; x=1748561866; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=Yvw3d371CPvGh9bouzujsmuFtM9ubEnaNp4RJbDym00=; b=OBDZEZYQmDwOLRKSZwNsW28DSwjvDWyV23UkvWVi5CECxBh5Kh0Ai8yh8CjZfO+ckX I7BrBek0Jgz0SCh20BLmzC9U7YwkMsjKCgao1UXtkSir2f9rwfDdb9b34QIpWb9lUUdI HHyFvtl5VOZ/fcewLQACFfBkKsJ1T4yw9o+UnAI0CvLki+Eeoflpy6vIXGrABDq2Lp0q TyZjgdWP1Oye03l7K2viLYl7S5PgazOuPNuMmMQBRymH/1HeHTI2IiUTFBowy9rxb6Fy 7e6GhDoxa7wKe3YkmWU6FQKDKckEvMz+0pIuxT6C5Bar+nijalJXQ245Br/CaAmWvGrf 94OQ== X-Gm-Message-State: AOJu0YwbF9hylmpQvSHErjwiN2RGnrUrZkIbE30FGUU2kOAjegBZaT0M gfXLepX7GC3J45x28b5awQewEXAK1f7GtJCXdqWbTSNNJfRNjzfWmVqZVsE4Nl43YJJbzr2IXzI P+GglYw== X-Google-Smtp-Source: AGHT+IGuDScws7kpg/friq+b634gqXx+BwD3tk0suRuGvu3JjtUiXgJphwYoBlYj4PDUcCLFI+H5bjTOQbU= X-Received: from pjbsp5.prod.google.com ([2002:a17:90b:52c5:b0:2ff:5516:6add]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3d44:b0:30e:7d09:2a7 with SMTP id 98e67ed59e1d1-30e83107ed7mr42352428a91.14.1747957066514; Thu, 22 May 2025 16:37:46 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 22 May 2025 16:37:27 -0700 In-Reply-To: <20250522233733.3176144-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250522233733.3176144-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1151.ga128411c76-goog Message-ID: <20250522233733.3176144-4-seanjc@google.com> Subject: [PATCH v3 3/8] x86, lib: Add WBNOINVD helper functions From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, Kevin Loughlin , Tom Lendacky , Kai Huang , Ingo Molnar , Zheyun Shen , Mingwei Zhang , Francesco Lavra Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kevin Loughlin In line with WBINVD usage, add WBONINVD helper functions. Explicitly fall back to WBINVD (via alternative()) if WBNOINVD isn't supported even though the instruction itself is backwards compatible (WBNOINVD is WBINVD with an ignored REP prefix), so that disabling X86_FEATURE_WBNOINVD behaves as one would expect, e.g. in case there's a hardware issue that affects WBNOINVD. Opportunsitically add comments explaining the architectural behavior of WBINVD and WBNOINVD, and to provide hints and pointers to uarch-specific behavior. Note, alternative() ensures compatibility with early boot code as needed. Signed-off-by: Kevin Loughlin Reviewed-by: Tom Lendacky Reviewed-by: Kai Huang Acked-by: Ingo Molnar Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson --- arch/x86/include/asm/smp.h | 6 ++++++ arch/x86/include/asm/special_insns.h | 32 +++++++++++++++++++++++++++- arch/x86/lib/cache-smp.c | 11 ++++++++++ 3 files changed, 48 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/smp.h b/arch/x86/include/asm/smp.h index 028f126018c9..e08f1ae25401 100644 --- a/arch/x86/include/asm/smp.h +++ b/arch/x86/include/asm/smp.h @@ -113,6 +113,7 @@ void native_play_dead(void); void play_dead_common(void); void wbinvd_on_cpu(int cpu); void wbinvd_on_all_cpus(void); +void wbnoinvd_on_all_cpus(void); =20 void smp_kick_mwait_play_dead(void); void __noreturn mwait_play_dead(unsigned int eax_hint); @@ -153,6 +154,11 @@ static inline void wbinvd_on_all_cpus(void) wbinvd(); } =20 +static inline void wbnoinvd_on_all_cpus(void) +{ + wbnoinvd(); +} + static inline struct cpumask *cpu_llc_shared_mask(int cpu) { return (struct cpumask *)cpumask_of(0); diff --git a/arch/x86/include/asm/special_insns.h b/arch/x86/include/asm/sp= ecial_insns.h index 6266d6b9e0b8..f2240c4ac0ea 100644 --- a/arch/x86/include/asm/special_insns.h +++ b/arch/x86/include/asm/special_insns.h @@ -115,9 +115,39 @@ static inline void wrpkru(u32 pkru) } #endif =20 +/* + * Write back all modified lines in all levels of cache associated with th= is + * logical processor to main memory, and then invalidate all caches. Depe= nding + * on the micro-architecture, WBINVD (and WBNOINVD below) may or may not a= ffect + * lower level caches associated with another logical processor that share= s any + * level of this processor=E2=80=99s cache hierarchy. + * + * Note, AMD CPUs enumerate the behavior or WB{NO}{INVD} with respect to o= ther + * logical, non-originating processors in CPUID 0x8000001D.EAX[N:0]. + */ static __always_inline void wbinvd(void) { - asm volatile("wbinvd": : :"memory"); + asm volatile("wbinvd" : : : "memory"); +} + +/* Instruction encoding provided for binutils backwards compatibility. */ +#define ASM_WBNOINVD _ASM_BYTES(0xf3,0x0f,0x09) + +/* + * Write back all modified lines in all levels of cache associated with th= is + * logical processor to main memory, but do NOT explicitly invalidate cach= es, + * i.e. leave all/most cache lines in the hierarchy in non-modified state. + */ +static __always_inline void wbnoinvd(void) +{ + /* + * Explicitly encode WBINVD if X86_FEATURE_WBNOINVD is unavailable even + * though WBNOINVD is backwards compatible (it's simply WBINVD with an + * ignored REP prefix), to guarantee that WBNOINVD isn't used if it + * needs to be avoided for any reason. For all supported usage in the + * kernel, WBINVD is functionally a superset of WBNOINVD. + */ + alternative("wbinvd", ASM_WBNOINVD, X86_FEATURE_WBNOINVD); } =20 static inline unsigned long __read_cr4(void) diff --git a/arch/x86/lib/cache-smp.c b/arch/x86/lib/cache-smp.c index 079c3f3cd32c..1789db5d8825 100644 --- a/arch/x86/lib/cache-smp.c +++ b/arch/x86/lib/cache-smp.c @@ -19,3 +19,14 @@ void wbinvd_on_all_cpus(void) on_each_cpu(__wbinvd, NULL, 1); } EXPORT_SYMBOL(wbinvd_on_all_cpus); + +static void __wbnoinvd(void *dummy) +{ + wbnoinvd(); +} + +void wbnoinvd_on_all_cpus(void) +{ + on_each_cpu(__wbnoinvd, NULL, 1); +} +EXPORT_SYMBOL(wbnoinvd_on_all_cpus); --=20 2.49.0.1151.ga128411c76-goog From nobody Sun Dec 14 06:40:54 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 06B5D2BF3D9 for ; Thu, 22 May 2025 23:37:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747957070; cv=none; b=umpN173hf8uStDQCajxzxO1hNa0jMcItnphyat3Jby04epKWCbpxum2fx/GGXYXm2UBMxwAofSNC/8PXK78vaKwU4hsqmtFiDq9+jv+HEbhZV7di1Rc5RJdggcL9k7dnl760YjcBuj47wtVDnJW6obXfuIGa4Kzq74Y8ZxLOB9w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747957070; c=relaxed/simple; bh=LdrFxL1TgLnHv60eOYGpxumD1HywpJAHkaMLkE2on1g=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=koA3SpTq2gLXvxoCTeEEPgNACoToFWPtDZ9OfWZ1doDq56TgJ6zHNl74E15cKwZVL/ABb1M6L+SNCuq+VcEJvt1lMyFTsFuKJVISojuWSp2EDawIsRD4hqogdbZ/fCc7ASGrhPopbWj4gcjBBl2sMvGsrUcmZN6d43f3E2bFHLo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=xRAYAsoC; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="xRAYAsoC" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-310a0668968so2191921a91.0 for ; Thu, 22 May 2025 16:37:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747957068; x=1748561868; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=He1Fb8XWIhOrf7jG2epQVScv14uLcqGVLp5FewijIAc=; b=xRAYAsoCsf3MW5Sgw1xt4TXOUixbzuOwUv+4EXc05iRdWUqjAoOFrVIaSJ5BOyhBmq D37Q39Wtm0/9Ila9KKJdowyA1wSCaCqb7cklhHEPVpCsPDcmAtJ0Ebo1yd7SUebfqTgI f3PGK2Mb1VHjQENbDES3+QaeggsZ/vF2iQTCN2s7tAP87Nx0E2yvBqshJ1/pPQh0gg5Y MsXMS6uimtdBStkLxwsXh8BMnZXd+ouBtSUvy9vWHK0c7hFhDzsc9bQMJR6kmDhBaECZ X+V86FBW8MrnCc0bdMyx74SX4rz21Pr4/GgragoiBfBdtkENcLTd13CIF77BT5ldlHFd glRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747957068; x=1748561868; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=He1Fb8XWIhOrf7jG2epQVScv14uLcqGVLp5FewijIAc=; b=f/omT97LXp3Oc28m04N+s1tWED7oJSQpLEcNsFoCg3Ql+VU0mLJGZAt1i8YomP5PGB 26QBwRHcL+EOhBS5IlfYxK19SekZIRoCbgfdLJqyPYY1kSAZUdU5nYKVnGBl5lRP7tfM RbXobGTM59+HKTbRMXKlrGY/dile+FbvztC8dmTBX3wy7X8oIn/Fbsc3l1peVkixxnU6 FJCc8F7HWI9Eg20zgrdB3uo/INjbmdDEoETAfV4UdvwtxPLsnuTQ0dr4AfYLGDVDKUzW WXs2Y1MbaiKWNbktDZ50GO2UzhCiQPcanGYNxkxDafvJlws3zxD8uu5XMm+z+dLlFY/P 2ptw== X-Gm-Message-State: AOJu0YzpZoCM8/5/NNuXhEEPjmZ5GbUbacTcKInPYMwYOYMjsqWrTTFx /khVe8xUOCuZPhzJvJs1T2oq7pXmLnZxQGpXTMVHQwvyevzeaQ5fk+dI1S7jFs8Zd7YmffJca2C Nxwz7Ng== X-Google-Smtp-Source: AGHT+IEY13ev1fFg8g1+XuY0Q3OScdgExPi1jAv5crcNvfE+k43onG6hdR5YtXe6pz4sFc3h4oa0wS9XN+4= X-Received: from pjbdj5.prod.google.com ([2002:a17:90a:d2c5:b0:310:89bf:401]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5750:b0:30e:3737:7c71 with SMTP id 98e67ed59e1d1-30e7d57e4e6mr38834925a91.20.1747957068219; Thu, 22 May 2025 16:37:48 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 22 May 2025 16:37:28 -0700 In-Reply-To: <20250522233733.3176144-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250522233733.3176144-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1151.ga128411c76-goog Message-ID: <20250522233733.3176144-5-seanjc@google.com> Subject: [PATCH v3 4/8] x86, lib: Add wbinvd and wbnoinvd helpers to target multiple CPUs From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, Kevin Loughlin , Tom Lendacky , Kai Huang , Ingo Molnar , Zheyun Shen , Mingwei Zhang , Francesco Lavra Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Zheyun Shen Extract KVM's open-coded calls to do writeback caches on multiple CPUs to common library helpers for both WBINVD and WBNOINVD (KVM will use both). Put the onus on the caller to check for a non-empty mask to simplify the SMP=3Dn implementation, e.g. so that it doesn't need to check that the one and only CPU in the system is present in the mask. Signed-off-by: Zheyun Shen Reviewed-by: Tom Lendacky Link: https://lore.kernel.org/r/20250128015345.7929-2-szy0127@sjtu.edu.cn [sean: move to lib, add SMP=3Dn helpers, clarify usage] Acked-by: Kai Huang Signed-off-by: Sean Christopherson --- arch/x86/include/asm/smp.h | 12 ++++++++++++ arch/x86/kvm/x86.c | 3 +-- arch/x86/lib/cache-smp.c | 12 ++++++++++++ 3 files changed, 25 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/smp.h b/arch/x86/include/asm/smp.h index e08f1ae25401..22bfebe6776d 100644 --- a/arch/x86/include/asm/smp.h +++ b/arch/x86/include/asm/smp.h @@ -113,7 +113,9 @@ void native_play_dead(void); void play_dead_common(void); void wbinvd_on_cpu(int cpu); void wbinvd_on_all_cpus(void); +void wbinvd_on_cpus_mask(struct cpumask *cpus); void wbnoinvd_on_all_cpus(void); +void wbnoinvd_on_cpus_mask(struct cpumask *cpus); =20 void smp_kick_mwait_play_dead(void); void __noreturn mwait_play_dead(unsigned int eax_hint); @@ -154,11 +156,21 @@ static inline void wbinvd_on_all_cpus(void) wbinvd(); } =20 +static inline void wbinvd_on_cpus_mask(struct cpumask *cpus) +{ + wbinvd(); +} + static inline void wbnoinvd_on_all_cpus(void) { wbnoinvd(); } =20 +static inline void wbnoinvd_on_cpus_mask(struct cpumask *cpus) +{ + wbnoinvd(); +} + static inline struct cpumask *cpu_llc_shared_mask(int cpu) { return (struct cpumask *)cpumask_of(0); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index be7bb6d20129..1d0e9180148d 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8245,8 +8245,7 @@ static int kvm_emulate_wbinvd_noskip(struct kvm_vcpu = *vcpu) int cpu =3D get_cpu(); =20 cpumask_set_cpu(cpu, vcpu->arch.wbinvd_dirty_mask); - on_each_cpu_mask(vcpu->arch.wbinvd_dirty_mask, - wbinvd_ipi, NULL, 1); + wbinvd_on_cpus_mask(vcpu->arch.wbinvd_dirty_mask); put_cpu(); cpumask_clear(vcpu->arch.wbinvd_dirty_mask); } else diff --git a/arch/x86/lib/cache-smp.c b/arch/x86/lib/cache-smp.c index 1789db5d8825..b45e8baa69bb 100644 --- a/arch/x86/lib/cache-smp.c +++ b/arch/x86/lib/cache-smp.c @@ -20,6 +20,12 @@ void wbinvd_on_all_cpus(void) } EXPORT_SYMBOL(wbinvd_on_all_cpus); =20 +void wbinvd_on_cpus_mask(struct cpumask *cpus) +{ + on_each_cpu_mask(cpus, __wbinvd, NULL, 1); +} +EXPORT_SYMBOL_GPL(wbinvd_on_cpus_mask); + static void __wbnoinvd(void *dummy) { wbnoinvd(); @@ -30,3 +36,9 @@ void wbnoinvd_on_all_cpus(void) on_each_cpu(__wbnoinvd, NULL, 1); } EXPORT_SYMBOL(wbnoinvd_on_all_cpus); + +void wbnoinvd_on_cpus_mask(struct cpumask *cpus) +{ + on_each_cpu_mask(cpus, __wbnoinvd, NULL, 1); +} +EXPORT_SYMBOL_GPL(wbnoinvd_on_cpus_mask); --=20 2.49.0.1151.ga128411c76-goog From nobody Sun Dec 14 06:40:54 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 646272494FB for ; Thu, 22 May 2025 23:37:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747957071; cv=none; b=IKyfLjXrstiX97UtRbtCb7ZXEN2/8t1k8MrtmhN/aBePKg6fkpKPI9WYBrwzaH5Tt9m1p+1UeCzCC4ftNmQ8GTe3aw0FXSX4a8PAgb79wyMnEfC/toJpyv34TELONspob2017lZV5nihOCQq2IZf1Ceh1DEVcM2WhWRQHaAYhX8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747957071; c=relaxed/simple; bh=rKtp5xfoIimuXjCjQBqX07AVfylVXbINV44Xdk84ous=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=K7f+LNg0V27fmfa+gcbeumpdl3Mp0Zruh4cI9YPIyEiLSV22pWEp6dbvAyQKfuDgerWdaU9uQyPiyzHAfQNFyT3N5CdhX8PZCQEXeXYBlmBnqZKB/q6dvjhVecHIH2JuU5x5PHofrYFXR7g4bp/F2DdgPFhIe/pMXfrFpZaxfjw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=QeE7hTOb; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="QeE7hTOb" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-30ec5cc994eso4767408a91.2 for ; Thu, 22 May 2025 16:37:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747957070; x=1748561870; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=DbY8RCgDJTqdxQLDkHfni+7ks/LJfloWj75h6P2Hb4U=; b=QeE7hTOb64D9guwsXs+QkxPqYqYyi4tckCb6dBIQCy/QwDk3F3zWNriozymP22c87p esHTVkdLqxi5iK1L+g5FlfwBhstHQiQulee1DtNVui1J9q5F19l170gzDd4gIwfAYtS0 qpqI21bdXIJ5BQvfGzsq5EU6/SVI8EWAr9mKea1SOxdbPHnUFjQ+crwtSHmXyufQtePA 0WXfyxc8QD1l1g16s3Vkb1rVR6WjChtrPNYAcOiNNwPVSW/khx6L34+P7gw9TVs3zyyP xobfBwCARAQSh67DCUGiqXxbF/H8OC4l2stYu4YsdqgB9baPa6bKWqP5T1x8MYLCq8Ke oKMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747957070; x=1748561870; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=DbY8RCgDJTqdxQLDkHfni+7ks/LJfloWj75h6P2Hb4U=; b=LnJGva7pxuKOawzxoVXV4Dviq4N5z18wbmvAbtH/fJoPsLPB/Ri4F3bJGP6tzjY3OP CFHFacn5FSTzhrEJcosum25d26SRChPDnhXqkgLp7kx0xjQFJYzP3O9+Y8uyB3GDMNZV MfZRto6BhsBK4Gw1Ofa3eMIGP1mRVGgzWeIYdSG0Z/AuboOqJN42xY7oCvkLUZAvYY+7 Q2S1xWCfSMVc5GCupIqClvRxuA/saiDVI0zfXYqQFzGJUEYstgpMxdD+4dLhFLU20kwg sG5tn4uk0u9JtifnLtdfQBnHiIH3IEW5HltUsAk3VvKCCY1Jxx5Qe/vnyKYXmKAbAXqO yUDw== X-Gm-Message-State: AOJu0YzB9BCZEQnwJvX5ewUafqQhJv40r3yA3p6Xk1o1C+LMLNxMBIL0 mg9UQdUNVjiT3G61CM8Cv6Suulouh/7o+gO15f1PZdxQ45fkv16jboEjLo/xT0hDTx+6Siw32td 0m0hbnQ== X-Google-Smtp-Source: AGHT+IGjzBQd6j4J8ysyd/Avsyzv8cw/ixuwdZ+zXrYI9Hkoyaw+vaTqL9WLjC62S+SSSB53dg47wGK8xzs= X-Received: from pjbpt3.prod.google.com ([2002:a17:90b:3d03:b0:2ff:852c:ceb8]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4f4e:b0:310:8d9a:eb1c with SMTP id 98e67ed59e1d1-3108d9aeb4fmr12062419a91.21.1747957069850; Thu, 22 May 2025 16:37:49 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 22 May 2025 16:37:29 -0700 In-Reply-To: <20250522233733.3176144-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250522233733.3176144-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1151.ga128411c76-goog Message-ID: <20250522233733.3176144-6-seanjc@google.com> Subject: [PATCH v3 5/8] KVM: x86: Use wbinvd_on_cpu() instead of an open-coded equivalent From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, Kevin Loughlin , Tom Lendacky , Kai Huang , Ingo Molnar , Zheyun Shen , Mingwei Zhang , Francesco Lavra Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use wbinvd_on_cpu() to target a single CPU instead of open-coding an equivalent, and drop KVM's wbinvd_ipi() now that all users have switched to x86 library versions. No functional change intended. Reviewed-by: Tom Lendacky Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 1d0e9180148d..d63a2c27e058 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -4965,11 +4965,6 @@ long kvm_arch_dev_ioctl(struct file *filp, return r; } =20 -static void wbinvd_ipi(void *garbage) -{ - wbinvd(); -} - static bool need_emulate_wbinvd(struct kvm_vcpu *vcpu) { return kvm_arch_has_noncoherent_dma(vcpu->kvm); @@ -4991,8 +4986,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cp= u) if (kvm_x86_call(has_wbinvd_exit)()) cpumask_set_cpu(cpu, vcpu->arch.wbinvd_dirty_mask); else if (vcpu->cpu !=3D -1 && vcpu->cpu !=3D cpu) - smp_call_function_single(vcpu->cpu, - wbinvd_ipi, NULL, 1); + wbinvd_on_cpu(vcpu->cpu); } =20 kvm_x86_call(vcpu_load)(vcpu, cpu); --=20 2.49.0.1151.ga128411c76-goog From nobody Sun Dec 14 06:40:54 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 478D32D29CA for ; Thu, 22 May 2025 23:37:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747957074; cv=none; b=p0482fKa/SV/Ggm8jNKwx6AfdliFNHapJNMuqmzSNB3lIT0AGcHzMysRM9GPL0YSYze/o0Clz7+CtgDdE9gkB6oHtfk9GOtmBnCOgg9nTWGuAayf4nE8JTNzhVWb9KLlIjMeILzfsaQIDe7YvdKsNFbSAMy6tCUg6REAK/EZiAc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747957074; c=relaxed/simple; bh=vRgPvO16SMJb3ieXUqlhCqh8srFYVsp7VBXX2an5x6Q=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=oKecsJpZI7kfhJgY/MbHFc6+OMlJyc89V8UucJ//EB51hpeXbwLDwR/ssv4EY1JMqysfZYHJXpzF63uibBs80q2B9GGbXcaBUq47j0sOomttAoRBl0ip9Uq57aNIKT3mw01t4dTGOVWnhiLFaLnwvDOvNnrjfs3QkOk/9uFXjEQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=h/xb7X3R; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="h/xb7X3R" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-30ebfc28a1eso8041228a91.2 for ; Thu, 22 May 2025 16:37:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747957071; x=1748561871; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=33x8wnTKFUGsCYrD9G+PbnP8XxuzA6yDnz1QlzvR1XE=; b=h/xb7X3RbYuTFn7O6B4aDWF/Tp5v1c9qVDWQ3pur9N1SB1pCSCzbI05MZljUhdky2C ffTq+eLBQiWmAnmbDjEMQV1aDjKJXq1yDOktRNVyhwB8nLebrU3AwRrNg7k3YQsZjaGo 613miveoPTYRa60qavFZRlKK0m+v1NgRfaR0C6NuAh/LIYYB0tyfd6/wo3QfSxD/B0k9 Filuc7VALmcXGVESORq2gTUBwMAS/iOetMr5OYHMjyGX/5NLmTY0+w1fk/JoxrIN+0TU PTyDY6QkuwUGAA+5gJCP9Ro0Vueh1j9W/LGGhbI9qj6m+BLgml0Fg5XMPSOx1rP/Uy+/ Tj/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747957071; x=1748561871; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=33x8wnTKFUGsCYrD9G+PbnP8XxuzA6yDnz1QlzvR1XE=; b=NS4+cKmVyxKKJPzmC8XwhDV3/kgA6QORUav/atukK/GBXffLaqfpgSaoObe3LTrE7C AxqT8TMldP0E+HrBUoDcI2mqB1SwCErNL7PW6naekbFP2cFTNocfnE0Q95yJeuLeQHS6 hTRHgtCM5A5BugjbcOyLafysNaN+hNG0qhyz38i7Z6qeAxOSzEhL8D3IsDVq7qz2+npk 4Da5eJJiTaceuSH0G2h23IjkvKJOIc4zbrLtRja8rAkrbYas87WFXTn3lmRS0WMalw5e GokB7/mYmAGs5eH3zk6f3b1npLpWSCMkEz/DD2Z2v99gDcBIkjQwxDnpU6M0U3skIlem gt8A== X-Gm-Message-State: AOJu0Yymix+djJr4eVu57jPwdArS60gzgJvrezrLIeHMhiIMoT6RILiG wGnWUYMPnQfWdCxdbie4P/Y1qmKpD5/b5c5il9GuzKPfM0sqzCEiFCDFh1cH721r3I6+R+A35kr 1HG0xNA== X-Google-Smtp-Source: AGHT+IErYR2fX0uqEW1Jt/hDr6m5ozY/eqMBmYEYZGX5WIbNApF2YfAu30cVJqx2+qm/KLiZMSEgVzM0R+8= X-Received: from pjp15.prod.google.com ([2002:a17:90b:55cf:b0:2ea:5084:5297]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1dd1:b0:2ff:6167:e92d with SMTP id 98e67ed59e1d1-30e8323ee6emr34142590a91.32.1747957071505; Thu, 22 May 2025 16:37:51 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 22 May 2025 16:37:30 -0700 In-Reply-To: <20250522233733.3176144-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250522233733.3176144-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1151.ga128411c76-goog Message-ID: <20250522233733.3176144-7-seanjc@google.com> Subject: [PATCH v3 6/8] KVM: SVM: Remove wbinvd in sev_vm_destroy() From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, Kevin Loughlin , Tom Lendacky , Kai Huang , Ingo Molnar , Zheyun Shen , Mingwei Zhang , Francesco Lavra Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Zheyun Shen Before sev_vm_destroy() is called, kvm_arch_guest_memory_reclaimed() has been called for SEV and SEV-ES and kvm_arch_gmem_invalidate() has been called for SEV-SNP. These functions have already handled flushing the memory. Therefore, this wbinvd_on_all_cpus() can simply be dropped. Suggested-by: Sean Christopherson Signed-off-by: Zheyun Shen Reviewed-by: Tom Lendacky Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/sev.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index a7a7dc507336..be70c8401c9b 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -2879,12 +2879,6 @@ void sev_vm_destroy(struct kvm *kvm) return; } =20 - /* - * Ensure that all guest tagged cache entries are flushed before - * releasing the pages back to the system for use. CLFLUSH will - * not do this, so issue a WBINVD. - */ - wbinvd_on_all_cpus(); =20 /* * if userspace was terminated before unregistering the memory regions --=20 2.49.0.1151.ga128411c76-goog From nobody Sun Dec 14 06:40:54 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BDC0E2D29D4 for ; Thu, 22 May 2025 23:37:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747957075; cv=none; b=GBrkvkdl1Jcc6vBkac9lX0s49v591JxiTAr/jSJOHxsZuVgL/qGL5lHPPhi4MsJSv+QyMf2CR7FGRbsTXmA6v1oY/Um/SbAw0H8Wpfgur6ZF0kKF0rAHFVj0yb6gRGxfqjzXH2VTRMlFTzSzBAVluOgxXf0GDArTi8+pVtMI78Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747957075; c=relaxed/simple; bh=bknyA+PyGb2jdOkVdNsNZUNIAgBV0hNEsZUHJd96il0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=geMhMAtoo5YbgYXit/02saNZfKNC/X3quzuL6QRVJCqJw/Ry+4ZJPRJ9FSTlqmWLeK+OAHRkb81x6OkeJ6Q1J3U2ngVVKDqeKYn0Or4VZ+HXllAbzJ520Dfq2cVusM0/aBRY4HQW0x7boDld/8xyNwLsw3ZgdeQHZqjs40Licys= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=pylouOSo; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="pylouOSo" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-22e7e097ef7so64019825ad.2 for ; Thu, 22 May 2025 16:37:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747957073; x=1748561873; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=seZN2EGAtTjhEaK9XXu9a0BU/Kezpabke/BGyffe3tE=; b=pylouOSozt50nOm3ZTG0whXgufYHmRmn2F8DpPPFshTiBD33myHZOTFS0YlIG0M4Lk v/V9ZkSn+SpLBuOHvDIr6sCK7sGkceQW5hqXzZfCVgSIUmDM3R13fZwYA+gIfmrKIYy9 RyIt8tRxxi48dMh2AFNUYtsytE1JmeBnIsvDbE4MIh51Jrp3/rRiHQB2np8yverjyZJU YcGQdMLklyAKCpSXMvxo9kQstA5VadUnwJVOIVLwkb6pL+3OA7Ks7x3B19PPJpwj2TDY hK20l3mvrK9B7+nPFU3/RXSM/27CZbIklP/XtsoSjuBQsUPUPs7Hxjox/1uHWuHoH+32 046w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747957073; x=1748561873; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=seZN2EGAtTjhEaK9XXu9a0BU/Kezpabke/BGyffe3tE=; b=oqdL6/g+T+QtRhxbVF3sQNCTonDQLqUDKyI4VaWGEBJgGw9F0HKQyKtNKTIXDI0bsA boANQOV5W+nO78fSrJFSW0ZANHfVciWKIwAUGSAXnGKZGrtSVMURxRBnriBNixZiuOjI v0ZJsyjUKgsCRt4nI2WDkHyT1Cq7aAS6PEQpIfqxrYJWrmlXiYjnCWpkUtyQHr8HaibX +65H48FbJ2RqbQAIgcdknXBNYJ2LXtiEm+aSh12vLv+yFh+pEQq1Kgo1BV6d4aWadIiW ywlgsR0H6WTmV0TybLhXycI1i7JWoNOd7ThU6jD+dDhCGBGvGqJNVDGcLamcMKzZw/So GyDA== X-Gm-Message-State: AOJu0YyLJML3V/dJn2BJUrkMiow/r3sDep3pT/NtjMykWMntKiVsS+Kr bw7mXBOm65VcyJ0Le9Z9rtCO8jmXyEKLf8ltJ6Yq3uLJcnRtYjvnyQd1vDwwK63MYNecisUWy1d g3HRUeg== X-Google-Smtp-Source: AGHT+IFOOrwLLJCq1Z+4dRHMqtaDLThdmKHgmx1QXHqioZeimXXjRObTtkWDRHa0hE0Qmb1cyO8UAcflGvE= X-Received: from pjbso5.prod.google.com ([2002:a17:90b:1f85:b0:2fa:1481:81f5]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ec89:b0:231:b1c8:3591 with SMTP id d9443c01a7336-231de36bb99mr418195725ad.24.1747957073118; Thu, 22 May 2025 16:37:53 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 22 May 2025 16:37:31 -0700 In-Reply-To: <20250522233733.3176144-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250522233733.3176144-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1151.ga128411c76-goog Message-ID: <20250522233733.3176144-8-seanjc@google.com> Subject: [PATCH v3 7/8] KVM: SEV: Prefer WBNOINVD over WBINVD for cache maintenance efficiency From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, Kevin Loughlin , Tom Lendacky , Kai Huang , Ingo Molnar , Zheyun Shen , Mingwei Zhang , Francesco Lavra Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kevin Loughlin AMD CPUs currently execute WBINVD in the host when unregistering SEV guest memory or when deactivating SEV guests. Such cache maintenance is performed to prevent data corruption, wherein the encrypted (C=3D1) version of a dirty cache line might otherwise only be written back after the memory is written in a different context (ex: C=3D0), yielding corruption. However, WBINVD is performance-costly, especially because it invalidates processor caches. Strictly-speaking, unless the SEV ASID is being recycled (meaning the SNP firmware requires the use of WBINVD prior to DF_FLUSH), the cache invalidation triggered by WBINVD is unnecessary; only the writeback is needed to prevent data corruption in remaining scenarios. To improve performance in these scenarios, use WBNOINVD when available instead of WBINVD. WBNOINVD still writes back all dirty lines (preventing host data corruption by SEV guests) but does *not* invalidate processor caches. Note that the implementation of wbnoinvd() ensures fall back to WBINVD if WBNOINVD is unavailable. In anticipation of forthcoming optimizations to limit the WBNOINVD only to physical CPUs that have executed SEV guests, place the call to wbnoinvd_on_all_cpus() in a wrapper function sev_writeback_caches(). Signed-off-by: Kevin Loughlin Reviewed-by: Mingwei Zhang Reviewed-by: Tom Lendacky Link: https://lore.kernel.org/r/20250201000259.3289143-3-kevinloughlin@goog= le.com [sean: tweak comment regarding CLFUSH] Cc: Francesco Lavra Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/sev.c | 45 ++++++++++++++++++++++++------------------ 1 file changed, 26 insertions(+), 19 deletions(-) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index be70c8401c9b..2676be2b121d 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -116,6 +116,7 @@ static int sev_flush_asids(unsigned int min_asid, unsig= ned int max_asid) */ down_write(&sev_deactivate_lock); =20 + /* SNP firmware requires use of WBINVD for ASID recycling. */ wbinvd_on_all_cpus(); =20 if (sev_snp_enabled) @@ -705,6 +706,18 @@ static void sev_clflush_pages(struct page *pages[], un= signed long npages) } } =20 +static void sev_writeback_caches(void) +{ + /* + * Ensure that all dirty guest tagged cache entries are written back + * before releasing the pages back to the system for use. CLFLUSH will + * not do this without SME_COHERENT, and flushing many cache lines + * individually is slower than blasting WBINVD for large VMs, so issue + * WBNOINVD (or WBINVD if the "no invalidate" variant is unsupported). + */ + wbnoinvd_on_all_cpus(); +} + static unsigned long get_num_contig_pages(unsigned long idx, struct page **inpages, unsigned long npages) { @@ -2753,12 +2766,7 @@ int sev_mem_enc_unregister_region(struct kvm *kvm, goto failed; } =20 - /* - * Ensure that all guest tagged cache entries are flushed before - * releasing the pages back to the system for use. CLFLUSH will - * not do this, so issue a WBINVD. - */ - wbinvd_on_all_cpus(); + sev_writeback_caches(); =20 __unregister_enc_region_locked(kvm, region); =20 @@ -3110,30 +3118,29 @@ static void sev_flush_encrypted_page(struct kvm_vcp= u *vcpu, void *va) =20 /* * VM Page Flush takes a host virtual address and a guest ASID. Fall - * back to WBINVD if this faults so as not to make any problems worse - * by leaving stale encrypted data in the cache. + * back to full writeback of caches if this faults so as not to make + * any problems worse by leaving stale encrypted data in the cache. */ if (WARN_ON_ONCE(wrmsrl_safe(MSR_AMD64_VM_PAGE_FLUSH, addr | asid))) - goto do_wbinvd; + goto do_sev_writeback_caches; =20 return; =20 -do_wbinvd: - wbinvd_on_all_cpus(); +do_sev_writeback_caches: + sev_writeback_caches(); } =20 void sev_guest_memory_reclaimed(struct kvm *kvm) { /* * With SNP+gmem, private/encrypted memory is unreachable via the - * hva-based mmu notifiers, so these events are only actually - * pertaining to shared pages where there is no need to perform - * the WBINVD to flush associated caches. + * hva-based mmu notifiers, i.e. these events are explicitly scoped to + * shared pages, where there's no need to flush caches. */ if (!sev_guest(kvm) || sev_snp_guest(kvm)) return; =20 - wbinvd_on_all_cpus(); + sev_writeback_caches(); } =20 void sev_free_vcpu(struct kvm_vcpu *vcpu) @@ -3897,9 +3904,9 @@ void sev_snp_init_protected_guest_state(struct kvm_vc= pu *vcpu) * From this point forward, the VMSA will always be a guest-mapped page * rather than the initial one allocated by KVM in svm->sev_es.vmsa. In * theory, svm->sev_es.vmsa could be free'd and cleaned up here, but - * that involves cleanups like wbinvd_on_all_cpus() which would ideally - * be handled during teardown rather than guest boot. Deferring that - * also allows the existing logic for SEV-ES VMSAs to be re-used with + * that involves cleanups like flushing caches, which would ideally be + * handled during teardown rather than guest boot. Deferring that also + * allows the existing logic for SEV-ES VMSAs to be re-used with * minimal SNP-specific changes. */ svm->sev_es.snp_has_guest_vmsa =3D true; @@ -4892,7 +4899,7 @@ void sev_gmem_invalidate(kvm_pfn_t start, kvm_pfn_t e= nd) =20 /* * SEV-ES avoids host/guest cache coherency issues through - * WBINVD hooks issued via MMU notifiers during run-time, and + * WBNOINVD hooks issued via MMU notifiers during run-time, and * KVM's VM destroy path at shutdown. Those MMU notifier events * don't cover gmem since there is no requirement to map pages * to a HVA in order to use them for a running guest. While the --=20 2.49.0.1151.ga128411c76-goog From nobody Sun Dec 14 06:40:54 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 885492D3A6B for ; Thu, 22 May 2025 23:37:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747957079; cv=none; b=Ql6wH77oSdFFOsYloj+Z9b48ueOvmOeJJCjcxF9tG6hw1Vli837pJrOyKrBmuJYMaMAL5Viz8Z4TvDuumrqHRtCbPyrIVbKBlpKxsTJKv8mtHFV2jaVBdoorKIwg3hLebk5O5kfmMdRkmiWhib3YAKJvnQJuNA0qAarsyHbfON8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747957079; c=relaxed/simple; bh=EJSWXtll+OllBjJufDyDEa/BI4x/BrdIZLYuFPsRMNI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=emaovdfg49W7VZEr8AY3B8zwX+D2eEMljcKqMIYCXRsQj4J4kk03HRz9f+M610+nBOW4mVCGcbIiSgKHCcpoZPaOfPns5Gx4nToohLx5F25dDm/eBZKXYWJd2C4pwllR8JWEQh3pPPCjmaukWZzFnX9h4AS5vBLEfb2m97PHsTY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=CWGUyIll; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="CWGUyIll" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-310a0668968so2192008a91.0 for ; Thu, 22 May 2025 16:37:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747957075; x=1748561875; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=f9Xn4KYWiGCbjLdjL8speXTpf75EGP3mA2cTcsP7kRE=; b=CWGUyIllGTIwpPH7p8CFIjoEkt97HcqLw5uADcx4SE614xhGo/A3c1Yu3oMzqJFDcf D9FnC31gdE7Sa0tSdvBo5t+zeuYRTGvmuiEnD0X9batPtvloRWgtGMS3uEhFepeSTDoz q4Q1egFeKDi7/wQyjMeHKTIImzG5btcewh579SpZBwpG3By6kk1dJsU/O+M5NH0wfBhd QlktKvBMZCEeCsfVSlJQsl40X7IQn08vlhmMll+xJino3L6U/xla0cnrUXo2Gu2q1a/s 7DTajzFlGYWDN6NQ/KW6zXK9MHTAGWXGM9vFjSTjaw8dYCIeqwph55fa6+h2FFFj7Trv 24dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747957075; x=1748561875; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=f9Xn4KYWiGCbjLdjL8speXTpf75EGP3mA2cTcsP7kRE=; b=bN/f2EsvYQ39FaMhHnS3XCidJmzaIH0OeL6gfOCSANSJEMJCEqq9wv1RhKTNigEGQN VOAOne474sGQbDp+BQn7ZCvPGI9aWajG0a/W2Tg/NKtccz7jNwXgunS3l/eVhJ/SADg6 a0tMOGhvTB/4l9564Uln8KKlvk23L8hneS6eQLJYH1QyLXhiP/sGmbWIvQOEn2l5+68l 6szdgRuFgcwwNvDNk4SunjXEHHkl0N0hZoXjELPI7lUBC1PpllLNKxUa9q0bgL0aktYn vJ9eSu2Ot2gUNFn9IoWWii32GegPm+YjBg5ZhIbzb4FXD6O1uYxYV0bijAytbr6UvGB8 z1rg== X-Gm-Message-State: AOJu0Yy1HXZGL8PxX7/WzBsJ7tGjlJIoZf7Y80iO31xeIFZLPM6Geq3W Bljo2mGpdfRHdK5WP9nuTLg51VeXbdju8sQNzwrbynWtTmOo1pBo+mqVayGTyLC0Th1O+45mDNA 3LluOUA== X-Google-Smtp-Source: AGHT+IF5Lb8CXIC70Vyva66+17O+yrwnyFqJbN7erK3jdHXzkFdxO3FG0VdX++4AGAOrzJkMYErRjBNX2mI= X-Received: from pjg13.prod.google.com ([2002:a17:90b:3f4d:b0:2ee:4a90:3d06]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3f4c:b0:2fe:994d:613b with SMTP id 98e67ed59e1d1-30e7d5be445mr37978564a91.35.1747957074752; Thu, 22 May 2025 16:37:54 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 22 May 2025 16:37:32 -0700 In-Reply-To: <20250522233733.3176144-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250522233733.3176144-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1151.ga128411c76-goog Message-ID: <20250522233733.3176144-9-seanjc@google.com> Subject: [PATCH v3 8/8] KVM: SVM: Flush cache only on CPUs running SEV guest From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, Kevin Loughlin , Tom Lendacky , Kai Huang , Ingo Molnar , Zheyun Shen , Mingwei Zhang , Francesco Lavra Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Zheyun Shen On AMD CPUs without ensuring cache consistency, each memory page reclamation in an SEV guest triggers a call to do WBNOINVD/WBINVD on all CPUs, thereby affecting the performance of other programs on the host. Typically, an AMD server may have 128 cores or more, while the SEV guest might only utilize 8 of these cores. Meanwhile, host can use qemu-affinity to bind these 8 vCPUs to specific physical CPUs. Therefore, keeping a record of the physical core numbers each time a vCPU runs can help avoid flushing the cache for all CPUs every time. Reviewed-by: Tom Lendacky Signed-off-by: Zheyun Shen Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/sev.c | 46 +++++++++++++++++++++++++++++++++++------- arch/x86/kvm/svm/svm.h | 1 + 2 files changed, 40 insertions(+), 7 deletions(-) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 2676be2b121d..c3ddcca9fdce 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -446,7 +446,12 @@ static int __sev_guest_init(struct kvm *kvm, struct kv= m_sev_cmd *argp, init_args.probe =3D false; ret =3D sev_platform_init(&init_args); if (ret) - goto e_free; + goto e_free_asid; + + if (!zalloc_cpumask_var(&sev->have_run_cpus, GFP_KERNEL_ACCOUNT)) { + ret =3D -ENOMEM; + goto e_free_asid; + } =20 /* This needs to happen after SEV/SNP firmware initialization. */ if (vm_type =3D=3D KVM_X86_SNP_VM) { @@ -464,6 +469,8 @@ static int __sev_guest_init(struct kvm *kvm, struct kvm= _sev_cmd *argp, return 0; =20 e_free: + free_cpumask_var(sev->have_run_cpus); +e_free_asid: argp->error =3D init_args.error; sev_asid_free(sev); sev->asid =3D 0; @@ -706,16 +713,31 @@ static void sev_clflush_pages(struct page *pages[], u= nsigned long npages) } } =20 -static void sev_writeback_caches(void) +static void sev_writeback_caches(struct kvm *kvm) { + /* + * Note, the caller is responsible for ensuring correctness if the mask + * can be modified, e.g. if a CPU could be doing VMRUN. + */ + if (cpumask_empty(to_kvm_sev_info(kvm)->have_run_cpus)) + return; + /* * Ensure that all dirty guest tagged cache entries are written back * before releasing the pages back to the system for use. CLFLUSH will * not do this without SME_COHERENT, and flushing many cache lines * individually is slower than blasting WBINVD for large VMs, so issue - * WBNOINVD (or WBINVD if the "no invalidate" variant is unsupported). + * WBNOINVD (or WBINVD if the "no invalidate" variant is unsupported) + * on CPUs that have done VMRUN, i.e. may have dirtied data using the + * VM's ASID. + * + * For simplicity, never remove CPUs from the bitmap. Ideally, KVM + * would clear the mask when flushing caches, but doing so requires + * serializing multiple calls and having responding CPUs (to the IPI) + * mark themselves as still running if they are running (or about to + * run) a vCPU for the VM. */ - wbnoinvd_on_all_cpus(); + wbnoinvd_on_cpus_mask(to_kvm_sev_info(kvm)->have_run_cpus); } =20 static unsigned long get_num_contig_pages(unsigned long idx, @@ -2766,7 +2788,7 @@ int sev_mem_enc_unregister_region(struct kvm *kvm, goto failed; } =20 - sev_writeback_caches(); + sev_writeback_caches(kvm); =20 __unregister_enc_region_locked(kvm, region); =20 @@ -2914,6 +2936,7 @@ void sev_vm_destroy(struct kvm *kvm) } =20 sev_asid_free(sev); + free_cpumask_var(sev->have_run_cpus); } =20 void __init sev_set_cpu_caps(void) @@ -3127,7 +3150,7 @@ static void sev_flush_encrypted_page(struct kvm_vcpu = *vcpu, void *va) return; =20 do_sev_writeback_caches: - sev_writeback_caches(); + sev_writeback_caches(vcpu->kvm); } =20 void sev_guest_memory_reclaimed(struct kvm *kvm) @@ -3140,7 +3163,7 @@ void sev_guest_memory_reclaimed(struct kvm *kvm) if (!sev_guest(kvm) || sev_snp_guest(kvm)) return; =20 - sev_writeback_caches(); + sev_writeback_caches(kvm); } =20 void sev_free_vcpu(struct kvm_vcpu *vcpu) @@ -3472,6 +3495,15 @@ int pre_sev_run(struct vcpu_svm *svm, int cpu) if (sev_es_guest(kvm) && !VALID_PAGE(svm->vmcb->control.vmsa_pa)) return -EINVAL; =20 + /* + * To optimize cache flushes when memory is reclaimed from an SEV VM, + * track physical CPUs that enter the guest for SEV VMs and thus can + * have encrypted, dirty data in the cache, and flush caches only for + * CPUs that have entered the guest. + */ + if (!cpumask_test_cpu(cpu, to_kvm_sev_info(kvm)->have_run_cpus)) + cpumask_set_cpu(cpu, to_kvm_sev_info(kvm)->have_run_cpus); + /* Assign the asid allocated with this SEV guest */ svm->asid =3D asid; =20 diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index f16b068c4228..45d564c674ef 100644 --- a/arch/x86/kvm/svm/svm.h +++ b/arch/x86/kvm/svm/svm.h @@ -112,6 +112,7 @@ struct kvm_sev_info { void *guest_req_buf; /* Bounce buffer for SNP Guest Request input */ void *guest_resp_buf; /* Bounce buffer for SNP Guest Request output */ struct mutex guest_req_mutex; /* Must acquire before using bounce buffers= */ + cpumask_var_t have_run_cpus; /* CPUs that have done VMRUN for this VM. */ }; =20 struct kvm_svm { --=20 2.49.0.1151.ga128411c76-goog