From nobody Fri Dec 19 19:17:06 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB742280034 for ; Fri, 16 May 2025 21:30:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747431007; cv=none; b=AanNJomtQ7PPSWA8PKywRdoqXzjQvzg6SAl7J7Dcq2IiYDpQqJkYrkJ2D+bZVCNqFH7CqTw+1QSib7CpVN9VGFMTv0Msms0PEduFPW1Jfba2cAHt+1Wr8CTJUhabhQdqQRzkkuC6PFMQWxLj28cudYjk0zrVqZPg6C4hH3VgybE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747431007; c=relaxed/simple; bh=rU/OOmXSxzdbXeVGOVp3Atr0buHumvJkkL63QEVbDk8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=E7hogexMa77wJ4vRoe3ZxERFgoCKbFcnhaUq08gvz4NG7YYMmVlavRTQdrYub35BfRDqNYPxWtxhF2fI8gRW0zDurMQGCMPbE6dwZONNhdsbUzk72b/H+2fVUxjaAQyzPUqSNfdcN59gxpxLB8HEjWYW4tbOcfHtOGjFOmQAnjM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=uGU1WcHa; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="uGU1WcHa" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2318043effaso27996005ad.1 for ; Fri, 16 May 2025 14:30:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747431005; x=1748035805; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Cl8sKId1p4fnSWsbXS5nPLz/0J3dr2L1BPCgdcqnxp0=; b=uGU1WcHaBipogCUU3hXvzA0CjoyVheHr8s7lhmBElXt/4lqJ+xeSMWTTFgpQ1cTcJ5 xdSiF6eAqkOWnP9Yp4Yh/vxnuULaOUP/S0CWN5z/sAWy+BAU6R3/+UUh9SXQsK01i7Op iiEajK5eFgFJbsjlbzYS5xeKg3PDBj95Njz9HqZHO4FxidMooGVurKudehAT107J4rNk u7RG6pwaILeAz00HkynkC2rNqjHyTwpRfrhoKZ1uE0OI1YyoLzj8yBfZA6Jb1OlO0W/5 t6a6tcDipvEeJbkEVGg+baWU0Czhr6Yp8aqhmLYS0JdWF49hNwF0arCgFd1X2Ggysn8Y NPug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747431005; x=1748035805; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Cl8sKId1p4fnSWsbXS5nPLz/0J3dr2L1BPCgdcqnxp0=; b=cfAvqJ4JsCkrcAE9s+LcheTLbryPp4xz82f5U60YiSXVOZF+VciSQFeZ1vtOtU2C1K sWTOZu5udkq4T2ba/EvX1rfi9PDL1uIzaaKZ6dC3CGUb4Ij1mvFfb2/mbyDSot07dwXw zmGyjv5N2w+P/9oz/VLksZl4BAT32FGywfiQIr7jrIKPuHCZ/6t2uK/TTKWOdJmbBIip uWkscqhl41gAbB80R4I2pvRYn7TODrsWCzJs6iIiAY8YKoEd+Dbj5V+sdo7zCJZ/2vYb oLaDrmvZ1jGtmWOmmccJFuy7rn6Vk8x0acJFeZ/by570FMx6T8sXwD72ZFlJbxr/Hyd0 IYlg== X-Gm-Message-State: AOJu0YwM9l1ZCauNqNoNzbJJunomzzvfTR/8Lth1QpuJoDkT4XmwDQPZ LbjnbFZdBPpp6LRrYteOSC0dPET4AP1umBxfZZoQ0zitQKZx//IaIY1iBqx9xVEx6DXLSvtlbPS t03Hnug== X-Google-Smtp-Source: AGHT+IF6I89C9mEIARSlw1bcJepJA60kvrJai+jIj3fEWudVeM5rH1kouHgj/taWzfpRfkrGgw+/t4fBuRo= X-Received: from pjbsw16.prod.google.com ([2002:a17:90b:2c90:b0:2ff:84e6:b2bd]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:3d07:b0:224:a96:e39 with SMTP id d9443c01a7336-231d438b420mr68386655ad.9.1747431004975; Fri, 16 May 2025 14:30:04 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 16 May 2025 14:28:26 -0700 In-Reply-To: <20250516212833.2544737-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250516212833.2544737-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1112.g889b7c5bd8-goog Message-ID: <20250516212833.2544737-2-seanjc@google.com> Subject: [PATCH v2 1/8] KVM: SVM: Remove wbinvd in sev_vm_destroy() From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, Zheyun Shen , Tom Lendacky , Kevin Loughlin , Kai Huang , Mingwei Zhang Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Zheyun Shen Before sev_vm_destroy() is called, kvm_arch_guest_memory_reclaimed() has been called for SEV and SEV-ES and kvm_arch_gmem_invalidate() has been called for SEV-SNP. These functions have already handled flushing the memory. Therefore, this wbinvd_on_all_cpus() can simply be dropped. Suggested-by: Sean Christopherson Signed-off-by: Zheyun Shen Reviewed-by: Tom Lendacky Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/sev.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 978a0088a3f1..e9fd3bcec37a 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -2883,12 +2883,6 @@ void sev_vm_destroy(struct kvm *kvm) return; } =20 - /* - * Ensure that all guest tagged cache entries are flushed before - * releasing the pages back to the system for use. CLFLUSH will - * not do this, so issue a WBINVD. - */ - wbinvd_on_all_cpus(); =20 /* * if userspace was terminated before unregistering the memory regions --=20 2.49.0.1112.g889b7c5bd8-goog From nobody Fri Dec 19 19:17:06 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9AA83280007 for ; Fri, 16 May 2025 21:30:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747431009; cv=none; b=GGq7jLRqbF+xKYgG1/OG0Loz+uYpXdyXFGbqYKMQzILcPJWwg4cnnplZHdQ3L28kFAtJORJYuFu/vY+OlEfdceF+vmaOVvVVfOqVehSmAsrIupB3gYxSgjVdtxqQ7b/nuBsoINP1HJl0HU7yIYoLx8vdul63h9AU1Os/ATPYum8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747431009; c=relaxed/simple; bh=xaIvUFnWYSzEf61zEmzdhPL+zDmYyFmwy5vgctv1DT0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=YEfW9bwbbPQihTyv17ZSDipHK5QJe7hwmRNiD+xNdOgB6MNhEBgPw2V+9cDbj6nXx3nQNt+GUjVEvbmIGEQ7EIRMi/xzM/BW5UqO7HMqPH96ibKxItHTQf/BdziigiXDZoWAtiROmz+2slIyRvd5xar/5N4aFMxi4ce2AZc/LII= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=0mKDonTb; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="0mKDonTb" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-30e57a372dcso2757585a91.2 for ; Fri, 16 May 2025 14:30:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747431007; x=1748035807; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=lKGqoll3RbNHREN0eIRYM+wz/B1kXBEjsywi3uO05do=; b=0mKDonTbAUE8lWWpi9bJRNxpckz5TYKIgs5XX7KHxClCQUYqcrDYlZSZolvQ6kZ+gm Y00ZNWuH7IqhbGlhrsU1FzOnBcPmdBVs1SZJ/7UZJZjX+7+2qSPJP+7A23YPx27U5RQR ttYi8oQcDygfzuevtqaXV5t1z0bi3Wl1yF8570f/1wJpgAcAjZMBnKNbPB/9AJpa8Jhe lAlPDJKhRwfKqjsIVz7wyhRfisDQXk3oZiuUQQmiccPKpUPrZsxP9an71A0KeJExdrTu 9sl2+JPIhrtMlQ9e7prc4J9lK6yNdz//m66I/awxFTKTgDh5PwMRo55TfKKvFAJJ1V2B Cpdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747431007; x=1748035807; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=lKGqoll3RbNHREN0eIRYM+wz/B1kXBEjsywi3uO05do=; b=NBfO6F+dbMRf1lvTSFREDmBZX5LAdH1ZkrcPLRNeUx5hzBP5B9icE6l7xkurrCXoOt VgFymiYJpBe7fxJ41h8SIGbdsr3KoOMjg9KIn5zFxArpgbdWjhS8F4Wm86BVWEXXGZWd r/Nc9+NFtwKcH5HQS+RdIWZsTnfDWtj3LExO++HRR1M/qZF8VEg79gYyyvY73goTVcaW RvsAExavSn8MBNFDpNqyDFwnDAOBQRwquqrMIgMroJCIET21tJue2q0zSeFDq+fy9XkH zWRoEENsbpq7ZEZDnEOxZBuPB/D1Dz3iLjTkS+PnIrnTBHV3A42B5VJ14DblOfVwDEzh IHhA== X-Gm-Message-State: AOJu0Yzs9c5XheE3bRM+pFGL0KliD+97WwBZ8wWNet+WOHJJg6R7UAzS j8x7AnAPTV3f1NWYyYrF2oEY6FqyAWZG4rBibjKQbkHYOmq88cSbOve7ABsSFzF3AFXJFWvfdQ+ ZhL+g/g== X-Google-Smtp-Source: AGHT+IH1jrrId9MVi2SBTvtJJTyzh8YzXge4AbpmDoeFZoyCxndzCIEwgvXMwRA0R+s3WnDYdARhk8o3l0Y= X-Received: from pjyd13.prod.google.com ([2002:a17:90a:dfcd:b0:30e:7f04:f467]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5746:b0:2ee:f687:6acb with SMTP id 98e67ed59e1d1-30e830fc346mr5582137a91.13.1747431006822; Fri, 16 May 2025 14:30:06 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 16 May 2025 14:28:27 -0700 In-Reply-To: <20250516212833.2544737-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250516212833.2544737-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1112.g889b7c5bd8-goog Message-ID: <20250516212833.2544737-3-seanjc@google.com> Subject: [PATCH v2 2/8] drm/gpu: Remove dead checks on wbinvd_on_all_cpus()'s return value From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, Zheyun Shen , Tom Lendacky , Kevin Loughlin , Kai Huang , Mingwei Zhang Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove the checks and associated pr_err() on wbinvd_on_all_cpus() failure, as the helper has unconditionally returned 0/success since commit caa759323c73 ("smp: Remove smp_call_function() and on_each_cpu() return values"). Signed-off-by: Sean Christopherson --- drivers/gpu/drm/drm_cache.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/drm/drm_cache.c b/drivers/gpu/drm/drm_cache.c index 7051c9c909c2..ea1d2d5d2c66 100644 --- a/drivers/gpu/drm/drm_cache.c +++ b/drivers/gpu/drm/drm_cache.c @@ -93,8 +93,7 @@ drm_clflush_pages(struct page *pages[], unsigned long num= _pages) return; } =20 - if (wbinvd_on_all_cpus()) - pr_err("Timed out waiting for cache flush\n"); + wbinvd_on_all_cpus(); =20 #elif defined(__powerpc__) unsigned long i; @@ -139,8 +138,7 @@ drm_clflush_sg(struct sg_table *st) return; } =20 - if (wbinvd_on_all_cpus()) - pr_err("Timed out waiting for cache flush\n"); + wbinvd_on_all_cpus(); #else WARN_ONCE(1, "Architecture has no drm_cache.c support\n"); #endif @@ -172,8 +170,7 @@ drm_clflush_virt_range(void *addr, unsigned long length) return; } =20 - if (wbinvd_on_all_cpus()) - pr_err("Timed out waiting for cache flush\n"); + wbinvd_on_all_cpus(); #else WARN_ONCE(1, "Architecture has no drm_cache.c support\n"); #endif --=20 2.49.0.1112.g889b7c5bd8-goog From nobody Fri Dec 19 19:17:06 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E2F5228030D for ; Fri, 16 May 2025 21:30:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747431011; cv=none; b=gAfw7znpDHguGFKtBsGkAfs4Vu/D9HhxIY6BPBXZv9WO3fnm7R9WsP/7M42JNOTLiaf9E7/xMRePWUtLlDhB8hAQaqJluOjVvvCtS1rbrfKMZ/H4L1dFJQEjJETjf8L+1HidC+94tAmfJjYXcg/rNIG9obvNBbQhyCLobQDQwkw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747431011; c=relaxed/simple; bh=DIrrcPlHnq5EeZRAr8HvqCuhe1CzBiwfAT2oWIINOSE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=GLbyrUGvq/7hAHyBri5oZ6lEp2/XyZkwjaq04ZtrY2Yl2kDkxbiaHODKjzeFeFJa2os5OdqJyRBh8dd0X1PRXagHWnu1Yrpjeu9OupgK9iGajDbKFLwPD1x7F57kpQB49fAqnHcgIU1kA2NMQevssksVuK4lHGq2nn09LO2a15I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=EHwsbHb6; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="EHwsbHb6" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-231d7f590ffso9292275ad.1 for ; Fri, 16 May 2025 14:30:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747431009; x=1748035809; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=NqEHUZ2TPYAtGToYJi37fXESd2ZGwlNQjgbzr/1LJdY=; b=EHwsbHb6b4nydBI5mtDrWco7DDoaRbTr4XyYo+L9P6RrxK4689dEWQMgG6G6m2DatD GAnN+P0CNRAglsp3KI6G7O1CSYGIY7lG8caLVxo3eqgRZykG4RHjg3ScUaiCFSyd0PNt vCKKpzYQxbC8TcGL+rocujSmtSv8gTiCYfzWCEVPoCu8xFoOPPlKmXvAahXNDK5+Y0Mo 0CWrU/uqwoINJ3/2OKm5kh2zAz/wdC3mBrviC0DDrhMfwjTb4eQvF0p9glvbfD+pYuyu X/l3/hS9NZzEuM40CyDqodKpdXmZ1fqifZ+aAqJMSmlJjp24BUJk1HM0XKZ4jGDFZH9p uoGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747431009; x=1748035809; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=NqEHUZ2TPYAtGToYJi37fXESd2ZGwlNQjgbzr/1LJdY=; b=U44G27K00gD/1SMIEq4AHgWbtLTl+UYtzQAJjq5vLOH3GQB4JrxN+p+gbhUr4OCHb2 Yv8ym+zCQSswCSvPaYn5bG7R/0BrLCr8kTYamrM8OSoizmjjzxeIhP1voTVoRM6HFYoT OAPlfBAOPGO752UB21Y4Qrk+lqDD7jQtcdq13A5JekOFouvuGJhZYEG2aDIKmSxR1hYE fHoc9Oe0wMiWXyXeWb5QIV5JpgP9nZl4laAaQL5lMXB/8bYdIKKI0+f/XsPWfDxO9FrL 5NN0ya1WILFuAz0SD0rUaEzK0mhcuMjD3DEABdpiJxsPxOzhSTza4VJHI9Bulf4Ci+Qx 02yg== X-Gm-Message-State: AOJu0YzVpMPdvRlmTh3GM/Y6IRbsr5UwZuBDZ8leSLz8c2Rwc6wVq+r6 7gGaj2Zr0v5fMaQAxnabEhPMZVo47T7o8ifAr9GPKLuBzeF6JyiGVaJsXlH4u0gvHLy2JdoOYRy QHFzM+Q== X-Google-Smtp-Source: AGHT+IHTq80VcqezatgmWlMunNaSOj4mkdrzwcQ+8bhQSnlwEViznyKrv/5A/SubVqL2ubXidQVjO6bdruo= X-Received: from plbp12.prod.google.com ([2002:a17:903:174c:b0:223:52c5:17f6]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:d4c5:b0:223:faf5:c82 with SMTP id d9443c01a7336-231de2e6c76mr46558255ad.8.1747431008643; Fri, 16 May 2025 14:30:08 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 16 May 2025 14:28:28 -0700 In-Reply-To: <20250516212833.2544737-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250516212833.2544737-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1112.g889b7c5bd8-goog Message-ID: <20250516212833.2544737-4-seanjc@google.com> Subject: [PATCH v2 3/8] x86, lib: Drop the unused return value from wbinvd_on_all_cpus() From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, Zheyun Shen , Tom Lendacky , Kevin Loughlin , Kai Huang , Mingwei Zhang Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Drop wbinvd_on_all_cpus()'s return value; both the "real" version and the stub always return '0', and none of the callers check the return. Signed-off-by: Sean Christopherson Reviewed-by: Ingo Molnar --- arch/x86/include/asm/smp.h | 5 ++--- arch/x86/lib/cache-smp.c | 3 +-- 2 files changed, 3 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/smp.h b/arch/x86/include/asm/smp.h index 0c1c68039d6f..028f126018c9 100644 --- a/arch/x86/include/asm/smp.h +++ b/arch/x86/include/asm/smp.h @@ -112,7 +112,7 @@ void __noreturn hlt_play_dead(void); void native_play_dead(void); void play_dead_common(void); void wbinvd_on_cpu(int cpu); -int wbinvd_on_all_cpus(void); +void wbinvd_on_all_cpus(void); =20 void smp_kick_mwait_play_dead(void); void __noreturn mwait_play_dead(unsigned int eax_hint); @@ -148,10 +148,9 @@ static inline struct cpumask *cpu_l2c_shared_mask(int = cpu) =20 #else /* !CONFIG_SMP */ #define wbinvd_on_cpu(cpu) wbinvd() -static inline int wbinvd_on_all_cpus(void) +static inline void wbinvd_on_all_cpus(void) { wbinvd(); - return 0; } =20 static inline struct cpumask *cpu_llc_shared_mask(int cpu) diff --git a/arch/x86/lib/cache-smp.c b/arch/x86/lib/cache-smp.c index 7af743bd3b13..079c3f3cd32c 100644 --- a/arch/x86/lib/cache-smp.c +++ b/arch/x86/lib/cache-smp.c @@ -14,9 +14,8 @@ void wbinvd_on_cpu(int cpu) } EXPORT_SYMBOL(wbinvd_on_cpu); =20 -int wbinvd_on_all_cpus(void) +void wbinvd_on_all_cpus(void) { on_each_cpu(__wbinvd, NULL, 1); - return 0; } EXPORT_SYMBOL(wbinvd_on_all_cpus); --=20 2.49.0.1112.g889b7c5bd8-goog From nobody Fri Dec 19 19:17:06 2025 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A4E4E28135A for ; Fri, 16 May 2025 21:30:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747431013; cv=none; b=eIZlhZ9InbSl49T+SLxuIqpTf5b8xdAapYPfiVLtpASX1vKKTLgBQT5EvpNDyzjxTceUqV6rINsNvK4TA+dhQNiGDraDUBhC2QYE2KjJVmxMgno8FLIHiT9YUX3urs4XQ/9NrGgFnwPHYJEMx8itH+xQ9aYXxod262tQ/cyb+7Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747431013; c=relaxed/simple; bh=Dc2CblWGVBVuskP0Q7s2EWDSw3Z8JzeBhNUqDFQE4jw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=d1OWdm+OpOPVLXtiUjI2bguzkW9Wuc5LEbqZ/IvAvPVNX0uoT1C80i/gv/hRowDLCHa8UpTAp2yMiGhrFzM8v0vfvZW6j1pCLDfJhIVgSrItoFXI4g8ieARo7rs1U1RMrtAkx0ITJ6AVsYFg3ERCmhKHDc2SGkBtJh+PdQobpbk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Fg3daL2E; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Fg3daL2E" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-b26cdc70befso1615008a12.1 for ; Fri, 16 May 2025 14:30:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747431011; x=1748035811; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=XXHpJVmCrSZ9FxACIyOchlyq3sCWnKzqZMy4NYSrXEo=; b=Fg3daL2EmSzDtxi1iGsTR2AfLI12QOImjoKbHZG9XV2WKWh115YQVmvyyEW2UGZEiU HdwYR0JnJH2PYG6Np3DSuHDmvAOoHk7hJq1OPMjKkzqNgkJ/kO1Mb9Ix+gR2iLjzIPtu zzRCd00xbIfMfL7S1/Xe97A46CJ5ThX7CllQf3NVCWjERV0p5TRgUI/lBIfbajsDq0CE 51qhaFsCtYq0p7GW62T1Sbc7GnsNW9UPIT2qbKEKo/8E2qdeBardjF/pXSCorgvku3kx IGRYBIjEyBbvLEQ60zw+5Ioq+4a9hjRSUpzSjsw+QYfkiEpDfgAianTrR5wCqvaX+dRr Yv7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747431011; x=1748035811; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=XXHpJVmCrSZ9FxACIyOchlyq3sCWnKzqZMy4NYSrXEo=; b=YDrd52+PL3gkgn9iH197iTXXWmzjhTN8ns2Uoh1ZwIrAzcjg/yPlLTQU2jzIyzopQT g+qhTziVmrSR1wDXU/Zmm8/wDg3SJR2uj9QFw6AMBVzQ9pNu44WoLhTiZRfXhTeFoTIw rpiH46fPnCPkJlUTWy6Bes3iZbmSHZEreOqln+wF7NOm4o7WNDQ0WZMnn04qLWTM67cw w+YpVU8Qh80iMwuuo8ORGWaK7CeUDil3PcRDxc7jKfv1JsYcN2WOrbWfGHUhgytNTn1i jOUrbLPYculTe8/lg4Xnj39Hz4OWjyMRGSg2a6OufrwTHSZyQXbsk//zLwICD3HC4Qfp O01Q== X-Gm-Message-State: AOJu0YzbF4YftWQcdpHhaPchrD1NgAvyCpGRsML036GNcYmF2nXV4O5B dbE/sEkkBSSQG5xlkznecPmDbQXTZ+nz2xva16K6WZkGgg8TH+at+1VVxyStjZ6qHzRzSZkQQpc wLkguTg== X-Google-Smtp-Source: AGHT+IEeTLUSaLphMvwH3ICiNx6NcrDzw6LvXIQfL11JWb1+SBH0kSl8HCWC4BRH4JIZt1VlaecUPLeiPHQ= X-Received: from pjb12.prod.google.com ([2002:a17:90b:2f0c:b0:30a:a05c:6e7d]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2644:b0:309:e195:59d4 with SMTP id 98e67ed59e1d1-30e7d52b166mr9407454a91.12.1747431010844; Fri, 16 May 2025 14:30:10 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 16 May 2025 14:28:29 -0700 In-Reply-To: <20250516212833.2544737-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250516212833.2544737-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1112.g889b7c5bd8-goog Message-ID: <20250516212833.2544737-5-seanjc@google.com> Subject: [PATCH v2 4/8] x86, lib: Add WBNOINVD helper functions From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, Zheyun Shen , Tom Lendacky , Kevin Loughlin , Kai Huang , Mingwei Zhang Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kevin Loughlin In line with WBINVD usage, add WBONINVD helper functions. Fall back to WBINVD (via alternative()) if WBNOINVD isn't supported, as WBINVD provides a superset of functionality, just more slowly. Note, alternative() ensures compatibility with early boot code as needed. Signed-off-by: Kevin Loughlin Reviewed-by: Tom Lendacky [sean: massage changelog and comments, use ASM_WBNOINVD and _ASM_BYTES] Reviewed-by: Kai Huang Signed-off-by: Sean Christopherson Acked-by: Ingo Molnar --- arch/x86/include/asm/smp.h | 6 ++++++ arch/x86/include/asm/special_insns.h | 19 ++++++++++++++++++- arch/x86/lib/cache-smp.c | 11 +++++++++++ 3 files changed, 35 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/smp.h b/arch/x86/include/asm/smp.h index 028f126018c9..e08f1ae25401 100644 --- a/arch/x86/include/asm/smp.h +++ b/arch/x86/include/asm/smp.h @@ -113,6 +113,7 @@ void native_play_dead(void); void play_dead_common(void); void wbinvd_on_cpu(int cpu); void wbinvd_on_all_cpus(void); +void wbnoinvd_on_all_cpus(void); =20 void smp_kick_mwait_play_dead(void); void __noreturn mwait_play_dead(unsigned int eax_hint); @@ -153,6 +154,11 @@ static inline void wbinvd_on_all_cpus(void) wbinvd(); } =20 +static inline void wbnoinvd_on_all_cpus(void) +{ + wbnoinvd(); +} + static inline struct cpumask *cpu_llc_shared_mask(int cpu) { return (struct cpumask *)cpumask_of(0); diff --git a/arch/x86/include/asm/special_insns.h b/arch/x86/include/asm/sp= ecial_insns.h index 6266d6b9e0b8..46b3961e3e4b 100644 --- a/arch/x86/include/asm/special_insns.h +++ b/arch/x86/include/asm/special_insns.h @@ -117,7 +117,24 @@ static inline void wrpkru(u32 pkru) =20 static __always_inline void wbinvd(void) { - asm volatile("wbinvd": : :"memory"); + asm volatile("wbinvd" : : : "memory"); +} + +/* Instruction encoding provided for binutils backwards compatibility. */ +#define ASM_WBNOINVD _ASM_BYTES(0xf3,0x0f,0x09) + +/* + * Cheaper version of wbinvd(). Call when caches need to be written back b= ut + * not invalidated. + */ +static __always_inline void wbnoinvd(void) +{ + /* + * If WBNOINVD is unavailable, fall back to the compatible but + * more destructive WBINVD (which still writes the caches back + * but also invalidates them). + */ + alternative("wbinvd", ASM_WBNOINVD, X86_FEATURE_WBNOINVD); } =20 static inline unsigned long __read_cr4(void) diff --git a/arch/x86/lib/cache-smp.c b/arch/x86/lib/cache-smp.c index 079c3f3cd32c..1789db5d8825 100644 --- a/arch/x86/lib/cache-smp.c +++ b/arch/x86/lib/cache-smp.c @@ -19,3 +19,14 @@ void wbinvd_on_all_cpus(void) on_each_cpu(__wbinvd, NULL, 1); } EXPORT_SYMBOL(wbinvd_on_all_cpus); + +static void __wbnoinvd(void *dummy) +{ + wbnoinvd(); +} + +void wbnoinvd_on_all_cpus(void) +{ + on_each_cpu(__wbnoinvd, NULL, 1); +} +EXPORT_SYMBOL(wbnoinvd_on_all_cpus); --=20 2.49.0.1112.g889b7c5bd8-goog From nobody Fri Dec 19 19:17:06 2025 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 73F2F281505 for ; Fri, 16 May 2025 21:30:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747431015; cv=none; b=hiLuf+SYMmenA4caHVW1B7JnDkSkv/k+VypW/fI3lNaMGybVe/FBCfIzoHNYffAtRLYCrGQ17GhG0elkGlYXPBw84PlHv6WmFGPAS5AXGjxBmZ263Q8UlZr/Fn042scXMvGWdzBq7k3ADRhqPWJxHI9iAnAaKqv2nayqgIDqpZw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747431015; c=relaxed/simple; bh=flYWJPPU3bRx3+dVKKMz70hC4D0LNvPpc7j5HWWPtvE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=oveO8DJgUQFIccJ1xZFc2umWJg7Qa7DCDhF07CPE0ssnIKclS6DPat/fmhA/zu0dIewvPuQXlmcMPk5KV9PQspHddGgaXxGFuCw2/IYUcGlrHpALuQfWDQuWUjNS3X8sjh4TneZvN5zl3Eth2UIJKUl5fziVQh1X3+ueOsN5+Cw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=I58OX6ZK; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="I58OX6ZK" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-b26e38174e5so1984247a12.1 for ; Fri, 16 May 2025 14:30:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747431013; x=1748035813; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=vApJs4VkDB5QlNVHwT4ZvbZJAzsc+75w07odne9W4Q4=; b=I58OX6ZKW0LXzuLqKDZjxw6XjIveh9DwU1Wee0aiWkf5YDMJyNc518rkQvZ5OSf4MW 6xqZ/rX5rNCkTm8IODC/Y/+MC4RzMt/5KvcpB8Y97y3dMGoRGOVfzL+BcAwGf2emmWq0 naaIE2IYRDxMP5N6slH5vRQ37jv9cizGGr4tmGAlH4YR7XqY9Iyn+0Jh7vVBGti4D08i 2J3v1xWLxqfJwBP42jog1yDXjMBolXowGgCCK7PBXvknT/2pn80NMa7xOFgIyeqUiWaf HCzR0ia72s8mgHgLSn/jpKMG0ZMRa4ltBFeoqAUR4vU+JkjrvxpzEtMG7VSX3hJWXX8+ LECg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747431013; x=1748035813; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=vApJs4VkDB5QlNVHwT4ZvbZJAzsc+75w07odne9W4Q4=; b=lJJqJLJeyqMo4Ct0GdMuJZVAqMMFczbFcJRBcJYDILY1apoCDO75C43hO3+q41TVCw xFHXhIFFZ5QnEyxwuU1mfWC/qYY7lHGOR2bVSWQMXfIZ9s37/jVWUw65Bq1Pk99ROwjB utD42HIyypuSfEu70/y3cxHDU1h8v4trxns2EW4RUnDeHkKGMtEPooAnOyv34QiPeQpf 3EFCxhXHstTGcDPmOlbDDo2nSagyhbZ5R0Lpg+aerWYjE8wFJaE7EN8d3mqyvHL7S6lC pCmDCsYOrtU9wTD179sXVNIXkzwmXfcLP9xsfbB5UfQniN596VpBl6fcuhl03zCqrWHX p+NA== X-Gm-Message-State: AOJu0YxRVxk6OA90Szf64YpXtIAaZbw2FIa9d9X70vhvOYQlHxbtM0eV vQp++JTeP+yLFYtsvGRrX3f0H/L3ei6WY11u2sGRb/Fb70QivQ/UkPpq5HMGrCvrwYj2N+8rwR2 vVSIePw== X-Google-Smtp-Source: AGHT+IGb68toJP8qmeSh29S453pBC6YH3dAmwTf87WeOmAYT6NO/sBgl0udqJdDz9JEtnHFNAAvfH+MPaDk= X-Received: from pjyp5.prod.google.com ([2002:a17:90a:e705:b0:2f4:465d:5c61]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:a12a:b0:215:e08d:7953 with SMTP id adf61e73a8af0-2162192b844mr7618494637.24.1747431012754; Fri, 16 May 2025 14:30:12 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 16 May 2025 14:28:30 -0700 In-Reply-To: <20250516212833.2544737-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250516212833.2544737-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1112.g889b7c5bd8-goog Message-ID: <20250516212833.2544737-6-seanjc@google.com> Subject: [PATCH v2 5/8] KVM: SEV: Prefer WBNOINVD over WBINVD for cache maintenance efficiency From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, Zheyun Shen , Tom Lendacky , Kevin Loughlin , Kai Huang , Mingwei Zhang Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kevin Loughlin AMD CPUs currently execute WBINVD in the host when unregistering SEV guest memory or when deactivating SEV guests. Such cache maintenance is performed to prevent data corruption, wherein the encrypted (C=3D1) version of a dirty cache line might otherwise only be written back after the memory is written in a different context (ex: C=3D0), yielding corruption. However, WBINVD is performance-costly, especially because it invalidates processor caches. Strictly-speaking, unless the SEV ASID is being recycled (meaning the SNP firmware requires the use of WBINVD prior to DF_FLUSH), the cache invalidation triggered by WBINVD is unnecessary; only the writeback is needed to prevent data corruption in remaining scenarios. To improve performance in these scenarios, use WBNOINVD when available instead of WBINVD. WBNOINVD still writes back all dirty lines (preventing host data corruption by SEV guests) but does *not* invalidate processor caches. Note that the implementation of wbnoinvd() ensures fall back to WBINVD if WBNOINVD is unavailable. In anticipation of forthcoming optimizations to limit the WBNOINVD only to physical CPUs that have executed SEV guests, place the call to wbnoinvd_on_all_cpus() in a wrapper function sev_writeback_caches(). Signed-off-by: Kevin Loughlin Reviewed-by: Mingwei Zhang Reviewed-by: Tom Lendacky Link: https://lore.kernel.org/r/20250201000259.3289143-3-kevinloughlin@goog= le.com [sean: tweak comment regarding CLFUSH] Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/sev.c | 41 ++++++++++++++++++++++++----------------- 1 file changed, 24 insertions(+), 17 deletions(-) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index e9fd3bcec37a..9dcdeea954d3 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -116,6 +116,7 @@ static int sev_flush_asids(unsigned int min_asid, unsig= ned int max_asid) */ down_write(&sev_deactivate_lock); =20 + /* SNP firmware requires use of WBINVD for ASID recycling. */ wbinvd_on_all_cpus(); =20 if (sev_snp_enabled) @@ -707,6 +708,18 @@ static void sev_clflush_pages(struct page *pages[], un= signed long npages) } } =20 +static void sev_writeback_caches(void) +{ + /* + * Ensure that all dirty guest tagged cache entries are written back + * before releasing the pages back to the system for use. CLFLUSH will + * not do this without SME_COHERENT, and flushing many cache lines + * individually is slower than blasting WBINVD for large VMs, so issue + * WBNOINVD (or WBINVD if the "no invalidate" variant is unsupported). + */ + wbnoinvd_on_all_cpus(); +} + static unsigned long get_num_contig_pages(unsigned long idx, struct page **inpages, unsigned long npages) { @@ -2757,12 +2770,7 @@ int sev_mem_enc_unregister_region(struct kvm *kvm, goto failed; } =20 - /* - * Ensure that all guest tagged cache entries are flushed before - * releasing the pages back to the system for use. CLFLUSH will - * not do this, so issue a WBINVD. - */ - wbinvd_on_all_cpus(); + sev_writeback_caches(); =20 __unregister_enc_region_locked(kvm, region); =20 @@ -3114,30 +3122,29 @@ static void sev_flush_encrypted_page(struct kvm_vcp= u *vcpu, void *va) =20 /* * VM Page Flush takes a host virtual address and a guest ASID. Fall - * back to WBINVD if this faults so as not to make any problems worse - * by leaving stale encrypted data in the cache. + * back to full writeback of caches if this faults so as not to make + * any problems worse by leaving stale encrypted data in the cache. */ if (WARN_ON_ONCE(wrmsrl_safe(MSR_AMD64_VM_PAGE_FLUSH, addr | asid))) - goto do_wbinvd; + goto do_sev_writeback_caches; =20 return; =20 -do_wbinvd: - wbinvd_on_all_cpus(); +do_sev_writeback_caches: + sev_writeback_caches(); } =20 void sev_guest_memory_reclaimed(struct kvm *kvm) { /* * With SNP+gmem, private/encrypted memory is unreachable via the - * hva-based mmu notifiers, so these events are only actually - * pertaining to shared pages where there is no need to perform - * the WBINVD to flush associated caches. + * hva-based mmu notifiers, i.e. these events are explicitly scoped to + * shared pages, where there's no need to flush caches. */ if (!sev_guest(kvm) || sev_snp_guest(kvm)) return; =20 - wbinvd_on_all_cpus(); + sev_writeback_caches(); } =20 void sev_free_vcpu(struct kvm_vcpu *vcpu) @@ -3901,7 +3908,7 @@ void sev_snp_init_protected_guest_state(struct kvm_vc= pu *vcpu) * From this point forward, the VMSA will always be a guest-mapped page * rather than the initial one allocated by KVM in svm->sev_es.vmsa. In * theory, svm->sev_es.vmsa could be free'd and cleaned up here, but - * that involves cleanups like wbinvd_on_all_cpus() which would ideally + * that involves cleanups like flushing caches, which would ideally be * be handled during teardown rather than guest boot. Deferring that * also allows the existing logic for SEV-ES VMSAs to be re-used with * minimal SNP-specific changes. @@ -4899,7 +4906,7 @@ void sev_gmem_invalidate(kvm_pfn_t start, kvm_pfn_t e= nd) =20 /* * SEV-ES avoids host/guest cache coherency issues through - * WBINVD hooks issued via MMU notifiers during run-time, and + * WBNOINVD hooks issued via MMU notifiers during run-time, and * KVM's VM destroy path at shutdown. Those MMU notifier events * don't cover gmem since there is no requirement to map pages * to a HVA in order to use them for a running guest. While the --=20 2.49.0.1112.g889b7c5bd8-goog From nobody Fri Dec 19 19:17:06 2025 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8E8E280326 for ; Fri, 16 May 2025 21:30:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747431017; cv=none; b=ZDfA6uh/WBPr/4Tb8RxD81iFMlLO5VBmdrxWuMJge58bb1igNfnWH2ldKzyK/4PB5Zjs/szJT30X/La0/12KagIs60GJgP/obb38SbYE8dWpWDLNe1O1rW+jM2dOx7UIN6o1SIbk33FG2JbptwbB8vdv19EPo8a3MsVrUzkprIU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747431017; c=relaxed/simple; bh=NZvCSJRzuz1KajlGW5YiPv7YN+M07Tdox9/J35tpIEQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=K1zI7fhqH4IP5y/saFmG5ZCBb5qSNDeXQbBIe3cp/eZ2p3Ej5DAEF5shhdu53B/kDtAU0jEXVZZ6oQH9pXFZzE54Dvfsdv+apLeL67lSy8DSRPFsgrcC+04KhSEIa5Uw5oB5ue4b7GiWdKm7FZnRrlsIWMtUbYaVjmAnbFfbvz4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=0d0RHOkz; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="0d0RHOkz" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-b26e33ae9d5so2375643a12.1 for ; Fri, 16 May 2025 14:30:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747431015; x=1748035815; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=7Dga/slcZZ6fKTxbenqYjayajDYVeAnUirYr9Z43Kqg=; b=0d0RHOkzaqcA9atnHe/JJClOr0TmFScvdbmO9kJtssIykG1Hg5H5wldUnFjcgH3cKj kPWzJ+5s9bfbp2/OPBZYwoh9GHTzuEdj7OaaOEyZ5y2T+tX52CKCCNt/h9wgZ48VZEy/ d5pdP0vL9h3kIg+seXAx9s4en8TqGf6jB9pHUMeYdGk8+ftd0+V1V67cx+1/I+33CL3a hVURrEJnyllWoJaiCNbey7xvxoROBdeJcvgz+5FdnjkAfNwIRt3UT4YkYqp9M/saUTIQ Df0D/XpZjil2qDN/5A0vH03R10W9fK9CJMtOurvhrALHLXNe1Ll9Czk+LSgeUjHui6w3 cYqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747431015; x=1748035815; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=7Dga/slcZZ6fKTxbenqYjayajDYVeAnUirYr9Z43Kqg=; b=YCUBD6b8mdZ5izEwIAmXuCvd0rNKW21vg02h+CwDq4JPLPweuI/hygCANb903qL+hS 2Pu0Z6IcaxATYK5lwhpVITNU2P31yblvod8R4bxAKWR+pIdVa0lEvT03bLTTfsOo/uuj e5RHsmb9+oNloz+/D8l5R3kAbs7SfCPs9jc23YX9IWrODXPQtqMB8SSB8bNMoRwkknjG Li4IvQp31GP/vfwoKUEhTPGqd8Gb1LVyK6GklPX1nbOX2TX9dJl68BGU6CA2rb3xAvg6 dqcQ8wpkzHDxM6nxT3MNkhn9+GDXmT+f9VD1cQDBK8Bzunz6DXRbfGSPuKI+/847j690 RP9g== X-Gm-Message-State: AOJu0Yx8xAQlQsZoJBbEPwr30SMLhhQzADqN9PVe665PSGCIKjFwRBTu fACifkPRigGBsN3BFejSs6GWMsy8XC+N2J2zq7zqg5BuWEjLww1hv8x03UKeMtNSXBvhJLrSq1a Yj7iS7Q== X-Google-Smtp-Source: AGHT+IEmIT1oJ5BehpO/x/61kMltmH/PXqlkOkfjC0VyUyY4O6dvRDZ3QTMaLlJYbCRtNBQEKJ+Vu2+xC7I= X-Received: from pjbpv17.prod.google.com ([2002:a17:90b:3c91:b0:30c:4b1f:78ca]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:3409:b0:214:f8b1:e385 with SMTP id adf61e73a8af0-216fb192726mr7093001637.2.1747431015121; Fri, 16 May 2025 14:30:15 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 16 May 2025 14:28:31 -0700 In-Reply-To: <20250516212833.2544737-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250516212833.2544737-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1112.g889b7c5bd8-goog Message-ID: <20250516212833.2544737-7-seanjc@google.com> Subject: [PATCH v2 6/8] KVM: x86: Use wbinvd_on_cpu() instead of an open-coded equivalent From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, Zheyun Shen , Tom Lendacky , Kevin Loughlin , Kai Huang , Mingwei Zhang Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use wbinvd_on_cpu() to target a single CPU instead of open-coding an equivalent. In addition to deduplicating code, this will allow removing KVM's wbinvd_ipi() once the other usage is gone. No functional change intended. Reviewed-by: Tom Lendacky Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index f9f798f286ce..b8b72e8dac6e 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -5009,8 +5009,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cp= u) if (kvm_x86_call(has_wbinvd_exit)()) cpumask_set_cpu(cpu, vcpu->arch.wbinvd_dirty_mask); else if (vcpu->cpu !=3D -1 && vcpu->cpu !=3D cpu) - smp_call_function_single(vcpu->cpu, - wbinvd_ipi, NULL, 1); + wbinvd_on_cpu(vcpu->cpu); } =20 kvm_x86_call(vcpu_load)(vcpu, cpu); --=20 2.49.0.1112.g889b7c5bd8-goog From nobody Fri Dec 19 19:17:06 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 55796281515 for ; Fri, 16 May 2025 21:30:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747431018; cv=none; b=VoVX/7pOdKK2/vDHMqMWVlNvbH3FXAlXOW+xT/+6ZazrnDFfq+SQvPii8w8OgvnH7FuIWY9XtRroHiFS9RavxGmtixhZpA6rqmrpFkybJ+1FST7/W9WPF4OTt3y1IJtkCd+Ir0uXD80VxaCUl0XaKSwXlLNjbGPZLKo+JmHyPRU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747431018; c=relaxed/simple; bh=IYav/1dDjcX5ip4JkAY9bSkG1zmgQsJeTQlI5viDyAg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Uqosvv8Kb4QXlLALYFPpVZv3hAgq+xjiQHLoGfFeuvJnqcfVvDVRTjrN5drCRHutcPk6GuSvq4VdrLVmxDqAhQtYU0Nkq5ahjqgDOvCE7RhXLBtiIW0Z1jJEdlx6lG4fCjNz+CEAoXOcmPCU4arl5qzYruqKuWpyaypecBboWz4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Ez2pxuOF; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Ez2pxuOF" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-30e86c46eadso940992a91.2 for ; Fri, 16 May 2025 14:30:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747431017; x=1748035817; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=VVF3vRzj16BIxVOrDVzAz6hJgiBEUt5vcCXrSP3D1EE=; b=Ez2pxuOFaahrXB+H6bp9rl9nv4ex1UmxLFGRJXxh6CKFwT4Dy35QzBaXUqEKQ909Cl riBuIh6kNP/Az9aYEa3sFB5yj38ZdO4x3bU+oS3vctg/CYVa1RODEMcOhfBMP2Ud2osy HIWzCTtsecJpfXiv847eRn/iRip6Bwf53HXVLJ5fUSefqs8puxUAeTn/Fn0W8XtJxJ4V iJrZbWPlrUetQaDloQVi9HktEjL2jT1K2n/PeAJ0U54r6aQAPXZI7HkeI82LPsbmBNtS byeFgh7zYEQFxKEHc62RMcToU/u9DkcOt0PlXZ6ToZyIWekpPgr5yohlJumq/65n4CnV wrjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747431017; x=1748035817; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=VVF3vRzj16BIxVOrDVzAz6hJgiBEUt5vcCXrSP3D1EE=; b=JOr5B9pk9lu33wnU2/v7glIN/cFHzHKW8gzqtz76Rf4UZDOBKmGyTOei82il1PgDhk hQIYCFFs6Ta0ixvuKz8xCOwOFprK1/ISMlRPH0nSjeFx9a2ODJQ6+4nyxd1Ry2GmcAYt GV/chHqgAkJaOcu6RApNJ9EG2i0UAVEysuelz9s1OxMayEiDBG6AS7qvE1xrXng7vbxB 0+AgqSxhgS/vhToLtKbLSrI+8a59WfZjhJ4mJHf89KIB1qkMKPvQo/u9EG+gzMiSHL/p pDR8UyyZBs60WA4lOEmWgos+ow8d08oXm9OC95kAUSTki27vUb0b0KGjxy0/8C9C/koK 3POg== X-Gm-Message-State: AOJu0YyD6J35+A8W/PXklCuvXRD8Kc2XSxOLcsGkdn7ocs10PpDQ8eTU XA4edq3oOdLzAmseZKA7tLMUJsVr7rqBeCxGaFyseFSj7zq6PrHlycoy32eW0YbZdhL1Iyxb5Jf mXbn4Ew== X-Google-Smtp-Source: AGHT+IHkPYaluTndmSlfUiXGyyaJLXx4X+SBO5Q/4uaYOgx+OJqKuWd6RzYI5mr8azAci2LMYsf58XYaMmI= X-Received: from pjbtb8.prod.google.com ([2002:a17:90b:53c8:b0:309:f831:28e0]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:56cc:b0:2fe:a8b1:7d8 with SMTP id 98e67ed59e1d1-30e7d5acb07mr7192751a91.25.1747431016716; Fri, 16 May 2025 14:30:16 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 16 May 2025 14:28:32 -0700 In-Reply-To: <20250516212833.2544737-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250516212833.2544737-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1112.g889b7c5bd8-goog Message-ID: <20250516212833.2544737-8-seanjc@google.com> Subject: [PATCH v2 7/8] x86, lib: Add wbinvd and wbnoinvd helpers to target multiple CPUs From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, Zheyun Shen , Tom Lendacky , Kevin Loughlin , Kai Huang , Mingwei Zhang Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Zheyun Shen Extract KVM's open-coded calls to do writeback caches on multiple CPUs to common library helpers for both WBINVD and WBNOINVD (KVM will use both). Put the onus on the caller to check for a non-empty mask to simplify the SMP=3Dn implementation, e.g. so that it doesn't need to check that the one and only CPU in the system is present in the mask. Signed-off-by: Zheyun Shen Reviewed-by: Tom Lendacky Link: https://lore.kernel.org/r/20250128015345.7929-2-szy0127@sjtu.edu.cn [sean: move to lib, add SMP=3Dn helpers, clarify usage] Acked-by: Kai Huang Signed-off-by: Sean Christopherson --- arch/x86/include/asm/smp.h | 12 ++++++++++++ arch/x86/kvm/x86.c | 8 +------- arch/x86/lib/cache-smp.c | 12 ++++++++++++ 3 files changed, 25 insertions(+), 7 deletions(-) diff --git a/arch/x86/include/asm/smp.h b/arch/x86/include/asm/smp.h index e08f1ae25401..fe98e021f7f8 100644 --- a/arch/x86/include/asm/smp.h +++ b/arch/x86/include/asm/smp.h @@ -113,7 +113,9 @@ void native_play_dead(void); void play_dead_common(void); void wbinvd_on_cpu(int cpu); void wbinvd_on_all_cpus(void); +void wbinvd_on_many_cpus(struct cpumask *cpus); void wbnoinvd_on_all_cpus(void); +void wbnoinvd_on_many_cpus(struct cpumask *cpus); =20 void smp_kick_mwait_play_dead(void); void __noreturn mwait_play_dead(unsigned int eax_hint); @@ -154,11 +156,21 @@ static inline void wbinvd_on_all_cpus(void) wbinvd(); } =20 +static inline void wbinvd_on_many_cpus(struct cpumask *cpus) +{ + wbinvd(); +} + static inline void wbnoinvd_on_all_cpus(void) { wbnoinvd(); } =20 +static inline void wbnoinvd_on_many_cpus(struct cpumask *cpus) +{ + wbnoinvd(); +} + static inline struct cpumask *cpu_llc_shared_mask(int cpu) { return (struct cpumask *)cpumask_of(0); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index b8b72e8dac6e..e00a4b3a0e8c 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -4981,11 +4981,6 @@ long kvm_arch_dev_ioctl(struct file *filp, return r; } =20 -static void wbinvd_ipi(void *garbage) -{ - wbinvd(); -} - static bool need_emulate_wbinvd(struct kvm_vcpu *vcpu) { return kvm_arch_has_noncoherent_dma(vcpu->kvm); @@ -8286,8 +8281,7 @@ static int kvm_emulate_wbinvd_noskip(struct kvm_vcpu = *vcpu) int cpu =3D get_cpu(); =20 cpumask_set_cpu(cpu, vcpu->arch.wbinvd_dirty_mask); - on_each_cpu_mask(vcpu->arch.wbinvd_dirty_mask, - wbinvd_ipi, NULL, 1); + wbinvd_on_many_cpus(vcpu->arch.wbinvd_dirty_mask); put_cpu(); cpumask_clear(vcpu->arch.wbinvd_dirty_mask); } else diff --git a/arch/x86/lib/cache-smp.c b/arch/x86/lib/cache-smp.c index 1789db5d8825..ebbc91b3ac67 100644 --- a/arch/x86/lib/cache-smp.c +++ b/arch/x86/lib/cache-smp.c @@ -20,6 +20,12 @@ void wbinvd_on_all_cpus(void) } EXPORT_SYMBOL(wbinvd_on_all_cpus); =20 +void wbinvd_on_many_cpus(struct cpumask *cpus) +{ + on_each_cpu_mask(cpus, __wbinvd, NULL, 1); +} +EXPORT_SYMBOL_GPL(wbinvd_on_many_cpus); + static void __wbnoinvd(void *dummy) { wbnoinvd(); @@ -30,3 +36,9 @@ void wbnoinvd_on_all_cpus(void) on_each_cpu(__wbnoinvd, NULL, 1); } EXPORT_SYMBOL(wbnoinvd_on_all_cpus); + +void wbnoinvd_on_many_cpus(struct cpumask *cpus) +{ + on_each_cpu_mask(cpus, __wbnoinvd, NULL, 1); +} +EXPORT_SYMBOL_GPL(wbnoinvd_on_many_cpus); --=20 2.49.0.1112.g889b7c5bd8-goog From nobody Fri Dec 19 19:17:06 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 28FE8283149 for ; Fri, 16 May 2025 21:30:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747431020; cv=none; b=QYr0WNS+QFx1dYtKka6ZnkyTZqliKCruFNpwAMl+eIMA14I3qxc0YXNb+0zktfR6RzsaCIeX+pMzLcuN2dTDCOx5Z05APK5vk3w2yRdqw20iw5+YVA86dvP83TBHKtOJKWjPSe052hKbe4hF4lvHfQwk25zUU9zlnxlDYOox2f0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747431020; c=relaxed/simple; bh=9gjRLei6v1SxXphoOx9HIqBlLl2425GFsp2GVeMc6XM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ejJRzBdWL7/pVSnTlNp6xVci6PUhuwiguL3kEfXFSxHCt7zIkISi1YUSBqD6Rg/rrOpZ5bz04oZ+SI71hxh07ip284g5HHFspM0czmu/DRRBqIkdKjSkrGeLyXqudcp7pL0Es+LN1Xev6eiQF29W4VBZ01uAdyl+UYZnhszejAE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=sqT+I+ep; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="sqT+I+ep" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-30e9659a391so483096a91.0 for ; Fri, 16 May 2025 14:30:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747431018; x=1748035818; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=SXRj+NGuc6iHnmhzt+utocsfWr9n0NYuxoyZ1Z1GpaI=; b=sqT+I+ep5uO8mvrrkLJ5c+S5nHhqEt3jF7Gwmy1nacZ5/y6MXulRf0CmwONxdUCpMC 2bSioLGHT/1SHiZYqf7V7rPuZXpjZVopdQ9xEjECdF9cyqnURI4V+ndmOtuZ5ur1xmcs Z5PhY89FnSThmwl0R1YhV0PLBSSjePbKztTf6IiJZRKHpaynOtz2Dhg4aaTGcBXRY+ms /Y51Ue9GgBDYQODgFFhaRrOO6vGyVBWxmmiNS+6Nn9i45UEPux13bJIVKUKIoi8dYtvK fUZiGDfFIh7uIK8Hr3ZdYtKqXucs0uA/4v8jeRuquN/dcIIwZO2CWUEZ1LQi8vuJRwh+ PL1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747431018; x=1748035818; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=SXRj+NGuc6iHnmhzt+utocsfWr9n0NYuxoyZ1Z1GpaI=; b=VrkITz3AcXi5f/uRwGqOTphH6luqciEBiY6AFlcOFoXDjeX0M2+cikYmFpsPzsNUjW XfPgr1jzGCkDe0uo9gZNVFBJm5zeWsfrp4Gibx/OUw4dj3HHxuqwfqXsyuZe4DRDEpsY REhd84sY7HuEjaNnHLtf62Vnjsm7NGWOXbLZ1xbDzaovBd+43TNjizpie/iXhDlU4wvV Mof1wdzhqE4pPb5Z88sTsungyQXEfhFAcNqPTikGn7sKYis9kt1HIjM5oj5bJbN0zCzy hoJmaIqUkvYEuHR23FJ10ig8h6agib9TiiJT0UK5O/gjqggY83x7VqHkwF5XSUTMFNK4 uTkg== X-Gm-Message-State: AOJu0Yx+XtKo+xXYRxFA3GMAiqUrTMjhfEkxWoWxa2Ih1COblR1+Lkgf zH0gD1SoNbHpqS85DSt5RguC7WRmWf5wsoaQ7GIRS6xT4HUsWwQx09I8k/nQoAHe908coyq2os+ KwaOIRg== X-Google-Smtp-Source: AGHT+IHYL3O/Cbv2jm09q+kj8jdu7I3+wzJtsaAiUexntrMOt9TdJlvYVGl+QVySwxtGwHh68HxcdY/49Ac= X-Received: from pjg13.prod.google.com ([2002:a17:90b:3f4d:b0:301:1bf5:2f07]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2c85:b0:30a:2196:e654 with SMTP id 98e67ed59e1d1-30e8312bb11mr6078548a91.15.1747431018405; Fri, 16 May 2025 14:30:18 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 16 May 2025 14:28:33 -0700 In-Reply-To: <20250516212833.2544737-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250516212833.2544737-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1112.g889b7c5bd8-goog Message-ID: <20250516212833.2544737-9-seanjc@google.com> Subject: [PATCH v2 8/8] KVM: SVM: Flush cache only on CPUs running SEV guest From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, Zheyun Shen , Tom Lendacky , Kevin Loughlin , Kai Huang , Mingwei Zhang Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Zheyun Shen On AMD CPUs without ensuring cache consistency, each memory page reclamation in an SEV guest triggers a call to do WBNOINVD/WBINVD on all CPUs, thereby affecting the performance of other programs on the host. Typically, an AMD server may have 128 cores or more, while the SEV guest might only utilize 8 of these cores. Meanwhile, host can use qemu-affinity to bind these 8 vCPUs to specific physical CPUs. Therefore, keeping a record of the physical core numbers each time a vCPU runs can help avoid flushing the cache for all CPUs every time. Signed-off-by: Zheyun Shen Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Reviewed-by: Tom Lendacky --- arch/x86/kvm/svm/sev.c | 46 +++++++++++++++++++++++++++++++++++------- arch/x86/kvm/svm/svm.h | 1 + 2 files changed, 40 insertions(+), 7 deletions(-) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 9dcdeea954d3..e40fb2105dbc 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -446,7 +446,12 @@ static int __sev_guest_init(struct kvm *kvm, struct kv= m_sev_cmd *argp, init_args.probe =3D false; ret =3D sev_platform_init(&init_args); if (ret) - goto e_free; + goto e_free_asid; + + if (!zalloc_cpumask_var(&sev->have_run_cpus, GFP_KERNEL_ACCOUNT)) { + ret =3D -ENOMEM; + goto e_free_asid; + } =20 /* This needs to happen after SEV/SNP firmware initialization. */ if (vm_type =3D=3D KVM_X86_SNP_VM) { @@ -464,6 +469,8 @@ static int __sev_guest_init(struct kvm *kvm, struct kvm= _sev_cmd *argp, return 0; =20 e_free: + free_cpumask_var(sev->have_run_cpus); +e_free_asid: argp->error =3D init_args.error; sev_asid_free(sev); sev->asid =3D 0; @@ -708,16 +715,31 @@ static void sev_clflush_pages(struct page *pages[], u= nsigned long npages) } } =20 -static void sev_writeback_caches(void) +static void sev_writeback_caches(struct kvm *kvm) { + /* + * Note, the caller is responsible for ensuring correctness if the mask + * can be modified, e.g. if a CPU could be doing VMRUN. + */ + if (cpumask_empty(to_kvm_sev_info(kvm)->have_run_cpus)) + return; + /* * Ensure that all dirty guest tagged cache entries are written back * before releasing the pages back to the system for use. CLFLUSH will * not do this without SME_COHERENT, and flushing many cache lines * individually is slower than blasting WBINVD for large VMs, so issue - * WBNOINVD (or WBINVD if the "no invalidate" variant is unsupported). + * WBNOINVD (or WBINVD if the "no invalidate" variant is unsupported) + * on CPUs that have done VMRUN, i.e. may have dirtied data using the + * VM's ASID. + * + * For simplicity, never remove CPUs from the bitmap. Ideally, KVM + * would clear the mask when flushing caches, but doing so requires + * serializing multiple calls and having responding CPUs (to the IPI) + * mark themselves as still running if they are running (or about to + * run) a vCPU for the VM. */ - wbnoinvd_on_all_cpus(); + wbnoinvd_on_many_cpus(to_kvm_sev_info(kvm)->have_run_cpus); } =20 static unsigned long get_num_contig_pages(unsigned long idx, @@ -2770,7 +2792,7 @@ int sev_mem_enc_unregister_region(struct kvm *kvm, goto failed; } =20 - sev_writeback_caches(); + sev_writeback_caches(kvm); =20 __unregister_enc_region_locked(kvm, region); =20 @@ -2918,6 +2940,7 @@ void sev_vm_destroy(struct kvm *kvm) } =20 sev_asid_free(sev); + free_cpumask_var(sev->have_run_cpus); } =20 void __init sev_set_cpu_caps(void) @@ -3131,7 +3154,7 @@ static void sev_flush_encrypted_page(struct kvm_vcpu = *vcpu, void *va) return; =20 do_sev_writeback_caches: - sev_writeback_caches(); + sev_writeback_caches(vcpu->kvm); } =20 void sev_guest_memory_reclaimed(struct kvm *kvm) @@ -3144,7 +3167,7 @@ void sev_guest_memory_reclaimed(struct kvm *kvm) if (!sev_guest(kvm) || sev_snp_guest(kvm)) return; =20 - sev_writeback_caches(); + sev_writeback_caches(kvm); } =20 void sev_free_vcpu(struct kvm_vcpu *vcpu) @@ -3476,6 +3499,15 @@ int pre_sev_run(struct vcpu_svm *svm, int cpu) if (sev_es_guest(kvm) && !VALID_PAGE(svm->vmcb->control.vmsa_pa)) return -EINVAL; =20 + /* + * To optimize cache flushes when memory is reclaimed from an SEV VM, + * track physical CPUs that enter the guest for SEV VMs and thus can + * have encrypted, dirty data in the cache, and flush caches only for + * CPUs that have entered the guest. + */ + if (!cpumask_test_cpu(cpu, to_kvm_sev_info(kvm)->have_run_cpus)) + cpumask_set_cpu(cpu, to_kvm_sev_info(kvm)->have_run_cpus); + /* Assign the asid allocated with this SEV guest */ svm->asid =3D asid; =20 diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index e6f3c6a153a0..a7c6f07260cf 100644 --- a/arch/x86/kvm/svm/svm.h +++ b/arch/x86/kvm/svm/svm.h @@ -113,6 +113,7 @@ struct kvm_sev_info { void *guest_req_buf; /* Bounce buffer for SNP Guest Request input */ void *guest_resp_buf; /* Bounce buffer for SNP Guest Request output */ struct mutex guest_req_mutex; /* Must acquire before using bounce buffers= */ + cpumask_var_t have_run_cpus; /* CPUs that have done VMRUN for this VM. */ }; =20 #define SEV_POLICY_NODBG BIT_ULL(0) --=20 2.49.0.1112.g889b7c5bd8-goog