From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5112215AD9B for ; Fri, 9 Aug 2024 19:03:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230210; cv=none; b=d0BTnQle8x3j4RI/FfofYIzygyvBxz0HNOLD0dkDcQGyQalgx6FHb0nL7d9JNhWaFpZ1KTrwfhCOWT9V9j41YjThucx/pXvnkl2sVvFJXSEz9GMIjGDNR0/0KHFa1pLAgGqnfx5e/zpAPgk/0t2If9xSYkDKZkS8/6THsccOSeg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230210; c=relaxed/simple; bh=+mKRLKzoADpOE20G4o9BQCTbkCSsOug+v9doDlmgPOw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=B/GKmq+H+UZjq9xPfKRkiF7BvyJZLlhFeF4LcWBiBtDGXzAD+2AUx8aZrNzf2WL59T3r9lUBbT0wLC1vqZHrCRYzNGKLszlbjFx6szVlb4mL1U95zTnjNgot6AxGbwjjbJ2/QVrrBh2RbDHaDFfOce7CQeNhScXJRC5I0097pWw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=MRRt48XY; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="MRRt48XY" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-6c8f99fef10so2782251a12.3 for ; Fri, 09 Aug 2024 12:03:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230208; x=1723835008; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=GsIoGeUgslSIDQu3stBnO5MiBTFOAiP1ETUsvE40Zzk=; b=MRRt48XYz6NoJWHb17AFlSJga+T1iOTrY5u389ZIr9h/aiPb8vVqnFyX4Uk5kY5PL0 khxdE7MjLkoTvBqEBVJu67yNn8atyJCSqrBLoQKO2nmI0dkbDvvq+rq+NCdI6bqZcZTT p1sKiWigmVuQvWvUtzID/9MLjxkSsuLe4yNbYOHKaUZYMe0xWXXPVQOnoQWkGdt1znJB D3jpPKPcRLgoVA9I6Ir3hdNkllZvlqBQKD3PLX+e/PA9cpYS4eCazzMdS2pGTvYV8Yts 2K0ZMHlwYhfd0brWQWuQPnTl7ZdWZ8SR5jMjRBbT7dFfDi4WLxmZoHUtqIBdXYU+ZQdq 2jWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230208; x=1723835008; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=GsIoGeUgslSIDQu3stBnO5MiBTFOAiP1ETUsvE40Zzk=; b=OJTQsJGYupwkjlZ7wwuuBrBTjm4vj0chby+x7jgmnfTtsMU86FtPkW8yhpvHd1POlk NDgx5c2/rg1pjVGlLMwagCHFlJJ+2dTtsJqxLGIWLfRFRj9TVn7bHuq9R8i0lsRrjsQd 3ntfoxNj6bAJ1GMjLtTEfloZXuN6CV4rXls8DFvEwBFoEOhbptTl4fCWbXtbb2unoouM h/YSaMDmPwoBvn9tRPubHM7aTybKK1Ob16pB3SiKOqHd21jvA6ZtRhVYEidjShosidnN vBv3+bHZV6KgFqIlfhHhNVHm/hISKZaTD1L75Jb/cxDAw/4YHTh5FP2PhU/Xccaz7wQn vVmw== X-Forwarded-Encrypted: i=1; AJvYcCWMXKZy29LyiCBABDUUC73KxBAT/R9OyGOspf1PTUf4DzF/OrDF33j+VJzirC6qtCigklay7fWcE1ZjX+Yn3DUrOmALzHt2/31mjBJ1 X-Gm-Message-State: AOJu0YzMysO3ITKrNRqJpyj9rjoD2Ef2wXIJ8PXSK48RyDaggAiGfJE0 vOTgZt2fQDPL1asVLAPeuO9gj9j5TnuBOouAio5Q24Z3vt/YEWXXZqwIKdZXYToU60hHA31HqkI vcg== X-Google-Smtp-Source: AGHT+IE3eXOVlwsDRuB2+lPmtn5qtlNh4cfa2iZm6g38l3Rqzh9FsNEd+cI5zpswhamioOgLLQz+IgMJaoM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a63:8c52:0:b0:7c1:271a:c780 with SMTP id 41be03b00d2f7-7c3d2acc41emr4853a12.0.1723230208428; Fri, 09 Aug 2024 12:03:28 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:02:58 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-2-seanjc@google.com> Subject: [PATCH 01/22] KVM: x86: Disallow read-only memslots for SEV-ES and SEV-SNP (and TDX) From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Disallow read-only memslots for SEV-{ES,SNP} VM types, as KVM can't directly emulate instructions for ES/SNP, and instead the guest must explicitly request emulation. Unless the guest explicitly requests emulation without accessing memory, ES/SNP relies on KVM creating an MMIO SPTE, with the subsequent #NPF being reflected into the guest as a #VC. But for read-only memslots, KVM deliberately doesn't create MMIO SPTEs, because except for ES/SNP, doing so requires setting reserved bits in the SPTE, i.e. the SPTE can't be readable while also generating a #VC on writes. Because KVM never creates MMIO SPTEs and jumps directly to emulation, the guest never gets a #VC. And since KVM simply resumes the guest if ES/SNP guests trigger emulation, KVM effectively puts the vCPU into an infinite #NPF loop if the vCPU attempts to write read-only memory. Disallow read-only memory for all VMs with protected state, i.e. for upcoming TDX VMs as well as ES/SNP VMs. For TDX, it's actually possible to support read-only memory, as TDX uses EPT Violation #VE to reflect the fault into the guest, e.g. KVM could configure read-only SPTEs with RX protections and SUPPRESS_VE=3D0. But there is no strong use case for supporting read-only memslots on TDX, e.g. the main historical usage is to emulate option ROMs, but TDX disallows executing from shared memory. And if someone comes along with a legitimate, strong use case, the restriction can always be lifted for TDX. Don't bother trying to retroactively apply the restriction to SEV-ES VMs that are created as type KVM_X86_DEFAULT_VM. Read-only memslots can't possibly work for SEV-ES, i.e. disallowing such memslots is really just means reporting an error to userspace instead of silently hanging vCPUs. Trying to deal with the ordering between KVM_SEV_INIT and memslot creation isn't worth the marginal benefit it would provide userspace. Fixes: 26c44aa9e076 ("KVM: SEV: define VM types for SEV and SEV-ES") Fixes: 1dfe571c12cf ("KVM: SEV: Add initial SEV-SNP support") Cc: Peter Gonda Cc: Michael Roth Cc: Vishal Annapurve Cc: Ackerly Tng Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 2 ++ include/linux/kvm_host.h | 7 +++++++ virt/kvm/kvm_main.c | 5 ++--- 3 files changed, 11 insertions(+), 3 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 950a03e0181e..37c4a573e5fb 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -2191,6 +2191,8 @@ void kvm_configure_mmu(bool enable_tdp, int tdp_force= d_root_level, #define kvm_arch_has_private_mem(kvm) false #endif =20 +#define kvm_arch_has_readonly_mem(kvm) (!(kvm)->arch.has_protected_state) + static inline u16 kvm_read_ldt(void) { u16 ldt; diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 689e8be873a7..62a3d1c0cc07 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -715,6 +715,13 @@ static inline bool kvm_arch_has_private_mem(struct kvm= *kvm) } #endif =20 +#ifndef kvm_arch_has_readonly_mem +static inline bool kvm_arch_has_readonly_mem(struct kvm *kvm) +{ + return IS_ENABLED(CONFIG_HAVE_KVM_READONLY_MEM); +} +#endif + struct kvm_memslots { u64 generation; atomic_long_t last_used_slot; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index d0788d0a72cc..fad2d5932844 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -1578,15 +1578,14 @@ static int check_memory_region_flags(struct kvm *kv= m, if (mem->flags & KVM_MEM_GUEST_MEMFD) valid_flags &=3D ~KVM_MEM_LOG_DIRTY_PAGES; =20 -#ifdef CONFIG_HAVE_KVM_READONLY_MEM /* * GUEST_MEMFD is incompatible with read-only memslots, as writes to * read-only memslots have emulated MMIO, not page fault, semantics, * and KVM doesn't allow emulated MMIO for private memory. */ - if (!(mem->flags & KVM_MEM_GUEST_MEMFD)) + if (kvm_arch_has_readonly_mem(kvm) && + !(mem->flags & KVM_MEM_GUEST_MEMFD)) valid_flags |=3D KVM_MEM_READONLY; -#endif =20 if (mem->flags & ~valid_flags) return -EINVAL; --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0D64615FD15 for ; Fri, 9 Aug 2024 19:03:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230212; cv=none; b=nwxn1+owur9o1awnkEelRrAf2vIgiZhJj/KNeWNunGDLpyRu/cOUG05US5sGVsfLGveFvet9yoms83wT09n6+BhKryA5BbLH/jViVLiKwAPVU6YW7E0yEcMtJgB+duAn+KbwUOnb/JXbWHVaaarucZ1cKGdkw7pLQPQ8+iBkHGI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230212; c=relaxed/simple; bh=/3Mc0aYUnYlmMvDT2VZdsvGshabqaL6eclOqKW0yriU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=sa8r55LlZjovK5guagtHk2d9Q4F9k0R/tSfktaXx1dF+vhvhUtdhGPmBLWtLQJNBmTqTBk9/b4Vy9jTPAIl++wjwn7jn63IaFkJEZy79DaqsIu5hc5hK1eSnGWhobAegcIni4QpL5ZE93VJnMMDSVOhY3bxyuF4UeN7M8r/CciE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=oDK1iV3p; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="oDK1iV3p" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-70d34fa1726so2497865b3a.1 for ; Fri, 09 Aug 2024 12:03:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230210; x=1723835010; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=NqYKYc5ZaZDxsjSNj47554NGsl1XvrBnLwjmIYW0Wgc=; b=oDK1iV3purwsR54IVPgw81HLbhgJHjrtb77iJTs3+5eF2ctuvhuFITBCMxYca4DIlQ 9qZztRVNSIXlxffv6qBQErqEt8YU86uDzVcIBC4DHfKBn6IFt3QI8DkXQVN+QTLVAgSK ciE5Sh+gutRUC/0VHpfdsK44jQZ8PhNeBPB7sP2r7W/d+iXC9QPGAvFk6GjJ4FDpWGmK m3pCezCnruPMFGO4BX4MeFC0EUzAg6H9RvrUByGg+VfAP2SEoFu+6rlniXKLsVB0ffri o+SStKmeZ1PZoamq+dZCMQ8lYn/kghkWjDHhtbsRg3KXPZoYE3srKLk7KuKMe/T19g9g l/2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230210; x=1723835010; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=NqYKYc5ZaZDxsjSNj47554NGsl1XvrBnLwjmIYW0Wgc=; b=XQsqit4fLw9yqTY+YL5bnFej/qXfMGVprRq9ZnOEnnaavSIUd1EYRoUDqkeUMpIhVp R7Z2ckJ18wlj7XxWx4QIv+14ndS55FDXIsX80DnnlWqxiLsgiImk1qlip8a3Yx5WZxqz 1C+EoYaMYCYsFnvPpnoWmFrUghXYV+tOY0Exbp0j8EyZOk+lm1EeEdbxifyYROCZ2LCH uxupiqz36NJM+LV7FOdrgrOOc6XcYeK27d+ZY8BkanUv6XyxtmYgXn2wSxIExgrwxoaZ a/ZCksEV1O1+nohQoCtLv+bknA/fp6zbeizdFIvEOknczm96r+yWfYXUPQ6ja96c4QfC ZyCg== X-Forwarded-Encrypted: i=1; AJvYcCXpLsp0HN0FOqMB4v2gRu1zf4ljdTQwsQTzoMzOZQdZLuoqTJfqwyCxAAzn91vdDhDhx+DlvokWXnCrs4A8RXLR3TctKLBKOEHhkaER X-Gm-Message-State: AOJu0YwbyOgEauHcJddTfTCnPOYfz0BPyT5r2qTkUa+lANGZNwDi4L59 WOeqNunNyivz2k/jQKS5VQW1xV3QqSymmIDGS44WPxoHZ1Xs2+CiD5vCvD/r9POFOB1v/hzQEgM Ehw== X-Google-Smtp-Source: AGHT+IHgCacj5m1MLlQUENYq6BFjimFqJz5RZsNs8W7hnjalJaUAINnEvpiTyC8RkzL0KTMAd+i6sITJmiw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6a00:66d7:b0:706:3421:7406 with SMTP id d2e1a72fcca58-710dc62cb1bmr198733b3a.1.1723230210279; Fri, 09 Aug 2024 12:03:30 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:02:59 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-3-seanjc@google.com> Subject: [PATCH 02/22] KVM: VMX: Set PFERR_GUEST_{FINAL,PAGE}_MASK if and only if the GVA is valid From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Set PFERR_GUEST_{FINAL,PAGE}_MASK based on EPT_VIOLATION_GVA_TRANSLATED if and only if EPT_VIOLATION_GVA_IS_VALID is also set in exit qualification. Per the SDM, bit 8 (EPT_VIOLATION_GVA_TRANSLATED) is valid if and only if bit 7 (EPT_VIOLATION_GVA_IS_VALID) is set, and is '0' if bit 7 is '0'. Bit 7 (a.k.a. EPT_VIOLATION_GVA_IS_VALID) Set if the guest linear-address field is valid. The guest linear-address field is valid for all EPT violations except those resulting from an attempt to load the guest PDPTEs as part of the execution of the MOV CR instruction and those due to trace-address pre-translation Bit 8 (a.k.a. EPT_VIOLATION_GVA_TRANSLATED) If bit 7 is 1: =E2=80=A2 Set if the access causing the EPT violation is to a guest-phy= sical address that is the translation of a linear address. =E2=80=A2 Clear if the access causing the EPT violation is to a paging-= structure entry as part of a page walk or the update of an accessed or dirty bi= t. Reserved if bit 7 is 0 (cleared to 0). Failure to guard the logic on GVA_IS_VALID results in KVM marking the page fault as PFERR_GUEST_PAGE_MASK when there is no known GVA, which can put the vCPU into an infinite loop due to kvm_mmu_page_fault() getting false positive on its PFERR_NESTED_GUEST_PAGE logic (though only because that logic is also buggy/flawed). In practice, this is largely a non-issue because so GVA_IS_VALID is almost always set. However, when TDX comes along, GVA_IS_VALID will *never* be set, as the TDX Module deliberately clears bits 12:7 in exit qualification, e.g. so that the faulting virtual address and other metadata that aren't practically useful for the hypervisor aren't leaked to the untrusted host. When exit is due to EPT violation, bits 12-7 of the exit qualification are cleared to 0. Fixes: eebed2438923 ("kvm: nVMX: Add support for fast unprotection of neste= d guest page tables") Cc: stable@vger.kernel.org Signed-off-by: Sean Christopherson Reviewed-by: Yuan Yao --- arch/x86/kvm/vmx/vmx.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index f18c2d8c7476..52de013550e9 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -5804,8 +5804,9 @@ static int handle_ept_violation(struct kvm_vcpu *vcpu) error_code |=3D (exit_qualification & EPT_VIOLATION_RWX_MASK) ? PFERR_PRESENT_MASK : 0; =20 - error_code |=3D (exit_qualification & EPT_VIOLATION_GVA_TRANSLATED) !=3D = 0 ? - PFERR_GUEST_FINAL_MASK : PFERR_GUEST_PAGE_MASK; + if (error_code & EPT_VIOLATION_GVA_IS_VALID) + error_code |=3D (exit_qualification & EPT_VIOLATION_GVA_TRANSLATED) ? + PFERR_GUEST_FINAL_MASK : PFERR_GUEST_PAGE_MASK; =20 /* * Check that the GPA doesn't exceed physical memory limits, as that is --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE02F166312 for ; Fri, 9 Aug 2024 19:03:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230214; cv=none; b=L/qg9dO74rMAYPFiuANy9aUiXIpiJ+1EhNXRMGnS/zpwcimgB6tvAic+dyQ7rYOSGaqBhcefX8iVCtizVDMKqWIuzKIO26LHJ5MvhEPuSXjs16dFDWgftT8OBAxKNXOEaVuKxlsqn6BwE6xuDkxD/2A21i5Alh3MyX9XChdeP/k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230214; c=relaxed/simple; bh=Tp+lr11Yh7ehyhJwxgWROxZWPiF2r5kHK9R10+ISb28=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=h4K+Hyrixkk6bmaO8KM63W8e1ug3N4MGVYB6gfptLeLArkW6o0b9niJJYkvJT7kdxPViCTtx6DALtG84FL83umUZroyjgscc/nO+x/5eTl6V4Nht6qqi97aj60LTdVe8f7/e3mHhX/UnL4MTEFbLV69tF0O2e0IgXOJomENb7mU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=cVCJjKqV; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="cVCJjKqV" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-70d1cc32058so2273211b3a.2 for ; Fri, 09 Aug 2024 12:03:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230212; x=1723835012; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=q+FRSzKgzmzlpy3KAiAZeqdbkMvFgdEpyfJ8A8CS9Hc=; b=cVCJjKqV6sMP4cfvdSeVUgBlASouMcV5bFJtif7ONxi77H4hsswecOFjsUi3WOBxtS Q6UbHyEmgiF21+KU9cpU0BRTHdFRnjMBsXgD0mLNL7cCZJhYCgV8BtSVIRJ0Q2wOppFA bHc6DCvB1RsOyqzkaHpVR5uv6k5K+X3nY2o/nWs5otxUZNgQQZPdxasbTMgQSfPuhvKr UHQ5BdPIOctKUfeKbGRp73m17EB52cstBgMo2465Q3oG5mpV4UBEPBAa4jXrKLg+68JN Q1hoV8SvT8yg3dX2W36RR4kK5IRfgY+DwTi/1damIU4kOxScq5+bvJ8k7rFvCTmYcVbw t11g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230212; x=1723835012; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=q+FRSzKgzmzlpy3KAiAZeqdbkMvFgdEpyfJ8A8CS9Hc=; b=t9hKCGXBNNGrnUXcjo9JSXnA0vI0dHUhoNtMMsA7K/dDH1vKRfP2zGGvyVGoGH/WHt M+0uqA+1ppashwB/uEQxEqFqEP/rNmcy3vLwbL1PFK/gjUZoZvbYxG//udJoyVc8RQ9j uIklj4qzFxUWnxySV/5UcHfYwQvumvLHhKlQudQXhg//lZsdMcKpslfjP07/C5+KUqls QkGE/nvMFEHP+aA1hOg9wIf91BjaI94N3VcyYPAxplapzBplRw5dRDysyLgSVIAZolqi 57to+iXgHZbd0FuypakgMuKxWrSN6mgma27DRIXZgpAktYrdhbB6zP/I+9oCp3+dW0T8 5SMQ== X-Forwarded-Encrypted: i=1; AJvYcCXIfPeyVXs5MnjtPLCV0lQM69syWeI+VdGtlR37StZ2Jb1hvBApWp9hutBomfuOPNXxjV/L03nF8IyDFhQ=@vger.kernel.org X-Gm-Message-State: AOJu0Ywkwv3hItW4cQ6+zs63cH/4HBA9c7g+Q4t2PI5U00XF1sETTdsn AKU6+veIAri3UpApguUicyHqEUQCgo6spm0BobOf8sYPhV8R6/RZ0zBcZ6SYoWk27Xeax8SoimF PNw== X-Google-Smtp-Source: AGHT+IHxQ+GgudCpTB9fCFij4ArNeNlFVa8zm1S935oIh/wd+bm22sj5O3d+Nthc3rB2/1PkOghLapua7zM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6a00:138b:b0:70f:84b6:8634 with SMTP id d2e1a72fcca58-710dc2cfb49mr128985b3a.0.1723230212170; Fri, 09 Aug 2024 12:03:32 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:03:00 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-4-seanjc@google.com> Subject: [PATCH 03/22] KVM: x86/mmu: Trigger unprotect logic only on write-protection page faults From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Trigger KVM's various "unprotect gfn" paths if and only if the page fault was a write to a write-protected gfn. To do so, add a new page fault return code, RET_PF_WRITE_PROTECTED, to explicitly and precisely track such page faults. If a page fault requires emulation for any MMIO (or any reason besides write-protection), trying to unprotect the gfn is pointless and risks putting the vCPU into an infinite loop. E.g. KVM will put the vCPU into an infinite loop if the vCPU manages to trigger MMIO on a page table walk. Fixes: 147277540bbc ("kvm: svm: Add support for additional SVM NPF error co= des") Cc: stable@vger.kernel.org Signed-off-by: Sean Christopherson Reviewed-by: Yuan Yao --- arch/x86/kvm/mmu/mmu.c | 78 +++++++++++++++++++-------------- arch/x86/kvm/mmu/mmu_internal.h | 3 ++ arch/x86/kvm/mmu/mmutrace.h | 1 + arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/mmu/tdp_mmu.c | 6 +-- 5 files changed, 53 insertions(+), 37 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 901be9e420a4..e3aa04c498ea 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2914,10 +2914,8 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struc= t kvm_memory_slot *slot, trace_kvm_mmu_set_spte(level, gfn, sptep); } =20 - if (wrprot) { - if (write_fault) - ret =3D RET_PF_EMULATE; - } + if (wrprot && write_fault) + ret =3D RET_PF_WRITE_PROTECTED; =20 if (flush) kvm_flush_remote_tlbs_gfn(vcpu->kvm, gfn, level); @@ -4549,7 +4547,7 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, s= truct kvm_page_fault *fault return RET_PF_RETRY; =20 if (page_fault_handle_page_track(vcpu, fault)) - return RET_PF_EMULATE; + return RET_PF_WRITE_PROTECTED; =20 r =3D fast_page_fault(vcpu, fault); if (r !=3D RET_PF_INVALID) @@ -4642,7 +4640,7 @@ static int kvm_tdp_mmu_page_fault(struct kvm_vcpu *vc= pu, int r; =20 if (page_fault_handle_page_track(vcpu, fault)) - return RET_PF_EMULATE; + return RET_PF_WRITE_PROTECTED; =20 r =3D fast_page_fault(vcpu, fault); if (r !=3D RET_PF_INVALID) @@ -4726,6 +4724,9 @@ static int kvm_tdp_map_page(struct kvm_vcpu *vcpu, gp= a_t gpa, u64 error_code, case RET_PF_EMULATE: return -ENOENT; =20 + case RET_PF_WRITE_PROTECTED: + return -EPERM; + case RET_PF_RETRY: case RET_PF_CONTINUE: case RET_PF_INVALID: @@ -5960,6 +5961,41 @@ void kvm_mmu_track_write(struct kvm_vcpu *vcpu, gpa_= t gpa, const u8 *new, write_unlock(&vcpu->kvm->mmu_lock); } =20 +static int kvm_mmu_write_protect_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or= _gpa, + u64 error_code, int *emulation_type) +{ + bool direct =3D vcpu->arch.mmu->root_role.direct; + + /* + * Before emulating the instruction, check if the error code + * was due to a RO violation while translating the guest page. + * This can occur when using nested virtualization with nested + * paging in both guests. If true, we simply unprotect the page + * and resume the guest. + */ + if (direct && + (error_code & PFERR_NESTED_GUEST_PAGE) =3D=3D PFERR_NESTED_GUEST_PAGE= ) { + kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(cr2_or_gpa)); + return RET_PF_FIXED; + } + + /* + * The gfn is write-protected, but if emulation fails we can still + * optimistically try to just unprotect the page and let the processor + * re-execute the instruction that caused the page fault. Do not allow + * retrying MMIO emulation, as it's not only pointless but could also + * cause us to enter an infinite loop because the processor will keep + * faulting on the non-existent MMIO address. Retrying an instruction + * from a nested guest is also pointless and dangerous as we are only + * explicitly shadowing L1's page tables, i.e. unprotecting something + * for L1 isn't going to magically fix whatever issue cause L2 to fail. + */ + if (!mmio_info_in_cache(vcpu, cr2_or_gpa, direct) && !is_guest_mode(vcpu)) + *emulation_type |=3D EMULTYPE_ALLOW_RETRY_PF; + + return RET_PF_EMULATE; +} + int noinline kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u= 64 error_code, void *insn, int insn_len) { @@ -6005,6 +6041,10 @@ int noinline kvm_mmu_page_fault(struct kvm_vcpu *vcp= u, gpa_t cr2_or_gpa, u64 err if (r < 0) return r; =20 + if (r =3D=3D RET_PF_WRITE_PROTECTED) + r =3D kvm_mmu_write_protect_fault(vcpu, cr2_or_gpa, error_code, + &emulation_type); + if (r =3D=3D RET_PF_FIXED) vcpu->stat.pf_fixed++; else if (r =3D=3D RET_PF_EMULATE) @@ -6015,32 +6055,6 @@ int noinline kvm_mmu_page_fault(struct kvm_vcpu *vcp= u, gpa_t cr2_or_gpa, u64 err if (r !=3D RET_PF_EMULATE) return 1; =20 - /* - * Before emulating the instruction, check if the error code - * was due to a RO violation while translating the guest page. - * This can occur when using nested virtualization with nested - * paging in both guests. If true, we simply unprotect the page - * and resume the guest. - */ - if (vcpu->arch.mmu->root_role.direct && - (error_code & PFERR_NESTED_GUEST_PAGE) =3D=3D PFERR_NESTED_GUEST_PAGE= ) { - kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(cr2_or_gpa)); - return 1; - } - - /* - * vcpu->arch.mmu.page_fault returned RET_PF_EMULATE, but we can still - * optimistically try to just unprotect the page and let the processor - * re-execute the instruction that caused the page fault. Do not allow - * retrying MMIO emulation, as it's not only pointless but could also - * cause us to enter an infinite loop because the processor will keep - * faulting on the non-existent MMIO address. Retrying an instruction - * from a nested guest is also pointless and dangerous as we are only - * explicitly shadowing L1's page tables, i.e. unprotecting something - * for L1 isn't going to magically fix whatever issue cause L2 to fail. - */ - if (!mmio_info_in_cache(vcpu, cr2_or_gpa, direct) && !is_guest_mode(vcpu)) - emulation_type |=3D EMULTYPE_ALLOW_RETRY_PF; emulate: return x86_emulate_instruction(vcpu, cr2_or_gpa, emulation_type, insn, insn_len); diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_interna= l.h index 1721d97743e9..50d2624111f8 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -258,6 +258,8 @@ int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, struct kv= m_page_fault *fault); * RET_PF_CONTINUE: So far, so good, keep handling the page fault. * RET_PF_RETRY: let CPU fault again on the address. * RET_PF_EMULATE: mmio page fault, emulate the instruction directly. + * RET_PF_WRITE_PROTECTED: the gfn is write-protected, either unprotected = the + * gfn and retry, or emulate the instruction direc= tly. * RET_PF_INVALID: the spte is invalid, let the real page fault path updat= e it. * RET_PF_FIXED: The faulting entry has been fixed. * RET_PF_SPURIOUS: The faulting entry was already fixed, e.g. by another = vCPU. @@ -274,6 +276,7 @@ enum { RET_PF_CONTINUE =3D 0, RET_PF_RETRY, RET_PF_EMULATE, + RET_PF_WRITE_PROTECTED, RET_PF_INVALID, RET_PF_FIXED, RET_PF_SPURIOUS, diff --git a/arch/x86/kvm/mmu/mmutrace.h b/arch/x86/kvm/mmu/mmutrace.h index 195d98bc8de8..f35a830ce469 100644 --- a/arch/x86/kvm/mmu/mmutrace.h +++ b/arch/x86/kvm/mmu/mmutrace.h @@ -57,6 +57,7 @@ TRACE_DEFINE_ENUM(RET_PF_CONTINUE); TRACE_DEFINE_ENUM(RET_PF_RETRY); TRACE_DEFINE_ENUM(RET_PF_EMULATE); +TRACE_DEFINE_ENUM(RET_PF_WRITE_PROTECTED); TRACE_DEFINE_ENUM(RET_PF_INVALID); TRACE_DEFINE_ENUM(RET_PF_FIXED); TRACE_DEFINE_ENUM(RET_PF_SPURIOUS); diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 69941cebb3a8..a722a3c96af9 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -805,7 +805,7 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, str= uct kvm_page_fault *fault =20 if (page_fault_handle_page_track(vcpu, fault)) { shadow_page_table_clear_flood(vcpu, fault->addr); - return RET_PF_EMULATE; + return RET_PF_WRITE_PROTECTED; } =20 r =3D mmu_topup_memory_caches(vcpu, true); diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index c7dc49ee7388..8bf44ac9372f 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -1046,10 +1046,8 @@ static int tdp_mmu_map_handle_target_level(struct kv= m_vcpu *vcpu, * protected, emulation is needed. If the emulation was skipped, * the vCPU would have the same fault again. */ - if (wrprot) { - if (fault->write) - ret =3D RET_PF_EMULATE; - } + if (wrprot && fault->write) + ret =3D RET_PF_WRITE_PROTECTED; =20 /* If a MMIO SPTE is installed, the MMIO will need to be emulated. */ if (unlikely(is_mmio_spte(vcpu->kvm, new_spte))) { --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 092B91684AB for ; Fri, 9 Aug 2024 19:03:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230216; cv=none; b=hL6bRo1iATtQ5l8LBJMlAKZGBFde3EeJGi/v2Zj9EqgRGDiN7XOIyIc7ypNPmkHiN+P0kKfoI+TM5ZrQaJb41NOi4bylyyDORhdvuxYnvS0Gfj26a52mRuTFoAnzjuHfekepZ6aG7/mRgmkM8yypm1noBENe5zSUIf/jhnz0ZMo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230216; c=relaxed/simple; bh=z+MxKCE9NuIwIw+7Qzhm3RReleKGu2hQZGUj5H5VOPg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Nkfv//9dsZaVv90oeopetfiliGThSGU2+DxEe4UuQMoWNiArqFqXApf3PggeoicshqcEU9ZllP70ZQqiehmTDtvsIWuk/ulSRRPL1oCeNm0mU3LOTmliKwdKSIv1Ri6ZwAWA3Cz/IkcMeIdA81uILX6ia8e+3RfTu9ECm034Bc8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=qLU9QVMy; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="qLU9QVMy" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-7a28f78c67aso2435020a12.1 for ; Fri, 09 Aug 2024 12:03:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230214; x=1723835014; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=OPGLZbFKP8CWiTTck6ADt4ijjePzDJ7PBKgxWrucHOI=; b=qLU9QVMyrGlfx9zlGzdJJRiuoYpyh0h/2JIr3QeGWq9NuGsphvy/yoBEfVBi9nlADP VGJLNNyIu1Y79ARtSZ+rPOgwZccUGubROkNFqYmKvfGFlLq7z0X6q7PE3R7FomXynYtF p/nYqPDv2oVUNjF5qLixp+p5ftX4UeWtgnNbEYenjyLC7s3Qii/RhEVVrpwTCpkSNhO9 aWQcokWuY+TT7QkOrWKNYHQIBXOc2bdZAuN1UTs/mLmlgvJOZSeyA54EE4KslecthvYV +x25wqtPfKE6x4fGbA1pM/44H2EaSEs63VOnPbNsLJroA25Fn4w/NwOM4iG+Lkz3zuEv gJ3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230214; x=1723835014; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OPGLZbFKP8CWiTTck6ADt4ijjePzDJ7PBKgxWrucHOI=; b=JzZH+klpczTZ0BkF6PT2ftHT2qVP94tLtpDHujw6DmA/dKGcpFf5NH+/LZjaW+1zSy JJuremTgVWKMzQewwCAtM3arAI1CRyDDrrNwCuQMAxKbpJk79F8LZ2xyhvvsNm2nzILY hs/tS77nnXAkc3FSlkHekaym+3kOaLgxmuiyCJKSBM5F70SVZ5jgmTefTcReL6XGEzwX 7JxzKmOzyzdsq4xlVnGkUDNAB0rLG5hbLGHKMFYzm/Z50eDQ8IxAoM7/bppisUDqCKq5 Nh+tFeA07LOV/7gLsoQtPlrgYbrOggjk1I7noGEvzbtOYOkojs1tTu5OYQuLJyRMJxZf bV0w== X-Forwarded-Encrypted: i=1; AJvYcCXWvnvJ3WvjsC4h+D6PT1PAd00MOZbtfmoZCf2zBblzlWT31RfgjkGhXgJxizksvR36taBzr7nwMPg7Xqk=@vger.kernel.org X-Gm-Message-State: AOJu0YxIhsfIMtL3HY38tFOtC2+43JHbLIeLqGf6oZPTv1QslgpleNV8 2YZsEn3JevLHlIMHwagw0A9rfjL2P2lQCUEZSGpd9ycV5sKpCSLuhYrv9VR8AHoYXgZcN9qNd7M q5g== X-Google-Smtp-Source: AGHT+IGhq7PH0PBhBiiGOey+3zlw6D8WzCjhgNtCKlfYkonI7WV9WfpMY5Brvwz4Erhp7Q90FZZ+mmQPmnQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:903:189:b0:200:98ed:3622 with SMTP id d9443c01a7336-200ae4e10damr1799675ad.6.1723230213995; Fri, 09 Aug 2024 12:03:33 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:03:01 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-5-seanjc@google.com> Subject: [PATCH 04/22] KVM: x86/mmu: Skip emulation on page fault iff 1+ SPs were unprotected From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When doing "fast unprotection" of nested TDP page tables, skip emulation if and only if at least one gfn was unprotected, i.e. continue with emulation if simply resuming is likely to hit the same fault and risk putting the vCPU into an infinite loop. Note, it's entirely possible to get a false negative, e.g. if a different vCPU faults on the same gfn and unprotects the gfn first, but that's a relatively rare edge case, and emulating is still functionally ok, i.e. the risk of putting the vCPU isn't an infinite loop isn't justified. Fixes: 147277540bbc ("kvm: svm: Add support for additional SVM NPF error co= des") Cc: stable@vger.kernel.org Signed-off-by: Sean Christopherson Reviewed-by: Yuan Yao --- arch/x86/kvm/mmu/mmu.c | 28 ++++++++++++++++++++-------- 1 file changed, 20 insertions(+), 8 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index e3aa04c498ea..95058ac4b78c 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -5967,17 +5967,29 @@ static int kvm_mmu_write_protect_fault(struct kvm_v= cpu *vcpu, gpa_t cr2_or_gpa, bool direct =3D vcpu->arch.mmu->root_role.direct; =20 /* - * Before emulating the instruction, check if the error code - * was due to a RO violation while translating the guest page. - * This can occur when using nested virtualization with nested - * paging in both guests. If true, we simply unprotect the page - * and resume the guest. + * Before emulating the instruction, check to see if the access may be + * due to L1 accessing nested NPT/EPT entries used for L2, i.e. if the + * gfn being written is for gPTEs that KVM is shadowing and has write- + * protected. Because AMD CPUs walk nested page table using a write + * operation, walking NPT entries in L1 can trigger write faults even + * when L1 isn't modifying PTEs, and thus result in KVM emulating an + * excessive number of L1 instructions without triggering KVM's write- + * flooding detection, i.e. without unprotecting the gfn. + * + * If the error code was due to a RO violation while translating the + * guest page, the current MMU is direct (L1 is active), and KVM has + * shadow pages, then the above scenario is likely being hit. Try to + * unprotect the gfn, i.e. zap any shadow pages, so that L1 can walk + * its NPT entries without triggering emulation. If one or more shadow + * pages was zapped, skip emulation and resume L1 to let it natively + * execute the instruction. If no shadow pages were zapped, then the + * write-fault is due to something else entirely, i.e. KVM needs to + * emulate, as resuming the guest will put it into an infinite loop. */ if (direct && - (error_code & PFERR_NESTED_GUEST_PAGE) =3D=3D PFERR_NESTED_GUEST_PAGE= ) { - kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(cr2_or_gpa)); + (error_code & PFERR_NESTED_GUEST_PAGE) =3D=3D PFERR_NESTED_GUEST_PAGE= && + kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(cr2_or_gpa))) return RET_PF_FIXED; - } =20 /* * The gfn is write-protected, but if emulation fails we can still --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E100D16A935 for ; Fri, 9 Aug 2024 19:03:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230218; cv=none; b=s6kBV6tjLGLkOLmKB2gulgjK5W7NuTp2lnKfXb9eQFWHYKo8pSsybjyTXgyt7BJdg6ak9QDS8O70ryxcL63rwVf7WifhupiUBKbYMaPZvijvW5zktDrXgqNsinbYSfsUELQ7WWWwUlFskJqO29jc23j1wtCGkXuN5M/BdLrsm68= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230218; c=relaxed/simple; bh=BWJKmkxdJpIr0WZAUrQe0T0TZqNNa6fnGorGum5Bq48=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=KTTgC27mKESND3PXl9qk9igVq91ANHm2YFTyuRqm0V/lWDc59HRCOw3ryQb48iQcyJMqCpKjai25VciRaVk7P8yXE+8l9nomUS039q/TAb6DniYc7H7kveyki30bTfr7PucJ05q1XC71tvXFp38rEGpZoRFAWL2cCRPhK+HmSGg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=JLyNomoJ; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="JLyNomoJ" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2cb81c562edso2687836a91.2 for ; Fri, 09 Aug 2024 12:03:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230216; x=1723835016; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=mzGayWAqBVAh7SBo8RHpGvCYGXHGAMMHoDF3FfeyQIE=; b=JLyNomoJzKMOk3xBSQvgz5RN0lcS5ZE4ln6rsJ/dcwYClcq8fNVVlpJflw8+JRP7B5 2viuYYgkhZTuw4VhhVEOy0vLLq45uev/DyRhY/r41xqIGcJunyDWgCFh91ko6QEl8Yii SJegWlQKrBgXNydtEyVzHs+FxIIi3o2siyDzOTMjDK+goGpGp8UN06cTzakjkHsP76xl e/NHeHk+MUhBS4LoNLgpH5sd4NSLlQkwajjwZxFZeHnodH1w1DL/hkidvri8xvzXkmVi ZgqTTuj5SYSn5kD42u1EQgdimk25+IBoXR6MAqev2AlI4yVBHU2R08p66rJWrxblUATb WYPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230216; x=1723835016; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=mzGayWAqBVAh7SBo8RHpGvCYGXHGAMMHoDF3FfeyQIE=; b=l6japEJOJJbv7WkBeynjFn61PD+2SiSgzo2IhlhkOmlJ1xn4B48e+V8oyyqDQRoEnJ Og6RhEBCBm8X85Nyr3Ivb/MmVmGrZMoMAZME8/vXNsifJQ6282VDeO85lLMAQi6JQeln 0wuMXxcfbJy5fb8CubR2nsK0n0VxfAhGVA8LXEFJSxaGByQDa4KoKn2muPQX4P4rQP1m GbFnIYoltyH6e4k7pEFHo7kGMTer9nJnZWcxyxvi4gNw+xUha1Ibf/FmT0sX/tCrasYE ETa17N8Nh0+OTQGHvLxjQY8udo4S1QtcwoCNTgs7Ayti/JJi8oMxD4rLmtBDNeJP42sk eAtQ== X-Forwarded-Encrypted: i=1; AJvYcCUlX6mc11+wWcvO4mTmeq0TaYvWdRLQ+NKxkwtCu4mWln4VUFkMDxV/G8rrI5MlDkUn+7OsJDfSAwi0N5R2c/drZuP7cymWGHHBiaIm X-Gm-Message-State: AOJu0Yx3WQg7b9vGK+w99Bs3IO3osZQ9dZvWVoJU3knQbqsGFLs3x5w7 RAL303NkAW78HK3SDxE2pwKF8rJcIl/4ZOJzgxpVLQx57s9XHbSFaC4WfyP+gcs29bMbL1Bhe4X 11Q== X-Google-Smtp-Source: AGHT+IFwoW6NpOd0vaQoEiZzjXJyMSsJDryKzJdxexhUyUmPk7TNdNT4VJMc4BZ52dupRQWvikDJT0Z/Pa8= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:90a:749:b0:2c9:6504:6787 with SMTP id 98e67ed59e1d1-2d1e8044b48mr5297a91.3.1723230215949; Fri, 09 Aug 2024 12:03:35 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:03:02 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-6-seanjc@google.com> Subject: [PATCH 05/22] KVM: x86: Retry to-be-emulated insn in "slow" unprotect path iff sp is zapped From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Resume the guest and thus skip emulation of a non-PTE-writing instruction if and only if unprotecting the gfn actually zapped at least one shadow page. If the gfn is write-protected for some reason other than shadow paging, attempting to unprotect the gfn will effectively fail, and thus retrying the instruction is all but guaranteed to be pointless. This bug has existed for a long time, but was effectively fudged around by the retry RIP+address anti-loop detection. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index af6c8cf6a37a..2072cceac68f 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8967,14 +8967,14 @@ static bool retry_instruction(struct x86_emulate_ct= xt *ctxt, if (ctxt->eip =3D=3D last_retry_eip && last_retry_addr =3D=3D cr2_or_gpa) return false; =20 + if (!vcpu->arch.mmu->root_role.direct) + gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, cr2_or_gpa, NULL); + + if (!kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa))) + return false; + vcpu->arch.last_retry_eip =3D ctxt->eip; vcpu->arch.last_retry_addr =3D cr2_or_gpa; - - if (!vcpu->arch.mmu->root_role.direct) - gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, cr2_or_gpa, NULL); - - kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa)); - return true; } =20 --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D7C3E16B751 for ; Fri, 9 Aug 2024 19:03:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230220; cv=none; b=HukfvXqkWtMxYXOaqQxmw5+syU+ghJC8gWPl+d5Xn9AerOdePdrbIfm8q8CB4+fUgOQdaA+ISIZwJPYdY6mybJGhT3K8gLEK0zE0m7vKI25juWgWFqFaIyPAI74ke7KWajk1ImBrozSeSXff+5IUUMI5aIspZ3PDVcS1Qr8wspU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230220; c=relaxed/simple; bh=21Oqbef94jP18zcmD++heqnas/lAgBbqRkP0cSKsWBk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=S84kbQhJdnWWCbbhA0fCVyd3l4DurMRAhONYTpR41Zz6lNIbIwj5P4/usKZ4+AZQYaOoLCNwj3pPcg+VZBuCpESYCg4nkHhlSVrKMAG0JRVntBGl1Xsj0+oBnCoMJaeaM0kkrfjJX3zC6GwFf4YozIuudSLfYei7nkXWPFxk4sE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=CsE49SMl; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="CsE49SMl" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-1fee7c9e4a4so21209975ad.2 for ; Fri, 09 Aug 2024 12:03:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230218; x=1723835018; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=PEo03FT/iIpJNEkcG69iehsrgcwll+jefs5K49ylrzM=; b=CsE49SMlL7nWanmZvTPdH6LvkdOzHU26mJ5JlEsfPgTJXu4AOFYZkq10s7hO9mqG8t RZQrWRqY3nsHSzXneNzBfQ0u3xcvtJ9RyFQqyJj6jjEYc4/kz+6OWhVPavppyppxA1BM 83yvDojTNxcGyAb522UrBuMO61CDyUBTzNY6ZFAXPYBp201pErBD+5GpLZjqPjuq2+/r 5pepx/FKt4oO3OKQgjf3wVaOzKdY/QaR89mIKFcxRhfm2210RHEGRZYLiL6ft8V6+YiN qy1w3BTkPXxlIva1y+4yCGX5vnB9xQL7HsnV5cMaWzjSH0155izC4OklzaOXC5d/ex2t uHYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230218; x=1723835018; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=PEo03FT/iIpJNEkcG69iehsrgcwll+jefs5K49ylrzM=; b=VEEpJ2Sb1wLsRZQ7VT004w0u3fb2XVRZbmWTtaCOnp7q6V926MErM5tqC0v4jaNsqi NS2YuiFKQX7ODQ71dYosEIYaAt4mHSXFU+NOl53lM3qwUgFgGVG/KGwgWYx/+WhQjKmj r+yJjHy5jVswsfdw4sTliZcmcOPw0PGU7j3NEttV9PQl56tqGJTGstr/XO/z2fZ9ScWR s1176J9AGj55ywGdtNCNX7ImBkiSQsmKdoJ60EZw92UaYhHGQNPyhfjTAkGas9HUUtuV ICfhLBQmBS+zU0ID0oK1wugu5DIjsb2gSWiLr8VamQHcAskZY/SSpHok//TSeK2L7jp/ 6tXw== X-Forwarded-Encrypted: i=1; AJvYcCWuN7n5w8dbsATAD/YpVkVbYySlsIeEzyBB54xZzQOmgrmPhHGFfYx5StYQBfSHFUkt98OfG6yJnYbe8xTm62uzLIriNRsxGMxB91Wl X-Gm-Message-State: AOJu0YyqwsstDQbjO5/FwLXJFTSaK5Nt1aoPLQPGPEdLaM4VlYEGfAtV xj2KIVGVZACIgi0Qd1F9dSGx/Zchjv0XsvIRhlN+f3uJ5HTr3iw7jMIgnjrE5ft3lLI4dF/Y5dt GkA== X-Google-Smtp-Source: AGHT+IH5LOjJ12m73lV027giJooc8yKy5h6vOUlHboXG6c+B/Zn9yoHT1SD7ZeiYbKu/63dWHfYsnexMC/M= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:e74b:b0:1f7:3e75:20cf with SMTP id d9443c01a7336-200ae5cb75amr1696075ad.8.1723230217981; Fri, 09 Aug 2024 12:03:37 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:03:03 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-7-seanjc@google.com> Subject: [PATCH 06/22] KVM: x86: Get RIP from vCPU state when storing it to last_retry_eip From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Read RIP from vCPU state instead of pulling it from the emulation context when filling last_retry_eip, which is part of the anti-infinite-loop protection used when unprotecting and retrying instructions that hit a write-protected gfn. This will allow reusing the anti-infinite-loop protection in flows that never make it into the emulator. This is a glorified nop as ctxt->eip is set to kvm_rip_read() in init_emulate_ctxt(), and EMULTYPE_PF emulation is mutually exclusive with EMULTYPE_NO_DECODE and EMULTYPE_SKIP, i.e. always goes through x86_decode_emulated_instruction() and hasn't advanced ctxt->eip (yet). Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 2072cceac68f..372ed3842732 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8973,7 +8973,7 @@ static bool retry_instruction(struct x86_emulate_ctxt= *ctxt, if (!kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa))) return false; =20 - vcpu->arch.last_retry_eip =3D ctxt->eip; + vcpu->arch.last_retry_eip =3D kvm_rip_read(vcpu); vcpu->arch.last_retry_addr =3D cr2_or_gpa; return true; } --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A3D99192B94 for ; Fri, 9 Aug 2024 19:03:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230221; cv=none; b=TlTcnRt02q/69sjl0KSX/JoVojzh8BncWLlq9ynwD2TmznFcMXQ4vBXDElxdnkc44Wm8HwWwuBHIdg6l0MD40yW6pRcpXM67fdDiiRdZQSlnJZBJtSp/TqbVU/PlHGjkK2W+SKKXp2zbSR1BJyZnf+ZfHa9ZGkcJeAPIeBIy4xU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230221; c=relaxed/simple; bh=sRrTbOJcfiSrIsMETnqBjIT3jUMPYwkaFHnukschxi8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Jd5doONuV2ZwYRy6Y07uXpG28q3MjPuSMEFXmszZxMUd5DXgK13tzTanyp8zlQBLbgoLVvqLg3aVlRNZXo6vwqq8Q6tXIJA7t4JbgG39h0Awfj8v1blQJkA9BY0pfqDzyRs6HBwINTMyE8Qj8X2zyn1Cxno5nUGprhvlNWdoW3I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=sbnrb9ZM; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="sbnrb9ZM" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2d1bc2352e1so2313176a91.0 for ; Fri, 09 Aug 2024 12:03:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230220; x=1723835020; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Dmb967qkQZW0D4SkRpUMqgdJRb4TcheZ9woSO8g40qM=; b=sbnrb9ZMAfulI93tmzCOU+a/I/BhuB3dybTTWS2Zgl4v8ppUHlGs9IYAjb2DXXQVw0 7e+V+xCtv84Ms2c/WZBrbwE7Fe/Uu6lNHh05lOCCaO9qp6xffRdtedkzjW8iyNZQ+19F sxdM+BugfmxiNkoKvnLOFeHupznER39PGd+Y74g+d9xjDLo7IvqR/1eYAmEuUpivZRla nSSc8jqQKsUcaMC/2ykHk+M7CCEZLwbbbzLPZ4Gqxm493diqQ4D7rfSR8C56NtQUnvzq g48BuEKfhJUWyqnZ/4T26yk2IDHynii2RVRyRkDz8kGjPuNGfRcTFvRcZU4MmcRdETAb gUdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230220; x=1723835020; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Dmb967qkQZW0D4SkRpUMqgdJRb4TcheZ9woSO8g40qM=; b=G4HHSyL3Z9/TzRegRphikXN3aD2HRjvvlGDryD8ZXgtPFKMlwacDA0niypiraayklJ Hx1N1aioxS++fIZ4uEObV0OlFGErCAp0YVLd3ni2HwXL236rD0dujpYgZbbqXNA0kltQ x86TUm474I1XQNuZHBG8P5Ygmi/wk+Z1+oJHVW6yzdwm1YW3aTigiZiN9tEKy1+//l/T YNvKbzoXfgWkwlGXHryru5i7gtb2p7ZEZn/CZP4ZV0jbqO9O5Oqbjv8VAr+fnwdy9kHJ nTL+OUj4hvRClrTVVArujkCvVHeLWtC7WA+2n2CoAf84XcQdxZE0KPEIvj7W14HrlF4a nsQA== X-Forwarded-Encrypted: i=1; AJvYcCXKaryK5bPt8QrEkaVXUY1W6Isgif2qt8HldWxAD/O7cHZ7HziXf6CHMVSf1UvYVzKJTyYfzGm4fYM/SBH3j7dKDhYJZpIUq2GTW3EP X-Gm-Message-State: AOJu0Yx6Efj/IznKdsmjGdFoxm6yZA9sWQeIZk9L68DcpRMETwNtKO+3 XXf/UK5ILojtGNh/o9Sir1V3H/2Zi8u1F0NwWdIfFwPq4/GitaPkW57G8Y4rmgIq+ugK/PrIgix 0FA== X-Google-Smtp-Source: AGHT+IGV4VVBAjWqDuf8M8TDt3Ik+GDvR+wC8XLlmWU8kgrPMhiCKChR2E/SCFm3L0IlV7VJrRflltDSgTc= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:90a:114f:b0:2d0:11a1:8013 with SMTP id 98e67ed59e1d1-2d1c4c2e3cbmr42172a91.2.1723230219818; Fri, 09 Aug 2024 12:03:39 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:03:04 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-8-seanjc@google.com> Subject: [PATCH 07/22] KVM: x86: Store gpa as gpa_t, not unsigned long, when unprotecting for retry From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Store the gpa used to unprotect the faulting gfn for retry as a gpa_t, not an unsigned long. This fixes a bug where 32-bit KVM would unprotect and retry the wrong gfn if the gpa had bits 63:32!=3D0. In practice, this bug is functionally benign, as unprotecting the wrong gfn is purely a performance issue (thanks to the anti-infinite-loop logic). And of course, almost no one runs 32-bit KVM these days. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 372ed3842732..4c3493ffce0b 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8934,7 +8934,8 @@ static bool retry_instruction(struct x86_emulate_ctxt= *ctxt, gpa_t cr2_or_gpa, int emulation_type) { struct kvm_vcpu *vcpu =3D emul_to_vcpu(ctxt); - unsigned long last_retry_eip, last_retry_addr, gpa =3D cr2_or_gpa; + unsigned long last_retry_eip, last_retry_addr; + gpa_t gpa =3D cr2_or_gpa; =20 last_retry_eip =3D vcpu->arch.last_retry_eip; last_retry_addr =3D vcpu->arch.last_retry_addr; --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9312F15EFA4 for ; Fri, 9 Aug 2024 19:03:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230224; cv=none; b=odDJ/X2M0gYMv3lFzBTG3RVcQtf1vCSG1Xrtvu+TusxXAaCRvrKMMn8Nn7GZr3b1UfUJm0zpxVt7fBXPVenradPfLkKArW5S+L76r3VYVlrmORcFoi3p5k0RRuUHK3b/SWUUsrRYjSn9IsGO6WZ5MBtIfjFQnyvCCMjwdiXV4oY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230224; c=relaxed/simple; bh=/ZfmRIYXnF67feXuVaBMt5yLUsLq1ywXFEJMJjRy+jg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=PRaVfvxFjj4mSgvE0L9QdzaHgUnTj5tbnwpTXDZ4vJKt+C9Po7hA5eyfAVWzuo7Y5xGmuobRslIhzpRrFL8uB01Eyz9bB6GmnXNU9xHFzVd6FBErOki94HaxQgHEzIDVvuRkmxrnOA25DdYy77NM8Ar27bbbY3HY4c77Di0KRko= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=edZOanDY; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="edZOanDY" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2cb5847ff53so2915395a91.2 for ; Fri, 09 Aug 2024 12:03:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230222; x=1723835022; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=J1R1XfNeOqF5tGdf3uHftS07JnGIcD76EsjpdvLzH1c=; b=edZOanDYrrfy3P3xSjTabgH8QXAIQJxSOrzaUQy7T/y4on4vNzPk7EY3+KpGt/P4dJ BoIh+YuydrOZmYOmXw3mhv1WPLONP/D3Epfrrwr/UdxK/Y7KgqvggjWlJDQCliC/oixs do57R+CE5wtLl54hgzMBgepKvMUNS2VQZOvQKXoGQ1JCBgbP9D1caJR93eJCef+CoRfy gBzHF7gl0MiNzntcGTDhg44F4hKwqy7PEp+kvp7VezzNgxZDNYqK79Prl5e42S338/nn 0NVDTYpG5A4HHdh0jwm5qNL/sItK6yum1/saN/5xofxFICBBplfsfHFDwAlmAA+ADuwJ vEjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230222; x=1723835022; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=J1R1XfNeOqF5tGdf3uHftS07JnGIcD76EsjpdvLzH1c=; b=rNtDMrKA8W0wSjxRlpMqHPlwE6zfUzEoPG0M3ucDb0xbMaSaW6WIKTQnnVbwrTTc2t YZGUAqCNk7n07qfLYgoFeXfFlt5EPPEIr20fEn+7hdeiPSNI2PzvbEaM3jbKIXirkHvU nfaND4nkkVDLiquVX3qYjlS08Is/ry4rW5xKUN11/uCN+1K+v7kd9ZN8qYrn7K1HQD+J 7i5aYqoh8BvOpgzPuRlK0/piEJG/zlYij38kkTeuSiC7lfJC7CYd2QQCXM/FhdNH1bex EqcTlBYM1mBcoi6s0Rby7hDZ2tNUDgCSmQrmaLl6gXg3qKUw9o790pcRHnlLgX/l1+45 chzg== X-Forwarded-Encrypted: i=1; AJvYcCVDiAUVGm1VK5IlribcSoNLx3gv0qrT2+RvMnVZRZ4t6o4xax7ZJm+D35DD2j00occh4ad1IrTO+dIUXap/DSwj4YHpb/m+kmsa2spo X-Gm-Message-State: AOJu0Yw1KCGBLhbX2osduBu7M/+XGEy5rS5h0QVofD6WLiAJzxCNVJN1 VGMDTBV3BzcNuthmZeIK4bgD84mn4+O9Gj4Ea+nCsAcjCPHxzu01wCD8cLZEL2l3suRXMyEMIsZ GVQ== X-Google-Smtp-Source: AGHT+IHa9+G1paE0vB/FK76ehSmHJcxLcXIjNwSssE1mwAYWJibido2K20yHJjwk7mERqEVMW8GyutB6AoM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:90a:9cb:b0:2cf:6730:9342 with SMTP id 98e67ed59e1d1-2d1e7fa3090mr13329a91.1.1723230221816; Fri, 09 Aug 2024 12:03:41 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:03:05 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-9-seanjc@google.com> Subject: [PATCH 08/22] KVM: x86/mmu: Apply retry protection to "fast nTDP unprotect" path From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move the anti-infinite-loop protection provided by last_retry_{eip,addr} into kvm_mmu_write_protect_fault() so that it guards unprotect+retry that never hits the emulator, as well as reexecute_instruction(), which is the last ditch "might as well try it" logic that kicks in when emulation fails on an instruction that faulted on a write-protected gfn. Add a new helper, kvm_mmu_unprotect_gfn_and_retry(), to set the retry fields and deduplicate other code (with more to come). Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/mmu/mmu.c | 39 ++++++++++++++++++++++++++++++++- arch/x86/kvm/x86.c | 27 +---------------------- 3 files changed, 40 insertions(+), 27 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 37c4a573e5fb..10b47c310ff9 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -2136,6 +2136,7 @@ int kvm_get_nr_pending_nmis(struct kvm_vcpu *vcpu); void kvm_update_dr7(struct kvm_vcpu *vcpu); =20 int kvm_mmu_unprotect_page(struct kvm *kvm, gfn_t gfn); +bool kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu, gpa_t cr2_or_g= pa); void kvm_mmu_free_roots(struct kvm *kvm, struct kvm_mmu *mmu, ulong roots_to_free); void kvm_mmu_free_guest_mode_roots(struct kvm *kvm, struct kvm_mmu *mmu); diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 95058ac4b78c..09a42dc1fe5a 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2731,6 +2731,22 @@ int kvm_mmu_unprotect_page(struct kvm *kvm, gfn_t gf= n) return r; } =20 +bool kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu, gpa_t cr2_or_g= pa) +{ + gpa_t gpa =3D cr2_or_gpa; + bool r; + + if (!vcpu->arch.mmu->root_role.direct) + gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, cr2_or_gpa, NULL); + + r =3D kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa)); + if (r) { + vcpu->arch.last_retry_eip =3D kvm_rip_read(vcpu); + vcpu->arch.last_retry_addr =3D cr2_or_gpa; + } + return r; +} + static int kvm_mmu_unprotect_page_virt(struct kvm_vcpu *vcpu, gva_t gva) { gpa_t gpa; @@ -5966,6 +5982,27 @@ static int kvm_mmu_write_protect_fault(struct kvm_vc= pu *vcpu, gpa_t cr2_or_gpa, { bool direct =3D vcpu->arch.mmu->root_role.direct; =20 + /* + * Do not try to unprotect and retry if the vCPU re-faulted on the same + * RIP with the same address that was previously unprotected, as doing + * so will likely put the vCPU into an infinite. E.g. if the vCPU uses + * a non-page-table modifying instruction on the PDE that points to the + * instruction, then unprotecting the gfn will unmap the instruction's + * code, i.e. make it impossible for the instruction to ever complete. + */ + if (vcpu->arch.last_retry_eip =3D=3D kvm_rip_read(vcpu) && + vcpu->arch.last_retry_addr =3D=3D cr2_or_gpa) + return RET_PF_EMULATE; + + /* + * Reset the unprotect+retry values that guard against infinite loops. + * The values will be refreshed if KVM explicitly unprotects a gfn and + * retries, in all other cases it's safe to retry in the future even if + * the next page fault happens on the same RIP+address. + */ + vcpu->arch.last_retry_eip =3D 0; + vcpu->arch.last_retry_addr =3D 0; + /* * Before emulating the instruction, check to see if the access may be * due to L1 accessing nested NPT/EPT entries used for L2, i.e. if the @@ -5988,7 +6025,7 @@ static int kvm_mmu_write_protect_fault(struct kvm_vcp= u *vcpu, gpa_t cr2_or_gpa, */ if (direct && (error_code & PFERR_NESTED_GUEST_PAGE) =3D=3D PFERR_NESTED_GUEST_PAGE= && - kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(cr2_or_gpa))) + kvm_mmu_unprotect_gfn_and_retry(vcpu, cr2_or_gpa)) return RET_PF_FIXED; =20 /* diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 4c3493ffce0b..5377ca55161a 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8934,27 +8934,13 @@ static bool retry_instruction(struct x86_emulate_ct= xt *ctxt, gpa_t cr2_or_gpa, int emulation_type) { struct kvm_vcpu *vcpu =3D emul_to_vcpu(ctxt); - unsigned long last_retry_eip, last_retry_addr; - gpa_t gpa =3D cr2_or_gpa; - - last_retry_eip =3D vcpu->arch.last_retry_eip; - last_retry_addr =3D vcpu->arch.last_retry_addr; =20 /* * If the emulation is caused by #PF and it is non-page_table * writing instruction, it means the VM-EXIT is caused by shadow * page protected, we can zap the shadow page and retry this * instruction directly. - * - * Note: if the guest uses a non-page-table modifying instruction - * on the PDE that points to the instruction, then we will unmap - * the instruction and go to an infinite loop. So, we cache the - * last retried eip and the last fault address, if we meet the eip - * and the address again, we can break out of the potential infinite - * loop. */ - vcpu->arch.last_retry_eip =3D vcpu->arch.last_retry_addr =3D 0; - if (!(emulation_type & EMULTYPE_ALLOW_RETRY_PF)) return false; =20 @@ -8965,18 +8951,7 @@ static bool retry_instruction(struct x86_emulate_ctx= t *ctxt, if (x86_page_table_writing_insn(ctxt)) return false; =20 - if (ctxt->eip =3D=3D last_retry_eip && last_retry_addr =3D=3D cr2_or_gpa) - return false; - - if (!vcpu->arch.mmu->root_role.direct) - gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, cr2_or_gpa, NULL); - - if (!kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa))) - return false; - - vcpu->arch.last_retry_eip =3D kvm_rip_read(vcpu); - vcpu->arch.last_retry_addr =3D cr2_or_gpa; - return true; + return kvm_mmu_unprotect_gfn_and_retry(vcpu, cr2_or_gpa); } =20 static int complete_emulated_mmio(struct kvm_vcpu *vcpu); --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 622321990A3 for ; Fri, 9 Aug 2024 19:03:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230225; cv=none; b=WJkPCoKiw2luNwl6j+78wfE1DUUEdkboV1qZkMifc5/AHhjh8/4BBWnQEQGy2LBCByw23SNbQw5ixixJ/1ywji9r1CeHrQXCI5hFpOT9SRSuH14OzUg4reXdzPJI1+a6WoEIWOJL2fEtz7Tws/rRHysW0WTV1qmJqru89uQDTrE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230225; c=relaxed/simple; bh=yFBflmyh8QZBoiRzx8kLH3cDwLhEcq2bi6TGRbUHykE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=PsaSe6ozE6ludQaBUAM9wUT2tF1oNcyDwZGsNqCmO57pAYSUpR0mus/i3/aAPZjV+nPi6zw/62r7WRLvbJnWlx1EY4ix3fSu0D9ak2RcbsWaTxF4WeMZTkLdp5dm1yanIF0avqkOqoN4iptnjYnPRvnj9k4KI0JP1aF+AsqXuJ0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=3gC/un9b; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="3gC/un9b" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-7106fcb5543so2013717b3a.2 for ; Fri, 09 Aug 2024 12:03:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230224; x=1723835024; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=dpThAu81+XUaVEIhWdG34RqJeDJX1In8gD4UvwvDbc0=; b=3gC/un9bqirfM3RGD3v3jM7gptwynDEfbIfTwpGFigTG0DRsC/mIrR+hCATCfNiSCT YTvDUajzVi6VlZzJFTmLiZLsh0ZRfAt8RyJYlmKJJBQA/tqe4bsTetWlB1r6PWHzZ/M2 FE75/1jRldImmy0r8Ss4yIEsH9aXKy1ALMSdSjhG2M+swWYSCi+I6g7sR9C8prLeGfHs g8TK7Xu5H4r9JknXKrPclC/D1YJUMAovBw5IN24lfEUe6DuDUIwRfoGBulthFXzKK4dC aLEy+J/avsEBUrTMZW9WVfvDIPViDAGGOAQc3iUWJNgGmScAkt0P7VHs36jYiN+G5tzU 7OsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230224; x=1723835024; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=dpThAu81+XUaVEIhWdG34RqJeDJX1In8gD4UvwvDbc0=; b=e1PQIcdvux5waEZU6c9JKJczn9OMSEBIg08Z/joSXXLnZfuVp/rAsr6DjjgMzYsuXy f5yOKgytIXinoh/4e0UreX313jwRHStzqkKM1QuRtnOIcg2bSIMROgI5zDDBpCd1HG4u gWVmL8jz8W9qP1BFIOvb8aumFGkLBfuoZdFlRZAJIwscfvLqQu7oFWrxMy7jotPeovnU huDRWkNGlbRC7G1pKiJE2ZJ0f+qFAGGqFqt/cf+hTkBHUtDy9W0ybEqD3IXQo1toQZBB ppnf+aAYCY29S10gCCqoV04rhSUqQvBXNBuMvRfRvhQ0jeO6JvYcMJPrmKt8JSqVFvrm 0WCw== X-Forwarded-Encrypted: i=1; AJvYcCVSkK+ku6bWHzhtHwhv5aZ+Xk/aaLAw0xqQHvsXSCch41yyClWMVDw6g3Trh0SgUkBLHIdFyyATIiBJHyveHbkpzXELn8hHqC+g0cov X-Gm-Message-State: AOJu0Yw3qRMHV1+uuTjd0lI+NA0QPip7FVD5VDyrtgokCP/nQP4ucZi6 VXE/Kv/IP7Zg/6PkxExasVQ1LdU2qvK2vaqHSzpbdiuLuLQmfoMCZAeDpsE48eazhnW131R+RAq nOA== X-Google-Smtp-Source: AGHT+IEfQ8yvNuIaJdiTz+V3/PCjs/k37b1h4YSYcKmGqEYlmGjeaAnHP/jSzt4t2a2FLDINbWuuKtX0WoQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6a00:6f5c:b0:70e:9de1:992e with SMTP id d2e1a72fcca58-710dc629036mr28643b3a.1.1723230223713; Fri, 09 Aug 2024 12:03:43 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:03:06 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-10-seanjc@google.com> Subject: [PATCH 09/22] KVM: x86/mmu: Try "unprotect for retry" iff there are indirect SPs From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Try to unprotect shadow pages if and only if indirect_shadow_pages is non- zero, i.e. iff there is at least one protected such shadow page. Pre- checking indirect_shadow_pages avoids taking mmu_lock for write when the gfn is write-protected by a third party, i.e. not for KVM shadow paging, and in the *extremely* unlikely case that a different task has already unprotected the last shadow page. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 09a42dc1fe5a..358294889baa 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2736,6 +2736,9 @@ bool kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu = *vcpu, gpa_t cr2_or_gpa) gpa_t gpa =3D cr2_or_gpa; bool r; =20 + if (!vcpu->kvm->arch.indirect_shadow_pages) + return false; + if (!vcpu->arch.mmu->root_role.direct) gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, cr2_or_gpa, NULL); =20 --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2D2BC1991B0 for ; Fri, 9 Aug 2024 19:03:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230227; cv=none; b=rWkP6pltzpTDK/e2oJXk78ARlxLboo8GcUcAHeKDzdHcxMrEguiOyKpAbCaapAn6uB1iWdxNm1473wEp8sJrxV6hpR2oYXQXq5+xtPSk6NitHDl8yStf+wJ4Q+F8x5iwbBZ9Frb6yCX8wJZDAajex7ZRNThQwaHgYfEN4LFp8eU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230227; c=relaxed/simple; bh=6moPdz+KiBY3JiE58UG/PhJfbn2rfK0CZx0lw/tb5cc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=PqdAhBew9+ytztfln0g74XUKhsmyys7sOu5/wTpyFJaRGKM8mPReIlIN4G7oHo5nkkXrRhgVR/e5n3jgZvR83+cr/ZF6ugYQDykOPWosaGknrTTAfHuODPuzVM+Z7nSgVXjPvuHH+jlr6smkjpjpedYe148seBrpdaXzIkX5Q90= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Mc72H+Sb; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Mc72H+Sb" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2cb68c23a00so3310444a91.0 for ; Fri, 09 Aug 2024 12:03:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230225; x=1723835025; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=eOJjh+TwQWkp+dpnChOXrZQKD3CV6Wrx85zDT1JyEBg=; b=Mc72H+SbMQzX9cnM+JRa9cIRvf0mFv1s+COBI5BZgY03MI4Jhsxpx+8pKCHNPP0zUN VHph1yw44WYvjMvW5kirUIPxWMYLdobl42EeaKsubujRo23t0pBrSsALB176KwF8yINf KcAgUrUq88E1fVKDH/M/YeYlDQmCicWlH/KDrWAbgSVtiOxRSdaf5ZtJGZHK/ybuIM9o BvH62zoBUni2PLRZsX0lqNNsetsleWWtOWO+tUhJPQjgP3CBVd2+8pwbNA0ei0B49xZG dkNfaCzJBJEljyhIeaFV3Rbt6wdscnK57edS01pVokVwrnIX+bHCZbKQPLGPbtetW1JP Wcug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230225; x=1723835025; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=eOJjh+TwQWkp+dpnChOXrZQKD3CV6Wrx85zDT1JyEBg=; b=WeTZ+RNilUGICpfWiPKjdBNqa2zRGxuAxEgEolpVDcqvponYmG1FXrtFjH1+mIXHxm YJ2pOUj0wSkzJjMkr1yZRWmUE8i9Q3smdhng8WaH+VgDWXUPEHl6NUnopwqIPYPFJBH4 0tWp4QyZmTGOc3x4OsO8RypLzzSTSsvxe95sFn84YvyI0iLSG+IWdaxLhRs9wCE3P8Oj huGmkeL7D+vSQnRunmHxQ4X/kAhEB0bR8izPeE50aUBX8rRYUAMTTGOkEsKeXvXxMT05 MF3mPwo/6hfgd8uXruUJ279k+Uy8Pn3rIVDbDqHg42Ae1XCCTKw07rx6AA6JWz7rq6na iTqQ== X-Forwarded-Encrypted: i=1; AJvYcCXf5/JRB4Ds+CdhDB9owvcB2areVIR2GDd6vzqiCE5kyRDFkfKv14WxcA4uYrkMqHG5bc9gNV59yYLCyVKgAWaqfxI3Z8qlUhUIIY9h X-Gm-Message-State: AOJu0YxT/dFjfBs55tzRE7ehKPyRuBADWoTI9ZqXuVqcyrgwoq1zOTzG DfhZlcto6H1wM6eucF+OZ32DbVEAEpLn9Zz9o1IzCEyEvnEb3pUtYr1Th3092SqNPYesyaHV6Sf 0cA== X-Google-Smtp-Source: AGHT+IG1vcqcE4+XQ17AvIfAnOAspsC8HmWjhE4Wu1r1FCI+qECCKznOMFlKFBRxe0lpfAErcb1t+i41zVM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:90b:1c0d:b0:2cf:93dc:112d with SMTP id 98e67ed59e1d1-2d1e801daadmr66093a91.4.1723230225408; Fri, 09 Aug 2024 12:03:45 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:03:07 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-11-seanjc@google.com> Subject: [PATCH 10/22] KVM: x86/mmu: Replace PFERR_NESTED_GUEST_PAGE with a more descriptive helper From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Drop the globally visible PFERR_NESTED_GUEST_PAGE and replace it with a more appropriately named is_write_to_guest_page_table(). The macro name is misleading, because while all nNPT walks match PAGE|WRITE|PRESENT, the reverse is not true. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 4 ---- arch/x86/kvm/mmu/mmu.c | 10 ++++++++-- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 10b47c310ff9..25a3d84ca5e2 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -280,10 +280,6 @@ enum x86_intercept_stage; #define PFERR_PRIVATE_ACCESS BIT_ULL(49) #define PFERR_SYNTHETIC_MASK (PFERR_IMPLICIT_ACCESS | PFERR_PRIVATE_ACCE= SS) =20 -#define PFERR_NESTED_GUEST_PAGE (PFERR_GUEST_PAGE_MASK | \ - PFERR_WRITE_MASK | \ - PFERR_PRESENT_MASK) - /* apic attention bits */ #define KVM_APIC_CHECK_VAPIC 0 /* diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 358294889baa..065bb6180988 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -5980,6 +5980,13 @@ void kvm_mmu_track_write(struct kvm_vcpu *vcpu, gpa_= t gpa, const u8 *new, write_unlock(&vcpu->kvm->mmu_lock); } =20 +static bool is_write_to_guest_page_table(u64 error_code) +{ + const u64 mask =3D PFERR_GUEST_PAGE_MASK | PFERR_WRITE_MASK | PFERR_PRESE= NT_MASK; + + return (error_code & mask) =3D=3D mask; +} + static int kvm_mmu_write_protect_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or= _gpa, u64 error_code, int *emulation_type) { @@ -6026,8 +6033,7 @@ static int kvm_mmu_write_protect_fault(struct kvm_vcp= u *vcpu, gpa_t cr2_or_gpa, * write-fault is due to something else entirely, i.e. KVM needs to * emulate, as resuming the guest will put it into an infinite loop. */ - if (direct && - (error_code & PFERR_NESTED_GUEST_PAGE) =3D=3D PFERR_NESTED_GUEST_PAGE= && + if (direct && (is_write_to_guest_page_table(error_code)) && kvm_mmu_unprotect_gfn_and_retry(vcpu, cr2_or_gpa)) return RET_PF_FIXED; =20 --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BACA4199229 for ; Fri, 9 Aug 2024 19:03:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230229; cv=none; b=D2P/z5HMerptM9wSFojRuoCef3MVOPjJMIjFQbN685zYmm1EYEBCnH4WB4rVo4iO/aNZnk2vBdSOrXMvobinkjo1vDN2jK6apGatI/OZGHGGGlBi1/J7TlGk6vDbqQsB1AZkt2kOZeKsiPVN/cXa312RTXIFod5dxMmEhws3Q+g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230229; c=relaxed/simple; bh=H5VYBxwz21vxlZmMRXdu3AJVQuY7buVv49eVdc22C0U=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=PYUag3OTSIOEYXvu6XnVq5l625Yz+syIp3JQ7AKl9fV+NJdz1jKsTJ7zRED1H+k4RV9e10yuEyQ5JME36j8jzih5ElWz1xWm9cyd8mehDqcIs3LRdacs6OC2vWOEtv9FMMI9cxV2Qub7jdSazN6A9PDY3PKixQRG5hPb1+XxAjU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=3D/xdQ2D; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="3D/xdQ2D" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-1fca868b53cso22258085ad.1 for ; Fri, 09 Aug 2024 12:03:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230227; x=1723835027; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=wRBh1Yq0HIE0LrqUowwJYclZkwSPk2840j1yHgVdQPA=; b=3D/xdQ2DJzCQP3vEfTMmnPdy/OTKSFcO/8Y7EjP+OxrYnAK905f3r19QARJu/mCmpW k6TdEK/8Y+QwBcIsV7LCyyFFeuFYkOyB3Lb/m++kDAkxQ/WvAM4I8IA9c0mXfwQSpr4y rf/lytcCP+pQTgfRxsf7RvSvaIcMC3ckzukkQz3WYHlr/1HJrkLycTco0sxHvQB5Zq5z //4ys3CrCUO8QEpJkPxtUffVLBXPNHmpfeIWdNCLZ5dcw8pRqlb+b+WPUL79yxzE5kwY 9AsQMoFvZ/4dGKK1eNC56MzgOCrxQymfM/dHhrpwKAj8stDx5viKRLdYf/swqMAQnLlL N6zg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230227; x=1723835027; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=wRBh1Yq0HIE0LrqUowwJYclZkwSPk2840j1yHgVdQPA=; b=RLvgVDyKHReOo5qA0djNiXYoJiKtoLcMYW0zjc9wgz6NYH2XFKh6hnIFoxKUg6du4o /PK36UT0rKN4K42QbjvfqIiNGJCWre/dX60NGurG+rQEdDNURlVLTZIw/eqLUNsJQw9A FeOsM935aQBzwx4TWabDwGpmr8tF3YB37vj0viqKmrIGBNLTXOF+MeoTXj09fd5O0Wqc AWbIu5/zALM8e2CyZR1EmgsStDkToj0zm5eo8UBLU9tk1xGJrSzbuxI3QAncZ7lt3NCr 9Af6RZGqHGQctyyl+MaKxR0atSBLJeVrbR69vVD8NhK2zVjA9yzG+v6Y0WJqV6vxHcpn +bYw== X-Forwarded-Encrypted: i=1; AJvYcCXgrd6fTGoVtOl2W6qgeJHjvUlZ4xrBvPDM02+D507dn35m3pMfdSmOGHrpAWnmNdsbr5RxqsJgqYamZsm0nuHs43+4y5v4e0qV5wby X-Gm-Message-State: AOJu0YztEIgFZTDmIkNg0UbjNbfygeTnG3Joie/yyob5vy+097+kCBX5 zYTleaqsUPP1xampGHDU+kuyWf0uhKl7Xx3i3ys+tO44cqM/X+yISZ3k0BP8tbpctt92eezs3HW pog== X-Google-Smtp-Source: AGHT+IEFKdM8xue3Odmp4NwfLFd4bvZYSXUmHSlUZu20h4ZixmSV8G+oSrO/GTtZZlzBDf9TdiNKX+NlKww= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:903:2441:b0:1ff:4618:36b8 with SMTP id d9443c01a7336-200ae5a7a86mr1534465ad.11.1723230227071; Fri, 09 Aug 2024 12:03:47 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:03:08 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-12-seanjc@google.com> Subject: [PATCH 11/22] KVM: x86: Move EMULTYPE_ALLOW_RETRY_PF to x86_emulate_instruction() From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move the sanity checks for EMULTYPE_ALLOW_RETRY_PF to the top of x86_emulate_instruction(). In addition to deduplicating a small amount of code, this makes the connection between EMULTYPE_ALLOW_RETRY_PF and EMULTYPE_PF even more explicit, and will allow dropping retry_instruction() entirely. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 5377ca55161a..7e90c3b888c2 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8872,10 +8872,6 @@ static bool reexecute_instruction(struct kvm_vcpu *v= cpu, gpa_t cr2_or_gpa, if (!(emulation_type & EMULTYPE_ALLOW_RETRY_PF)) return false; =20 - if (WARN_ON_ONCE(is_guest_mode(vcpu)) || - WARN_ON_ONCE(!(emulation_type & EMULTYPE_PF))) - return false; - if (!vcpu->arch.mmu->root_role.direct) { /* * Write permission should be allowed since only @@ -8944,10 +8940,6 @@ static bool retry_instruction(struct x86_emulate_ctx= t *ctxt, if (!(emulation_type & EMULTYPE_ALLOW_RETRY_PF)) return false; =20 - if (WARN_ON_ONCE(is_guest_mode(vcpu)) || - WARN_ON_ONCE(!(emulation_type & EMULTYPE_PF))) - return false; - if (x86_page_table_writing_insn(ctxt)) return false; =20 @@ -9150,6 +9142,11 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, g= pa_t cr2_or_gpa, struct x86_emulate_ctxt *ctxt =3D vcpu->arch.emulate_ctxt; bool writeback =3D true; =20 + if ((emulation_type & EMULTYPE_ALLOW_RETRY_PF) && + (WARN_ON_ONCE(is_guest_mode(vcpu)) || + WARN_ON_ONCE(!(emulation_type & EMULTYPE_PF)))) + emulation_type &=3D ~EMULTYPE_ALLOW_RETRY_PF; + r =3D kvm_check_emulate_insn(vcpu, emulation_type, insn, insn_len); if (r !=3D X86EMUL_CONTINUE) { if (r =3D=3D X86EMUL_RETRY_INSTR || r =3D=3D X86EMUL_PROPAGATE_FAULT) --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 96C2E199395 for ; Fri, 9 Aug 2024 19:03:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230231; cv=none; b=JXKvsHjDPsepQDXOs3usC/PI0Of4xWSxRNTlUaHlgBF/8qpNbYtQhM5uXRawf8GSRLn7NbbvyoZ6IOKZorQy8ceQO5CCB9sC9EK/8O6FkyAE9mLgoydfWiUHk2xY4lJr6K11oGhzw++S5Q/87pZdIy1nGkB/PTvnIfW/G0bMwHM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230231; c=relaxed/simple; bh=sCYMMA2rJwBRxJrTSDkgNzKiSx1xbuccHFqpbX0NI+c=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=dc7mk5Vo1oZ90s2Fu+kJm0m7jKdigjrJYAIuVQcp5oYRHHaRmeuKUPpZcNQa2n0dQHKl8J5GVPxLMgmYcgAAl1tzUAWnaA6al1p4Xm97rFfHDubRaZmc1uSyBaJ+pNY1BRFgEUqV3YtvL1BUw73LJ59sJjGZZu8OGMrgs/2ksMU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ILoWDIGr; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ILoWDIGr" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-79028eac001so2522750a12.0 for ; Fri, 09 Aug 2024 12:03:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230229; x=1723835029; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=UL4xGW9Vpu7Q61ctxbdglUexBeIvbgO/x5cgODPhM5E=; b=ILoWDIGrpL5yE7HUgxssXh0ZcD0yqedEE0786kfhzOKs20v/2+aX2qM4+U8k6REoJW fhEEZQLH4utHVRCgRJfzwTr7ufKkUgi/8uWbbms2rR0RHlEgGEH+9ZzvHYMKgn+C9DLF /gAdsZq+I+Kbww8xgKOnEwb7uRSHgTj7dMk3k1AX7gbxcPLPBJYIDyBmTVvpe6Ixxw3f zejhsfU+KXLpKgRIRsHtfI6HOIgUV/ThE/IeP3qJEh0NioDJXguuRmhDlKohUg5mBsaK sW83UsA/wvoKkmLjkh4MoxNgt5EWZKeUFkAfjajuVGlbtGIoRP0f4cQXQLSbSRnIOhYn XswA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230229; x=1723835029; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=UL4xGW9Vpu7Q61ctxbdglUexBeIvbgO/x5cgODPhM5E=; b=a4FR0vyblB/OPA5v8HcE7x1POLy7b4X2V+U9ylgq1186jkPiZz6QTxNxCP5k7c+aCS tlmJu3eUhHUC3g1nfLbd+W32K14bneE8pY2rZU4+7oQbygrV5ipsf/nLm2e0kyhYobfU oHus6mBxtE2mOqfmGvjsvZbKlPhtaVMB7/jJ0JGvGv0qePtcq8nmLTZlo+vHE5gG+HH1 q7m/CpaC9P+KpvFoHAIw9lEfQDP+5g7Ir/Dfy0fiq9KhBuAuGcejP4QPF5ewDxg/aimr 30WFE220MCVMg0Mci0YnNEPafJx/2WEmNZl/yFhzXYv1RPqcJSUjxDT5jb8zEgAG6qzx gVIQ== X-Forwarded-Encrypted: i=1; AJvYcCV0owOR70FkGyZbmwQdanpORQdVbmAiIMJrIe/f1DQKO4Z7RoiGjPluKTAZmrDtg7o4MRzle8TJ5cCAArMpIPTECW0ZmCrBzzkAkJVE X-Gm-Message-State: AOJu0Yx5v1fVzQ5Z6S98i0mFsWtYgCJQulW/4eO+607KVJgQx3MXPqUs 3LFS2z/Xxxw1+rH2w2cgURoRXNQDSE/M816mLkyVPp6pYQU6fKe6rf2rIqab5ASc6Ctu0+fnEj2 cDg== X-Google-Smtp-Source: AGHT+IEWNryKfrDGJ6A8ssPAg413DCCTFIkgK0+hprcIqq7sfsduMvpY8oVW+ZWuuTwECASFLr+r3u7EGWQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:d490:b0:1f9:ddfe:fdde with SMTP id d9443c01a7336-200ae5cde47mr1585545ad.9.1723230228793; Fri, 09 Aug 2024 12:03:48 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:03:09 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-13-seanjc@google.com> Subject: [PATCH 12/22] KVM: x86: Fold retry_instruction() into x86_emulate_instruction() From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that retry_instruction() is reasonably tiny, fold it into its sole caller, x86_emulate_instruction(). In addition to getting rid of the absurdly confusing retry_instruction() name, handling the retry in x86_emulate_instruction() pairs it back up with the code that resets last_retry_{eip,address}. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 30 +++++++++--------------------- 1 file changed, 9 insertions(+), 21 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 7e90c3b888c2..771e67381fce 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8926,26 +8926,6 @@ static bool reexecute_instruction(struct kvm_vcpu *v= cpu, gpa_t cr2_or_gpa, return !(emulation_type & EMULTYPE_WRITE_PF_TO_SP); } =20 -static bool retry_instruction(struct x86_emulate_ctxt *ctxt, - gpa_t cr2_or_gpa, int emulation_type) -{ - struct kvm_vcpu *vcpu =3D emul_to_vcpu(ctxt); - - /* - * If the emulation is caused by #PF and it is non-page_table - * writing instruction, it means the VM-EXIT is caused by shadow - * page protected, we can zap the shadow page and retry this - * instruction directly. - */ - if (!(emulation_type & EMULTYPE_ALLOW_RETRY_PF)) - return false; - - if (x86_page_table_writing_insn(ctxt)) - return false; - - return kvm_mmu_unprotect_gfn_and_retry(vcpu, cr2_or_gpa); -} - static int complete_emulated_mmio(struct kvm_vcpu *vcpu); static int complete_emulated_pio(struct kvm_vcpu *vcpu); =20 @@ -9225,7 +9205,15 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, g= pa_t cr2_or_gpa, return 1; } =20 - if (retry_instruction(ctxt, cr2_or_gpa, emulation_type)) + /* + * If emulation was caused by a write-protection #PF on a non-page_table + * writing instruction, try to unprotect the gfn, i.e. zap shadow pages, + * and retry the instruction, as the vCPU is likely no longer using the + * gfn as a page table. + */ + if ((emulation_type & EMULTYPE_ALLOW_RETRY_PF) && + !x86_page_table_writing_insn(ctxt) && + kvm_mmu_unprotect_gfn_and_retry(vcpu, cr2_or_gpa)) return 1; =20 /* this is needed for vmware backdoor interface to work since it --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A495199E8D for ; Fri, 9 Aug 2024 19:03:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230232; cv=none; b=XMjsYgJwfwf9Gepse6RprLOAhYu2GJLtd1DHqe+k7sec+Su7XNCp944fEhRf1jxiD468nBALK1w9nUDzxxw0HE0vwWuoa6SmpEi7NlO2NIjY0ZE5SpO6R98mXk3MnE8gSOXgTG0D+OgY/te5r/s4bR1JrgHaeE79zb6km8ghW90= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230232; c=relaxed/simple; bh=y4e2ws5CmA6QmgH+LrpuQNDyjhqqtDpNs/vzxZUhBOE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=g/cpbvdcREOvlV0FLlF3Q/LJv8xCESdQ2yH1N87x6wl+yMXxr7Li63vgW/n8b3ww+pRb/d9ss47B+TaOOVQ2wspevbkr8sPCeM5Rp9BnlrD+cc5PVHO5kts2a/7UDafs/0RLxpuwdfxlcpENbIGVAjzDUUKLGLrTYUNmgXa9mkw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=sq0OnCz7; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="sq0OnCz7" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-1fd9a0efe4eso22444025ad.0 for ; Fri, 09 Aug 2024 12:03:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230231; x=1723835031; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=pC7nnCclNkH1a5h1FMz2WKmoD/v8iKeyKDxG+elCokA=; b=sq0OnCz7sU2kNc8p13jeYxANoV9w2vLjOiZteOoCUVOk+mYL/O1IXIqJGmf5h8Z3wP VfG1Ir0KzPv17c0kRHJNssfh4ISnU9PuQiqS596wL2OPHQJ+anjX8VRIUNAwLDFdYdn9 qaWvOgzUSB6wOiQ780GdhSFjWGQYuyp209WxRsM+O8IR8mVreZPfm89AHlSSBrEGHypQ 8yGLrq3OxtrBiZuiU2i7mFWKw6qHVonMfPysQCtd3QXiR9S1ZI7LjgbZFEROZJpuHLx3 QANAnlF/fP/3pZ9boW1hd9VonCCPIHdI7swWw4002tFvvBDNFsYFEKIUcFLQyGk9+MRO BFjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230231; x=1723835031; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=pC7nnCclNkH1a5h1FMz2WKmoD/v8iKeyKDxG+elCokA=; b=rSkHufNwUwkOPhy2O18GjCJC6+4ykZSOc/IqOtJKzRYt91u5xEwQi16B5ntlpBO1TZ EbKjOvASwel9cIjk26pMR9BpRM9zyI9pGAoSQgoHaFw5OKg/S3eT4gqZgqRh3T3WMUwW SVl031dbbVru5ehAe7VcoIFsSIgV6mmbcCdXXkedBXbTyVBR6QYeHtwRiDeXO73AMv/P p/SrmB/RIrrGY99cOuxlHKqaCeuRditY5Af81//mE0hx3i5RtA6YTjEgjI9dRI7w+vh0 oBOvdHXaHBC6WksZKyiWxdxW4xJs8PogNDOtAmzzQESiq1i7cQoyrMqFWVNy2HO56nTJ qu7g== X-Forwarded-Encrypted: i=1; AJvYcCXKYMy0OvmDyFmvrL0qaDZMwlrPdCUB5ppCRVjuJ1BGdv9K7AQ7ym6E0i5uGZtyX6+QK1MuKgX62W3+SpbcNLTrtjxJdxGaMoM3p0Yl X-Gm-Message-State: AOJu0Yxs+70QeYGD0Z7r/o6o1eksrWtOV93DjcvkZm84VCb3fQkgt6qT murvsNOAaQZK5HeaGMKmP7uMfaL7QEz/5T6HQ9BF3GHxlvEXyG72/WT9nFicwv0bfKu0svrh/wP OlQ== X-Google-Smtp-Source: AGHT+IEV0guIC48ohvagF6zYB+z3BO6oJse2IrkM1+rRSypDawH8Vs0UpeVAtkLq4rZ1vhdOd0D1zEgBHxo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:903:2445:b0:1f9:cbe5:e422 with SMTP id d9443c01a7336-200ae56e4f1mr55825ad.8.1723230230668; Fri, 09 Aug 2024 12:03:50 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:03:10 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-14-seanjc@google.com> Subject: [PATCH 13/22] KVM: x86/mmu: Don't try to unprotect an INVALID_GPA From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" If getting the gpa for a gva fails, e.g. because the gva isn't mapped in the guest page tables, don't try to unprotect the invalid gfn. This is mostly a performance fix (avoids unnecessarily taking mmu_lock), as for_each_gfn_valid_sp_with_gptes() won't explode on garbage input, it's simply pointless. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 065bb6180988..a5d1f6232f8c 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2739,8 +2739,11 @@ bool kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu= *vcpu, gpa_t cr2_or_gpa) if (!vcpu->kvm->arch.indirect_shadow_pages) return false; =20 - if (!vcpu->arch.mmu->root_role.direct) + if (!vcpu->arch.mmu->root_role.direct) { gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, cr2_or_gpa, NULL); + if (gpa =3D=3D INVALID_GPA) + return false; + } =20 r =3D kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa)); if (r) { @@ -2759,6 +2762,8 @@ static int kvm_mmu_unprotect_page_virt(struct kvm_vcp= u *vcpu, gva_t gva) return 0; =20 gpa =3D kvm_mmu_gva_to_gpa_read(vcpu, gva, NULL); + if (gpa =3D=3D INVALID_GPA) + return 0; =20 r =3D kvm_mmu_unprotect_page(vcpu->kvm, gpa >> PAGE_SHIFT); =20 --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 57F5C19A282 for ; Fri, 9 Aug 2024 19:03:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230234; cv=none; b=AibHgS6Mp0OKCkh+f0+T3BjvrXktPwhI90dCCcrLjCayFZ66RXLn3v10crz6Cddrky9lz0S9r264ohSyz/v8l7IL/ALHH2T6CPJF86Kys02TUFLqpSjfQ11aKVwR7B/N1nQoDblgpbT61MGrWFk3MYmc6BaeRU8JesD0YM9ySnU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230234; c=relaxed/simple; bh=bRCd13k6i1A0U1LnG9khVbeFxffh10XZUO3UGsbmPjg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=dSFkCvREagetQBalKLVk1XJOFyyrdFWP7lubCvByrpasMjmHuSdIzMr9cZbuVBngbFZaZUNI2IQr9wdG1LtlCSsjcEAX8GDZQExm+i2uwMna7DDL/Jc7z6K8pILRvNbxGPygBKchlLgWIh7wIeLnBCVfTn3xP0FwM7UkYBW1piM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=hgK0uaBd; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="hgK0uaBd" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-1ff192decb8so25498625ad.0 for ; Fri, 09 Aug 2024 12:03:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230233; x=1723835033; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=i6No2hwzIu6e4YAJZs7ugQt7CXH2dDKyvdiMgrvYWso=; b=hgK0uaBd1RE+lf3DyVFJOV9l/j2RQdxqNqi358JHDxqzPCD3Tx+ZS5r/leMQduUrQm DnmQXl/xK4eJJD6gkoPt38xa2dt45RY50vSlpNA3F6oO4EwCCNl/GaoroNSm0kSpGLp+ emlosudG2AzCn+Xh4gK/2eYKfFGUPafJQ7MWtDD4yowBQaEI/7hwX3sdjgiNYLf3fqa+ t4VDfJtK3YvqRrNTjWtDc69t8FaOuK9TO56L7MuozAtt3qyWb8VTcV9/sP3CKx1CeCIT nVhRmuXfUZWcq/wCpvT6Fy7E/qAiy1VKKfCH9L7MwzpyJlo6YDMP0OQOEMp/Pux1gLOG h1eA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230233; x=1723835033; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=i6No2hwzIu6e4YAJZs7ugQt7CXH2dDKyvdiMgrvYWso=; b=nJ89PAZDGeQJ1KG3j/WT/5fW0wT5IfxVONzMuGNngZ/J1bvwMTh+MJzJ8DCQy8+ZNE X3qg09jDllD48hZ4a4VKdbGksQ+PnKtaGjelXtLh2+YOnYS3Q21wV5QHtH4HendCHqYx TtgmHDIVKmBV+DKv8/MvRtKB6k3uxfIOTyvhue6Pc1RaZ9PugYCQs0hwKLZ9a0bnrmtQ +E/NKB+oB9uYH1138dmQ+xbxh1iZjEr1vCKHHKkOGMloLol8qjpRy8notHHjBsEfMlhW eQlWEjiIW8myWkJxI03V5obUxrbS44N/GZdxC/z5O9WTtLacutGHZ/w5o77ty1qG+Zt/ tUxQ== X-Forwarded-Encrypted: i=1; AJvYcCXATwVhM1+My15iz+xBx/P7k4vZFAFMqkvNDzew2HbwD0xbFOxgnAXbrcpH573WWKHRjsSmPPA6eXDhJE4=@vger.kernel.org X-Gm-Message-State: AOJu0YzNXsxgL5e3A+yC8KU65StZ6Y59SLvsbAe/UGqazmTVAWTsEig1 abYEhWNcBSAKa7t26i0KWBJrDEknw2Q/VSxVsp/I744t+XgMtjGGV/iCAacaPmQPC6msY/ii89z rug== X-Google-Smtp-Source: AGHT+IEiNLsgXHr0s14hEc+y88fytjsQ/uGLHuXkzmmisdcEk/UgFC8sD2BUQDaPdf5KBMzcHwv9BLUzTIc= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:fa8e:b0:1f6:2964:17b8 with SMTP id d9443c01a7336-200ae584d14mr701055ad.10.1723230232634; Fri, 09 Aug 2024 12:03:52 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:03:11 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-15-seanjc@google.com> Subject: [PATCH 14/22] KVM: x86/mmu: Always walk guest PTEs with WRITE access when unprotecting From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When getting a gpa from a gva to unprotect the associated gfn when an event is awating reinjection, walk the guest PTEs for WRITE as there's no point in unprotecting the gfn if the guest is unable to write the page, i.e. if write-protection can't trigger emulation. Note, the entire flow should be guarded on the access being a write, and even better should be conditioned on actually triggering a write-protect fault. This will be addressed in a future commit. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index a5d1f6232f8c..f64ad36ca9e0 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2761,7 +2761,7 @@ static int kvm_mmu_unprotect_page_virt(struct kvm_vcp= u *vcpu, gva_t gva) if (vcpu->arch.mmu->root_role.direct) return 0; =20 - gpa =3D kvm_mmu_gva_to_gpa_read(vcpu, gva, NULL); + gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, gva, NULL); if (gpa =3D=3D INVALID_GPA) return 0; =20 --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4D5EF19AA68 for ; Fri, 9 Aug 2024 19:03:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230236; cv=none; b=fpXEnaSQ3IM6CMHO0my+BEgHH/vyTZOYVznn6mPUliBhHZz+LjCeiBU+VKGTilZBcIRRkdFJhXWUxC2sVNKk3Ex/AWk1hl5XAs6E2FfrUWHOh8r7sHdURYhp729TZ6V4V6PcDOG5UxPvBnhqjVBBWb7iZhMMHXfAVXjf1qeXlQo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230236; c=relaxed/simple; bh=KQRLdya5684hxmBaKQMKjQM37jKdGg0h9k1yv0K/BzQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=AWb5fDXv6C3cDFNzAOBoOfOoQLjhWvXI8lqsAZeZ1crvp+0jFhEEITuXgeM9Xjo4YK1s9XT8G0OG3M3No2o1OCiaeXIhEMa7iSlvYJttV+DP/RpU/wtanDtdLOvB9nlzdKbxoI/68LD994VLNLOr/3HT1RPOtYu+vtAlv7llf6A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=eY/ryirv; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="eY/ryirv" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-1ff72ddb631so19953895ad.1 for ; Fri, 09 Aug 2024 12:03:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230235; x=1723835035; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=cEDePGFWarkq0p5GsDOTFsBMJqTP6fc9aq5ikFn/88E=; b=eY/ryirv0gekqLbSvXNIgRIDVVLESnaEOtcs5bBWyn/JTKnrTu7G77qrV5ehukr29R q0n8XHUDsFL1yYhFTxh11CeM8eG1azt++6JizKLF6d5DrTFl1aQykvjSgwpt9+AAAAQP HDzqLvQrHMuX98rDnxz6TB/B9sO2/hXrirGrhVUvrI9JfxLF8eoLVN6ChnDpRoOofWcJ Se/U27kFxDc/+NasBZYOhfnAjJ9evcKsdFM2z996b1FdPD1cgRy+SsAa85IvrZjqVqCF PUr2Df5lOyhBYw6nmywrY+riTK4aAPglze24U1DLzQKWpj/LysQhpUZe0zebipbvuTZ7 JDXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230235; x=1723835035; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=cEDePGFWarkq0p5GsDOTFsBMJqTP6fc9aq5ikFn/88E=; b=rAwoRiG4k6rsrkWc65L7kwJk1j+TQZbjEy0tfszfbTRHgeCkmASRodAOmYH5K1p1ae u8a9pwdA0rdTZTy0Bbm790chZ5iaA4ACyrgJt6K6sXOs4jwLAF5KAnv+zVYP8UkBWfJ9 y0B1VCFKglJBp55fPW6NoHo673NOnvBZxuNB25toPUN8iKR+Lw22wcmhDh8H9OPAHM0P UgIBNIC+O1hz4aa+/gahEapH0cRQfb8pqZ66A4LkKLrqiRdGfOXQrNnC+RjvcEygvwXN ThUsCYcQNbIDVk6sJA3Sb3Y4Wdh4ioI57rom1dCYvJBmR0rBquR8ghg/ZlfVyqy/bqWt gesA== X-Forwarded-Encrypted: i=1; AJvYcCV0/r1FJeb2iwaPPnQ6LETBR6GsTpKQBqLoUwPPp637fuegjxaEQUtIkaiHPcRbtwjV+R/GFSE5BlNHARK3ZJSbVd0ncDdaWf/ywPzX X-Gm-Message-State: AOJu0Yxlj97/RuxNUEp0Qz+jpujfdizbKtffFLDTlk+x43dwFkGGKfur UWzDidz+CvSM64jlLhKHntk0+JHi7vewF8aROQE9O9+fOZdKfxq7LRNtU/cXktRfqUNpBG9+Zuv Flg== X-Google-Smtp-Source: AGHT+IGcq0KW3D4A3WQypd2J8Gl2rA30J53XcmsY9Nzx5KvXDePBctQ+y9HgClPMmimHKkgZewLGtj2KVP8= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:c405:b0:1fb:82f5:6631 with SMTP id d9443c01a7336-200ae5aa8f5mr1239735ad.9.1723230234728; Fri, 09 Aug 2024 12:03:54 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:03:12 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-16-seanjc@google.com> Subject: [PATCH 15/22] KVM: x86/mmu: Move event re-injection unprotect+retry into common path From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move the event re-injection unprotect+retry logic into kvm_mmu_write_protect_fault(), i.e. unprotect and retry if and only if the #PF actually hit a write-protected gfn. Note, there is a small possibility that the gfn was unprotected by a different tasking between hitting the #PF and acquiring mmu_lock, but in that case, KVM will resume the guest immediately anyways because KVM will treat the fault as spurious. As a bonus, unprotecting _after_ handling the page fault also addresses the case where the installing a SPTE to handle fault encounters a shadowed PTE, i.e. *creates* a read-only SPTE. Opportunstically add a comment explaining what on earth the intent of the code is, as based on the changelog from commit 577bdc496614 ("KVM: Avoid instruction emulation when event delivery is pending"). Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 28 ++++++++-------------------- 1 file changed, 8 insertions(+), 20 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index f64ad36ca9e0..d3c0220ff7ee 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2753,23 +2753,6 @@ bool kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu= *vcpu, gpa_t cr2_or_gpa) return r; } =20 -static int kvm_mmu_unprotect_page_virt(struct kvm_vcpu *vcpu, gva_t gva) -{ - gpa_t gpa; - int r; - - if (vcpu->arch.mmu->root_role.direct) - return 0; - - gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, gva, NULL); - if (gpa =3D=3D INVALID_GPA) - return 0; - - r =3D kvm_mmu_unprotect_page(vcpu->kvm, gpa >> PAGE_SHIFT); - - return r; -} - static void kvm_unsync_page(struct kvm *kvm, struct kvm_mmu_page *sp) { trace_kvm_mmu_unsync_page(sp); @@ -4640,8 +4623,6 @@ int kvm_handle_page_fault(struct kvm_vcpu *vcpu, u64 = error_code, if (!flags) { trace_kvm_page_fault(vcpu, fault_address, error_code); =20 - if (kvm_event_needs_reinjection(vcpu)) - kvm_mmu_unprotect_page_virt(vcpu, fault_address); r =3D kvm_mmu_page_fault(vcpu, fault_address, error_code, insn, insn_len); } else if (flags & KVM_PV_REASON_PAGE_NOT_PRESENT) { @@ -6037,8 +6018,15 @@ static int kvm_mmu_write_protect_fault(struct kvm_vc= pu *vcpu, gpa_t cr2_or_gpa, * execute the instruction. If no shadow pages were zapped, then the * write-fault is due to something else entirely, i.e. KVM needs to * emulate, as resuming the guest will put it into an infinite loop. + * + * For indirect MMUs, i.e. if KVM is shadowing the current MMU, try to + * unprotect the gfn and retry if an event is awaiting reinjection. If + * KVM emulates multiple instructions before completing even injection, + * the event could be delayed beyond what is architecturally allowed, + * e.g. KVM could inject an IRQ after the TPR has been raised. */ - if (direct && (is_write_to_guest_page_table(error_code)) && + if (((direct && is_write_to_guest_page_table(error_code)) || + (!direct && kvm_event_needs_reinjection(vcpu))) && kvm_mmu_unprotect_gfn_and_retry(vcpu, cr2_or_gpa)) return RET_PF_FIXED; =20 --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5BAC919ADA4 for ; Fri, 9 Aug 2024 19:03:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230238; cv=none; b=p5b5X3ivFCPAjWp/rLbYsdWJz/gXowJANsGgiLYPwq3ZJp2wWQ1mo+YyKA7ut1Ef+k3YowKEixNShQ4mhG1I+A+IqiWF5NxzfkuapOpVYMAJYoWvtpKp8i9tO/b/7W7rQaSE3w6O9i/UZyjvCKng2rCb8Zepi488/W/LNSKiiQM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230238; c=relaxed/simple; bh=uGTn00vVoZxHd5+Z0XK3NYo3MeaA+VVaXsCWEnk+RDE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=HGt8j1iIZfojMSkbQpaGneIIBT6kQ8HLrPtIUXpwNzCY/98CDhd60d7Ftr/uy1tOrxYF4pdE4QUryHLD618SAG8Fg50zxZCNatVrVV3qZIhsk6XjHA1rfIC0Br85vwO8IdnGo4mm4DlaQb7EsCRPRc6M4ySvDZGVoP6OVRjKkTU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=vrDxC+nv; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="vrDxC+nv" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-692aa9db2d5so50366837b3.3 for ; Fri, 09 Aug 2024 12:03:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230236; x=1723835036; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=kvnI0Btlm0LRVdXYEsn4iCu6mP1I2mMf1I/or9DffZY=; b=vrDxC+nvwwNWBAP6KBMGI3+W3qqwHxOQKD5R8KmNLfKsx5oFAPABaRk7N0o+2HxFCK LiSwg1vDxzRTonMsioCJKOw/u7UoFr4Ol2Fr/Hkdvsiq/TmqPVRAvt6Yd01F/S1/A0xi LzoGGQdovshJIy3EtRg/UiHaIJ+KEt6PrxQtmMoVS9dQ0Vk1nSQ6YkjvMSW3BmDGwoyU RDA9tce7YCwSA12wCdWqGB3bgIgfbDqIw/UXvoqTH6a7BxOBNd7/HHCiFSOIIrj9YjzS aG612LIm3Yi4enaMHGWLicq1wpGbTlCguSh8fglTQKzPyoerxGM97oygofBhhZiHFGsH FlhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230236; x=1723835036; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=kvnI0Btlm0LRVdXYEsn4iCu6mP1I2mMf1I/or9DffZY=; b=vptB4tzeYIV2wK4y5K0NFj+I8rlLgqTdHv8tTRJD+9eakcmKu9fJQrpn27OowlEOLc FxMdak6AifOezseV3DnZLRjeAQs6rc7ALe6NADD5pRbrjyxFaGOdA8d+fZrAFJ6YnqdR 5mxG6hfbgjy2zuRECaZzlL/yqiMLqHFQZfjhczo+deonOnutydnZZGQd7ysBDT/b2Ozq 18o2VfCSndce2mpiOVYCPZCh4stwnIqcG1oQglfWlPldHThKAysLiLLPS8WVZotKcQFJ G4BkVbU5ILrkpmGMRudGOeZ397s2IZN0WJ0JAdLbRf/Z/udBqtNobTt5D2yMCZK3xWye flVg== X-Forwarded-Encrypted: i=1; AJvYcCWY0X9uInqvKgFHDiQpqoG8VNsJdjBFCwDwA58vMVMc7EIhhY7ygr9qPihI9kfSKeL33ftkzvkuI+On4Bj/lFsKH8xa4lV1kr0+Rv76 X-Gm-Message-State: AOJu0YyevPVpnBkf+XIqrVCVk8hL5lEnmFgjBmJURhZVBZwB1sBBBayw yn88A1mKCGPhjCK74Mt4HdJz1jZzS9QapxlKywAb25Af1enQFYLQ9hnleMTbFvPkGGMHSXBeCnZ 9oQ== X-Google-Smtp-Source: AGHT+IFL6bQgCxnimbHP4f5bmsF/aP/gxKIR0RtUrr1Bhzg5SSWFxmFRUrGcU3KsKu+jzdwgfD+rPT571b4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:690c:288f:b0:680:cd2b:90ed with SMTP id 00721157ae682-69ec4fda24dmr98747b3.3.1723230236426; Fri, 09 Aug 2024 12:03:56 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:03:13 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-17-seanjc@google.com> Subject: [PATCH 16/22] KVM: x86: Remove manual pfn lookup when retrying #PF after failed emulation From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Drop the manual pfn look when retrying an instruction that KVM failed to emulation in response to a #PF due to a write-protected gfn. Now that KVM sets EMULTYPE_PF if and only if the page fault it a write-protected gfn, i.e. if and only if there's a writable memslot, there's no need to redo the lookup to avoid retrying an instruction that failed on emulated MMIO (no slot, or a write to a read-only slot). Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 18 ------------------ 1 file changed, 18 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 771e67381fce..67f9871990fb 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8867,7 +8867,6 @@ static bool reexecute_instruction(struct kvm_vcpu *vc= pu, gpa_t cr2_or_gpa, int emulation_type) { gpa_t gpa =3D cr2_or_gpa; - kvm_pfn_t pfn; =20 if (!(emulation_type & EMULTYPE_ALLOW_RETRY_PF)) return false; @@ -8887,23 +8886,6 @@ static bool reexecute_instruction(struct kvm_vcpu *v= cpu, gpa_t cr2_or_gpa, return true; } =20 - /* - * Do not retry the unhandleable instruction if it faults on the - * readonly host memory, otherwise it will goto a infinite loop: - * retry instruction -> write #PF -> emulation fail -> retry - * instruction -> ... - */ - pfn =3D gfn_to_pfn(vcpu->kvm, gpa_to_gfn(gpa)); - - /* - * If the instruction failed on the error pfn, it can not be fixed, - * report the error to userspace. - */ - if (is_error_noslot_pfn(pfn)) - return false; - - kvm_release_pfn_clean(pfn); - /* * If emulation may have been triggered by a write to a shadowed page * table, unprotect the gfn (zap any relevant SPTEs) and re-enter the --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0FFFD19B3D8 for ; Fri, 9 Aug 2024 19:03:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230240; cv=none; b=nEwolwbDfeYbj6rAlord0rcTP1a1hkaB1weeqMexcBDn3JXyGfj9DuMZUkpzPBxD8os0V7IJ1ttoyu9/oxGpjmUKQnzmN2vtggnf31DRVtL9b3xW9yq+4S9jNixdYRQkuNHpCns3sAupvhopQXk+Yi3uslRGN1JNcwsWuIigMHI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230240; c=relaxed/simple; bh=RDqWoEaDDp768Fdnd/8fsk9c8H7F3taR/Wk1mJd5iLI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=TxHkl5oZRwup3KfKtyINBLgtF1nycuhCuOgpNuXI1RIQtpqzkNTzZEZhJJ4lnAD86jZkQUvNcwKvXFk4JwyCTISWJcvnVnl+0xsosemaSNf7KVobak5mMJVcyVKen9bghNlQLNpa3JwRF/eugeME9dyeHg274TL5R3ozB/DPsdo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=m4oatGom; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="m4oatGom" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-70d24aead3eso2414932b3a.0 for ; Fri, 09 Aug 2024 12:03:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230238; x=1723835038; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=G217uULw1eanng9SZ8z6eqUs2KV3XibdbxN7GEMWgng=; b=m4oatGomofSnTHFgxzLERGs3IYhaO5r0zr5H93xI77SWuquHadY+bRuFo6EYdh3LJt 12qTstYk/lz8m/6PyHnt5HwKgZ7TklVrawmwQiMKMOg8P/O2SV6oYntWlrEzP5zzKvzy pI8xk5bbebSh5/D8XSaeoxdD0ImHPmNNyyumviS3dxGmFTtaXwzdsP3La6cDNBEtZut4 B6TiWsC94P4ULrUYt3VOAzdumVajeRrhgLRJ772nL31FebFbJUcc/4pv0mukuXNoAKX5 vSaf4oN9+DMIWgpjlkueM0ln72lV3pmORW2Ow7MPH7eidDmd8sdewnX+UM5+7Io963gW 4AzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230238; x=1723835038; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=G217uULw1eanng9SZ8z6eqUs2KV3XibdbxN7GEMWgng=; b=cP/kbmBIIz2crhRv1P+/dXnR+Z+ENCXO+YtiLatO12duadaILCUVzMdZQojIV9ZywJ 3+JennaVyd1DyxSrGLpfJbr5KKIABhQ0iPI96/dU6kKYj+3C50Qzvf/inQxxn2QDTSKV TD2EWuk+jPfBxJgbUeTKOwoD2Ara+2bajf0r+p47cEWVWP9Z2uCnA8bOgu7EY4DPrJk9 xcAxlbbkBWf39GB+T0Bl8okdwL6BjJESLo0P2RMHN7obyzQFrcn/i1K7eVcoYhkpCgJS a5J5tus3Sl+FyG8kBdnSnwyGfsp8M8WyBZBDTOKX5G/m1JdC5N1FCQ7vEtcU7+Tj3tMo p5+A== X-Forwarded-Encrypted: i=1; AJvYcCWSK2LXYtz4eCy9aiZL41Vc/pd0Z3tE5Udietnaz466tJkg//siVc7WVG/3C0KrWKm+VUJxb70Hkjk/9woyqJ1znDe1jrmpCymtzwIn X-Gm-Message-State: AOJu0Yx5KNrQUXrs6CBgc0En3VjGkjeiSX4pa5bNClrrnSnxSwwd4A1s IEMn3zCWJQBQR1LpujPvZZJll+PEqGQKPgy/hECA5/ykNgBoTD+5fTk/GqN0ovab5NizIJadFWM nCQ== X-Google-Smtp-Source: AGHT+IFukHzCdY9uMRpg3fMRlm0qASRZ4Id9jFsAUf7dR9BX89bxw65n083WMS3QuIOPqFtD/0LNn1B1Ttk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6a00:840b:b0:710:4d3a:6bc9 with SMTP id d2e1a72fcca58-710dcb3c6b4mr54928b3a.3.1723230238188; Fri, 09 Aug 2024 12:03:58 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:03:14 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-18-seanjc@google.com> Subject: [PATCH 17/22] KVM: x86: Check EMULTYPE_WRITE_PF_TO_SP before unprotecting gfn From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Don't bother unprotecting the target gfn if EMULTYPE_WRITE_PF_TO_SP is set, as KVM will simply report the emulation failure to userspace. This will allow converting reexecute_instruction() to use kvm_mmu_unprotect_gfn_instead_retry() instead of kvm_mmu_unprotect_page(). Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 67f9871990fb..bbb63cf9fe2c 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8871,6 +8871,19 @@ static bool reexecute_instruction(struct kvm_vcpu *v= cpu, gpa_t cr2_or_gpa, if (!(emulation_type & EMULTYPE_ALLOW_RETRY_PF)) return false; =20 + /* + * If the failed instruction faulted on an access to page tables that + * are used to translate any part of the instruction, KVM can't resolve + * the issue by unprotecting the gfn, as zapping the shadow page will + * result in the instruction taking a !PRESENT page fault and thus put + * the vCPU into an infinite loop of page faults. E.g. KVM will create + * a SPTE and write-protect the gfn to resolve the !PRESENT fault, and + * then zap the SPTE to unprotect the gfn, and then do it all over + * again. Report the error to userspace. + */ + if (emulation_type & EMULTYPE_WRITE_PF_TO_SP) + return false; + if (!vcpu->arch.mmu->root_role.direct) { /* * Write permission should be allowed since only @@ -8896,16 +8909,13 @@ static bool reexecute_instruction(struct kvm_vcpu *= vcpu, gpa_t cr2_or_gpa, kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa)); =20 /* - * If the failed instruction faulted on an access to page tables that - * are used to translate any part of the instruction, KVM can't resolve - * the issue by unprotecting the gfn, as zapping the shadow page will - * result in the instruction taking a !PRESENT page fault and thus put - * the vCPU into an infinite loop of page faults. E.g. KVM will create - * a SPTE and write-protect the gfn to resolve the !PRESENT fault, and - * then zap the SPTE to unprotect the gfn, and then do it all over - * again. Report the error to userspace. + * Retry even if _this_ vCPU didn't unprotect the gfn, as it's possible + * all SPTEs were already zapped by a different task. The alternative + * is to report the error to userspace and likely terminate the guest, + * and the infinite loop detection logic will prevent retrying the page + * fault indefinitely, i.e. there's nothing to lose by retrying. */ - return !(emulation_type & EMULTYPE_WRITE_PF_TO_SP); + return true; } =20 static int complete_emulated_mmio(struct kvm_vcpu *vcpu); --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 11CF119CCE7 for ; Fri, 9 Aug 2024 19:04:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230242; cv=none; b=MNuofsRwYBzfpEZQlLkHo5r8t4fOv2ai3zo2LVEeds0CBB+tygnDp9ZaoRkvTWSRRFe9wk2vxNMrqJnsvKmgAiXPSZUNWqISGEkD7npmeNzvUYK07WgCSUUcnR19T7C3G4hyb49Rwbgnfd7gl+RywZUHXPJJlcpQmb0yzSwcn04= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230242; c=relaxed/simple; bh=OP2Xm/eCfjpFyHsXtR5gMPIrKC8fRopal9cAbGgO8xI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=DypeR1Era8INJCAA4gg3ok84QuYZs4nucjXvvDLJyOJBhWbPHC/qJbactnw0poKNsQYqrxw4Gj33hMA7ld+xFK/6dMcl7LhjSxM0byoEb7FrYayxXNUEjTNLzp5pxj3+pi9s8wFCu645cOJOHZK+XDFbG5O6vgLvc7g1c7Crpto= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=UrxlA6W2; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="UrxlA6W2" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-1fc658a161bso19774375ad.0 for ; Fri, 09 Aug 2024 12:04:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230240; x=1723835040; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=8EFwojLdgCXRArpaY4WrRJNxw1pZl6kTHVRAl3enDUk=; b=UrxlA6W2axWrmkJW/aK/C81Dro0T2orEkmqitkMhvYQoVge8LUz725tcTa27ArHi4i AisSFJ/RxN8abI8+BvlWFGo0RoMM0UJofAgI10YEdlKQcZduQFxnM58D/KlawXWKiDF9 QilZeWFRz2fEDBcn6mChX+1GvfOIk4axZJckLL8roYsezwMElX161+q950zCWAzN4AP4 zux18oYCPuHNKsRJGuw4bT8SNciJ9a559ZmgWkCpbVxnisIOHDKJ0b4jQbm4OSSXefUW GSC59q0pD85uCd/U4GUozkI+N5uGux33Bzt0P939WJVC1E/7fV/O9GHgfMxFEU4Hj0jx l17Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230240; x=1723835040; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=8EFwojLdgCXRArpaY4WrRJNxw1pZl6kTHVRAl3enDUk=; b=Yc0MJmq+btDVXsFHg9VbzegBncE+gotVdpYvH4YWgAKhEa9X2dDoCHYP5GIRTZ74we FgYIkUnq7QjTqgNWYrnV5gMI8fkNqdzS3xqb0ODCggGx/i1ZIBSXV3zyK/A6C5ooWMjg aA361kQVxxFzPCRz/zunKOIdrffUtfFoYfuLosWHByWbc4MPExGep3ZK4GZTFHmgzyT/ HrR8DDZ9hVmhzNDlsrWFX6gFDyqP3MfF8lKmyZR/LoQaneKzXNJsFGxGBnec3GrOAWYA EvMU3CmuSMtLkXnGihklkXB92qbBTtkw+kZ9ca2DV+KdFlpzH/EkFUf6DC7rMNAaA0g4 9YLw== X-Forwarded-Encrypted: i=1; AJvYcCUfQs26JRc38Y43UDcY2gqlYa30roja1ae/L6jrwVQm3n4oA2EFsrQ1TYWxoDZvGlF8K1M6XiOJpd8YipZuccVxCM1Sgb0Tc2F/4f9l X-Gm-Message-State: AOJu0YwiZVrUnBGQpXU0AvIqyVj1z5wguPNgDtLGHitRxt9T9MXyZbq9 xgn3hyJr5voTYaQW8lmEYwM9dkqUgQhuFArV9OMovnidBOeJd14bOe8RyPvtJzEIyUkSvIS+tRq 8PA== X-Google-Smtp-Source: AGHT+IHL18+pCNoijbD8KFo1R0+dsdp7SJvyPnWEUeMEzIHpCRLtjzjhtG6pvSImezAFhKPYfcjJa5aR9lY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:ec85:b0:1fb:72b4:8772 with SMTP id d9443c01a7336-200ae5d42ddmr1281035ad.10.1723230240209; Fri, 09 Aug 2024 12:04:00 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:03:15 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-19-seanjc@google.com> Subject: [PATCH 18/22] KVM: x86: Apply retry protection to "unprotect on failure" path From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use kvm_mmu_unprotect_gfn_and_retry() in reexecute_instruction() to pick up protection against infinite loops, e.g. if KVM somehow manages to encounter an unsupported instruction and unprotecting the gfn doesn't allow the vCPU to make forward progress. Other than that, the retry-on- failure logic is a functionally equivalent, open coded version of kvm_mmu_unprotect_gfn_and_retry(). Note, the emulation failure path still isn't fully protected, as KVM won't update the retry protection fields if no shadow pages are zapped (but this change is still a step forward). That flaw will be addressed in a future patch. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 20 +------------------- 1 file changed, 1 insertion(+), 19 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index bbb63cf9fe2c..ddeda91b0530 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8866,8 +8866,6 @@ static int handle_emulation_failure(struct kvm_vcpu *= vcpu, int emulation_type) static bool reexecute_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, int emulation_type) { - gpa_t gpa =3D cr2_or_gpa; - if (!(emulation_type & EMULTYPE_ALLOW_RETRY_PF)) return false; =20 @@ -8884,29 +8882,13 @@ static bool reexecute_instruction(struct kvm_vcpu *= vcpu, gpa_t cr2_or_gpa, if (emulation_type & EMULTYPE_WRITE_PF_TO_SP) return false; =20 - if (!vcpu->arch.mmu->root_role.direct) { - /* - * Write permission should be allowed since only - * write access need to be emulated. - */ - gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, cr2_or_gpa, NULL); - - /* - * If the mapping is invalid in guest, let cpu retry - * it to generate fault. - */ - if (gpa =3D=3D INVALID_GPA) - return true; - } - /* * If emulation may have been triggered by a write to a shadowed page * table, unprotect the gfn (zap any relevant SPTEs) and re-enter the * guest to let the CPU re-execute the instruction in the hope that the * CPU can cleanly execute the instruction that KVM failed to emulate. */ - if (vcpu->kvm->arch.indirect_shadow_pages) - kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa)); + kvm_mmu_unprotect_gfn_and_retry(vcpu, cr2_or_gpa); =20 /* * Retry even if _this_ vCPU didn't unprotect the gfn, as it's possible --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CFBF119CD0F for ; Fri, 9 Aug 2024 19:04:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230244; cv=none; b=AGkFWRDyIkdQDazv3Xobkcy8oE6uXsMwEBnwGbr7+K/qE0RFxKecWeNAYfXQFYagYels4MVKpUwBagGVE2sZTD/8mZnJHJYZvkohKUPxmotMOQt8BZSmYD1FxJTcfG0ulHFHqeyksbUaCNCTETmkSoj7EWfyeSAmXHcWG6xsES8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230244; c=relaxed/simple; bh=wdr9ikZ2JhMxx8FpdS0bIMEmE4PRqH0Dmh2wTfd07I4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=quR07azwxuU/n5ta3ZzOFquuD/wLtY17vcn1TiN27vGuFxYjWDsTuBDNuzw0vGTFfbDA/5T3ArLecmzbvrhLNXfeI6lsnMhJp0ABsZt2Fxnj9Ha9ne8PuM4uGOXVHnTc1CRCfiy+9dpAG2ThT8MLTqXFj74RRhOK5fz2gyKMLVs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=2aj6Uc0I; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="2aj6Uc0I" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-78e323b3752so1819584a12.0 for ; Fri, 09 Aug 2024 12:04:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230242; x=1723835042; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=hf4PmVOwr+xgGzCTD+kiITxVr7NtARshe+47djUw/5w=; b=2aj6Uc0IInxiJX19I7lJV4RpJs1v5/uEgbZQgFQkDhsJy88ojCNPP+nnq9l9yKT+mB b7DsWcVVuasuJ5W6oMUlMouj9lf6JxsowqjmKOWvGkmEs6dWEP/ow9MPOpPIRNYHQk9/ 9xiIrhBD1AAHx4inIH/3iFdo4mZnl0UYoA8UHbgr9amfpiPbwaXTGdQpjc9r1vmZF1QV 51hfj4D/SiwzawxRihQiJ1/aA9nBk07e5yfdyQrsMpmDN4sEO3ybcWoOXlHcNi+yXFgs G4gTBObGpyfJYzb/txflIgwFXtKJLKkJoxiepvIGe0wrgY1EDhI18IRQQ0yaeZOWd+su YcTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230242; x=1723835042; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=hf4PmVOwr+xgGzCTD+kiITxVr7NtARshe+47djUw/5w=; b=gimQvUqTWW6X6CLoNGpjUA/wCxznbGQ7oy6d4b1s2Jh0EJvTgomgSK4OTZpXFbuso2 3VA29wxcukX4OWTOIPAb01I/DUeJ9b5Tu8tsxmiPfF6UjAquc0PStIeCua39KXVtY2b5 0QG0FoJyzQPpohg7MGCuLgdc6QhnlVlqqWSHe+dku+xdQmZ4doGMZex/Z7VcN0ksRET2 q4/V3pAIVefeqfrWidW0S7+0QuYuSRCaH88VUAJeslj55aH21NIKZoGLYfRlapPhfI7e OCUTx24r518HBGsFTMRIHKZR47aN1cnxfl1ugXmHxupaf0X5GaMbHb5jvV5hPdwK9BCD 69yw== X-Forwarded-Encrypted: i=1; AJvYcCXn+bDh427FsWoe0R42MgATM5PfrOBtWpweTaVaTIARYmMIJ5ZUQTbrBr+rA7c9OURY6hM8ehPZ7/Ilk9D7DljRfPdFf849c/Bdek6/ X-Gm-Message-State: AOJu0YwNUitprThYyrgboRDMWP4yJiAtu3jJZpESh9+cfgyd7QHj8kdi IfeGIDE/bgoDlrlzQm/6G1ti+kFJRxKNZJGr4pItfy4qqRBOGGF+Q3tP7wMI1wr0Y8HXjKwgnOA jfw== X-Google-Smtp-Source: AGHT+IGF6pq1ifYV6Vpt8CaGEEk4qft1uIN5zn189JX1iQA8F6BtQ2QllI81BNCavpgKD3Lyc31lvkGgw54= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:fa8e:b0:1f6:2964:17b8 with SMTP id d9443c01a7336-200ae584d14mr701195ad.10.1723230242082; Fri, 09 Aug 2024 12:04:02 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:03:16 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-20-seanjc@google.com> Subject: [PATCH 19/22] KVM: x86: Update retry protection fields when forcing retry on emulation failure From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When retrying the faulting instruction after emulation failure, refresh the infinite loop protection fields even if no shadow pages were zapped, i.e. avoid hitting an infinite loop even when retrying the instruction as a last-ditch effort to avoid terminating the guest. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 10 +++++++++- arch/x86/kvm/mmu/mmu.c | 12 +++++++----- arch/x86/kvm/x86.c | 2 +- 3 files changed, 17 insertions(+), 7 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 25a3d84ca5e2..b3a2793fc89c 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -2132,7 +2132,15 @@ int kvm_get_nr_pending_nmis(struct kvm_vcpu *vcpu); void kvm_update_dr7(struct kvm_vcpu *vcpu); =20 int kvm_mmu_unprotect_page(struct kvm *kvm, gfn_t gfn); -bool kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu, gpa_t cr2_or_g= pa); +bool __kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu, gpa_t cr2_or= _gpa, + bool always_retry); + +static inline bool kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu, + gpa_t cr2_or_gpa) +{ + return __kvm_mmu_unprotect_gfn_and_retry(vcpu, cr2_or_gpa, false); +} + void kvm_mmu_free_roots(struct kvm *kvm, struct kvm_mmu *mmu, ulong roots_to_free); void kvm_mmu_free_guest_mode_roots(struct kvm *kvm, struct kvm_mmu *mmu); diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index d3c0220ff7ee..59af085a6e8e 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2731,22 +2731,24 @@ int kvm_mmu_unprotect_page(struct kvm *kvm, gfn_t g= fn) return r; } =20 -bool kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu, gpa_t cr2_or_g= pa) +bool __kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu, gpa_t cr2_or= _gpa, + bool always_retry) { gpa_t gpa =3D cr2_or_gpa; - bool r; + bool r =3D false; =20 if (!vcpu->kvm->arch.indirect_shadow_pages) - return false; + goto out; =20 if (!vcpu->arch.mmu->root_role.direct) { gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, cr2_or_gpa, NULL); if (gpa =3D=3D INVALID_GPA) - return false; + goto out; } =20 r =3D kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa)); - if (r) { +out: + if (r || always_retry) { vcpu->arch.last_retry_eip =3D kvm_rip_read(vcpu); vcpu->arch.last_retry_addr =3D cr2_or_gpa; } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index ddeda91b0530..65531768bb1e 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8888,7 +8888,7 @@ static bool reexecute_instruction(struct kvm_vcpu *vc= pu, gpa_t cr2_or_gpa, * guest to let the CPU re-execute the instruction in the hope that the * CPU can cleanly execute the instruction that KVM failed to emulate. */ - kvm_mmu_unprotect_gfn_and_retry(vcpu, cr2_or_gpa); + __kvm_mmu_unprotect_gfn_and_retry(vcpu, cr2_or_gpa, true); =20 /* * Retry even if _this_ vCPU didn't unprotect the gfn, as it's possible --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4DE1B19D075 for ; Fri, 9 Aug 2024 19:04:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230245; cv=none; b=HTXKA+tSY+G+9kO8kX+jUlZuPFgt24V3JK4E6UEUuK3Z0DcftMy4ny45rDrxBMCeBC22/S7RWCAnj1yynTMc4jd0rRNtyyFBSdfleoZU+BfJ9D+15qAb1G0BU0khMp5An72K5W2vtpo75O8n15Lt+UPI0lG2S0YiYDVMw9F3zBI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230245; c=relaxed/simple; bh=rL3+IDQm11patEvsiloUsWqSDQCzY8fDc8KY40xXf1M=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=EFBUA5fauM2yaEnSkPxljd+9EU3n773p9EyS9cPQNcq3F04YpAFW90WcsSShEI00S4k1yjy+r5jlfr4n2T7ey2knOYgV387sAqdlcGK72Ga/vekcRJNZ2fA8CjZrMoBoEiZcY/kGNMDLM9pFICzF0Q06FvqaQC5cVbBduqk+77Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=xhJNpf1j; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="xhJNpf1j" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-710ca162162so2116674b3a.0 for ; Fri, 09 Aug 2024 12:04:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230244; x=1723835044; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=zAoc9f/zQ674f2x3bQLKg9kULz1L+f8jr5RFvkl+QSU=; b=xhJNpf1jG8biD/CMsNeOEXm7qfDsq4e/aYc/J8CAoTWqbDccKWylPufMFLRY3K8Gef Rqgfmci2qygfNkSXmMAIx1vhDHXmJAFrP0WdGVHdYPeKFfhaKXpjyaInpTgPFTwBGrqN PnQ3Z89L46S1KSp3dq0Sg/x5Osi32fys2jhg6YAgs/SGFnP8QivF7U9zVukzeMT5xCBZ FgooekYw9Zzlfy+8E99vRavo67yKUpuRPC+kXZOhVSK/7v5s+VHVXBr2E4kfbauCgL2w czSsWCj/dV7jpbkEILqadRrvS7S7iFUbQvsAKKCXXwl3DKQfRfT30fXdQ3r8qzTp/fch BqCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230244; x=1723835044; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=zAoc9f/zQ674f2x3bQLKg9kULz1L+f8jr5RFvkl+QSU=; b=We64LNBY7xA+wNPQoyLVDhfmIpcAM1PmdMRCIEz9Oqp6dMjxowZQuaVQ//xplpLndK JUU8bTxNWAKYLE+d6n2EPWQcjqmpV92leEApjkN3X1b2SbE0eWdLahjTcTVfUBk1Oeow DH44hyM/QRyOt2dmvhNt3J7uS9TRXsk35FF08kIMEdHN0hcbv/byR/OTUoUjz9M0CdRD V46bCeFwabkfETyL4yR9/RI0zyfDrghF4F0aZUDZ04qvuGIKylIQxgjIIdQYS85IQFxV p16d9ejsztb3xdyhbl9nRrXNtiBLq6V9pv1l4/dR7B1NPRknAAUnMT9FRD7ftfoyXTfM bg0Q== X-Forwarded-Encrypted: i=1; AJvYcCVIu6O9l8QKqt4SAFP3qdZ00RLEhjhui+Wycg6enVsXYE8Ry/5H1fb1VfSD7jljmaCjQSIvyE2vaM/HFu8Lx80kCktoeQITgQw0SI0O X-Gm-Message-State: AOJu0YyszZRlMKnMYsKTEBhYCMPAzWkLF8Cwq7sbuIWwtvgoISrFUVko LTrh6u949Iu1CUPgmvysQRbmXwIkMCpTp7iFf4bcrUvPKax+gcu4GNdTE5AfjiTLNTox/j7GI3G DBA== X-Google-Smtp-Source: AGHT+IFWWDSkqm0UJUbqcujAzmPwVvL+hZvhhLf0wMFFrRq4oNl8ycj37ldJXO+dIolxfhHFVUMfScjgue4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6a00:9158:b0:710:4d39:c8f9 with SMTP id d2e1a72fcca58-710dcb62de8mr26720b3a.6.1723230243593; Fri, 09 Aug 2024 12:04:03 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:03:17 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-21-seanjc@google.com> Subject: [PATCH 20/22] KVM: x86: Rename reexecute_instruction()=>kvm_unprotect_and_retry_on_failure() From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rename reexecute_instruction() to kvm_unprotect_and_retry_on_failure() to make the intent and purpose of the helper much more obvious. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 65531768bb1e..2f4bb5028226 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8863,8 +8863,9 @@ static int handle_emulation_failure(struct kvm_vcpu *= vcpu, int emulation_type) return 1; } =20 -static bool reexecute_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, - int emulation_type) +static bool kvm_unprotect_and_retry_on_failure(struct kvm_vcpu *vcpu, + gpa_t cr2_or_gpa, + int emulation_type) { if (!(emulation_type & EMULTYPE_ALLOW_RETRY_PF)) return false; @@ -9131,8 +9132,8 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, gp= a_t cr2_or_gpa, kvm_queue_exception(vcpu, UD_VECTOR); return 1; } - if (reexecute_instruction(vcpu, cr2_or_gpa, - emulation_type)) + if (kvm_unprotect_and_retry_on_failure(vcpu, cr2_or_gpa, + emulation_type)) return 1; =20 if (ctxt->have_exception && @@ -9218,7 +9219,8 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, gp= a_t cr2_or_gpa, return 1; =20 if (r =3D=3D EMULATION_FAILED) { - if (reexecute_instruction(vcpu, cr2_or_gpa, emulation_type)) + if (kvm_unprotect_and_retry_on_failure(vcpu, cr2_or_gpa, + emulation_type)) return 1; =20 return handle_emulation_failure(vcpu, emulation_type); --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4DAB719D889 for ; Fri, 9 Aug 2024 19:04:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230247; cv=none; b=kfYmyglX+TUZ4Wf2O8zeg4UQ4ykbPMmk9FzvJezSu4A2A2ArBlSXAKIzXg+lYEkDOnwZ+ghjLm5tZvEYoi9yBLyg2nbU8S3iQZ/4agkj/NtwZ2ggEUigIQe+SkkZL6LahLS5pT1USYcmLgUAjMLtH45N+HodjZ9jpqqgasx7aTc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230247; c=relaxed/simple; bh=RAj0k26L7e6hItfq0DdkXBLRRrgYaa5pBwltIWlH7wA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=fcB+StWowb2IEMyqiqfzHzkwnp7fv8d5agmFxO/08+zSMad65+SvaqIDwMIgcrAiBmlvEs/MiaKsHN9/YfEEzOgs/ez9pCcyNWpQanUYdTIRJDOJqjVSLyTczL1te6BrEqQOsmzmUOfX7fkxn6V9aQ6LJaf5Z1hOjFfFn2l5Teo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=m1kAB4Wy; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="m1kAB4Wy" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7a994e332a8so2108115a12.3 for ; Fri, 09 Aug 2024 12:04:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230246; x=1723835046; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=G4BNuwtJ6NMA4YiBcM1PUSgleR458VStyd6GBZZF8w8=; b=m1kAB4WyqvGB5XwCXRyTuFYrHtxBGoNOXUcb6HR14TUPMOgJcXlNK2Bu5GyDZt/qGd jkP23gqlrOUwcA6SM3WWw5dipdSzj4GCLBcEkKULOrOexMmjzgDurbmJwjAyozXS1c5Y Ftg2l+bMOI1tM2b9qd/ddLvmEogJrRb3Nt5jGiR3xkdDWg6e11OqPx69xLB8Xkbn6kLD M8Z2BPmgelXxBUHx7abL6MioJTHEsHVLt6jA5uLnllkEpIJPDAUeobv8x8D++x2/P3N4 XFtK0tDcZ2r/xtKx2HkuaO/wptfHUo+8HfLASr6H6GT2ILZdKoLKnU/vYK23DEf8gfx7 CbJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230246; x=1723835046; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=G4BNuwtJ6NMA4YiBcM1PUSgleR458VStyd6GBZZF8w8=; b=wpi1+RkkDPLmwZ3f8b0jyYusRfo171qvdChKgYG6zBzoPC938SpinxiuVliAZl5mO/ JoYaqPYzk/HIzwyuTxAl877dVJy0KPL7V0xMuSs8YdMZtqPTwx7IZesdsavqopcix07J SHQYcpIU8+SWISOh6QYoqkEvWpOLerMWzSU/+3PAN4WTli9va+4r81ufCog6K/8Lag06 4QWXgDkzTVWORq+0uDN6I0qXihLZPjwqutrmQUTIIhxMSPWU8rORd3HPGwrnaRl2LFjf taf1EKAMceIHzzLtDmU4WuxCRMy6grB/yawiKVsqbe4Feij5F4wFEipO8nsI0LCJreiS Yclg== X-Forwarded-Encrypted: i=1; AJvYcCWXYjrHuAzIarNYpt7Gs9pJZ3XMmsNmin+lBT30vgoGy8Y63SlNE5nOL1wBI86/UyYHdyMM6KT8uAcz/eJglYvQtgzlPvQYGONqhOpp X-Gm-Message-State: AOJu0YxrS/t8xlCYhkwObMKnvUGvp1UYa84aIyupOPoG0Yyfx+P5gnuA b/dEo/8kILbnw+QepAKT6YAkY/XwpK2XJ4Va0uAyCRoTn1B05zSZD7FZew9AqnbBlqFIEiWuGgN 0Pw== X-Google-Smtp-Source: AGHT+IEDzFUCPnHj7AFwv0nC2uk2Vq5ql39xuChh1cZ88t8pY1ewqFSi2d5sUYKbJsHcj8NSBHzLWancOaM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a63:d64e:0:b0:7c1:89ee:a9fe with SMTP id 41be03b00d2f7-7c3d2c14841mr4639a12.8.1723230245533; Fri, 09 Aug 2024 12:04:05 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:03:18 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-22-seanjc@google.com> Subject: [PATCH 21/22] KVM: x86/mmu: Subsume kvm_mmu_unprotect_page() into the and_retry() version From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Fold kvm_mmu_unprotect_page() into kvm_mmu_unprotect_gfn_and_retry() now that all other direct usage is gone. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 1 - arch/x86/kvm/mmu/mmu.c | 33 +++++++++++++-------------------- 2 files changed, 13 insertions(+), 21 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index b3a2793fc89c..e2df07b3c411 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -2131,7 +2131,6 @@ int kvm_get_nr_pending_nmis(struct kvm_vcpu *vcpu); =20 void kvm_update_dr7(struct kvm_vcpu *vcpu); =20 -int kvm_mmu_unprotect_page(struct kvm *kvm, gfn_t gfn); bool __kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu, gpa_t cr2_or= _gpa, bool always_retry); =20 diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 59af085a6e8e..300a47801685 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2713,31 +2713,16 @@ void kvm_mmu_change_mmu_pages(struct kvm *kvm, unsi= gned long goal_nr_mmu_pages) write_unlock(&kvm->mmu_lock); } =20 -int kvm_mmu_unprotect_page(struct kvm *kvm, gfn_t gfn) -{ - struct kvm_mmu_page *sp; - LIST_HEAD(invalid_list); - int r; - - r =3D 0; - write_lock(&kvm->mmu_lock); - for_each_gfn_valid_sp_with_gptes(kvm, sp, gfn) { - r =3D 1; - kvm_mmu_prepare_zap_page(kvm, sp, &invalid_list); - } - kvm_mmu_commit_zap_page(kvm, &invalid_list); - write_unlock(&kvm->mmu_lock); - - return r; -} - bool __kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu, gpa_t cr2_or= _gpa, bool always_retry) { + struct kvm *kvm =3D vcpu->kvm; + LIST_HEAD(invalid_list); + struct kvm_mmu_page *sp; gpa_t gpa =3D cr2_or_gpa; bool r =3D false; =20 - if (!vcpu->kvm->arch.indirect_shadow_pages) + if (!kvm->arch.indirect_shadow_pages) goto out; =20 if (!vcpu->arch.mmu->root_role.direct) { @@ -2746,7 +2731,15 @@ bool __kvm_mmu_unprotect_gfn_and_retry(struct kvm_vc= pu *vcpu, gpa_t cr2_or_gpa, goto out; } =20 - r =3D kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa)); + r =3D false; + write_lock(&kvm->mmu_lock); + for_each_gfn_valid_sp_with_gptes(kvm, sp, gpa_to_gfn(gpa)) { + r =3D true; + kvm_mmu_prepare_zap_page(kvm, sp, &invalid_list); + } + kvm_mmu_commit_zap_page(kvm, &invalid_list); + write_unlock(&kvm->mmu_lock); + out: if (r || always_retry) { vcpu->arch.last_retry_eip =3D kvm_rip_read(vcpu); --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sat Feb 7 11:38:11 2026 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EBC7319D8AD for ; Fri, 9 Aug 2024 19:04:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230249; cv=none; b=vDjz0svskfsegNqy8Kq5K8RytZxBhQJZWgkKZokZiK0oK5AffEay1GpkrnYelrxMDjS9yA10IvFHMslZwp0Y3vr/2Q18z2VaQTDk8OaQEV7LOTWt6JoNnywzU2cn8ACmLoOfGkvCj/DfBB6jDtCXU4fb9nk7iiumUHveywgiGiE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723230249; c=relaxed/simple; bh=fQuuC8h+my/ip7xEVnWzpMj4lV5hpjnXHtbzdkoRqRI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=WR+sy2w5R/jC+tPM7m5Hhm8THjNZRz1XRo0XilStZtv90fMq0Hga2Q/WXa2UvYKlsmTRUdgjwvSeNwJJApjtjg697R7+13ikDXI4//QdM8B3YjBJRFLM8gWRaLWcyMl+67RMFNGLf6tu2U+0EMae9WzEqT7tuQrwDZ4XDJd+UIo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=BtQcNZHX; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="BtQcNZHX" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2d1da888717so1705007a91.2 for ; Fri, 09 Aug 2024 12:04:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723230247; x=1723835047; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=oWpScC2lPAOYQSvm8ofyxB/d7cOw2/4qi1QTT+90O+I=; b=BtQcNZHXhmSs64QAR2YOmmt44qJx16MN0wvYFpE4NKJQYAKOrch1mmZ4RJLN/JijtZ /54RyTnlbUhvzsFYk5SnfTosafZpskQk15P5lXSo4FGNyIf9xmey5RtbUYRaK2uC0qJe HZJtm65BKrobVAoUnbHk7hCevCsJhXbKs+HFo2bEIzl/rPnUC1U2iFzKj8Dye1Hx00Q9 axYvDCNfVienClKv9u8EkBVDRljsvtToxTQgNQ6Wfr04y8soD4ki5TNmoJe80nLaY3mo 3DKhFyX5fr6xG4EDnCzWW6u7+JQdiNzjI3UdMxx4LB4g0ISVG2F857CNIBCVBPV6g5LK wOyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723230247; x=1723835047; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=oWpScC2lPAOYQSvm8ofyxB/d7cOw2/4qi1QTT+90O+I=; b=epaPNQueGuXPzvU2enBayVCvXC91PflPC4wNfZCRPUG+06T62UjuRJXNhJiVzQUULT WzLGuxBEsXeZnf3NFizRD5o725CHPHw3EvXKXyZLyfY9inbxuNmCgEMctaWsuVUnj5q9 ok96wtvWr/xK539NEMArh8wvOcn6WlBhOjb95zGTetTykAXeOKn/P9otcqLbj0ZqO/RR iXXc/l6Sxwq61Fym8CTGDLMdtNWof7nlpYsc3y15kQWmlQfusNS1hjV7MQVcPSsuRD4v 8zGo4Cz+2Kd8qj4U+CMi7+LbRvCN/W7+mhLFtKTxsTfkME4qcnImDtloJQKTf3DmQUn1 LYGA== X-Forwarded-Encrypted: i=1; AJvYcCUwRb6n0DCAvJggDcTNZAvXtSOq/iD+ZFXpwl6p+t7NsLjuGlppYQmbWmmFQIuOghYzG+qVkOWy9W6KMpXF3hJJBX5WCp6MryfM0jdJ X-Gm-Message-State: AOJu0YzfXzyqNjKEmEafFr9XXvNukcr06j6l6oAPYPXNbAcNzNZxOvQs qRH0ZAeIrgKxkS7JJKGVFBB3M8Mfv+vsv3qmJEvYTun1mMVfMKyjTiGQ4yvgP6rzmzdPCrwaBMi cSw== X-Google-Smtp-Source: AGHT+IGa40bwUoG33bFhX/FYTpen2eZdAbi/a/MWhlBc3vfk78wwXJqpBLLJqIKqaXmEqi+kQcPOVyN0PcA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:90a:d812:b0:2cf:dafd:b793 with SMTP id 98e67ed59e1d1-2d1e80674c6mr10196a91.5.1723230247317; Fri, 09 Aug 2024 12:04:07 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 9 Aug 2024 12:03:19 -0700 In-Reply-To: <20240809190319.1710470-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240809190319.1710470-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240809190319.1710470-23-seanjc@google.com> Subject: [PATCH 22/22] KVM: x86/mmu: Detect if unprotect will do anything based on invalid_list From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Gonda , Michael Roth , Vishal Annapurve , Ackerly Tng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Explicitly query the list of to-be-zapped shadow pages when checking to see if unprotecting a gfn for retry has succeeded, i.e. if KVM should retry the faulting instruction. Add a comment to explain why the list needs to be checked before zapping, which is the primary motivation for this change. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 300a47801685..50695eb2ee22 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2731,12 +2731,15 @@ bool __kvm_mmu_unprotect_gfn_and_retry(struct kvm_v= cpu *vcpu, gpa_t cr2_or_gpa, goto out; } =20 - r =3D false; write_lock(&kvm->mmu_lock); - for_each_gfn_valid_sp_with_gptes(kvm, sp, gpa_to_gfn(gpa)) { - r =3D true; + for_each_gfn_valid_sp_with_gptes(kvm, sp, gpa_to_gfn(gpa)) kvm_mmu_prepare_zap_page(kvm, sp, &invalid_list); - } + + /* + * Snapshot the result before zapping, as zapping will remove all list + * entries, i.e. checking the list later would yield a false negative. + */ + r =3D !list_empty(&invalid_list); kvm_mmu_commit_zap_page(kvm, &invalid_list); write_unlock(&kvm->mmu_lock); =20 --=20 2.46.0.76.ge559c4bf1a-goog