From nobody Fri Dec 19 07:17:35 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 729DB4C8C for ; Sat, 31 Aug 2024 00:15:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063344; cv=none; b=erWxSOlUxcQHWX+iAp3zor8HgyzFPPN+NP+9+GpMCo1f3mYSQ78x4yZHQIODO79HUGkDib3JdhMM8cHfk3mwE/2fbQPWho/0PKO+VCXEHTANr0OitYBTW8LjJ2xBIkZWp3qX93G/34kCqbaf6sr2MdtYq3I2s0oqdLb3wJVawQI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063344; c=relaxed/simple; bh=YXptvY2Uza2H6tZmzonyv74pSte76Tmo4IfX2fwO+lI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=C9u6YSlOLDFJLFeHquS4MX6tpPSRGFONpk1bn6oot8kTQiLRs5VKX1uIMwBR4Q6GdeKalc3PXk2MfxbTZZaiyKLB7+eyTbWOLQkhs4qSEpOEaHGQstMBI4q5TmjX73eroyZQnRufA4dPZn/ijJQORw+7SuTkzSQlD45dRxoh1SY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=wVS4K4EC; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="wVS4K4EC" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-7142a78918bso2851653b3a.0 for ; Fri, 30 Aug 2024 17:15:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063343; x=1725668143; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=TE7D/Acr9LdQYMIXlBbpo74VePMtTsMcOWGqO6E6aPU=; b=wVS4K4ECSWnN4B2tXS5Xi+dfWOwdFk322Z3lzJgGYbNbPuUk1GVvTgQl1EF2ZORY38 ygjMeFPBXe6cWuHk/yvVr3aryruQGspP8HdJKGDqhdzE2KkA6mC7hnRFaTOKg9UwQywR SZRU1ya92RrqYrA2esJCvU9eMKpCMUyWhGONaTCBMWr6VF12KyVzLvTo7duQzHG9uynM zmtgYcP2hn40AYEZn1AmkgebR71OKQmZAzDhQh/Hf5we0VFvUK/UU7MOzG59mJ27jIku PDHgP49RazXClrJ2AzxVYkmKC09zntgMVqGjLnZIClTAsE6vFG6h5oMR5gOGB5aj+/iJ JkNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063343; x=1725668143; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=TE7D/Acr9LdQYMIXlBbpo74VePMtTsMcOWGqO6E6aPU=; b=aOhMjIEke0aDCpO0BixS1/rtpKwu/N47c0jl/vrIp0O8Z0CuCS3vvpVpY5yFIGQz1o 3zBgsdaRr6NNnI1Js0cQaP9iabinpG48s1JzKLpkAsPr3IX7WUUA/opOp0DoTO4lx08R N5uetE/bi4sePzO9+pNMbQX6gSHjlISISpG+TK/ndP+wWHgknqO/Ex07Rkpsl6u8uZEV tRDa9KxiKjTSIwlmXZGvaQHriENytDqzgjXSLL6mKQ6wfRgNWamN896/S1VGrmIUydWs AhbBFHNXo9vaEfut3P1SxfCL42rMUDsWoxiIvom/u6Qoe35cHb6buoXugh2mRGBUyp0R Et/w== X-Forwarded-Encrypted: i=1; AJvYcCWe8SjtyAD6BIVGKGq3VROtowBW9mWiMktetTZ66Q8l1szAvpWA4nrE3FJvQ58eXWr1uGPXkF5fat8F1qU=@vger.kernel.org X-Gm-Message-State: AOJu0YwZXs0ODCA8ZplJuRo/GQAC+OpWBUWs2WLC0N108Q3v6gtggoVH LkH+8XlgZ1kqbYs/VV8nRhwzSAPUvobwgNdNGf+Ct1WEB0ViLEtX0qNzx4AXmJcJZw4b2zWLsyf h5g== X-Google-Smtp-Source: AGHT+IHgRgu3nlEMwZydwng1Zdptt9mq0HfSF6rdSzaH8wMvVyFEIrgTbt4SBvsLsaty6rpdkiRrHOmEX/c= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6a00:739d:b0:714:200c:39b0 with SMTP id d2e1a72fcca58-717308ae005mr11477b3a.6.1725063342706; Fri, 30 Aug 2024 17:15:42 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:16 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-2-seanjc@google.com> Subject: [PATCH v2 01/22] KVM: VMX: Set PFERR_GUEST_{FINAL,PAGE}_MASK if and only if the GVA is valid From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Set PFERR_GUEST_{FINAL,PAGE}_MASK based on EPT_VIOLATION_GVA_TRANSLATED if and only if EPT_VIOLATION_GVA_IS_VALID is also set in exit qualification. Per the SDM, bit 8 (EPT_VIOLATION_GVA_TRANSLATED) is valid if and only if bit 7 (EPT_VIOLATION_GVA_IS_VALID) is set, and is '0' if bit 7 is '0'. Bit 7 (a.k.a. EPT_VIOLATION_GVA_IS_VALID) Set if the guest linear-address field is valid. The guest linear-address field is valid for all EPT violations except those resulting from an attempt to load the guest PDPTEs as part of the execution of the MOV CR instruction and those due to trace-address pre-translation Bit 8 (a.k.a. EPT_VIOLATION_GVA_TRANSLATED) If bit 7 is 1: =E2=80=A2 Set if the access causing the EPT violation is to a guest-phy= sical address that is the translation of a linear address. =E2=80=A2 Clear if the access causing the EPT violation is to a paging-= structure entry as part of a page walk or the update of an accessed or dirty bi= t. Reserved if bit 7 is 0 (cleared to 0). Failure to guard the logic on GVA_IS_VALID results in KVM marking the page fault as PFERR_GUEST_PAGE_MASK when there is no known GVA, which can put the vCPU into an infinite loop due to kvm_mmu_page_fault() getting false positive on its PFERR_NESTED_GUEST_PAGE logic (though only because that logic is also buggy/flawed). In practice, this is largely a non-issue because so GVA_IS_VALID is almost always set. However, when TDX comes along, GVA_IS_VALID will *never* be set, as the TDX Module deliberately clears bits 12:7 in exit qualification, e.g. so that the faulting virtual address and other metadata that aren't practically useful for the hypervisor aren't leaked to the untrusted host. When exit is due to EPT violation, bits 12-7 of the exit qualification are cleared to 0. Fixes: eebed2438923 ("kvm: nVMX: Add support for fast unprotection of neste= d guest page tables") Reviewed-by: Yuan Yao Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/vmx.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index f9fbc299126c..ad5c3f149fd3 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -5800,8 +5800,9 @@ static int handle_ept_violation(struct kvm_vcpu *vcpu) error_code |=3D (exit_qualification & EPT_VIOLATION_RWX_MASK) ? PFERR_PRESENT_MASK : 0; =20 - error_code |=3D (exit_qualification & EPT_VIOLATION_GVA_TRANSLATED) !=3D = 0 ? - PFERR_GUEST_FINAL_MASK : PFERR_GUEST_PAGE_MASK; + if (error_code & EPT_VIOLATION_GVA_IS_VALID) + error_code |=3D (exit_qualification & EPT_VIOLATION_GVA_TRANSLATED) ? + PFERR_GUEST_FINAL_MASK : PFERR_GUEST_PAGE_MASK; =20 /* * Check that the GPA doesn't exceed physical memory limits, as that is --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 997D4A930 for ; Sat, 31 Aug 2024 00:15:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063347; cv=none; b=kYDWh70pGAviZls7S2olT1hvVhlHR7INtYWA7NqeQTdhYbDe2NlpnJm2Oxs+Lnlum5p2JjfG5nm5AFTuIZL/kPnEHqF0JcZVZo77qszgG9pYz5DolGUvUfqPGbTrMtSJaYxYpZvhw6yNOverdbyl7SdLLNWgXXoauHu6PjRZd4o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063347; c=relaxed/simple; bh=Mo/ClTEYGuNJJb8dKR3/LwowRG+11WAmRzNI0wc6hIo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=o+cGJYj1i1vZl/t1gxZ3tWDkQawbENKDhdiZZbeN6NSHDRvG2xidtNJglHRnGBER24R+2A2DjCPU4tvQQYvVvwtJfkE01EpUaYf6Hvz6Q3HwzAX0i40BHjbAGIy0vEER6E8Bg7KBuQXymGYe/GaP2ivBKDvr9wSJV6A7V5FjHKc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=zUqpGxDA; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="zUqpGxDA" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6ad97b9a0fbso47398817b3.0 for ; Fri, 30 Aug 2024 17:15:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063344; x=1725668144; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=4GoAAkSfNvtumfQvjreZFP9Dt8KDDoJK+os4hnvMlFA=; b=zUqpGxDAkLHSy1m0G0VXcv28dH5UkPDbGfG9jD2oTaIMmsqfsXM8Og+S4qcxGjkeQi YXExGWU0qNI/1WMXxiRlPFJSS3S38G88u3bD5Ullu1Vha/o064Tpom9hCh+x2dSTzUcb Ogg7v1jobsRrTCHXFysi9i/6RUnYCQ/L0voUEFp3MJ0+YNYHNlSpALws7JISQqbUMTKu drpr6mVYv+QWvQZkoaTBR8OUy+JFFf8sADsPHIlodjy5LAQbD8SEbqvH1p7yD3c7RCtM o2ibQeOgeIMagHHTUV3WgVZvyyKAsKTUyn8m1HI142xhbKIBOsDyvXYVlEyTRGfZt9xo aOjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063344; x=1725668144; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=4GoAAkSfNvtumfQvjreZFP9Dt8KDDoJK+os4hnvMlFA=; b=K2JaDc5ufAii2ZjBELo9QpxirvjZH1uNd/qr6IAjeUTutqrlarbOajEV27SSp4kcXP l4bv/k+/gPd9xynAPerSjpOYKAV66Z5PojL1U614kQaaN/WDtKx2jU3FZFAnJowdqTB9 vgP3mxWNdfd2mFG1SfKUikNscN/30KmXSUGluTcq+MUDkeSTnEuqHmeFPcNzHECkk0Nx cw2E0gD1dkVvmvcU5qKRbkiPRWzoqoyjFd/d+BF4ptLXnDqJuPTYn+uHpPmfGC/7ldzQ 6A3GZtxmuyrIi6FW0Z0aRdY0MSPMrKRSUpa0HMxZkn8O0lyczapTcPxARFi6r3I8qfn2 Mpow== X-Forwarded-Encrypted: i=1; AJvYcCWUCxcUNAy9BNtJ6XAGaCNz+SPI48XMuYqLjKXKQjMNTG2h/OAidyEgyREKXxG8zDVr9fKCtVLZRQOH9PI=@vger.kernel.org X-Gm-Message-State: AOJu0YxH/Nl472qGnXqL+gewPg7wmvls+GMYm4AAVsW65Y24DxaAN/19 h3wbAAu2zJZhgWqr222y3falzLKoI+mHS1j77rmJSrLTvkzUFJ1QTo6ayXEsb4a1Hjt3Tnmrsft SUg== X-Google-Smtp-Source: AGHT+IErh4TFCGh75jwFCCbkJhJfx7XdDEylAQkQhEXoE6ZOSV4assjNprZddTeOdsaNpULeb35aupwsApg= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:690c:4182:b0:691:2f66:4b1c with SMTP id 00721157ae682-6d410103377mr464037b3.6.1725063344714; Fri, 30 Aug 2024 17:15:44 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:17 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-3-seanjc@google.com> Subject: [PATCH v2 02/22] KVM: x86/mmu: Replace PFERR_NESTED_GUEST_PAGE with a more descriptive helper From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Drop the globally visible PFERR_NESTED_GUEST_PAGE and replace it with a more appropriately named is_write_to_guest_page_table(). The macro name is misleading, because while all nNPT walks match PAGE|WRITE|PRESENT, the reverse is not true. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 4 ---- arch/x86/kvm/mmu/mmu.c | 9 ++++++++- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 1811a42fa093..62d19403d63c 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -280,10 +280,6 @@ enum x86_intercept_stage; #define PFERR_PRIVATE_ACCESS BIT_ULL(49) #define PFERR_SYNTHETIC_MASK (PFERR_IMPLICIT_ACCESS | PFERR_PRIVATE_ACCE= SS) =20 -#define PFERR_NESTED_GUEST_PAGE (PFERR_GUEST_PAGE_MASK | \ - PFERR_WRITE_MASK | \ - PFERR_PRESENT_MASK) - /* apic attention bits */ #define KVM_APIC_CHECK_VAPIC 0 /* diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index d25c2b395116..4ca01256143e 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -5947,6 +5947,13 @@ void kvm_mmu_track_write(struct kvm_vcpu *vcpu, gpa_= t gpa, const u8 *new, write_unlock(&vcpu->kvm->mmu_lock); } =20 +static bool is_write_to_guest_page_table(u64 error_code) +{ + const u64 mask =3D PFERR_GUEST_PAGE_MASK | PFERR_WRITE_MASK | PFERR_PRESE= NT_MASK; + + return (error_code & mask) =3D=3D mask; +} + int noinline kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u= 64 error_code, void *insn, int insn_len) { @@ -6010,7 +6017,7 @@ int noinline kvm_mmu_page_fault(struct kvm_vcpu *vcpu= , gpa_t cr2_or_gpa, u64 err * and resume the guest. */ if (vcpu->arch.mmu->root_role.direct && - (error_code & PFERR_NESTED_GUEST_PAGE) =3D=3D PFERR_NESTED_GUEST_PAGE= ) { + is_write_to_guest_page_table(error_code)) { kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(cr2_or_gpa)); return 1; } --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D2A0101F2 for ; Sat, 31 Aug 2024 00:15:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063349; cv=none; b=lHsMqcJTRKOjo/3DJQmMbZfyc46MSKW8V+NpMiMcaI7JqnvXKTjlr3V3jHKAs3UTL/BAQv4wruDFapguxnaBCECBzycTZalDZVx+xaLxoE8fIJfbpsY5jBqF9tiS9K39gFn/Kcuy3PECTe8BuGRh54vkck5oA6A5fnFBfFd0w5Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063349; c=relaxed/simple; bh=0S1/EC5bIW7JtKKmJwoso0teGFo94sSYh6FcpWHE3qM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=OdmfhB9tMDhLBwTcskjJYBJjwmv+4Fo229lbkmWglbtwgguRIoPDfmLmQDTGeF1LJFn+V8Y2LeB2/qdDxot+n6HG0pgnYxZrWnmw4HLp0Qcz3HCeC4bBDaZTh+Xdi4vdjtg+z+Mc/VfX0gILOVg6WubtYhz/Nm6HxW4knWpGWQE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=FotemT0a; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="FotemT0a" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-201e94a71a0so21133315ad.1 for ; Fri, 30 Aug 2024 17:15:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063347; x=1725668147; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=XxaFoHu5A4IbMNy5peQQugHxw56QXCkNO6VTfRtkiXo=; b=FotemT0aYrkpUXJK3O2KZVdcObrvQw+Bo1XoSIkxcl9m4ZxgL2WOJeMFodlDRQ1hHs EUjtcfL2XEApQq5gEk9iY+gx5vdae2koFcfDzBtuHlBbz0PE7GdJSMeJB0nPzO56ZDRI k7GD3xBJlG6bkIlU1m/PgVDcZs5Jvz/3Ly/6qJYZhxAvHYh+6aLUeqVds63fHpMhERA3 ceG21gxPhoFq/8lJOq7LxUWDOr69aFa7As1q0ou3ClYQcIdqDVIcviRT5Ss2Lx7JT7Om XQniYdthN2DFUseIEqJFHu7UKJgOYaxuH02LsXe9XBYG+7LtuJSmH67H8a8fbRM8Rp0n cyEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063347; x=1725668147; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=XxaFoHu5A4IbMNy5peQQugHxw56QXCkNO6VTfRtkiXo=; b=aG7RGX3DlsdUSyXFRqO3zn17bPo2S5p5Efa+q1CqfBdnC2tY/ODo2I55OjdyjB/szy u/srMeXX85yQeCGJ+d1b1RCGZf6HAVuGpvjSyyeD+9XSRJhtFAwXWHcxNpZlVPeFCWIl ag0x9JeAdyChVsROcLS5NwvCFAH5YkMSBxjRV171QoFA+jCHVAbhcNe/IC7/PVw6A1kn 6Ps3YggLPaYa70rZMcLey8l9S+axaQaVpo7xw3+9cnLWBNX54jgNlKna3UejMFirrSyu tUaST2SjtsLtt66LXJAaOr/f8KBc4pcsTvgl6qsuTTLeAVf/dpjit8SDPKrde8RB7mEh 1BoA== X-Forwarded-Encrypted: i=1; AJvYcCUeq7PzJZzn/rKnGFE/DfJAFt9sN5yOMuc4Qb2awLni35GNipxJCZ4+Pb62JB2/XH7zEdnoAHRqL/pwQXY=@vger.kernel.org X-Gm-Message-State: AOJu0YworpPebJ6j4keMFFPCA2447chlMMmybm6OS9Q8Pl5BWibeDUuv pVnO6RTzPBtZ30Xo7ZnMiOZ+zUk2ETx7/lOEi7pCCqfmIxa1+V4sOzfJWuWbgE+dhso5WoRfedz QUw== X-Google-Smtp-Source: AGHT+IGKNozOb7ZK1aStUW9JhhyXqjyJ0ZBSiMSMAq640/pmOcPj4mJYAE5X8noIc6nc/DNjGrR2ZttuBmw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:903:11c3:b0:1fa:1a78:b5bc with SMTP id d9443c01a7336-20527228c4emr2748435ad.0.1725063346583; Fri, 30 Aug 2024 17:15:46 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:18 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-4-seanjc@google.com> Subject: [PATCH v2 03/22] KVM: x86/mmu: Trigger unprotect logic only on write-protection page faults From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Trigger KVM's various "unprotect gfn" paths if and only if the page fault was a write to a write-protected gfn. To do so, add a new page fault return code, RET_PF_WRITE_PROTECTED, to explicitly and precisely track such page faults. If a page fault requires emulation for any MMIO (or any reason besides write-protection), trying to unprotect the gfn is pointless and risks putting the vCPU into an infinite loop. E.g. KVM will put the vCPU into an infinite loop if the vCPU manages to trigger MMIO on a page table walk. Fixes: 147277540bbc ("kvm: svm: Add support for additional SVM NPF error co= des") Reviewed-by: Yuan Yao Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 75 +++++++++++++++++++-------------- arch/x86/kvm/mmu/mmu_internal.h | 3 ++ arch/x86/kvm/mmu/mmutrace.h | 1 + arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/mmu/tdp_mmu.c | 6 +-- 5 files changed, 50 insertions(+), 37 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 4ca01256143e..57692d873f76 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2896,10 +2896,8 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struc= t kvm_memory_slot *slot, trace_kvm_mmu_set_spte(level, gfn, sptep); } =20 - if (wrprot) { - if (write_fault) - ret =3D RET_PF_EMULATE; - } + if (wrprot && write_fault) + ret =3D RET_PF_WRITE_PROTECTED; =20 if (flush) kvm_flush_remote_tlbs_gfn(vcpu->kvm, gfn, level); @@ -4531,7 +4529,7 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, s= truct kvm_page_fault *fault return RET_PF_RETRY; =20 if (page_fault_handle_page_track(vcpu, fault)) - return RET_PF_EMULATE; + return RET_PF_WRITE_PROTECTED; =20 r =3D fast_page_fault(vcpu, fault); if (r !=3D RET_PF_INVALID) @@ -4624,7 +4622,7 @@ static int kvm_tdp_mmu_page_fault(struct kvm_vcpu *vc= pu, int r; =20 if (page_fault_handle_page_track(vcpu, fault)) - return RET_PF_EMULATE; + return RET_PF_WRITE_PROTECTED; =20 r =3D fast_page_fault(vcpu, fault); if (r !=3D RET_PF_INVALID) @@ -4703,6 +4701,7 @@ static int kvm_tdp_map_page(struct kvm_vcpu *vcpu, gp= a_t gpa, u64 error_code, switch (r) { case RET_PF_FIXED: case RET_PF_SPURIOUS: + case RET_PF_WRITE_PROTECTED: return 0; =20 case RET_PF_EMULATE: @@ -5954,6 +5953,40 @@ static bool is_write_to_guest_page_table(u64 error_c= ode) return (error_code & mask) =3D=3D mask; } =20 +static int kvm_mmu_write_protect_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or= _gpa, + u64 error_code, int *emulation_type) +{ + bool direct =3D vcpu->arch.mmu->root_role.direct; + + /* + * Before emulating the instruction, check if the error code + * was due to a RO violation while translating the guest page. + * This can occur when using nested virtualization with nested + * paging in both guests. If true, we simply unprotect the page + * and resume the guest. + */ + if (direct && is_write_to_guest_page_table(error_code)) { + kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(cr2_or_gpa)); + return RET_PF_RETRY; + } + + /* + * The gfn is write-protected, but if emulation fails we can still + * optimistically try to just unprotect the page and let the processor + * re-execute the instruction that caused the page fault. Do not allow + * retrying MMIO emulation, as it's not only pointless but could also + * cause us to enter an infinite loop because the processor will keep + * faulting on the non-existent MMIO address. Retrying an instruction + * from a nested guest is also pointless and dangerous as we are only + * explicitly shadowing L1's page tables, i.e. unprotecting something + * for L1 isn't going to magically fix whatever issue cause L2 to fail. + */ + if (!mmio_info_in_cache(vcpu, cr2_or_gpa, direct) && !is_guest_mode(vcpu)) + *emulation_type |=3D EMULTYPE_ALLOW_RETRY_PF; + + return RET_PF_EMULATE; +} + int noinline kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u= 64 error_code, void *insn, int insn_len) { @@ -5999,6 +6032,10 @@ int noinline kvm_mmu_page_fault(struct kvm_vcpu *vcp= u, gpa_t cr2_or_gpa, u64 err if (r < 0) return r; =20 + if (r =3D=3D RET_PF_WRITE_PROTECTED) + r =3D kvm_mmu_write_protect_fault(vcpu, cr2_or_gpa, error_code, + &emulation_type); + if (r =3D=3D RET_PF_FIXED) vcpu->stat.pf_fixed++; else if (r =3D=3D RET_PF_EMULATE) @@ -6009,32 +6046,6 @@ int noinline kvm_mmu_page_fault(struct kvm_vcpu *vcp= u, gpa_t cr2_or_gpa, u64 err if (r !=3D RET_PF_EMULATE) return 1; =20 - /* - * Before emulating the instruction, check if the error code - * was due to a RO violation while translating the guest page. - * This can occur when using nested virtualization with nested - * paging in both guests. If true, we simply unprotect the page - * and resume the guest. - */ - if (vcpu->arch.mmu->root_role.direct && - is_write_to_guest_page_table(error_code)) { - kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(cr2_or_gpa)); - return 1; - } - - /* - * vcpu->arch.mmu.page_fault returned RET_PF_EMULATE, but we can still - * optimistically try to just unprotect the page and let the processor - * re-execute the instruction that caused the page fault. Do not allow - * retrying MMIO emulation, as it's not only pointless but could also - * cause us to enter an infinite loop because the processor will keep - * faulting on the non-existent MMIO address. Retrying an instruction - * from a nested guest is also pointless and dangerous as we are only - * explicitly shadowing L1's page tables, i.e. unprotecting something - * for L1 isn't going to magically fix whatever issue cause L2 to fail. - */ - if (!mmio_info_in_cache(vcpu, cr2_or_gpa, direct) && !is_guest_mode(vcpu)) - emulation_type |=3D EMULTYPE_ALLOW_RETRY_PF; emulate: return x86_emulate_instruction(vcpu, cr2_or_gpa, emulation_type, insn, insn_len); diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_interna= l.h index 1721d97743e9..50d2624111f8 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -258,6 +258,8 @@ int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, struct kv= m_page_fault *fault); * RET_PF_CONTINUE: So far, so good, keep handling the page fault. * RET_PF_RETRY: let CPU fault again on the address. * RET_PF_EMULATE: mmio page fault, emulate the instruction directly. + * RET_PF_WRITE_PROTECTED: the gfn is write-protected, either unprotected = the + * gfn and retry, or emulate the instruction direc= tly. * RET_PF_INVALID: the spte is invalid, let the real page fault path updat= e it. * RET_PF_FIXED: The faulting entry has been fixed. * RET_PF_SPURIOUS: The faulting entry was already fixed, e.g. by another = vCPU. @@ -274,6 +276,7 @@ enum { RET_PF_CONTINUE =3D 0, RET_PF_RETRY, RET_PF_EMULATE, + RET_PF_WRITE_PROTECTED, RET_PF_INVALID, RET_PF_FIXED, RET_PF_SPURIOUS, diff --git a/arch/x86/kvm/mmu/mmutrace.h b/arch/x86/kvm/mmu/mmutrace.h index 195d98bc8de8..f35a830ce469 100644 --- a/arch/x86/kvm/mmu/mmutrace.h +++ b/arch/x86/kvm/mmu/mmutrace.h @@ -57,6 +57,7 @@ TRACE_DEFINE_ENUM(RET_PF_CONTINUE); TRACE_DEFINE_ENUM(RET_PF_RETRY); TRACE_DEFINE_ENUM(RET_PF_EMULATE); +TRACE_DEFINE_ENUM(RET_PF_WRITE_PROTECTED); TRACE_DEFINE_ENUM(RET_PF_INVALID); TRACE_DEFINE_ENUM(RET_PF_FIXED); TRACE_DEFINE_ENUM(RET_PF_SPURIOUS); diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 69941cebb3a8..a722a3c96af9 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -805,7 +805,7 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, str= uct kvm_page_fault *fault =20 if (page_fault_handle_page_track(vcpu, fault)) { shadow_page_table_clear_flood(vcpu, fault->addr); - return RET_PF_EMULATE; + return RET_PF_WRITE_PROTECTED; } =20 r =3D mmu_topup_memory_caches(vcpu, true); diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 3c55955bcaf8..3b996c1fdaab 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -1046,10 +1046,8 @@ static int tdp_mmu_map_handle_target_level(struct kv= m_vcpu *vcpu, * protected, emulation is needed. If the emulation was skipped, * the vCPU would have the same fault again. */ - if (wrprot) { - if (fault->write) - ret =3D RET_PF_EMULATE; - } + if (wrprot && fault->write) + ret =3D RET_PF_WRITE_PROTECTED; =20 /* If a MMIO SPTE is installed, the MMIO will need to be emulated. */ if (unlikely(is_mmio_spte(vcpu->kvm, new_spte))) { --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A340817545 for ; Sat, 31 Aug 2024 00:15:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063352; cv=none; b=o/ZpozqMWQdao6diZQJdOPGfjO33WkyVoK9c+mBQgIM3VICJjRgMjnkWXgwPsJpn05bCC1Ucz+rkEQpnNFvN1zOajhSVuqyAaDTJmXqemiXokZwFsF8LSkudhAZ78YdNrlv3C/PbYxoh1IxHwacq8JelUdW28TFKzzvSndRgo9o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063352; c=relaxed/simple; bh=Vfn4sap7F12KSn6gEqe3OjOMjrYNlixw+DuYUTSraC8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=uHzvLBhlJKXxJahlIf9rsUIbMFFSxeoqlqPX8gLRMa5UgTfm1+YgDVvAldEHMYpMfLEVV73KL4R6ozFrUHlZYmtKyiCQhDCgRX503f1hyy+C6ccJnBwoveTw0GAz//FY++hCET5SgGRcVs+oqHq6aDB9YUZrssLMwT+17PEguEw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=PNhw/taa; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="PNhw/taa" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6b3fec974e5so45078897b3.1 for ; Fri, 30 Aug 2024 17:15:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063348; x=1725668148; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=r3dfY8/fDEjK6mVegBOh3Hcv+cOkuxjVRlEEYnL1bxw=; b=PNhw/taaNcf5TffgHlDIn99eeOHOCbf7GefvwEfAWI+G3nSOnzSbFzIm7vyt70zKEW 8yPck86N6nOHfU7fFxvwlJCfc/bJb9TmvW+gO4JI2YrN5X3J8a6+p3AcBh/R8FV6TyjZ Ym3g1U5uAGZgRTtpoQsbNEqU5ZBqL4On0KxoTpYcLRtK5+uqwwOQ8i1LuyLA60y0RJ7K A8ro/5/XCz3YQ82AgVduK6JCGKDoCOoLTyeFZKr7nO7j16xXBxRbOfEdeJpK1IyjBMbN xVOvcv/xdi+8VbOS+OkvIKconJF+wtqhc+uOsMnXokdxXik6e+GaYgdwtpXFWDeJYtNV sSog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063348; x=1725668148; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=r3dfY8/fDEjK6mVegBOh3Hcv+cOkuxjVRlEEYnL1bxw=; b=s/nX69jJq1BZYiZnsR5enbfhRCSy2szKmkdlDxuDXvGClIk4MlPKfBcQGKSVCyOJad ePgctcu07PfrMMqavAXgYtsZ2FYzkPR3XBfzJggWEwGfEKnlDZMCEw9jCTXCSA/TZMJ+ yDhTrSlFH9PjJTsK7haoIG/VvZx2/2MLcoSVUvZQcTxGsbNc3hIHutqtcpntjok7pWkF 8EKJWjtUJz7+FBHy+hAVlRF7gs8lEk9/3+48feRiPR1BUjGJfIU4NgW9J9KJ8VS2nwC4 ADCXmAptyzSECgRZVvVKT1OhByc8xyu9J8C2XNbKt3vjXm4LtSjhG7myYxcY3JEb4aIL fl2A== X-Forwarded-Encrypted: i=1; AJvYcCWk70qvMksO44fVLrwZzskEHztipC9Ii37dO5H4vIY5/zabGoVLfltKjXkCBr3afO3LsQ97C+k+QIqMnqo=@vger.kernel.org X-Gm-Message-State: AOJu0YzXGvu7S42vmWPSM/REG5T+RERvBOEsf458ehPZnlNkOk9ivNYA QDyr1PNYITb0SRIdBhXSwKKUqd+y5f/uzvR65EwX0xQst3OGYhnUFBaDToxBCUhSRPtLd1JEtCh ctQ== X-Google-Smtp-Source: AGHT+IEX38k43fNniAUT9zZ10fu//L1yrfO75qcNhigMQXRQOpLzwXTrY/RTolEek0hddCoQ2pigQLcJwJ0= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:690c:2884:b0:6b2:6cd4:7f98 with SMTP id 00721157ae682-6d4102f7068mr572167b3.8.1725063348631; Fri, 30 Aug 2024 17:15:48 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:19 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-5-seanjc@google.com> Subject: [PATCH v2 04/22] KVM: x86/mmu: Skip emulation on page fault iff 1+ SPs were unprotected From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When doing "fast unprotection" of nested TDP page tables, skip emulation if and only if at least one gfn was unprotected, i.e. continue with emulation if simply resuming is likely to hit the same fault and risk putting the vCPU into an infinite loop. Note, it's entirely possible to get a false negative, e.g. if a different vCPU faults on the same gfn and unprotects the gfn first, but that's a relatively rare edge case, and emulating is still functionally ok, i.e. saving a few cycles by avoiding emulation isn't worth the risk of putting the vCPU into an infinite loop. Opportunistically rewrite the relevant comment to document in gory detail exactly what scenario the "fast unprotect" logic is handling. Fixes: 147277540bbc ("kvm: svm: Add support for additional SVM NPF error co= des") Cc: Yuan Yao Signed-off-by: Sean Christopherson Reviewed-by: Yuan Yao --- arch/x86/kvm/mmu/mmu.c | 37 +++++++++++++++++++++++++++++-------- 1 file changed, 29 insertions(+), 8 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 57692d873f76..6b5f80f38a95 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -5959,16 +5959,37 @@ static int kvm_mmu_write_protect_fault(struct kvm_v= cpu *vcpu, gpa_t cr2_or_gpa, bool direct =3D vcpu->arch.mmu->root_role.direct; =20 /* - * Before emulating the instruction, check if the error code - * was due to a RO violation while translating the guest page. - * This can occur when using nested virtualization with nested - * paging in both guests. If true, we simply unprotect the page - * and resume the guest. + * Before emulating the instruction, check to see if the access was due + * to a read-only violation while the CPU was walking non-nested NPT + * page tables, i.e. for a direct MMU, for _guest_ page tables in L1. + * If L1 is sharing (a subset of) its page tables with L2, e.g. by + * having nCR3 share lower level page tables with hCR3, then when KVM + * (L0) write-protects the nested NPTs, i.e. npt12 entries, KVM is also + * unknowingly write-protecting L1's guest page tables, which KVM isn't + * shadowing. + * + * Because the CPU (by default) walks NPT page tables using a write + * access (to ensure the CPU can do A/D updates), page walks in L1 can + * trigger write faults for the above case even when L1 isn't modifying + * PTEs. As a result, KVM will unnecessarily emulate (or at least, try + * to emulate) an excessive number of L1 instructions; because L1's MMU + * isn't shadowed by KVM, there is no need to write-protect L1's gPTEs + * and thus no need to emulate in order to guarantee forward progress. + * + * Try to unprotect the gfn, i.e. zap any shadow pages, so that L1 can + * proceed without triggering emulation. If one or more shadow pages + * was zapped, skip emulation and resume L1 to let it natively execute + * the instruction. If no shadow pages were zapped, then the write- + * fault is due to something else entirely, i.e. KVM needs to emulate, + * as resuming the guest will put it into an infinite loop. + * + * Note, this code also applies to Intel CPUs, even though it is *very* + * unlikely that an L1 will share its page tables (IA32/PAE/paging64 + * format) with L2's page tables (EPT format). */ - if (direct && is_write_to_guest_page_table(error_code)) { - kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(cr2_or_gpa)); + if (direct && is_write_to_guest_page_table(error_code) && + kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(cr2_or_gpa))) return RET_PF_RETRY; - } =20 /* * The gfn is write-protected, but if emulation fails we can still --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6178D442F for ; Sat, 31 Aug 2024 00:15:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063352; cv=none; b=ZO6hQgtCaeFa9WVK6xy2KGEXaZrOIYMH91JdlP3+lRrJTPm8RWYmwk4OouDQ8AfDjtUWb9K5NZkI4H1VYHDkXTeUE8dPZu5yd5Gczde8d4UL+jmSmGgQ3/IjjvrPyJ8rD5jd9xYo2h2hDvbOi/ZZfYYaj2aujC0xPdQodr9UD4Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063352; c=relaxed/simple; bh=aTVAVwczKEP1sfRbeojsquqSLoy7yQdZzgoVJkxL3Hw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=qj5x0EH6uFRiIpJmfUFRU1yxcOW2WCmwc9+/ep8Ew23GRt5oaoYo2FhDcHRuKx3zcZoSsIVmIPBa6PTwRi/8VHYgHJH9KQFiTPVBRZcWvY+DxQBBhqZqxLv6TNZgYxJjtbOKZ9vxOXMK/PDv8LsHlqp/E76tkt8B5jShKOMyUFM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=u4V+jB3D; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="u4V+jB3D" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6d2c2cf7473so46338657b3.0 for ; Fri, 30 Aug 2024 17:15:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063350; x=1725668150; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=7EhMqLEmX3n6K2WXzV5Gm2EiCppXNLHfvg1qXCILft0=; b=u4V+jB3D9zcTcjsW2UOL9Mi5Ej3Pv9n02ZNBcq5tIbc4wZIy/Y3/sJ+/ZSwQj8k0I5 EMLdI+q2PRAZ2fX/OxAviCUbmL71FPgg24GEDkZUNcrLuOiSrLI4OREv9y8Js45pSDQm fsxr5Cp90dQ+pZ2FR+7PBnhKW8ZPQ17COOPVVzO1Kx+zk3Ou/5v2AFX6pvJhdhCCG/4R z+pdHAEsrOCsYrfjcMaQCvG6zrajXbciEPIic0cEVn8Hqekssum+l2S6/P1Z8d2RNnKM Gq/8biLDca4ndrxJKYO8nT5GT/tolSAlia0WvAvpfIa/h4XLH96zBsHctnT/f1EvHItN 4PTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063350; x=1725668150; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=7EhMqLEmX3n6K2WXzV5Gm2EiCppXNLHfvg1qXCILft0=; b=ZCehhMppSPo9K3OY7Zy4aQHyei0a71ni3qpgz3Aolb3lGh+RyMOSQQX9nLK8IbcBre VtPjmTnLAOdzi0aPRkH9WaLtnSPU5M6bYVRNQW5/16dfwwlnkmmWtYGzBA54szNvF0Fz 7kdTUG8OMKKy4T2l3PbMtNziWfZpAVgC33cKqTVTSzHaATVnR128gDHJqjpFBWj4xKud Zi5o3SBLobkn8tMahQYsHXu0JfIR3NFeYC/lx4U6NmUYGEHA5daUOCVadVXLckP8ZhXG Kr+pjllMM2fy9vPDnEg1onL3K/78Ft1mUyAYupSMzbg7E18T/80UpRbZcykJe2dvXt0o JmvA== X-Forwarded-Encrypted: i=1; AJvYcCWhVvlnaBt+Wn7mqTGhIBnoPCl+cKGFhLCAFIQ78+zGUpPeaLSkS9rvaBlBVuGRsSOwMPJFCL3E6+KY58o=@vger.kernel.org X-Gm-Message-State: AOJu0YyvJmJdwWON7DCQ5BoMVmB27OzOJoqYU2eHKoKWVzYhPhnp/o4U Nos+NjUbAoiFWS1WHvR0HqtAylxTUwlf7Ylp7UTTrjP8MwFUSou0M4vOuq9WcTl+tDeQXYfO72N U8Q== X-Google-Smtp-Source: AGHT+IH772L2wnK+bJh/Re03anXSfmUsTgTWEqSK2F8xhRla6k8xN8TqIB6/NPP9fx/gq1A3FL1yJgAK9+o= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:690c:4490:b0:650:a16c:91ac with SMTP id 00721157ae682-6d411290d2emr966657b3.8.1725063350558; Fri, 30 Aug 2024 17:15:50 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:20 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-6-seanjc@google.com> Subject: [PATCH v2 05/22] KVM: x86: Retry to-be-emulated insn in "slow" unprotect path iff sp is zapped From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Resume the guest and thus skip emulation of a non-PTE-writing instruction if and only if unprotecting the gfn actually zapped at least one shadow page. If the gfn is write-protected for some reason other than shadow paging, attempting to unprotect the gfn will effectively fail, and thus retrying the instruction is all but guaranteed to be pointless. This bug has existed for a long time, but was effectively fudged around by the retry RIP+address anti-loop detection. Signed-off-by: Sean Christopherson Reviewed-by: Yuan Yao --- arch/x86/kvm/x86.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 966fb301d44b..c4cb6c6d605b 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8961,14 +8961,14 @@ static bool retry_instruction(struct x86_emulate_ct= xt *ctxt, if (ctxt->eip =3D=3D last_retry_eip && last_retry_addr =3D=3D cr2_or_gpa) return false; =20 + if (!vcpu->arch.mmu->root_role.direct) + gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, cr2_or_gpa, NULL); + + if (!kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa))) + return false; + vcpu->arch.last_retry_eip =3D ctxt->eip; vcpu->arch.last_retry_addr =3D cr2_or_gpa; - - if (!vcpu->arch.mmu->root_role.direct) - gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, cr2_or_gpa, NULL); - - kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa)); - return true; } =20 --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F7D31CAA4 for ; Sat, 31 Aug 2024 00:15:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063355; cv=none; b=Xu5gp7y/Zy0IUZ1rBS00vbrZaSXhhxvbo+XIUNDGFq9n3A9Qe6O9r56qUiOAd0rN0vQrB2K8m5PBNYeJP2oV9FyI13IYamsoqthmz/eL0GHmu5iBf02np89/6QgLZYg10P99FkNzQCQUiO1rkGEDwiQVbWq/9FUazVpozyNfl9g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063355; c=relaxed/simple; bh=qX4frsSxKTQ6bojkC/e3dE7oNpANxrfnXEYYdQProQY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=BvZz+/CzQVDNnMoezz5DwE6P4+oYoA5nZ9BVNDlrlUVP2tyicoaqHRN2ntuSh37uCrSMJu6UfFKfQ33FeCx6qskpowlBztBa0XDOcbJ97TO0q5mLO2vQMfZ7ujEks7yK6dMsr4NvSpTYHMmVIcLn2J2tobkAVNO9ZGD73G8BdPM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=if74Vmvq; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="if74Vmvq" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-715a16b4206so2612709b3a.0 for ; Fri, 30 Aug 2024 17:15:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063353; x=1725668153; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=24VCpIEq4+Q+YXItNAgkyUyFi45YkZe3g1wKHaUykVY=; b=if74VmvqYni3eqNvaUmGHTJjg4PCvPPWMu3qF84E0YRUB6IwZ85rCzgNA5cSk2+1uN iqL2y7Vo40yuzfVazV2EJSZV3hFZrjUp5GilEw+am5A38qMmLh7WgnovBW1Htf1rwWd/ GzXegY51lL52fkRRvXif14JmIshL/LxgcFhU4DD+OuVXcLAnu0suFjbe4ohkcMbtap6n T2+Ywv20Wn7dwS+xz4adxjbn7lK0Emm1G9L1ZHBSh6L0TbsJOnOhknl3k0ow/d8lbc4X 0qU+V/bl4LVPGwnsjiSl1YUsUxnyJPsP+FU7dr8a9IfsN+piuzYhQzCw0uQk8fQ9gzxq ppdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063353; x=1725668153; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=24VCpIEq4+Q+YXItNAgkyUyFi45YkZe3g1wKHaUykVY=; b=NFA/dPAxbQHTv1t/d3dNw4QhjWwkQuJIeijcYGm0h8fKK/GW/XFecDFlYyHDEBMIpw tSREWBEYlK2bNOnyQdYGnrC8vVLCL8yVVrv7jKjff8dk7AyQOXJbbdOVNTLmQIEH8o1x DhskZOGMF9xyP8YaIRxrpiUwdSLlIwRXIJXktP1P4PmNBUl5pAdujJVVwNsLwMAlD4Qw fvJmbN5pYNDVv7vIqy61aZUUe8d7qjgwbS2nl94ZoR/rMibg6LufGW/odb1or5t9E8oL Veq76K+VrJo6leEyJrxSLLWqm3v6TtBnCv6w01n+Ig24Ylc2V1hvqPkgAix/VbicW2tZ Pt+Q== X-Forwarded-Encrypted: i=1; AJvYcCUIpMzmHzQ7Byiq9M9l0uVcqOQ23Yrc4T+bcMLUfniDUiErXzzj6YujGp2Idivk5MH5wMgaR2K7UawymIs=@vger.kernel.org X-Gm-Message-State: AOJu0YzDLcfmFrdX/NJYi6iDeW0bB0qYJz+SVBU2I/AZobhJs6tYGt7n +L7/+Oh3sJIxwenWFTiAE0Udb7W9eWPJVjEmoEB1VXdPR7z1mH3+uPsdH3Rb60HBMf8szb7NfcJ zcg== X-Google-Smtp-Source: AGHT+IEmNWc2IOu0f8CgmpJdBNIqSgvzCJlTfxhTsRLfVh/+amYRW9Ejbsh1qT3edScYRLUIfkgLKFvBqKo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6a00:91d7:b0:714:200c:39a2 with SMTP id d2e1a72fcca58-717307a3102mr7424b3a.6.1725063352881; Fri, 30 Aug 2024 17:15:52 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:21 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-7-seanjc@google.com> Subject: [PATCH v2 06/22] KVM: x86: Get RIP from vCPU state when storing it to last_retry_eip From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Read RIP from vCPU state instead of pulling it from the emulation context when filling last_retry_eip, which is part of the anti-infinite-loop protection used when unprotecting and retrying instructions that hit a write-protected gfn. This will allow reusing the anti-infinite-loop protection in flows that never make it into the emulator. No functional change intended, as ctxt->eip is set to kvm_rip_read() in init_emulate_ctxt(), and EMULTYPE_PF emulation is mutually exclusive with EMULTYPE_NO_DECODE and EMULTYPE_SKIP, i.e. always goes through x86_decode_emulated_instruction() and hasn't advanced ctxt->eip (yet). Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c4cb6c6d605b..a1f0f4dede55 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8967,7 +8967,7 @@ static bool retry_instruction(struct x86_emulate_ctxt= *ctxt, if (!kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa))) return false; =20 - vcpu->arch.last_retry_eip =3D ctxt->eip; + vcpu->arch.last_retry_eip =3D kvm_rip_read(vcpu); vcpu->arch.last_retry_addr =3D cr2_or_gpa; return true; } --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C5249208A7 for ; Sat, 31 Aug 2024 00:15:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063357; cv=none; b=mJzl8KrnX7vyaE2ESltIGyAR++vqrGa/D+FtW3XiCW8JcQDqWWIiv39l8xCA+FuqtMg+kInj/JW2yAdsfmjBXX4wIbuzj8rDcU+E6LQZZbOHKX1fmh0FRHVMEA6oKnpGmDUA/bSUej3cMGKwy94eNxShGwL2eKpPsaUH5y7e9iM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063357; c=relaxed/simple; bh=pD5WjQYntO47PlRaRqi7VIbbwgINoA1bCT/vA6bz3VQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=dPDd85S6W1GthyJa20Yc+gFr7x4GN/O5YDdcwfngbehjd8+uzFrjls123WinXQjRQLlznHlpAhBKcynotghLwA4nCNUju32GEyvp1TYZzhQHaHx5CjZmdMcU57sRuq2eS+AEzH16ilugDQL7hMmXKvDKjBH3VXuW4/ech2lF13w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=zg4wGD5x; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="zg4wGD5x" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2d8759255d4so1044586a91.3 for ; Fri, 30 Aug 2024 17:15:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063355; x=1725668155; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=+Qft+nZl+LjcqZqAniryaxEtRXUtCnTAy59Lv2DhjhA=; b=zg4wGD5xuUA3h+LdDwGof/mJtuSJEw13b9S35EQNGcCDy8uXbsfdzz4NcLX16XRdcS aoc8sWi/MglBjyiWUtm2NXpYc386Y1A6Ynte9I0ogyB1lm+GP26C0yaFfMNf8Hc29ViX PapNWp6AeSJBKPaQMJsKG62e+/Kn3t21vFvsxtjfRnLNIfe9G8CzIpcr9/gsAa8nSb13 AS2W6XQsZoHmr6JxbG9jHP7OdvlTjkVEi/EED+PsETeba9dRDh8NczH+Ns0nUtEY43wZ 7B6pgOOzInf3B7IVhJfzVXw6xSJ7viXSE/foXpenZsKMH3K6uzHXRaO/P5DScGWnJgqN cZhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063355; x=1725668155; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=+Qft+nZl+LjcqZqAniryaxEtRXUtCnTAy59Lv2DhjhA=; b=cDYBvbqc/5VruJ3X0yFHcaBF7IdZy3QS6ode8fI/Xa3+/miWriHucQ30vPngdk9ECn PpOxtXg28EBZFytsxah82WHYVGqnMgRg3OB/fa7HgswGYSjsQhdPN5aquFuCSd3rpLnK dZFowyAYedHzYMhKg7xHkgqEsUl7FPcl1dZi5yoGbbpS3Ckrsl0yuv3c6UBTxL7ytFcY 0FEW1wVeeXzD3tlOS9uXKjk0c2HFHKummN7PcBV+aTQz4Z2T1/dij1aYlThX0qR9cjnB hG90kwU1gGt/lO3a+5BACmUk8QtKBPUIVrDODKYtnfyYvfwDXUzZlAtCSMwBBRcPZoRA Glgg== X-Forwarded-Encrypted: i=1; AJvYcCXBCFQCXBCaqYOfIy98B8UjHSCNEvbHeC4R3ZuG14TVVVVNLcCqdmG2UCt1wcjBoz82LptWohKbsBT+pzQ=@vger.kernel.org X-Gm-Message-State: AOJu0YxmqXUn4ZT9L5AjW3Ea0UwOODFRkAn+Uv/fxvQbTMUBSDnr8ojr Yfduc5e6jlid86PZWR8Pf1WqJ+hApCbY5AgYxDa7yq3PLdi1DpGJp+BF+AzT/JGlCX7ABXT4O2Q IDQ== X-Google-Smtp-Source: AGHT+IGTQJIEJrg9n5PvoSMJ5tiETEAqsh2sRrvNvPq6jrtr32LGIiuli1Er3RpuQYOhVlsrghuEnFeBcIY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:90a:3f86:b0:2d8:8abe:cb03 with SMTP id 98e67ed59e1d1-2d88abed74emr3402a91.6.1725063354736; Fri, 30 Aug 2024 17:15:54 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:22 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-8-seanjc@google.com> Subject: [PATCH v2 07/22] KVM: x86: Store gpa as gpa_t, not unsigned long, when unprotecting for retry From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Store the gpa used to unprotect the faulting gfn for retry as a gpa_t, not an unsigned long. This fixes a bug where 32-bit KVM would unprotect and retry the wrong gfn if the gpa had bits 63:32!=3D0. In practice, this bug is functionally benign, as unprotecting the wrong gfn is purely a performance issue (thanks to the anti-infinite-loop logic). And of course, almost no one runs 32-bit KVM these days. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index a1f0f4dede55..c84f57e1a888 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8928,7 +8928,8 @@ static bool retry_instruction(struct x86_emulate_ctxt= *ctxt, gpa_t cr2_or_gpa, int emulation_type) { struct kvm_vcpu *vcpu =3D emul_to_vcpu(ctxt); - unsigned long last_retry_eip, last_retry_addr, gpa =3D cr2_or_gpa; + unsigned long last_retry_eip, last_retry_addr; + gpa_t gpa =3D cr2_or_gpa; =20 last_retry_eip =3D vcpu->arch.last_retry_eip; last_retry_addr =3D vcpu->arch.last_retry_addr; --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C144225779 for ; Sat, 31 Aug 2024 00:15:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063360; cv=none; b=R/o6Fvrr3NENOvXUskfnYy7X0yBGitbc8t384IZER+4/0J1KZ28fHF7FhA8ZWT331Bxr2Dc2JYWwjOCmiT5SV1NjWUt2mjXm0TeCypVewTxt1M7bbUHv5E3Apsxi+1ngjR9nwK9ewn+KrmNJVUe9SVNm0GgBp9CaZj8R+cHDQ1c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063360; c=relaxed/simple; bh=okOHlfjWRjUaRWCJVIfP0FRC3jaPnRUhU2Ljs35GXBk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Hv8RxCj3y6mzxyEPw3Hzj77Rwu2hGaAJSkHlOS+lIJhVzD7z76AhkBimKmI7kVot0TJ8vAhdU5dTF4VbTEUeXeQnvl9wnoF3oP/LJ3cUudvrFG9knXq79KL+25QT2Whk9y5eZK8WxgFTxFDrM0rs9THniqz0Mb8iTxhz6A0O0qM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=2+Irigdr; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="2+Irigdr" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e13c4519ed6so4650197276.0 for ; Fri, 30 Aug 2024 17:15:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063357; x=1725668157; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=20uh8Karf7HaNxExXLYXZVv2s3Odzh3UmkqbAlerb3Y=; b=2+Irigdr4bQIlzbvJeSbXDfosSil53gPtkW8kI4SodSmmNdyfHLwsU9F0nJNdZCafS VoEThpBorLnNYrx/JOkNpUlQssXapauDaudrOHW7T65sEaAwbKRJhLsw3oEHDp+F0g5N QnVxXbUf9yBGqhpEqWNNIHSZQK+P7GiRFCcuERb4vJjzZmgtwpG42eurHaUTZH15vyps KIKCEUgpMHrIMrJnYJ2v0RUwPHq5WIAnK2K7QA91CPSW+QNAu05mHVqlZbimWCh0ndky BUAQqEngDj0sS83iD5ycFmnIgCSvuybpR1/KjtzQ/wrYyRkA3YWibrV7LqIe4ITePh8f uOXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063357; x=1725668157; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=20uh8Karf7HaNxExXLYXZVv2s3Odzh3UmkqbAlerb3Y=; b=gPaq7HT7MYXTyzvh1PAd0SFhIUDkUepv4QqPrS91+vjhboXTSK2PUHQdeG3cMDOAW2 LyMeXOrNAYLDNQ/JRRLkKLWZ4H557SCWy94mA/WlKuvbPoTpNOklWrFBHaRlx4LyZuSm Xzf3twh23JBxlfY8b0xyJk21ML6Bm/ly5YKStM7USVneAC9eymHvW3HlzdB0dJjQV6ag Uezr4DLDyBPwg2SDhOFKQgsbn1aye5FydJw67Jj+rfvnHyFFqX0BwhoF44X49sEHhS00 CxIU3BJY/mCXM50BeSQyQeSskAN/xKzFoxgovA9b0/as75zePkWXTdCo5kw002LnMKU2 YG1w== X-Forwarded-Encrypted: i=1; AJvYcCVCWNJoR/yc1QfC3WCBxA2HXxb2IfMV65pMWB8XAvQ+S8sZgYoHROoh38DJro9V7DuXu/FUwNwJ+i5G/qg=@vger.kernel.org X-Gm-Message-State: AOJu0YwRHi1Xd4ELZaXDRUyyqwzY10nMwTfwYriSstoUJYLGQiDsIsg7 //taLx21A89o8tHFM9W/4+shim6j5Axfmp4QlXRQwPnygkzc/ZpTT/impeDB6igFF/1eb2PTRHR ZMA== X-Google-Smtp-Source: AGHT+IEtVf5oWPZeuNHrdt0XXTW0hZJaLnTQjVDPanMWRsjlMz9p2OLeYpGnhxaOIzgFrBVl4hdpYEQgBiU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:8d02:0:b0:e1a:7eff:f66b with SMTP id 3f1490d57ef6-e1a7efff840mr25477276.5.1725063356829; Fri, 30 Aug 2024 17:15:56 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:23 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-9-seanjc@google.com> Subject: [PATCH v2 08/22] KVM: x86/mmu: Apply retry protection to "fast nTDP unprotect" path From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move the anti-infinite-loop protection provided by last_retry_{eip,addr} into kvm_mmu_write_protect_fault() so that it guards unprotect+retry that never hits the emulator, as well as reexecute_instruction(), which is the last ditch "might as well try it" logic that kicks in when emulation fails on an instruction that faulted on a write-protected gfn. Add a new helper, kvm_mmu_unprotect_gfn_and_retry(), to set the retry fields and deduplicate other code (with more to come). Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/mmu/mmu.c | 39 ++++++++++++++++++++++++++++++++- arch/x86/kvm/x86.c | 27 +---------------------- 3 files changed, 40 insertions(+), 27 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 62d19403d63c..2c3f28331118 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -2135,6 +2135,7 @@ int kvm_get_nr_pending_nmis(struct kvm_vcpu *vcpu); void kvm_update_dr7(struct kvm_vcpu *vcpu); =20 int kvm_mmu_unprotect_page(struct kvm *kvm, gfn_t gfn); +bool kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu, gpa_t cr2_or_g= pa); void kvm_mmu_free_roots(struct kvm *kvm, struct kvm_mmu *mmu, ulong roots_to_free); void kvm_mmu_free_guest_mode_roots(struct kvm *kvm, struct kvm_mmu *mmu); diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 6b5f80f38a95..c34c8bbd61c8 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2713,6 +2713,22 @@ int kvm_mmu_unprotect_page(struct kvm *kvm, gfn_t gf= n) return r; } =20 +bool kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu, gpa_t cr2_or_g= pa) +{ + gpa_t gpa =3D cr2_or_gpa; + bool r; + + if (!vcpu->arch.mmu->root_role.direct) + gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, cr2_or_gpa, NULL); + + r =3D kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa)); + if (r) { + vcpu->arch.last_retry_eip =3D kvm_rip_read(vcpu); + vcpu->arch.last_retry_addr =3D cr2_or_gpa; + } + return r; +} + static int kvm_mmu_unprotect_page_virt(struct kvm_vcpu *vcpu, gva_t gva) { gpa_t gpa; @@ -5958,6 +5974,27 @@ static int kvm_mmu_write_protect_fault(struct kvm_vc= pu *vcpu, gpa_t cr2_or_gpa, { bool direct =3D vcpu->arch.mmu->root_role.direct; =20 + /* + * Do not try to unprotect and retry if the vCPU re-faulted on the same + * RIP with the same address that was previously unprotected, as doing + * so will likely put the vCPU into an infinite. E.g. if the vCPU uses + * a non-page-table modifying instruction on the PDE that points to the + * instruction, then unprotecting the gfn will unmap the instruction's + * code, i.e. make it impossible for the instruction to ever complete. + */ + if (vcpu->arch.last_retry_eip =3D=3D kvm_rip_read(vcpu) && + vcpu->arch.last_retry_addr =3D=3D cr2_or_gpa) + return RET_PF_EMULATE; + + /* + * Reset the unprotect+retry values that guard against infinite loops. + * The values will be refreshed if KVM explicitly unprotects a gfn and + * retries, in all other cases it's safe to retry in the future even if + * the next page fault happens on the same RIP+address. + */ + vcpu->arch.last_retry_eip =3D 0; + vcpu->arch.last_retry_addr =3D 0; + /* * Before emulating the instruction, check to see if the access was due * to a read-only violation while the CPU was walking non-nested NPT @@ -5988,7 +6025,7 @@ static int kvm_mmu_write_protect_fault(struct kvm_vcp= u *vcpu, gpa_t cr2_or_gpa, * format) with L2's page tables (EPT format). */ if (direct && is_write_to_guest_page_table(error_code) && - kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(cr2_or_gpa))) + kvm_mmu_unprotect_gfn_and_retry(vcpu, cr2_or_gpa)) return RET_PF_RETRY; =20 /* diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c84f57e1a888..862eed96cfd5 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8928,27 +8928,13 @@ static bool retry_instruction(struct x86_emulate_ct= xt *ctxt, gpa_t cr2_or_gpa, int emulation_type) { struct kvm_vcpu *vcpu =3D emul_to_vcpu(ctxt); - unsigned long last_retry_eip, last_retry_addr; - gpa_t gpa =3D cr2_or_gpa; - - last_retry_eip =3D vcpu->arch.last_retry_eip; - last_retry_addr =3D vcpu->arch.last_retry_addr; =20 /* * If the emulation is caused by #PF and it is non-page_table * writing instruction, it means the VM-EXIT is caused by shadow * page protected, we can zap the shadow page and retry this * instruction directly. - * - * Note: if the guest uses a non-page-table modifying instruction - * on the PDE that points to the instruction, then we will unmap - * the instruction and go to an infinite loop. So, we cache the - * last retried eip and the last fault address, if we meet the eip - * and the address again, we can break out of the potential infinite - * loop. */ - vcpu->arch.last_retry_eip =3D vcpu->arch.last_retry_addr =3D 0; - if (!(emulation_type & EMULTYPE_ALLOW_RETRY_PF)) return false; =20 @@ -8959,18 +8945,7 @@ static bool retry_instruction(struct x86_emulate_ctx= t *ctxt, if (x86_page_table_writing_insn(ctxt)) return false; =20 - if (ctxt->eip =3D=3D last_retry_eip && last_retry_addr =3D=3D cr2_or_gpa) - return false; - - if (!vcpu->arch.mmu->root_role.direct) - gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, cr2_or_gpa, NULL); - - if (!kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa))) - return false; - - vcpu->arch.last_retry_eip =3D kvm_rip_read(vcpu); - vcpu->arch.last_retry_addr =3D cr2_or_gpa; - return true; + return kvm_mmu_unprotect_gfn_and_retry(vcpu, cr2_or_gpa); } =20 static int complete_emulated_mmio(struct kvm_vcpu *vcpu); --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C6A052263A for ; Sat, 31 Aug 2024 00:15:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063361; cv=none; b=UelUK6t46mykYDs1rPJP2sdQ2PMLC4HiFe2TJBzojJv2IcjH3bsc0QWBteDE2Wet19k8Env4HiMWLHW+EMnLjtiup7VMayO9iSEkNRkIApMdVbF/cq1luZGdyi9EH9jESxv/RWFKst4JePrUtbadV2vtZ9uHJf/0jAEZxlle0lM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063361; c=relaxed/simple; bh=DjdRqrZVYaDjDoNjboOK73Yx3wVuE0JYZU3mSEZ1MxI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=aSVLyY7Y6X7ivhH/PIAhWo6cdSCsiTB6vlnn1pL/APOhYD48yCjmfsZ8C0phBQtgpTKZn0Bn2g+gfVNICTaMS7hBpianJl75R13e4FmyNmnvvB7xtjXVdelK1pYGhzePHXpfN7spyBh+GUXma1vMSrezoyQc0tCGwYAr4NHx7tI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=TjxQ0x6m; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="TjxQ0x6m" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6d52d811d1aso3586927b3.1 for ; Fri, 30 Aug 2024 17:15:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063359; x=1725668159; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=7IoutToPYHP78p+giZTIYSlAIHUDU+a9Cu0+YJ5R9Xs=; b=TjxQ0x6mO0qhLDz7PUmPBXJZFI6tjXSmkWi5TFrqGLmpDtFk7g7523cJ7jdnCbwpg9 BXFOJ2kc5WSYChLT4x0HqbWXQLsxoeP8OJkdNCb/MzFGLpMoE05Rz5mfxt/nhFJ4bptc v/LJO2riG041V68hCCwRT/15kMP5h+ex1lmJrMQRMUmnfrqi5TWL2aYP56ynv2R4qmUm 80RXuJd1u7AQExiumlw8SUzhcNksy0+/FYF/PYL2tQQZ2C7CvwN3MU9kt44IVMne8SkQ d07/vREGYuaiOVll+dAonMdaVgGCW6IrdMHZHdGCJZi0BYdKTM7BI7LgURCKgr5PvwvX 2f2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063359; x=1725668159; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=7IoutToPYHP78p+giZTIYSlAIHUDU+a9Cu0+YJ5R9Xs=; b=fFiegfaDczmS0pRqZv3ZmMymF/JVH6nf1qGRUJJNFc2MpGt6ALL+Lat1RiBdozKA6w B5y5971CmQtsMDfAbQr8Kri4ACBG850ET3EwR8CUuzbCLP907HTvy1NpNYxVbxhWUl67 w3Z5hDZnxL7ZdHZwyh0qCvulZ1/zDkVasWRpr5wgzHT5fmhV/TyP1ZiVZRSBP+I4Pu8Y e3du+eik19C6X5QAgesXMj/hMqybk58/YYnrdthbGtf7wK6zVZoiTh+EwAOSRnuSj5jY EebGP14MbaBs8g2Ic7Rxpvriao70Bt1VkiUHgckuTID/4OY+0qeojATmiXqZ034K8TKY bYEg== X-Forwarded-Encrypted: i=1; AJvYcCXEAegD8z97+emtsTkTe35oEHiqQn6JawAjuxRZ1M/n60JlkR0ltP3fQEpsoaT/pG3GSek5rW7usNR3Xos=@vger.kernel.org X-Gm-Message-State: AOJu0Yx+yUpDdOOdNdspOBgbL2MqHH2dYXwKbiKNDGaLqpoC+Pdn4WD0 VTg6+2hdzWIpPcZldj3y4LIXx/r5XZ5IcuhelfQwW1Jy00GBqnz10aOgN2SOsiSzqLU8O5zzgLL bxg== X-Google-Smtp-Source: AGHT+IHuUMDVgauY7GuvKPVxwwQlBkVToxZb6WAcgBa0Ma8vA/kxkgf3DqQCO/+ZLc7AXyR0g3JjVsDh5oM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6902:1367:b0:e0b:b36e:d73d with SMTP id 3f1490d57ef6-e1a5c8765b0mr58850276.4.1725063358899; Fri, 30 Aug 2024 17:15:58 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:24 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-10-seanjc@google.com> Subject: [PATCH v2 09/22] KVM: x86/mmu: Try "unprotect for retry" iff there are indirect SPs From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Try to unprotect shadow pages if and only if indirect_shadow_pages is non- zero, i.e. iff there is at least one protected such shadow page. Pre- checking indirect_shadow_pages avoids taking mmu_lock for write when the gfn is write-protected by a third party, i.e. not for KVM shadow paging, and in the *extremely* unlikely case that a different task has already unprotected the last shadow page. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index c34c8bbd61c8..dd62bd1e7657 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2718,6 +2718,17 @@ bool kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu= *vcpu, gpa_t cr2_or_gpa) gpa_t gpa =3D cr2_or_gpa; bool r; =20 + /* + * Bail early if there aren't any write-protected shadow pages to avoid + * unnecessarily taking mmu_lock lock, e.g. if the gfn is write-tracked + * by a third party. Reading indirect_shadow_pages without holding + * mmu_lock is safe, as this is purely an optimization, i.e. a false + * positive is benign, and a false negative will simply result in KVM + * skipping the unprotect+retry path, which is also an optimization. + */ + if (!READ_ONCE(vcpu->kvm->arch.indirect_shadow_pages)) + return false; + if (!vcpu->arch.mmu->root_role.direct) gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, cr2_or_gpa, NULL); =20 --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CFCBB2AE8C for ; Sat, 31 Aug 2024 00:16:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063363; cv=none; b=qC6I77gHKiO0j75KLdCG2C6ST93QssqXNkIJixdVQo0Zgz4kxqff9zlM0MS5Wc3EGpCab/g/iaZ7cWoM+z+UntR+fKwE+BJEr6llj82Vf08jlENoANqhVDZD2aXyGmSy8Na0ut91fWeEdUaBPPdQ1zoSSG4fRwpMS86jPmKI0+s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063363; c=relaxed/simple; bh=DCTzB3tz2/E8t0BLmCC1a+S2lgNSLEDLxXAB74yg2pc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Rqt510mIEno4jUGDxqQboGpBt2jttIK2NAvJ1xtHnNy6VuwlHmnySQ3lmMVkZVYMzm+1O8UBrx/tdsnt6Ahv5BRyH/qpQ3CMeiKc8JBy/1wi283s8eCG24gxv+1mT49W9ipX0ANRPd63Hy0jzWcNT3AKEqe4cTfeIpYezDMF8ME= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=jOjvqHEq; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="jOjvqHEq" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6d4f52a4069so10223497b3.3 for ; Fri, 30 Aug 2024 17:16:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063361; x=1725668161; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=If9fhF0Hkv4z0IGK7mMWOkmyhzNHFQEYRDvTkV0MKq0=; b=jOjvqHEq+XKUKiN1eGl8lkJ3pYAE5i/2mfM8UXFFxvr6LmknJDVZeYqb18NcyD0y1Z GOL32mRi8Kkin+X7bh5DnjtKL2gmOnuMV77DktMYX9pnHVEk21QrZ73+5WFeNQueK9ks B/O93/GVYPqqH1XTxY3RlHjYr/JA0VwBD5mFqOVcrCeK4nn1vXF8C5ekusA6ScMBWZ5r Ly16/M5lRvb7aaJ9R7ymrBLB7hnIC9Xus0WNoL49B71qgjGSrNXbbKTVJLeYAFeOfH1d H1AlZX0j9HSJDz3hdRQSV9bzAr1DERaKdQXMuhDxYtW8gTZdAKqeITYEOnHspcsUBpoJ ZgQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063361; x=1725668161; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=If9fhF0Hkv4z0IGK7mMWOkmyhzNHFQEYRDvTkV0MKq0=; b=JkfnqnV0EM3dR0dUJ8rz7vl+pf2GoW/VwAegOEOfJiiF3zFR1WGHA8YVV95d8e/L7r mjKT12tmE5eA07TSFfYfo4JBR+Ss6mIwHcaKXtHkbiufTPOkJ1JMWxaZ0SxGzhfuBXmP Y5mklOYACSDbkd+Ipk6dlpQ6ij9iyOul+vupgz4BgWOeOZsc95f8jQlKDYPMC0Showa6 V5c+Y+Pp9IYVGMAXaHz1ld71/zFxsHZ1vpluSqiWe+6nQC+TOgdhqeZQYapFd83+vxuq g2lBjjLWka4r7gmayKoFCtTZVgUOw0nTZFcz1t7Y7VPDMZItMk+aEL6kIRbI/7nsCGFD Irtg== X-Forwarded-Encrypted: i=1; AJvYcCXhBFyMT4Ow41hsId8M2S/yryexokkywh2btIfj2/N6C+Of1iIm21HLG8ZGj6PKJz5+c0pbF2OHjLXilA8=@vger.kernel.org X-Gm-Message-State: AOJu0YzdELZnpTyZ1qib0IC3s2JddUQItzWcCmWn/kAnUOwtTFHkObu0 IlWJeOu/LP9XKDxz4bHtz8kD6yDQy8f6JWN3zdMPbzGXOlS+IFetaIlhIs0YK4Wrj8BtueUh1X+ YJg== X-Google-Smtp-Source: AGHT+IGiIrJ4A5BQmPLk01OCG6uv19KvfBdqQ++zW9utvyZoUIEh7zcE7pTpIYxGuUvXyPiED4dAhLqQ79M= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a0d:c944:0:b0:691:55ea:85e6 with SMTP id 00721157ae682-6d410bbb5c2mr205897b3.7.1725063360840; Fri, 30 Aug 2024 17:16:00 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:25 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-11-seanjc@google.com> Subject: [PATCH v2 10/22] KVM: x86: Move EMULTYPE_ALLOW_RETRY_PF to x86_emulate_instruction() From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move the sanity checks for EMULTYPE_ALLOW_RETRY_PF to the top of x86_emulate_instruction(). In addition to deduplicating a small amount of code, this makes the connection between EMULTYPE_ALLOW_RETRY_PF and EMULTYPE_PF even more explicit, and will allow dropping retry_instruction() entirely. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 862eed96cfd5..7ddca8edf91b 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8866,10 +8866,6 @@ static bool reexecute_instruction(struct kvm_vcpu *v= cpu, gpa_t cr2_or_gpa, if (!(emulation_type & EMULTYPE_ALLOW_RETRY_PF)) return false; =20 - if (WARN_ON_ONCE(is_guest_mode(vcpu)) || - WARN_ON_ONCE(!(emulation_type & EMULTYPE_PF))) - return false; - if (!vcpu->arch.mmu->root_role.direct) { /* * Write permission should be allowed since only @@ -8938,10 +8934,6 @@ static bool retry_instruction(struct x86_emulate_ctx= t *ctxt, if (!(emulation_type & EMULTYPE_ALLOW_RETRY_PF)) return false; =20 - if (WARN_ON_ONCE(is_guest_mode(vcpu)) || - WARN_ON_ONCE(!(emulation_type & EMULTYPE_PF))) - return false; - if (x86_page_table_writing_insn(ctxt)) return false; =20 @@ -9144,6 +9136,11 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, g= pa_t cr2_or_gpa, struct x86_emulate_ctxt *ctxt =3D vcpu->arch.emulate_ctxt; bool writeback =3D true; =20 + if ((emulation_type & EMULTYPE_ALLOW_RETRY_PF) && + (WARN_ON_ONCE(is_guest_mode(vcpu)) || + WARN_ON_ONCE(!(emulation_type & EMULTYPE_PF)))) + emulation_type &=3D ~EMULTYPE_ALLOW_RETRY_PF; + r =3D kvm_check_emulate_insn(vcpu, emulation_type, insn, insn_len); if (r !=3D X86EMUL_CONTINUE) { if (r =3D=3D X86EMUL_RETRY_INSTR || r =3D=3D X86EMUL_PROPAGATE_FAULT) --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2EDA1364A4 for ; Sat, 31 Aug 2024 00:16:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063364; cv=none; b=KmwtM2TDlHGqTUyaIrf/hTmUgoCwZVgHj/2biBxjCQAwiTvQgXm0C4q9GYbMnsxnBMH+Qy6xExBGzL4DNyF8qinCqfJ5DHMuRX+NR4u5zxcQtuLkkC2SZWm1PLPwWMv/Wi5hBoc1QuLjoejT5p+IFf5UwRwC6pRxA5IQMjEMobo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063364; c=relaxed/simple; bh=Gt/hYCvtkqqT087VrypTyzp43782ZGBu82s/aGaFlhM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Wy9S3fs+vzaTOge++lSyxIywm9p4mLJPI+zuBCj5sWa/HkzeJIiKXhiaK0wr8UT3M978+wG4TUi34AQu1ekfXdAtu6AWCJFqGri04WYpr+dN4+dOEme9janBD7Y1ZPKVH/6v//J7uEzg1iMzr+J6misjYANcVODbzkFlBwTzU7Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=4P17CCHH; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="4P17CCHH" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2052918b4f4so12009515ad.1 for ; Fri, 30 Aug 2024 17:16:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063362; x=1725668162; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=REcqlnagGsiRzvNxOPvSFUigo/12XdRKebv9ZoVRwHA=; b=4P17CCHH9cUyr35Ptu3Cp5aJpTAhT6f9U8msd7v1QhWjWMIYHtwFCwn5P6Crt7cRRY N6E6oUzAs9H5EGTsbBwVJ4GuYUdbk2SRil7Da5W05L/pOvfOVhtQck2wzFvx/i22UA+Q hDDsIASA5z4ZCrPR6wdWd7PevV8Wu1PNoki6y5aICA+MENKVxIbn39ZO1UeAkutIOoKB nuEFtAwx48ihfeNr8K3ed+NA3P0NG5Nm+o472txGcT9tRYGg+Y3DvenRIP/DDXkoHvMJ YwQFncsundD13wItLYdaqn6hdZ5PHEHPC9jLri5C88/go6K8NkJFvrKhp0uVAGQrRL3E VB8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063362; x=1725668162; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=REcqlnagGsiRzvNxOPvSFUigo/12XdRKebv9ZoVRwHA=; b=ViJbHmxNkAi/n9a4vFxCJBkcb5n/KQavUy3gMm7qKzyPVrDy5v3X3TMaOIc8W6vQIL cQ9QAnPUPTC14vw2Y3KYqRgw2V7kqZFRXxbn8yPZaBKcVdQYUIQb6SFUO/Y/lAK5vJy+ mapVQq6YQMILcSEuxlGKes+/pNa8+k9gXL0XG996XteCWIoSuyrK+w8gu86Cmo1xkifL 8WM6+Cpr21JatdAqLgFvGEoAw166ZhXUEj+RXsxvsZ65N/8bnHWxxbu5JKz+sYJuLvKa PmIUaIqq/omsl+1bUJuWcVg5fusxM8eECegLLrGImRQPPV/DmrnveKU+dGvbU0XPGnEN erDQ== X-Forwarded-Encrypted: i=1; AJvYcCWkakv9Rjdncz5jlwDdPa+1ynRSbG3cuOpmTs2zR8DRYBdGhLwZZCLk9qgU0cyMFbtLWVCtu4Z6fL+MPxY=@vger.kernel.org X-Gm-Message-State: AOJu0Yylze6lLlkNh02kpQ4n91R6HeQCLOI+kcdS983JbrZ6b9cfmsHP omsC1Dkv+1SGfwmyi5H2UpkyFBqmXfM+NtGFu9g8pvxS5KvcRBzgdHSf1nT0pJwJiNDs8EFnwHE 7Jw== X-Google-Smtp-Source: AGHT+IH1Nilzksm2OfpsDzscQSy+ZeVt81YmGO0kRIhD+rE737Nd1fk81Tkpfa9/YV68oh+vNJ/7FWAw/OI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:fb0b:b0:202:232b:2dc6 with SMTP id d9443c01a7336-205276e1f97mr726825ad.5.1725063362502; Fri, 30 Aug 2024 17:16:02 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:26 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-12-seanjc@google.com> Subject: [PATCH v2 11/22] KVM: x86: Fold retry_instruction() into x86_emulate_instruction() From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that retry_instruction() is reasonably tiny, fold it into its sole caller, x86_emulate_instruction(). In addition to getting rid of the absurdly confusing retry_instruction() name, handling the retry in x86_emulate_instruction() pairs it back up with the code that resets last_retry_{eip,address}. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 30 +++++++++--------------------- 1 file changed, 9 insertions(+), 21 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 7ddca8edf91b..c873a587769a 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8920,26 +8920,6 @@ static bool reexecute_instruction(struct kvm_vcpu *v= cpu, gpa_t cr2_or_gpa, return !(emulation_type & EMULTYPE_WRITE_PF_TO_SP); } =20 -static bool retry_instruction(struct x86_emulate_ctxt *ctxt, - gpa_t cr2_or_gpa, int emulation_type) -{ - struct kvm_vcpu *vcpu =3D emul_to_vcpu(ctxt); - - /* - * If the emulation is caused by #PF and it is non-page_table - * writing instruction, it means the VM-EXIT is caused by shadow - * page protected, we can zap the shadow page and retry this - * instruction directly. - */ - if (!(emulation_type & EMULTYPE_ALLOW_RETRY_PF)) - return false; - - if (x86_page_table_writing_insn(ctxt)) - return false; - - return kvm_mmu_unprotect_gfn_and_retry(vcpu, cr2_or_gpa); -} - static int complete_emulated_mmio(struct kvm_vcpu *vcpu); static int complete_emulated_pio(struct kvm_vcpu *vcpu); =20 @@ -9219,7 +9199,15 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, g= pa_t cr2_or_gpa, return 1; } =20 - if (retry_instruction(ctxt, cr2_or_gpa, emulation_type)) + /* + * If emulation was caused by a write-protection #PF on a non-page_table + * writing instruction, try to unprotect the gfn, i.e. zap shadow pages, + * and retry the instruction, as the vCPU is likely no longer using the + * gfn as a page table. + */ + if ((emulation_type & EMULTYPE_ALLOW_RETRY_PF) && + !x86_page_table_writing_insn(ctxt) && + kvm_mmu_unprotect_gfn_and_retry(vcpu, cr2_or_gpa)) return 1; =20 /* this is needed for vmware backdoor interface to work since it --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7224942077 for ; Sat, 31 Aug 2024 00:16:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063366; cv=none; b=QSGS9ZVCUEh7BXxfhnZPr8OeinK9qfm3LCcKN5mx5p7mm1nx4Zo4lTG+B8FicqoXUXm1xnHkeo2N9k4Zh+rW52o5Rrac1vbq/Hw634r07bynvoXTUgqF+EcTCDqbWLeO2q+ioKmKb2XP8/Xp6i/ZDl7NdwO0ac9u9gpy9e2oUqY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063366; c=relaxed/simple; bh=0ugKPXm7f0Z9tkNBlydIBvCQhLXirirTmaxUHzxFYGc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=IPBPP2moioMe+mm1OdXIidjR1g0gw+lyxwHrAfOlBVzfM3IzLQ+rtt9zd6uSpqjld+2/9HxmUQyhtoXLdVSyTGaN4XDubfg1RxJ7LGobwi0z5g50dzoOls+mvgMBwkyNnScVCqnLn+XrShTI8BBqBcSX85qjE0CRHoqwE9fLU5Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=x6Ud8f+Y; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="x6Ud8f+Y" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-714290c2b34so2321098b3a.1 for ; Fri, 30 Aug 2024 17:16:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063365; x=1725668165; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=pFaEKpwsoQLgtn+xPOU/nmb2rgY2BlrNEgxoYJSjng0=; b=x6Ud8f+YfiNAppvZWkbOoTh7yNRBp/eZpMnrRSdIR8mbBUeHPL2tUDE41RtBZRqTKk 2m4wZj4k0gWHDEMvAE+rD2Rx9ihFFJTKoO8hEsteWIAfQxbwquLVysdznBoBIPKl9XIq 7AD9/vT/9fTg3Xp8nrZiAF5gn9l0djEkeAK9zf7xodZd8QZkgvFqhzqyFf+DnG4Q5Jn7 Y8c4j1tx0T96EHILB77ySJgUKmMX2BMjWp0cKLpM7DyAF4+WWZM5cp4JSjNSp7nqf7Ph f9vIieuWR7y/+fP3h185SmIir8jt4SmGf52ctXsCU2ywgmxW0/8+ALa+BsIskzn3jBtH A+JQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063365; x=1725668165; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=pFaEKpwsoQLgtn+xPOU/nmb2rgY2BlrNEgxoYJSjng0=; b=AcsOg+9KYP6PfRQGjKH9oS0C8EQU2t4+oJ+mYuHUOu8V5bExHJzpVLjtXPin2m3T1Z y+Mwx29jDtGvdCdtUNyygLLhrZ8Ht4A7Q4ZfQzZKfle/whZgSrwJqCFuqkHkNFG5Qamh Kpq6z4kP0pTUnp5725yfTs8UlL+A05YYUepCXN5z2UVmb2zUq9UvkLDusY/cYoGBkYy8 7VOKpsMfsmVP9N1ggVRk4QFjkC4+GRlmRX0yX1GGE9KHkLekGEu7wOqaABPwrUnIMyoB 86AisfWuaqkzGFvC0JPqEtOO9zQhuz6zpO+3nzbw4JZYj8j+2bDuBm5KcgLBzJFYIaQg Yp2w== X-Forwarded-Encrypted: i=1; AJvYcCVdTGz0KRGvSMcj5bEaSCCbw+3z/c6JOizULVpvvtmp39vdlkGyWCsEiYopWl5aGXw6O8vNykPQiiY8qzs=@vger.kernel.org X-Gm-Message-State: AOJu0Yy6LOQcdAcjbWiHMJrkX4/1fTwR2O10603dr4kca/ooB8QimmcG Ks/aEZaalW6B4ZIOvNeA9/GEzdEXnJU1OgKvWHBMTkqgxt/2XLCeV7BqtMy+A2aXVGBf/Chqh2B i0Q== X-Google-Smtp-Source: AGHT+IHeayohZ8P9bTfkvLJWiAXof8KEhjPXfE8aDRqDE3XirEUvuRZ9MRzo1AWngrkQZ4frV2qYEdsDGQ0= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6a00:870f:b0:714:37ca:ed6e with SMTP id d2e1a72fcca58-715e104ad90mr24129b3a.3.1725063364507; Fri, 30 Aug 2024 17:16:04 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:27 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-13-seanjc@google.com> Subject: [PATCH v2 12/22] KVM: x86/mmu: Don't try to unprotect an INVALID_GPA From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" If getting the gpa for a gva fails, e.g. because the gva isn't mapped in the guest page tables, don't try to unprotect the invalid gfn. This is mostly a performance fix (avoids unnecessarily taking mmu_lock), as for_each_gfn_valid_sp_with_gptes() won't explode on garbage input, it's simply pointless. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index dd62bd1e7657..ee288f8370de 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2729,8 +2729,11 @@ bool kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu= *vcpu, gpa_t cr2_or_gpa) if (!READ_ONCE(vcpu->kvm->arch.indirect_shadow_pages)) return false; =20 - if (!vcpu->arch.mmu->root_role.direct) + if (!vcpu->arch.mmu->root_role.direct) { gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, cr2_or_gpa, NULL); + if (gpa =3D=3D INVALID_GPA) + return false; + } =20 r =3D kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa)); if (r) { @@ -2749,6 +2752,8 @@ static int kvm_mmu_unprotect_page_virt(struct kvm_vcp= u *vcpu, gva_t gva) return 0; =20 gpa =3D kvm_mmu_gva_to_gpa_read(vcpu, gva, NULL); + if (gpa =3D=3D INVALID_GPA) + return 0; =20 r =3D kvm_mmu_unprotect_page(vcpu->kvm, gpa >> PAGE_SHIFT); =20 --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5823DAD2D for ; Sat, 31 Aug 2024 00:16:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063369; cv=none; b=EVjWlaNOWZP/k4ax2hVmIPQzLTwnt6YY1GGA1XC44Uw1DbZxpxDSybud+D2lwHcxaFpvmv0VmZmVCm0wB1sfRM0fp5bOGcsliZTACwAOkcNZj/zMhCEoG1fsTymyoab/bAXCYGLBV3i4vhU1iqjfpK3HDAxL6gGnj6rkgJj3A70= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063369; c=relaxed/simple; bh=LzpJ/JVED9Xy/kfec+E0sqmDsVm4jZ8PMfDy2LTQRhA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=cA22eJuhMyNRtA+4DXibdSa0t91QxzhKbDpZ8rE1IzR3rRkg41a9du59SEW9whl+LkkfMMba3zkan1A9GzyiajDhrp8x5DCNQYxWaPV7h/bkXEmLEL62U2A3rV5iY4r+zA9CxqjLOJJfBLrf5xWv58befWH9HdYPM+juBNQrFW8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=cGo2+jyr; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="cGo2+jyr" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-7144c67e3d0so2311957b3a.0 for ; Fri, 30 Aug 2024 17:16:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063368; x=1725668168; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=i1F/rHShfrDTgNHmv4cG1toMKdrxBDoSppyZEKY7y+E=; b=cGo2+jyrnJ7A6D0zsP/8TWXvElPKXRdHnVxx4M30gHzMwr6sjOoERCtL/GVa6EJIZt 3q4b4lkWMrPO8CFxRBjC0r6cP2AKMqWGiaEFxDhfPAcqw2+j5Pu94ZiTmXO/ks8THxev 52CFV4LvzYVEmIjaNVeF3eudhcI3fbIdIKYBYKoF7GJEJPTuVGTkReNq3yJj7OaP3OfI f5pWwse4xl4FtUcmJu2pwiZ5ursqgtKV42lHKzZvV4NCzX4K2dPgsASV/OLDv+2NjH97 3bW9KuD2O6R/3o3SzsSOgmPQBQbJ1KXZL8BOCLt3RypMpLDcxSjsz2nCVA3gABcALOEW 5n2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063368; x=1725668168; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=i1F/rHShfrDTgNHmv4cG1toMKdrxBDoSppyZEKY7y+E=; b=Ui7IMffFmrFl+kpoIj6kOBEOZvQf9y2wf6wsjButXK7Yi4f6blvNAb7eeE5SPSdz24 M9M0AQwu+I7Ig5QzwRveMx0hg9COcnYBIRqBi4dttJPNDCaAfawRGGMUKmZIfC9gLPlz WZ2Kj/nS/qX+nX8MjbxFJrXFVwfoOm0aKl79x+t/dccRZ5BZX6L7wHYGbiF3FUSaqBSj +Imjxc0XTNN5Z2gaJyDs/85MTk+PeXldaUg2DourEXReDW8j3omYfbdVNuCWy2tcLqVl vsQa6E+rKYdI0oZCA56Tc48o/MKjz0x057v6GDprDgzbOcgpKVbnO3+Q/TStauqfPjMA WHyg== X-Forwarded-Encrypted: i=1; AJvYcCX0+e5FPegaNgrEGrWv5awHxi0196ekwpAVrP89BwELD9/7x8mLnCFcm6H3svnp4ce5xnfzCMNTe/dse2Y=@vger.kernel.org X-Gm-Message-State: AOJu0Yy9sh2baikAZPdkzpZ7KRavti+YjGPWZcleZW6Vn422lwEqO5Ia hcdb6OB1BdFeKjyE9VYCfHmoV2NCfdmYqQia6pMIw65Y5fB4Vfqq9PjRws/rKwY6IaxAtvEsl+P zHA== X-Google-Smtp-Source: AGHT+IFRLF5vuwLCmCExGGa5XYbATHrgktWltU4k7xw5DL+Mn8fkJtnEkifE9EVPqwBnNdQM3Vq7ebbZ5c4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6a00:8983:b0:710:5cfc:8795 with SMTP id d2e1a72fcca58-7173045b0b6mr7558b3a.0.1725063366272; Fri, 30 Aug 2024 17:16:06 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:28 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-14-seanjc@google.com> Subject: [PATCH v2 13/22] KVM: x86/mmu: Always walk guest PTEs with WRITE access when unprotecting From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When getting a gpa from a gva to unprotect the associated gfn when an event is awating reinjection, walk the guest PTEs for WRITE as there's no point in unprotecting the gfn if the guest is unable to write the page, i.e. if write-protection can't trigger emulation. Note, the entire flow should be guarded on the access being a write, and even better should be conditioned on actually triggering a write-protect fault. This will be addressed in a future commit. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index ee288f8370de..b89e2c63b435 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2751,7 +2751,7 @@ static int kvm_mmu_unprotect_page_virt(struct kvm_vcp= u *vcpu, gva_t gva) if (vcpu->arch.mmu->root_role.direct) return 0; =20 - gpa =3D kvm_mmu_gva_to_gpa_read(vcpu, gva, NULL); + gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, gva, NULL); if (gpa =3D=3D INVALID_GPA) return 0; =20 --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A50B25C8FC for ; Sat, 31 Aug 2024 00:16:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063372; cv=none; b=rAb3D6sjElAJymryBcX0WiNeCksPIg2R112J1kTRVaJ1xY6lXdwEXl7PS/bICZfVJlaYoHi2poR914od6AsjNgakgarSBklsrH8mMG2gm3EhDWI7v2cV5fDCKxVlzHnzDc5UTEBAOBM+6l6cKSGDGhAUAb/e//KzxsN96ro1WnQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063372; c=relaxed/simple; bh=uB8TtNeHx27Ad7EM5biaxlZheZEshJ2mMvs5MEtYNi4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=KlPymuLhtGko8fSx3dcQqXF9O0akIj8km86HCeupbFDv7jKFllGUWcU/eMt49gvLNUf1vwpUDKo5GXOD4EwkK9coFGIGVYmNXV+qP1cjpPlKWgxVTMdJaROGyRgaLHfLivbFsXl1aqaxA9OH+dxEQp/3cm6IW33F/xrtT5URhCY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=jHFiApLp; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="jHFiApLp" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-714290c2b34so2321128b3a.1 for ; Fri, 30 Aug 2024 17:16:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063370; x=1725668170; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=9O4VEwldMzhpZYdn76eUeYyG7aK2ZiOdW5UELJTylQg=; b=jHFiApLpCRUo8xRYDgJtKyqNk7XnmPjUX4PCYfQuEKViL5h4HT6fkJwUzJ2Qm9uA61 a+jKdWS8ZREMNT9XJy9Opvm1HTYAnUshHZqLd66vYKp4oJPBLEBfcjr31nL6sKcTn8Df tDceGS7gVkbaEyqxPpOK5pzDNrsSktjEo1ScIp0QqqGsy52m1z8x+iEtKcbKHGZQj9rQ 8WEJTfARdI++iMlO3awEhNJRKcg7JM4w/BgNarqHRcXSI2ymivZF8TTh8x09zVgBy0vx 9hB6fkrIjaIAX5FftWDWy6jCpPsqzOxQBQKTbzcOCmVHm8dbmmTWcJtCJoD4rDNKem/4 W7Kw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063370; x=1725668170; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9O4VEwldMzhpZYdn76eUeYyG7aK2ZiOdW5UELJTylQg=; b=cktaQD7rtybJlzXC7Sy1vyyY6jG1E02N438QqSHrhDzLXr32Cq40Qh+r85evlmA9Wo turt2asm5ae9u26S9a7+50VuJ+uUqegCuPTQa1uoHIDmpEPQTj5briDTLNkgjRk+7nZG idlufVhAxYCHZS6zRkny8+Ye+5f/oZFRF5tNlGTVYcEAvMlxddFd6hFVpRuUOOTdQQdt sh32S+4KbMgtCGLgQLG+fHXv94x8z5E+BPLH0USZe+8kj+d91UCOhZFtNlZpbklvQx9L yFG+ewkSn4iAHUGsAYh0O95u07pKYUOL0GuctLM0Y9x6ynodjvPQmaOJEWY/Pa659XQc U8qA== X-Forwarded-Encrypted: i=1; AJvYcCVnP68gNGtf8I+i01cCaSII+w1gz60tcNMfmrO/6PWJJKG6lHltmssRd21OuVS5Eh8TeorjVsk464g9lsM=@vger.kernel.org X-Gm-Message-State: AOJu0Yx4mi3e939dGZl88Z5uPCHyMoDuzv4HTjjZ6kqwbd1pyWrFQA6A 3RadS6PBdoiFT9diQNJefUIjQePct/jt36oy/z2CrAXJojqIg/SNUI/J1rrrEjvEZ6JGg5DDQZ+ 2Rg== X-Google-Smtp-Source: AGHT+IGVv3c3jcB0r49a4HwlSnaEQWbUvEgEsWoVSM9vX8DyBoJnaCsk9OSnitGCy817N8L5rs5wpa/NxsM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:aa7:9383:0:b0:714:184b:b2de with SMTP id d2e1a72fcca58-7170677fd42mr15341b3a.1.1725063369831; Fri, 30 Aug 2024 17:16:09 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:29 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-15-seanjc@google.com> Subject: [PATCH v2 14/22] KVM: x86/mmu: Move event re-injection unprotect+retry into common path From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move the event re-injection unprotect+retry logic into kvm_mmu_write_protect_fault(), i.e. unprotect and retry if and only if the #PF actually hit a write-protected gfn. Note, there is a small possibility that the gfn was unprotected by a different tasking between hitting the #PF and acquiring mmu_lock, but in that case, KVM will resume the guest immediately anyways because KVM will treat the fault as spurious. As a bonus, unprotecting _after_ handling the page fault also addresses the case where the installing a SPTE to handle fault encounters a shadowed PTE, i.e. *creates* a read-only SPTE. Opportunstically add a comment explaining what on earth the intent of the code is, as based on the changelog from commit 577bdc496614 ("KVM: Avoid instruction emulation when event delivery is pending"). Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 28 ++++++++-------------------- 1 file changed, 8 insertions(+), 20 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index b89e2c63b435..4910ac3d7f83 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2743,23 +2743,6 @@ bool kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu= *vcpu, gpa_t cr2_or_gpa) return r; } =20 -static int kvm_mmu_unprotect_page_virt(struct kvm_vcpu *vcpu, gva_t gva) -{ - gpa_t gpa; - int r; - - if (vcpu->arch.mmu->root_role.direct) - return 0; - - gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, gva, NULL); - if (gpa =3D=3D INVALID_GPA) - return 0; - - r =3D kvm_mmu_unprotect_page(vcpu->kvm, gpa >> PAGE_SHIFT); - - return r; -} - static void kvm_unsync_page(struct kvm *kvm, struct kvm_mmu_page *sp) { trace_kvm_mmu_unsync_page(sp); @@ -4630,8 +4613,6 @@ int kvm_handle_page_fault(struct kvm_vcpu *vcpu, u64 = error_code, if (!flags) { trace_kvm_page_fault(vcpu, fault_address, error_code); =20 - if (kvm_event_needs_reinjection(vcpu)) - kvm_mmu_unprotect_page_virt(vcpu, fault_address); r =3D kvm_mmu_page_fault(vcpu, fault_address, error_code, insn, insn_len); } else if (flags & KVM_PV_REASON_PAGE_NOT_PRESENT) { @@ -6039,8 +6020,15 @@ static int kvm_mmu_write_protect_fault(struct kvm_vc= pu *vcpu, gpa_t cr2_or_gpa, * Note, this code also applies to Intel CPUs, even though it is *very* * unlikely that an L1 will share its page tables (IA32/PAE/paging64 * format) with L2's page tables (EPT format). + * + * For indirect MMUs, i.e. if KVM is shadowing the current MMU, try to + * unprotect the gfn and retry if an event is awaiting reinjection. If + * KVM emulates multiple instructions before completing event injection, + * the event could be delayed beyond what is architecturally allowed, + * e.g. KVM could inject an IRQ after the TPR has been raised. */ - if (direct && is_write_to_guest_page_table(error_code) && + if (((direct && is_write_to_guest_page_table(error_code)) || + (!direct && kvm_event_needs_reinjection(vcpu))) && kvm_mmu_unprotect_gfn_and_retry(vcpu, cr2_or_gpa)) return RET_PF_RETRY; =20 --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C9D2212AAFD for ; Sat, 31 Aug 2024 00:16:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063374; cv=none; b=GZ8Rrs4Nn5mQVj/peh7frHgTG8yODwKWYfDjeI9IMxnc2/8or+dagWPr6LqT+Yc+g9Fv9tlFV6FmzKynYdIoct7/KHRSOThOXOGFvPepSewo8LVSEuCHKEmTf2jLcoXnhrNbjLqpK+UWA/N1WNfDbfz1j95SiVBDZnQsD1GshQs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063374; c=relaxed/simple; bh=oLv8xcV7npfhCNOsBP8X1Fn7qxQks0etkV0j5AxfzOM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=fhTzwT3rucV8NBTCSVjLoLp4k3rfgq3qSwjggvS+8gTYNBQWtCjNbEquwAA4KXIo8wKn/JEqNkzrCBopaDzlGFS/e6/zumTt9JU/PX1Y0lE3ZhJr8/OhNjsA/02+cAPJH5jjDYbG99JPaDiPN5g7wZRf7va2nKqS4gBg7kT0tWc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=XyncVTSp; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="XyncVTSp" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6b351a76beeso47784557b3.0 for ; Fri, 30 Aug 2024 17:16:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063372; x=1725668172; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=u3jsbE6XP6TTJxd7m+VzZ7IpksW0atkO59oovcuh+cU=; b=XyncVTSp9OOQ1hTBku60LBAwfl6MAAxDKr4rDWj8Ap5Tlwj+RvpIgXkjVvVBLjSNQe leSkZLfcw1Nz4h7kH/iJGvdOjp5Exa2BcGf+kKvZ+jM2ZVGBXBEiKKMz6/lLr4pe8vh6 GunlKsH/MWIMUfoJLkZL0U8JupRljNPFK60flwqEJfSK0YSy66O1YY3QfoWanI9/n+aB 5fKwSZaGxXK72XdmgnfsBdUsj2vcpczy1eGe5yfeJh2Iq0JsAuTaqPle2mqA+v8cHHkO AkElyQX+g+43wdOvNyY0wE6eRbUHnQQ0mgnk8dqNGPkHWcNgKVMnvmXcKG7el3uDYou1 xMxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063372; x=1725668172; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=u3jsbE6XP6TTJxd7m+VzZ7IpksW0atkO59oovcuh+cU=; b=nrtCTWec8ybD7JGYZzhL1h6kBiBxun8g/G/QVJdjxpB8q3156K4eNF49CX4cFF4nvX SqGNh7IjJ5o5KzIX8Fz3f4yg6wEFPqDPL3uSDCDJvzTjc7XptR44OLmcsD/jIfDJ67fI 3q9DJY9eVfhk2FH/HkP+W9v9R3F6rBlkd5PMqgYpI305jRGZFMHS+neOVJDFyk6Vf7Qx bhwHm/r1OBIfpDadpkPoM4/Hpu7PGPhIkheuUZ/QAgcA1hy9zMXHVPIco0Bgs63/P/xn rjKkhbdV/vCVXj/vYTecJ32yr/PqYFvgFQ81IJskhLyz608EEdC10y53Ta9Dw4/w4mtI a59w== X-Forwarded-Encrypted: i=1; AJvYcCVYT1pkVNZwFogDIAfBkOWt+pZJ7B3KNrX5IjXVK9odMo5iSVG0UGdYvmZL0yAAgWmrFtvybpT1He1MRyc=@vger.kernel.org X-Gm-Message-State: AOJu0Yxa8pfFB37zfiP/7i3yijRyu+tI9ySwg/L33h/4NA6uc0Umagsn dbuHBz4mgDbvAGQcqeaEwuzHwfvnANPLs5FvvNbfOg9pmot1DzhAQjTumXprhg0pX4B1kKqk8Sr qVw== X-Google-Smtp-Source: AGHT+IH9ZaBmo0cTRqD8QCtPipkvkJ/y4RHQnHtriu8u41kiihvKrtMLF9S4g7I+ks6ZXYZUgSjRk+7s+Mw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a5b:9c9:0:b0:e16:50d2:3d39 with SMTP id 3f1490d57ef6-e1a7a1771dbmr6330276.9.1725063371775; Fri, 30 Aug 2024 17:16:11 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:30 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-16-seanjc@google.com> Subject: [PATCH v2 15/22] KVM: x86: Remove manual pfn lookup when retrying #PF after failed emulation From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Drop the manual pfn look when retrying an instruction that KVM failed to emulation in response to a #PF due to a write-protected gfn. Now that KVM sets EMULTYPE_ALLOW_RETRY_PF if and only if the page fault hit a write- protected gfn, i.e. if and only if there's a writable memslot, there's no need to redo the lookup to avoid retrying an instruction that failed on emulated MMIO (no slot, or a write to a read-only slot). I.e. KVM will never attempt to retry an instruction that failed on emulated MMIO, whereas that was not the case prior to the introduction of RET_PF_WRITE_PROTECTED. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 18 ------------------ 1 file changed, 18 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c873a587769a..23be5384d5a5 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8861,7 +8861,6 @@ static bool reexecute_instruction(struct kvm_vcpu *vc= pu, gpa_t cr2_or_gpa, int emulation_type) { gpa_t gpa =3D cr2_or_gpa; - kvm_pfn_t pfn; =20 if (!(emulation_type & EMULTYPE_ALLOW_RETRY_PF)) return false; @@ -8881,23 +8880,6 @@ static bool reexecute_instruction(struct kvm_vcpu *v= cpu, gpa_t cr2_or_gpa, return true; } =20 - /* - * Do not retry the unhandleable instruction if it faults on the - * readonly host memory, otherwise it will goto a infinite loop: - * retry instruction -> write #PF -> emulation fail -> retry - * instruction -> ... - */ - pfn =3D gfn_to_pfn(vcpu->kvm, gpa_to_gfn(gpa)); - - /* - * If the instruction failed on the error pfn, it can not be fixed, - * report the error to userspace. - */ - if (is_error_noslot_pfn(pfn)) - return false; - - kvm_release_pfn_clean(pfn); - /* * If emulation may have been triggered by a write to a shadowed page * table, unprotect the gfn (zap any relevant SPTEs) and re-enter the --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BAFEA1369B6 for ; Sat, 31 Aug 2024 00:16:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063376; cv=none; b=Bdad4twijbxlUqRBjONTGCa1781bhFhti09vk3z4Pa2qSsIgjhtDliZ0O3PKGMSvSkfsYWXevdMiQ/vFX0ifK8rIhsw3NcS2VRydsGFcyORcHQntXp+LAnDI4MiAi1stEP2v5jamwj4XDM+1TryvR8fo0bPQiZ7Wzt1QazPUH14= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063376; c=relaxed/simple; bh=03HGP+MpByrZ7kzxrpdb3wvh41fpQldermKrn3cjMQg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=JmdizYTA4OZj8TYjCOb3oiJtfwmHujtD1mO3F7W4UDP1Kt/gT6LvJRCtJ8cM4V5dCa0Qcp/ILUnw+4sGPyU2B+Xs37kGli9qwjDe0XsKbAm5HJ1etQlW0/c6YQ2DyVuojVfnm+4t2AcIDesGMDpa7hukqoNSxAGSgUA8oMmq9+w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=OLlVQGX4; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="OLlVQGX4" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-7142a78918bso2852264b3a.0 for ; Fri, 30 Aug 2024 17:16:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063374; x=1725668174; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=NOgdVNwQvUHFMsxLXjZ7EYJ7UIxi1DX2us/omkckfEs=; b=OLlVQGX4/ynkZOD+R7thBD3Acongqwxk1ZaX2reavQ+yWutoQ8sucH+A9BtccsKz26 W5bOfSw7BidX4CyHQ0OGYU07NKxpmMXzNT/8XRELr08HAsBLGgEUk2QYVJonUvuUOssX BW8NHZtmHjhhL/J2t2ORjGZM+tuAj11l/AEGOImE4wyvCh2cR69gGwV02Jnl32zYEF54 WuC39hI3Et1Pt+9W0l+/9x19gjMWB+Qb8H8McniOyv8AkyJsGxLUBV97PJgFt+1Da2UM eA+3j9HlnSsfOuTUU5jy8RZp6qYy6H3s37GlMSB5GSfHtzCESfJ9/jt0ewQWx7Rtfn2f mlpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063374; x=1725668174; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=NOgdVNwQvUHFMsxLXjZ7EYJ7UIxi1DX2us/omkckfEs=; b=fbnbGStKK2Mcd9atW2ey7EdnvHG38AYuBvY6SEANoiHcDAUkNVOCkM+qcRKi0fF4M3 TwbCUCoHYQoqvxNGzMnlPiyIoNPR240+DRykn9o+HU6mAp7wXKEhGdFw2luNk2eBVtKo efynTOxGDyKB4zIwDcNn4NGtj67rRZH1jWqxqRLgDLHErpdzk5eVlHDetTMgGXwkAMk8 WIUDQ8CBHczSdJ9fcZlMGSbDi3yEnmvSWOuPN7s5816m1Qfo239bIuruAKYh80sCVRbA nPfC2wiFVMTvyHLRJQ8yGhuIQEeShwfoSsFj+remqZWi/Q6rM1glq2TqqvYjLQ2PnWFY IG2w== X-Forwarded-Encrypted: i=1; AJvYcCVbiBVVlcukTxHujwmhpX+efLGs0A8ZhNld5YowNHWRb5dFmbFs9qOBDubGDEpqh0UonDgSp4KC4ZVJImo=@vger.kernel.org X-Gm-Message-State: AOJu0YyY1jqE+tSfvgBOOl3e4jkvFndGFZoMK1lPbliBn3IN3xecHmna 6y25KLDSSbTxi7eyBWT+CPzPY65hP7NJBqwL4aGglwLRtaMGWtHNwAK8OojkeZt3X9Ilw7fbi18 wZQ== X-Google-Smtp-Source: AGHT+IHOxjcI5NDjaRDWXOQy4s+IAV1B1/GPpRSOPNEGZ+Qxs8WEg4mv+/mZzSMwknbgpQWTaR+XmVIYezg= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6a00:9444:b0:70d:2a6e:31cb with SMTP id d2e1a72fcca58-7173072f5a5mr11018b3a.3.1725063373855; Fri, 30 Aug 2024 17:16:13 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:31 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-17-seanjc@google.com> Subject: [PATCH v2 16/22] KVM: x86: Check EMULTYPE_WRITE_PF_TO_SP before unprotecting gfn From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Don't bother unprotecting the target gfn if EMULTYPE_WRITE_PF_TO_SP is set, as KVM will simply report the emulation failure to userspace. This will allow converting reexecute_instruction() to use kvm_mmu_unprotect_gfn_instead_retry() instead of kvm_mmu_unprotect_page(). Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 23be5384d5a5..ad457487971c 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8865,6 +8865,19 @@ static bool reexecute_instruction(struct kvm_vcpu *v= cpu, gpa_t cr2_or_gpa, if (!(emulation_type & EMULTYPE_ALLOW_RETRY_PF)) return false; =20 + /* + * If the failed instruction faulted on an access to page tables that + * are used to translate any part of the instruction, KVM can't resolve + * the issue by unprotecting the gfn, as zapping the shadow page will + * result in the instruction taking a !PRESENT page fault and thus put + * the vCPU into an infinite loop of page faults. E.g. KVM will create + * a SPTE and write-protect the gfn to resolve the !PRESENT fault, and + * then zap the SPTE to unprotect the gfn, and then do it all over + * again. Report the error to userspace. + */ + if (emulation_type & EMULTYPE_WRITE_PF_TO_SP) + return false; + if (!vcpu->arch.mmu->root_role.direct) { /* * Write permission should be allowed since only @@ -8890,16 +8903,13 @@ static bool reexecute_instruction(struct kvm_vcpu *= vcpu, gpa_t cr2_or_gpa, kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa)); =20 /* - * If the failed instruction faulted on an access to page tables that - * are used to translate any part of the instruction, KVM can't resolve - * the issue by unprotecting the gfn, as zapping the shadow page will - * result in the instruction taking a !PRESENT page fault and thus put - * the vCPU into an infinite loop of page faults. E.g. KVM will create - * a SPTE and write-protect the gfn to resolve the !PRESENT fault, and - * then zap the SPTE to unprotect the gfn, and then do it all over - * again. Report the error to userspace. + * Retry even if _this_ vCPU didn't unprotect the gfn, as it's possible + * all SPTEs were already zapped by a different task. The alternative + * is to report the error to userspace and likely terminate the guest, + * and the last_retry_{eip,addr} checks will prevent retrying the page + * fault indefinitely, i.e. there's nothing to lose by retrying. */ - return !(emulation_type & EMULTYPE_WRITE_PF_TO_SP); + return true; } =20 static int complete_emulated_mmio(struct kvm_vcpu *vcpu); --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 49A661386D8 for ; Sat, 31 Aug 2024 00:16:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063378; cv=none; b=Itu7zSDRza1N2SmTjob2sKdUzqFpiSvy1SkSskry+/AJCibItqN6Cm2MsVZf6VFceuWRIl+12rOZZsMneMmkuBQZ56gGf7SRnMZlSckRamqon1PWLy40oPtrhYlq/WyxtjdX4/mERqUdn6ZsRAHp+gkMOLZ7RfbxzLxM8pxRK5A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063378; c=relaxed/simple; bh=QaJOICfdwQ30ELkiG5OwkLM3NF9kzLFQgtTBFvcj7SY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=l50dp6EYs1JaUodDnVMg41V+ShZlF+9YWKN+g61JXOx77hpsIBFQ5oH1BxLezDNmei6MIopcNlBZAT9lLKDkRRt0g4xjTLU+AeruG3YSiMUdRpRtFepziy2vefjvgzSSGZbCk4edMelkBRDV6s2SifATzveXLTtvDZhAo3iCbQ8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=AfMQ2ACM; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="AfMQ2ACM" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6b2822668c2so45868237b3.1 for ; Fri, 30 Aug 2024 17:16:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063376; x=1725668176; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=WjHitxqWq6khw0o+yUtW8qx1qHM1xiloEtLNGgpGHyQ=; b=AfMQ2ACMQmx1Uc7lboshFETqa+gcvHqtsdxWZSxx1DXbCbX6BEUV4zrKF0vFxNVxIk KuSgRougIoYa8Pw4/+jTXAhttinwklYSNuRgW0fioobIanvSH+wxOv2zL1FV4OB4NUi/ focr48ZselAyGouzsrtnlDjZZk+2UnKjsdSowEAIqfSFVnZRIpjrJvAm6cGTv/2w18qB L0T4dBMgboCKE9unK4I1aBOkfKUAh7HpL++cDe+ex7xFcPhnWqhK9JmZ8tLxosIaXUFx 1A45gIIr0bXxFipPU+gPG3iP6g2Lo0I+QcW3aRcE35l33our+3vQCS6b24V/m+xnHxOW ufUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063376; x=1725668176; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=WjHitxqWq6khw0o+yUtW8qx1qHM1xiloEtLNGgpGHyQ=; b=hQhm1fqpI8FkNfuWz5f6XIsyLcJggaSN3Hflh1LTts6Fu1ww8ktIbq1Jxi6tlsxhEV FIiRTqNNKbpkM6BptpESv23T9NoKLcTyetocot8LWBwDN/mFzwaSaqqqO8dTxAMnfixs 2JPTjM2CSJCJXmEJ4DfrUFxwL1LxbOFLSRUjcnJUXj399JJdZBWAS5ZozGezWHgVbQg6 lSSQEzYEHQKhuQX+h0Z0+ejWrhNNmlYarr+lbqVActhWeo7B+r8f4Sdi0oL5vWjYrP03 ql1gljoxpf7FvXPXStlCeoJcxs3QyWVsf6vkuU6A3GI/218UZ5E5JN9EPL5dH/w/gwaZ bNdQ== X-Forwarded-Encrypted: i=1; AJvYcCWhItdkXKOxXf+vJVhsYfV1n1x0J4XfiZqHaG19NOF+AUO6HM8rBXdKV5axdfOUjKqxziowl1X922mxKqU=@vger.kernel.org X-Gm-Message-State: AOJu0Yxoskg9K1BLx/ZPT0Rni2XYfKltHq5YiScgrr9Zl9w9Jf20yWw/ FCudrr6vtwKjiIo/Z7gesXb+kO1+4Mom/6yYYiwOTgcHhGrXvHYQU2xf2RvaL83rConT9rNO+JD v6Q== X-Google-Smtp-Source: AGHT+IGo0xQgoGTC0KvVhguAmOCzE28x29RmlZpo2tFA6rgbVMj8w/GoFAp/vSeEnp4ivUddGUq0H+jFDzk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:7156:0:b0:e11:5da7:337 with SMTP id 3f1490d57ef6-e1a79ff9892mr8887276.3.1725063375993; Fri, 30 Aug 2024 17:16:15 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:32 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-18-seanjc@google.com> Subject: [PATCH v2 17/22] KVM: x86: Apply retry protection to "unprotect on failure" path From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use kvm_mmu_unprotect_gfn_and_retry() in reexecute_instruction() to pick up protection against infinite loops, e.g. if KVM somehow manages to encounter an unsupported instruction and unprotecting the gfn doesn't allow the vCPU to make forward progress. Other than that, the retry-on- failure logic is a functionally equivalent, open coded version of kvm_mmu_unprotect_gfn_and_retry(). Note, the emulation failure path still isn't fully protected, as KVM won't update the retry protection fields if no shadow pages are zapped (but this change is still a step forward). That flaw will be addressed in a future patch. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 20 +------------------- 1 file changed, 1 insertion(+), 19 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index ad457487971c..09fc43699b15 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8860,8 +8860,6 @@ static int handle_emulation_failure(struct kvm_vcpu *= vcpu, int emulation_type) static bool reexecute_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, int emulation_type) { - gpa_t gpa =3D cr2_or_gpa; - if (!(emulation_type & EMULTYPE_ALLOW_RETRY_PF)) return false; =20 @@ -8878,29 +8876,13 @@ static bool reexecute_instruction(struct kvm_vcpu *= vcpu, gpa_t cr2_or_gpa, if (emulation_type & EMULTYPE_WRITE_PF_TO_SP) return false; =20 - if (!vcpu->arch.mmu->root_role.direct) { - /* - * Write permission should be allowed since only - * write access need to be emulated. - */ - gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, cr2_or_gpa, NULL); - - /* - * If the mapping is invalid in guest, let cpu retry - * it to generate fault. - */ - if (gpa =3D=3D INVALID_GPA) - return true; - } - /* * If emulation may have been triggered by a write to a shadowed page * table, unprotect the gfn (zap any relevant SPTEs) and re-enter the * guest to let the CPU re-execute the instruction in the hope that the * CPU can cleanly execute the instruction that KVM failed to emulate. */ - if (vcpu->kvm->arch.indirect_shadow_pages) - kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa)); + kvm_mmu_unprotect_gfn_and_retry(vcpu, cr2_or_gpa); =20 /* * Retry even if _this_ vCPU didn't unprotect the gfn, as it's possible --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2700313C9CA for ; Sat, 31 Aug 2024 00:16:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063380; cv=none; b=NtiYFkSmWYx+akN0RsMP+8CezbQh21mDc0116z0jdAtNY/YgvtDmqX+pzCWNIO1jmV1Kpz/DFGl5Ox7oSQS7B3Bemcr0VegvjKktRB3tlR4gquOgO0SDBK0L7g8xdHj4gvoNl7dISpk7TMUhLkFPKUdN/RpRj9LKVUn1U3+W3m0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063380; c=relaxed/simple; bh=nHlj+xD/ONShNKxAH4EaHG4dggMKZLPwwDJ8pS5wCL4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=gqXbDTGgmHJsA2MqZxZzOEAnATkHt8rEzCE8H/BZHuRdT8X4LFqpLCJ6dfzU8ljOUq6ok8jkXJHkf0MkRpU7HCgzZjqh9W3gvBsS/u/w8wV5J/oz3cLXlrRjoLAtLMjrPCJ1K/kKDNrrCQsLrX4fydGLWt28EjCtIHgYiAl0YB0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=FrIfmNBw; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="FrIfmNBw" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2039405b567so20229315ad.3 for ; Fri, 30 Aug 2024 17:16:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063378; x=1725668178; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=hU77OYTkUMLfRxkahmddxieOilGttKEAZ+XNkDI94lk=; b=FrIfmNBwOJ/nIvezraVFZufKwvOVxQm1l8/QO+M9mFHsqLxlCK9p75qcKmyXEmfgA0 wBSG6N0VwE2b8qshg+LYXO/XWXbWVREW+OinUSKFWDO3flfDWkJG+tWnAKPozOtzbtoS qUUDHH/emk6lvMftMWezRoik4HA8WZUiLPoZb/3FLkq9uqg26uB1VqRobM/a5rrMlCuR u4wifjqarxdc+A+lGa9R58RXuVfPjnEM0FdEo1tUeH3Q8L9wChZ371DlQfe5AIu3jR2J f4kWaKrNtz8vDcmzJyrmDp1itu9+KdFkcBm+VDruDl9UzbukGbjJwJ3Upvxky9bh/H5z 1u0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063378; x=1725668178; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=hU77OYTkUMLfRxkahmddxieOilGttKEAZ+XNkDI94lk=; b=T5u41oW7gfn+Ev7BOspG3sQw9nevcysEtQ2uxnpgjoXfTs5jR1XBfVQG9ighgFgmH0 PF+2UVgWxn40BMztXghaz4OWGHPd3yHYCoo2Jw9/7rUpa2bdqziJlBoTxUkATf2f+9WN 4NLVQAzfHV7qJ3jGpK4HkjA/EUJNpDGQU2S1dfMg8d8mdKEXTtHytIw0f5HB8+cw2lMA xWDWMhetObzYKrWYlt9A/r3lECJg8hNRObolGvDEjZwIeDwcrlJ1eFNF/VXK9v3uP5Ml lT8S+2UhktCMLX2o3Ij93BwU14KtqvB4GHTxa4ZGf8crSNXNPuuTsEiREuzBNS3BVBZj xSyA== X-Forwarded-Encrypted: i=1; AJvYcCWHR8eRQ4tAHW5zjchsxBHHY6rdBMAW6j2b6TqE0XAbMvPkO3h0yQHIk2LqABYMMcQH2CdM9T8BqGPjxxQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yzbn7ZGHieF4uCue6xrfB010OOv0WKVjCJemXLB9wko+IC4JYqR gQkgsDLxWX1JtjrmAnKgBeuKklpnNTVsYYh9Nro5z/IYQ1AQE3n57ZE8JFoWSKdh9MF9zqNAKJZ gRg== X-Google-Smtp-Source: AGHT+IHblxeHkSJzB63Sh5fAMNGdMNQ2SfoqcaQ/5BRDjPe/OytJTEsD0IQrqIW87OaHEx0tHfAmrT3IG7M= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:903:1d1:b0:202:70f:641a with SMTP id d9443c01a7336-20527626efdmr976465ad.2.1725063378157; Fri, 30 Aug 2024 17:16:18 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:33 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-19-seanjc@google.com> Subject: [PATCH v2 18/22] KVM: x86: Update retry protection fields when forcing retry on emulation failure From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When retrying the faulting instruction after emulation failure, refresh the infinite loop protection fields even if no shadow pages were zapped, i.e. avoid hitting an infinite loop even when retrying the instruction as a last-ditch effort to avoid terminating the guest. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 10 +++++++++- arch/x86/kvm/mmu/mmu.c | 12 +++++++----- arch/x86/kvm/x86.c | 2 +- 3 files changed, 17 insertions(+), 7 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 2c3f28331118..4aa10db97f6f 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -2135,7 +2135,15 @@ int kvm_get_nr_pending_nmis(struct kvm_vcpu *vcpu); void kvm_update_dr7(struct kvm_vcpu *vcpu); =20 int kvm_mmu_unprotect_page(struct kvm *kvm, gfn_t gfn); -bool kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu, gpa_t cr2_or_g= pa); +bool __kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu, gpa_t cr2_or= _gpa, + bool always_retry); + +static inline bool kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu, + gpa_t cr2_or_gpa) +{ + return __kvm_mmu_unprotect_gfn_and_retry(vcpu, cr2_or_gpa, false); +} + void kvm_mmu_free_roots(struct kvm *kvm, struct kvm_mmu *mmu, ulong roots_to_free); void kvm_mmu_free_guest_mode_roots(struct kvm *kvm, struct kvm_mmu *mmu); diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 4910ac3d7f83..aabed77f35d4 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2713,10 +2713,11 @@ int kvm_mmu_unprotect_page(struct kvm *kvm, gfn_t g= fn) return r; } =20 -bool kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu, gpa_t cr2_or_g= pa) +bool __kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu, gpa_t cr2_or= _gpa, + bool always_retry) { gpa_t gpa =3D cr2_or_gpa; - bool r; + bool r =3D false; =20 /* * Bail early if there aren't any write-protected shadow pages to avoid @@ -2727,16 +2728,17 @@ bool kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcp= u *vcpu, gpa_t cr2_or_gpa) * skipping the unprotect+retry path, which is also an optimization. */ if (!READ_ONCE(vcpu->kvm->arch.indirect_shadow_pages)) - return false; + goto out; =20 if (!vcpu->arch.mmu->root_role.direct) { gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, cr2_or_gpa, NULL); if (gpa =3D=3D INVALID_GPA) - return false; + goto out; } =20 r =3D kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa)); - if (r) { +out: + if (r || always_retry) { vcpu->arch.last_retry_eip =3D kvm_rip_read(vcpu); vcpu->arch.last_retry_addr =3D cr2_or_gpa; } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 09fc43699b15..081ac4069666 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8882,7 +8882,7 @@ static bool reexecute_instruction(struct kvm_vcpu *vc= pu, gpa_t cr2_or_gpa, * guest to let the CPU re-execute the instruction in the hope that the * CPU can cleanly execute the instruction that KVM failed to emulate. */ - kvm_mmu_unprotect_gfn_and_retry(vcpu, cr2_or_gpa); + __kvm_mmu_unprotect_gfn_and_retry(vcpu, cr2_or_gpa, true); =20 /* * Retry even if _this_ vCPU didn't unprotect the gfn, as it's possible --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1CFE11097B for ; Sat, 31 Aug 2024 00:16:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063382; cv=none; b=GgoKIGIz4lbie+pwVDr8u99dQNHOAQMKXPvvf4HVUPaNLSwipHnjshlqcYG+C7NYxQWQZlvqIlkgro6snVoq77K9yioATUQ9BG3fuI/lDEC6vnsAaRj6r/LbUIK2tZguT8+IJPu/c9mXzKhgRoslAN8lN6UKF4JgrWGzaWR68S8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063382; c=relaxed/simple; bh=ccmI6fgXaS6ULZCMeBWyQaXyCbh7+cmAtQZQ72hgJko=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=OW2ahEw02ZCqEMpsyqCLIerXSnoIj2fUcMEUPTKQNtQCKH1gWHot1akOLrhvib4QwpOPof9x60dGZxZyaEATxtoTkNMOUW4Xvw4lYk3rogRZRRU2e7bni5tj9DczWfqzyRgqV2GcXMplTjHm5GGXla8FfPAdRI8nG4VUkgKgLX0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=fhaWi+E1; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="fhaWi+E1" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6b8d96aa5ebso45066967b3.1 for ; Fri, 30 Aug 2024 17:16:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063380; x=1725668180; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=heJP/gE8sNgyHlf3GBWAi72kmE/mtNKhFddnYBUVTOE=; b=fhaWi+E106CCFd5rgQtGX4joRjEjC0gmVzUlr+pTI5dSIPC7JrJoYsPFdc4PK8EFVt XKXw6KPkvSrYdFC0bNPtLpxTtE2paLsNrSHNoEBHfzjBpw1IvHmlx75IkqFufOKbUJlI yv6wHagtf98IUSXyoXqXnzxkz+mTcwh4Y++lnVlKvXKheb21V9q5uTXhbZuTZdw/tUfI cHZb8vH+y3lkUZGxXWS7YQ8+VLvM1ntiaFdNR5/U+Ofc5CHqNAMyvUnmKSVyD8NkJvVq VszHbjvT4g/Oogn1S/+m2xKGKbUpQmRYML14h48qy2b/DYAGRUTR9/8Jfo5i6nqD4V1A pWeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063380; x=1725668180; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=heJP/gE8sNgyHlf3GBWAi72kmE/mtNKhFddnYBUVTOE=; b=AEHqCedzf7GGDdm4aHFrccH37jf1C2XIlQL5oRRt4ODo+8BZ7dbmt6S4ZIE/++8gyV uLvoJvUnYkiBAVUVCICf5pSAQxhE3bc7qU67TWlhEJiXUcW0dUJoMyWdhCR++T3jvQNR nXWdjUsi3WR6qxGo0thjjcl484IhnPfqyVjOCIbp8PcA9ybTey0eZmqjvPi8AfMs0vT8 s3CX9AyT6mo6HcwMFVgmHF4u8PLxrKaTCnBQnifpLDGChcdovNM3zPE7lt20t/nun7Mo EU1WCWVmZ63DbNeMGSUWfZgpnrwgj86nETbmmDrzQiXuf6nBeY7VfeYrZJnZWZj1PcEe fTSw== X-Forwarded-Encrypted: i=1; AJvYcCWGyI5l4z1IwLGLNJftQ1KoZufDWS8iv1lJ/LJ9EI0oz5NO4aN9QLKnQprvb4hVQDDe/B0eW3PPGisw/jg=@vger.kernel.org X-Gm-Message-State: AOJu0YxtLBlLbQQ2M/wC4aw829obYkuWY6SEBMHz69FzajBEvQv+FjDS H/G2KGiEr7Xt7Yq5DiNXSQnB6wUxAkSgfi9ZWNCc+fLaK+zTfydqMVX6xqMH+HtPlUUl54yGC6S V7A== X-Google-Smtp-Source: AGHT+IHiBKbgkYqyJRDMcIF9+XogmVSPipW/TOpEzV4NmLy2XzHmDX3Zwx+y82tht7UQeTFU4iZAMrMnrmI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:690c:6103:b0:62c:ea0b:a447 with SMTP id 00721157ae682-6d40d88f5d9mr2178957b3.2.1725063380216; Fri, 30 Aug 2024 17:16:20 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:34 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-20-seanjc@google.com> Subject: [PATCH v2 19/22] KVM: x86: Rename reexecute_instruction()=>kvm_unprotect_and_retry_on_failure() From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rename reexecute_instruction() to kvm_unprotect_and_retry_on_failure() to make the intent and purpose of the helper much more obvious. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 081ac4069666..450db5cec088 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8857,8 +8857,9 @@ static int handle_emulation_failure(struct kvm_vcpu *= vcpu, int emulation_type) return 1; } =20 -static bool reexecute_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, - int emulation_type) +static bool kvm_unprotect_and_retry_on_failure(struct kvm_vcpu *vcpu, + gpa_t cr2_or_gpa, + int emulation_type) { if (!(emulation_type & EMULTYPE_ALLOW_RETRY_PF)) return false; @@ -9125,8 +9126,8 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, gp= a_t cr2_or_gpa, kvm_queue_exception(vcpu, UD_VECTOR); return 1; } - if (reexecute_instruction(vcpu, cr2_or_gpa, - emulation_type)) + if (kvm_unprotect_and_retry_on_failure(vcpu, cr2_or_gpa, + emulation_type)) return 1; =20 if (ctxt->have_exception && @@ -9212,7 +9213,8 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, gp= a_t cr2_or_gpa, return 1; =20 if (r =3D=3D EMULATION_FAILED) { - if (reexecute_instruction(vcpu, cr2_or_gpa, emulation_type)) + if (kvm_unprotect_and_retry_on_failure(vcpu, cr2_or_gpa, + emulation_type)) return 1; =20 return handle_emulation_failure(vcpu, emulation_type); --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 924B114EC5E for ; Sat, 31 Aug 2024 00:16:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063384; cv=none; b=dBUk2UICHbv7RmQ7TwmWkqA/HqwnHoczEicCi84IQv9RTJOpYIqL8aAVLContX1ETHCuBODj0nKhVMfedWpo62Fno/2Sl9tzseoSSpUyjA3JW6gFmItdBRp7Rji82kjKovTno3ys3wxU1v6dVtjugbL9sJZAQvbGDbiOgwOXKkw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063384; c=relaxed/simple; bh=jbNYq8nf4cToP9nG3R9NtHXCFO1UUWSTXyZK1MFowyM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=jnYiiCf07EXT/C8jcprAMvUC4at2IU8x0m9xwauL4fXn1W/8qPzR1jGJSQXaBMiiNNnREIkUoVO3xu3Gb52L4ipXi7xoTD6DmtOsIVNidNRwEVXbSQpE+fWsuQewVTWZoimsQOrFnzX08YSPiFNimrzWZS9uAEo6tYMgIfXLrRE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=gqxJs9zS; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="gqxJs9zS" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2d406dc42f4so2287029a91.3 for ; Fri, 30 Aug 2024 17:16:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063382; x=1725668182; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=cR1WOtXSuetpHOXK2S+96cMV73XM9Whtcc3KrMXdeXM=; b=gqxJs9zS7eFen9ZACMiMMglKRYkC+Xb7ZK7pYkLyKdJTjggbmQvEHqEXPGXI9Nxq41 IdogRgDcL4iawguzYoGtGTKXVLrvIqW/c8GXBXgLxgecgUmmGQYociWuQDHLL6CebDle P3FZhsPvFOyLbtb+CoyYSP1zdWTZ4k+bF0bzf7Kx6Kwix8t2eSd9xD1XRlup0fsrdcju SzBaE9maQ65q7Fc06D6/GP2fxGzJDePck5H/kNgLrT+n48Ju5OBy3QgW5GCztmx1ansu IQjrxRvH6WvaIwSO51tzHxZ8jy37nGqXSXVyGY/y9S6K80UjVL58lPrr0AuO6KaREsLo VTHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063382; x=1725668182; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=cR1WOtXSuetpHOXK2S+96cMV73XM9Whtcc3KrMXdeXM=; b=dWsdR1O+V4JkuJDefUZOAtBx/1lB+r8sgkFmfzha4q/xdHIP6kUZLrhIJLAf9g6SJb ZAsP/Wl0T7buGitYz0ylpCvPS8LMq87GHwvbnm4f4jtGk/99JkovR/VqzyD259tE/gzW OZ9BFQk5Shohi8besWwyXARkVkg05a21WCbUOlxYnMGm8OwjLRcCMUCtqbZZeHBDxLw0 VSWNPx4BOqtDmLVO8p/hDKrkWTIW7ndmgr1D6EZR1NppRquCzyoq1uJl8GN6kf0shJg0 YOo0SDmAzXAXlNYnrlxVB3quc+fAe2QECaLyci8m/BF2GGENMzSu6N4yaA309wu3oV/m 2YMw== X-Forwarded-Encrypted: i=1; AJvYcCU5vZjx15Va8ePw4pIQaY3jkweoHaAoyXlY7h6D0NO3pJnp4HIZQ6CFZFV2t6WhTKHZPX4uwBAtXNknEQ4=@vger.kernel.org X-Gm-Message-State: AOJu0YxkuUoWdnrsTtC29x/dB++gNhJZOaeS2Nq5ItaLPhdgMvveipeQ sGBQIOhb78m7oDtB6d+GnG/MYEIUvLCacByES6Fddue/bdfu8vCPO0yj8Z9o+G2QqrmCtDYryKX aew== X-Google-Smtp-Source: AGHT+IGfs7gXN6qWH4tGlZzsztZguerRTLfMi+oiBPqhIYFJkNIRpCAXsHN1MLCUuyCoUdzMs9CiDW0dVcI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:90a:bc85:b0:2d8:94d4:5845 with SMTP id 98e67ed59e1d1-2d894d4658amr1195a91.0.1725063381728; Fri, 30 Aug 2024 17:16:21 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:35 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-21-seanjc@google.com> Subject: [PATCH v2 20/22] KVM: x86/mmu: Subsume kvm_mmu_unprotect_page() into the and_retry() version From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Fold kvm_mmu_unprotect_page() into kvm_mmu_unprotect_gfn_and_retry() now that all other direct usage is gone. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 1 - arch/x86/kvm/mmu/mmu.c | 33 +++++++++++++-------------------- 2 files changed, 13 insertions(+), 21 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 4aa10db97f6f..0fbde3ca8d1a 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -2134,7 +2134,6 @@ int kvm_get_nr_pending_nmis(struct kvm_vcpu *vcpu); =20 void kvm_update_dr7(struct kvm_vcpu *vcpu); =20 -int kvm_mmu_unprotect_page(struct kvm *kvm, gfn_t gfn); bool __kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu, gpa_t cr2_or= _gpa, bool always_retry); =20 diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index aabed77f35d4..d042874b0a3b 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2695,27 +2695,12 @@ void kvm_mmu_change_mmu_pages(struct kvm *kvm, unsi= gned long goal_nr_mmu_pages) write_unlock(&kvm->mmu_lock); } =20 -int kvm_mmu_unprotect_page(struct kvm *kvm, gfn_t gfn) -{ - struct kvm_mmu_page *sp; - LIST_HEAD(invalid_list); - int r; - - r =3D 0; - write_lock(&kvm->mmu_lock); - for_each_gfn_valid_sp_with_gptes(kvm, sp, gfn) { - r =3D 1; - kvm_mmu_prepare_zap_page(kvm, sp, &invalid_list); - } - kvm_mmu_commit_zap_page(kvm, &invalid_list); - write_unlock(&kvm->mmu_lock); - - return r; -} - bool __kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu, gpa_t cr2_or= _gpa, bool always_retry) { + struct kvm *kvm =3D vcpu->kvm; + LIST_HEAD(invalid_list); + struct kvm_mmu_page *sp; gpa_t gpa =3D cr2_or_gpa; bool r =3D false; =20 @@ -2727,7 +2712,7 @@ bool __kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcp= u *vcpu, gpa_t cr2_or_gpa, * positive is benign, and a false negative will simply result in KVM * skipping the unprotect+retry path, which is also an optimization. */ - if (!READ_ONCE(vcpu->kvm->arch.indirect_shadow_pages)) + if (!READ_ONCE(kvm->arch.indirect_shadow_pages)) goto out; =20 if (!vcpu->arch.mmu->root_role.direct) { @@ -2736,7 +2721,15 @@ bool __kvm_mmu_unprotect_gfn_and_retry(struct kvm_vc= pu *vcpu, gpa_t cr2_or_gpa, goto out; } =20 - r =3D kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa)); + r =3D false; + write_lock(&kvm->mmu_lock); + for_each_gfn_valid_sp_with_gptes(kvm, sp, gpa_to_gfn(gpa)) { + r =3D true; + kvm_mmu_prepare_zap_page(kvm, sp, &invalid_list); + } + kvm_mmu_commit_zap_page(kvm, &invalid_list); + write_unlock(&kvm->mmu_lock); + out: if (r || always_retry) { vcpu->arch.last_retry_eip =3D kvm_rip_read(vcpu); --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 826BE11712 for ; Sat, 31 Aug 2024 00:16:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063385; cv=none; b=HGMBjuRt6BliyIBA80InImuz3vp/z61o7ovKoslPmKO+YgpDdj0ANgzzAYP0RlM75cC1epmztBNnyzsSeNwwuixBYVnK9F+F5lPTKvOxDVNjhrCNI9XAVF+ddpMCIp4wPi6XRNwJNuVcS8SDopYByWUWQZRvtf6aWlwyxoXPp3M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063385; c=relaxed/simple; bh=0jH6aGUfJ6nXVSowsfjSHGcTxPjzpE2W+Gg/9Z921x8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=lKv5SEM0x6yBdPqsaRJsAt/stUKp52sPi8czfAFUOeIiAOK6Cs1XcUhloQyBiknuO5ZvgcILJ/vUJxbb3rqLvA6PzEDOI8ZM7NdGIIfLAr73ocbOjyF1m3SnnLRyAjMylpaqCqaBSlvOBch/alzfiMZsitVCTUzqQcjAHXAGolg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Mfp6PDTM; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Mfp6PDTM" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-7cd93173175so1272721a12.1 for ; Fri, 30 Aug 2024 17:16:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063384; x=1725668184; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=DYB5QhEfP12JFKsEt0nbZ8L014oKTm3YJf7/Et6tdYI=; b=Mfp6PDTMjZtwF95ui7Km+FWChJKeWWZT+yE1ZWOqar5RRPsgXglDh7YnKZ8e1tUCdS Z1YKJw9s4gmLmacMPNxmIhhSyT91I3w6EkkD2HKMwaE7/HIzS3PBsPXgR2lT6XllzoQ8 wy3Um6tjd+5x7vh+J+57J0YS7Ktjjb9LAhZ5IoAJTjg8VqN39mTZ1j9F1xHU6dAG6qXj jsHPOd5/6GdpTV0ljohtsWiuZFd4PhIs1NZkBrHGoJznxYKVXI2cB4a3iScfLu5d/IL6 i7fvHI3HuFxjj4PWMUTAVXp2o2vMJlhLhD2c9Ge0hhVi7xnIF5FVkhl4LnqK0njd1jCj 0HKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063384; x=1725668184; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=DYB5QhEfP12JFKsEt0nbZ8L014oKTm3YJf7/Et6tdYI=; b=SD2xWnllXQG4XENxLqIxGSQDk7tpEL3qdhB6V51oLwGZYSr1fiL7GkcIukpqRF7Msl HWbuxc4ypAV76FG7RmUL3DzkTbDaKsedRPmYlKANMkC43+uJYF3C6cFJpxYDtubzb0/k G8Gy1gIBrEpzjzIuUpWoDaLge3pLQ30XGB7DIyCZXmJ7Gpke6VK/3dEdNsEkYUg/aAri XMUGeRQZXT4+edUWEAMXXbqHJaYdHsduAkOgLTTcVw8TP3SyX+8o5tgJEA/Gsyr6Xn1E qwrUhUbiRdkWtjVZ1jsQooYtoCtJt6XiMOY988PDn9ooW5k/fbyDQPh8+US6dBNCwoue hhqw== X-Forwarded-Encrypted: i=1; AJvYcCXR/SDHKEtiT+brz5pg5cFHBwnS1FmyQgh1wqabHEa2S3866crCWHGR7Pc/yHglxDbFfZxaTXVMhgTxQVQ=@vger.kernel.org X-Gm-Message-State: AOJu0YxjAtmw8PIJ+AcK0+QZQCFnzab3ULGe2anyvyvL2lZxKFyfWRXN V6nCW0oRFtzqxPhOpMVYIFCS0fqJjzeUoczjjTHQB1suEzGg3HWYdBV/fKxECbqSa7dU7BAhIvc W4A== X-Google-Smtp-Source: AGHT+IG9Caf/Ygx0zgVnN7lUuRoljiEi/rAgaAN7Ize+jtz+IuGMkKmmeQheo0RyBKXbT93DJK9xVvlhHb4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:e5c7:b0:1fb:716e:819e with SMTP id d9443c01a7336-20527669412mr2026595ad.4.1725063383865; Fri, 30 Aug 2024 17:16:23 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:36 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-22-seanjc@google.com> Subject: [PATCH v2 21/22] KVM: x86/mmu: Detect if unprotect will do anything based on invalid_list From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Explicitly query the list of to-be-zapped shadow pages when checking to see if unprotecting a gfn for retry has succeeded, i.e. if KVM should retry the faulting instruction. Add a comment to explain why the list needs to be checked before zapping, which is the primary motivation for this change. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index d042874b0a3b..be5c2c33b530 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2721,12 +2721,15 @@ bool __kvm_mmu_unprotect_gfn_and_retry(struct kvm_v= cpu *vcpu, gpa_t cr2_or_gpa, goto out; } =20 - r =3D false; write_lock(&kvm->mmu_lock); - for_each_gfn_valid_sp_with_gptes(kvm, sp, gpa_to_gfn(gpa)) { - r =3D true; + for_each_gfn_valid_sp_with_gptes(kvm, sp, gpa_to_gfn(gpa)) kvm_mmu_prepare_zap_page(kvm, sp, &invalid_list); - } + + /* + * Snapshot the result before zapping, as zapping will remove all list + * entries, i.e. checking the list later would yield a false negative. + */ + r =3D !list_empty(&invalid_list); kvm_mmu_commit_zap_page(kvm, &invalid_list); write_unlock(&kvm->mmu_lock); =20 --=20 2.46.0.469.g59c65b2a67-goog From nobody Fri Dec 19 07:17:35 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C71C516DC01 for ; Sat, 31 Aug 2024 00:16:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063388; cv=none; b=hlRf0DBYOx5NkJsqzPUQBnSBKQvwfoaqpvOWS3KfgSdwl/7sUL/1WYvksUzIArWvH8GgX9PzEYqEDcAfdEVsSE1i4FIulDgYdCRlGbbnsvrx+JSyHvuPWW1qvpfaz1SooJeK0GmMDswLZjWc5LqOsMhGdiokUvN/vENJuCgPUjk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725063388; c=relaxed/simple; bh=wP2W7KNm4uaZDYWeNWlAdBMglkxhSfv/7EU+XUEuEpI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=q4fX4kwjUd3aKFefiivmao3t2/qYHahk9s7aKT1qwGx2MdbelPpoKtnct96mz4xDRjQgE6lJo8zDDKXNZqCLGWXvJ2LNxIA0XJeDs5d+rCnVuwsufjd4c30yr44UCNFRUX4SegxyCPkqgG54wKP3P6bPJuyk0znoAy2lVeP9Tn8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Ksu3PKTO; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Ksu3PKTO" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-205428e6facso5173465ad.2 for ; Fri, 30 Aug 2024 17:16:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725063386; x=1725668186; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=gP7K1Sev7aMPsK1RhVpHEhmgUKxxDPKWFSHlKDQHYWk=; b=Ksu3PKTOOO4mxuAo6jd9hV464GRAdKdPpv/NZVShb8FV+EIBJH7n4cduy6DLOAtgyc Je3KWlcDmVEHo2gI7bj/zw96UV0EDNiXNh+foPKA54kr07ZVkC/fy672Jqp0vLbqNnXh UgFZk4Zvu5IG5kq2AoHGtO/4gUCO0/3/c5IwzB7lUw/taTvqBjVPurLsSYai+P2E3D+k lKA9ubIk7QKIper7UQJdo5dUvaYoTPFUSLZPssWgF8xLESR96XQOSjNUyRawOAEc7AJu NCpVVklH0zp5OPCwf9VqqyybNFfgND5eJIXFgpZm6t/bKQMef8DSHCI6d3R1CizKvoth zt/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725063386; x=1725668186; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=gP7K1Sev7aMPsK1RhVpHEhmgUKxxDPKWFSHlKDQHYWk=; b=ZO070hTfGNhJn2Oi/OvPfgskQnNgocnkmEtBSb/EyDLwlOy5F3FQLGmsDH41ZISIyJ hHNCy25LPRGXEWHkp61uZXznlrSYdLcRje7X2axtRIBYmN9sS6xQyk4igS5ouZ2fCW5F 6SCXs9ZGMiU9xmnT6nev/vgsGcHrcfQ4zQeOtRZXM4zvTTwuKnXV/5liuBwJUaQHHO52 f+aYmHoJMuMa4YctP6q8UXs1fz0oiQHeRz1RKCm0K898Xpk3943aZkN2kk1r+82au5Eu OjHOATiBa2nk7eyuwZH8HfDeKK1VcstC9t/jkKamVnc9KBC6q60ut55g6Ab3GM1eUujx 1+ow== X-Forwarded-Encrypted: i=1; AJvYcCU2M69GOskMLCLa1qvKQFwnVeU96tU9CBYskeEGqbA1TDs6zm8DKxbrAK7t+VvJl2AZLvsZocYyAIxATXo=@vger.kernel.org X-Gm-Message-State: AOJu0YyxH+ewshNg+weOkLv/WXKgDrNYnxHuiVTGwSH3uuTBUziFbckC zCuFhUoj/4TyjEzonNQ9IyC+7mSCBGCfqdDfXao57VX+0b51J9ZfjDfXZW8lWgrCuXGD6v72JF8 dTw== X-Google-Smtp-Source: AGHT+IETKE6MP4mUQ/pQIHpSlTqVQLzv82JDtckTBZhGegQc2MolI+RlmMC71D58MpTAuF2igBH8CH7kJwU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:fb0e:b0:1fb:7f2c:5642 with SMTP id d9443c01a7336-2052764073dmr769545ad.4.1725063386022; Fri, 30 Aug 2024 17:16:26 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 30 Aug 2024 17:15:37 -0700 In-Reply-To: <20240831001538.336683-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240831001538.336683-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240831001538.336683-23-seanjc@google.com> Subject: [PATCH v2 22/22] KVM: x86/mmu: WARN on MMIO cache hit when emulating write-protected gfn From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Yuan Yao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" WARN if KVM gets an MMIO cache hit on a RET_PF_WRITE_PROTECTED fault, as KVM should return RET_PF_WRITE_PROTECTED if and only if there is a memslot, and creating a memslot is supposed to invalidate the MMIO cache by virtue of changing the memslot generation. Keep the code around mainly to provide a convenient location to document why emulated MMIO should be impossible. Suggested-by: Yuan Yao Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index be5c2c33b530..c9cea020aad6 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -5990,6 +5990,18 @@ static int kvm_mmu_write_protect_fault(struct kvm_vc= pu *vcpu, gpa_t cr2_or_gpa, vcpu->arch.last_retry_eip =3D 0; vcpu->arch.last_retry_addr =3D 0; =20 + /* + * It should be impossible to reach this point with an MMIO cache hit, + * as RET_PF_WRITE_PROTECTED is returned if and only if there's a valid, + * writable memslot, and creating a memslot should invalidate the MMIO + * cache by way of changing the memslot generation. WARN and disallow + * retry if MMIO is detected, as retrying MMIO emulation is pointless + * and could put the vCPU into an infinite loop because the processor + * will keep faulting on the non-existent MMIO address. + */ + if (WARN_ON_ONCE(mmio_info_in_cache(vcpu, cr2_or_gpa, direct))) + return RET_PF_EMULATE; + /* * Before emulating the instruction, check to see if the access was due * to a read-only violation while the CPU was walking non-nested NPT @@ -6031,17 +6043,15 @@ static int kvm_mmu_write_protect_fault(struct kvm_v= cpu *vcpu, gpa_t cr2_or_gpa, return RET_PF_RETRY; =20 /* - * The gfn is write-protected, but if emulation fails we can still - * optimistically try to just unprotect the page and let the processor + * The gfn is write-protected, but if KVM detects its emulating an + * instruction that is unlikely to be used to modify page tables, or if + * emulation fails, KVM can try to unprotect the gfn and let the CPU * re-execute the instruction that caused the page fault. Do not allow - * retrying MMIO emulation, as it's not only pointless but could also - * cause us to enter an infinite loop because the processor will keep - * faulting on the non-existent MMIO address. Retrying an instruction - * from a nested guest is also pointless and dangerous as we are only - * explicitly shadowing L1's page tables, i.e. unprotecting something - * for L1 isn't going to magically fix whatever issue cause L2 to fail. + * retrying an instruction from a nested guest as KVM is only explicitly + * shadowing L1's page tables, i.e. unprotecting something for L1 isn't + * going to magically fix whatever issue caused L2 to fail. */ - if (!mmio_info_in_cache(vcpu, cr2_or_gpa, direct) && !is_guest_mode(vcpu)) + if (!is_guest_mode(vcpu)) *emulation_type |=3D EMULTYPE_ALLOW_RETRY_PF; =20 return RET_PF_EMULATE; --=20 2.46.0.469.g59c65b2a67-goog