From nobody Mon Nov 25 19:51:37 2024 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4EC281CFAC for ; Wed, 28 Feb 2024 02:41:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088114; cv=none; b=OjNyTwx8y5LdjgT2rl47rdfBzqRA7XKBDls8OqFC5ckVXS4C+9UIwJ9qcJeQpmZGZMG9CmMxzH4Vjn94H9xTBrppXKDjDdhO4DDYmeEfqfJSFyXJNHdpkoTp5u91o1BA4qZyU21N2WYDkank9y7ZZLlk3nqW5IZHqxwu/by0sAY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088114; c=relaxed/simple; bh=0nYnhkfXEoXgiOzmvAD6QsOU4Ex9riioYW0HkqLKI3I=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=hnMgwFSSD0R32rqz796HHIT8rK+plpr8j1pkd7agnbtMhwgqD29l6mUrQrwwNI1ks1w8yA5+4KECfyoZGeIaeWBF0GhbFG/cE6NUryWI/AzE2gPsRm5tNlMMWh+XTCOSgvjhC7kO5lsyIgYDNAl3f98rciqlkktohuMjVoPxyeg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ENgPAUD1; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ENgPAUD1" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-1dc157e4778so45668085ad.0 for ; Tue, 27 Feb 2024 18:41:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709088112; x=1709692912; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=yZ0WtCwVDO8+ewSeZMQs1E3cZ245daVAR5Xu2LLMu24=; b=ENgPAUD1amsVNQu56qAD34wlUMNbFKf4FUuqYOieghOpiD6XxA9B3ujNsnRkzk8BGv k+7q8++BCQD+M9Sad+Qqbl3/dMOd0b29tnWiLI+l8/A/fh73Xf2KURapAtkVz4QWI/7F 41AAxtd2MA+TARP2Q/RW9KcYpjurGnExevLPkEcXNxFK8KyDINeLWtACErB78A1mLzmZ e+xpy6nNyolRoZLUlC5PeBe20brCbSnm7gIc0cKEoHL3m7tLExb5K9g75RGv6xxuRsIb pFX1O2PYJOQfJ2x4XpcJT/rdq2k6AxOkSMtMMRTToMbbUruZ1RJVjjAkH5UZQLSJ+RYo GEgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709088112; x=1709692912; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=yZ0WtCwVDO8+ewSeZMQs1E3cZ245daVAR5Xu2LLMu24=; b=ePVBFikoqNfu2H4yKB78+NkMhNk87L38RZaIbXIhg10NNQbZBcCLYvZrXdnxyIRpz1 DGu5Q8rBNwiajSUXXdWiTU0yVWMlRdwWsY8L+Pm7RRReIcWsyd0/j6AJSNP3bwWUnY0b gzHgsZbMJy0c3xs0Y5oRhCuh48Uon6u7SSZthDdTPiXb/fVyns2xF/AOYg8oRtZZZW6z P7EIgBbwpEc+q+RS/cSeXrKyMBSyxVOOFXWT8u2K4mg/+v7gERbQ3iyU8VmWdk8CI54T PPEa4qC4GXGRZ83Xtq4SG+Eim3OOCy3eJht7WrHWz25x5oofd2oWDP1AnjWWpLxp02y8 zcjA== X-Forwarded-Encrypted: i=1; AJvYcCWufI5mNlnPYbqoNz0gnNtIJ2Qfpix1SV8kBigVTcFPHpWbYfWD/tAPA5ETbI6he81d07lJUUfFHmGfE4G1WCr+nfXgSdEPdebOZw8Y X-Gm-Message-State: AOJu0Ywv58T2E6n5UPU8kfq3/u6SxyEIblstsiQGhSV6nNVAUC1YZJhX cRlcGwVYvOy/BL6UUj3ED+4wLiPIdx584acffQ+0jBE6ipmTkg/283ZdRIlV9fBnErU+xjHA9Kz IIA== X-Google-Smtp-Source: AGHT+IHT+n9eH6+34c02CKv+xvYBYlsjdHG/QXVBuMrn39hwmZ9Kl8Uiyv0qzfOxyiZ/4SfXMTOE04r4lGw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:dac9:b0:1dc:b424:56e0 with SMTP id q9-20020a170902dac900b001dcb42456e0mr343078plx.3.1709088112680; Tue, 27 Feb 2024 18:41:52 -0800 (PST) Reply-To: Sean Christopherson Date: Tue, 27 Feb 2024 18:41:32 -0800 In-Reply-To: <20240228024147.41573-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240228024147.41573-1-seanjc@google.com> X-Mailer: git-send-email 2.44.0.278.ge034bb2e1d-goog Message-ID: <20240228024147.41573-2-seanjc@google.com> Subject: [PATCH 01/16] KVM: x86/mmu: Exit to userspace with -EFAULT if private fault hits emulation From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yan Zhao , Isaku Yamahata , Michael Roth , Yu Zhang , Chao Peng , Fuad Tabba , David Matlack Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Exit to userspace with -EFAULT / KVM_EXIT_MEMORY_FAULT if a private fault triggers emulation of any kind, as KVM doesn't currently support emulating access to guest private memory. Practically speaking, private faults and emulation are already mutually exclusive, but there are edge cases upon edge cases where KVM can return RET_PF_EMULATE, and adding one last check to harden against weird, unexpected combinations is inexpensive. Suggested-by: Yan Zhao Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 8 -------- arch/x86/kvm/mmu/mmu_internal.h | 13 +++++++++++++ 2 files changed, 13 insertions(+), 8 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index e4cc7f764980..e2fd74e06ff8 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4309,14 +4309,6 @@ static inline u8 kvm_max_level_for_order(int order) return PG_LEVEL_4K; } =20 -static void kvm_mmu_prepare_memory_fault_exit(struct kvm_vcpu *vcpu, - struct kvm_page_fault *fault) -{ - kvm_prepare_memory_fault_exit(vcpu, fault->gfn << PAGE_SHIFT, - PAGE_SIZE, fault->write, fault->exec, - fault->is_private); -} - static int kvm_faultin_pfn_private(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_interna= l.h index 0669a8a668ca..0eea6c5a824d 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -279,6 +279,14 @@ enum { RET_PF_SPURIOUS, }; =20 +static inline void kvm_mmu_prepare_memory_fault_exit(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault) +{ + kvm_prepare_memory_fault_exit(vcpu, fault->gfn << PAGE_SHIFT, + PAGE_SIZE, fault->write, fault->exec, + fault->is_private); +} + static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_o= r_gpa, u32 err, bool prefetch, int *emulation_type) { @@ -320,6 +328,11 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcp= u *vcpu, gpa_t cr2_or_gpa, else r =3D vcpu->arch.mmu->page_fault(vcpu, &fault); =20 + if (r =3D=3D RET_PF_EMULATE && fault.is_private) { + kvm_mmu_prepare_memory_fault_exit(vcpu, &fault); + return -EFAULT; + } + if (fault.write_fault_to_shadow_pgtable && emulation_type) *emulation_type |=3D EMULTYPE_WRITE_PF_TO_SP; =20 --=20 2.44.0.278.ge034bb2e1d-goog From nobody Mon Nov 25 19:51:37 2024 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 837441F95A for ; Wed, 28 Feb 2024 02:41:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088117; cv=none; b=efQ9DhFg7AHb+vMN0FkCf4KjK/vWxh/+j+kzLut31ip0ptUG35ZWtC5kdftafdJZXJMf9dNgVjzxP//Z7ljL2XCJN5d1vDv7pTrFV+PCesccwrCz5t2hR4dx3omggobMhMGZjm5J2MMbbaaxR80Lkvwq7pFiq5rJfUVFv406xZ0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088117; c=relaxed/simple; bh=Pbt+JF3UA/8dMtWQFjeGy4m6XEf0EGAmPbiziPcskhM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=stoRyUFRaPbPnb0x9uC2mhJOuzmuKuaNg2MYxkGkBttR8kAZAVVFqWa2PjsrCDYRd5G3upgPuRz2uq2Hq3nB+NN/646AnXRdjYdXnWzXRqlu+OJJpIvL32NQCSkWU8a34Xuhi/WoXz/d6muXtW64NgYjYrucs2f6ReeH8rY8vR4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=20r5g6XC; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="20r5g6XC" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-dcc58cddb50so8337985276.0 for ; Tue, 27 Feb 2024 18:41:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709088114; x=1709692914; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=SB7VJZrxLLlFGrRBdKBxwQcdc7hfIE4p4aRvNnIMDrw=; b=20r5g6XCXbfOojC6XcQ3lbgZbmsVRuzdMnK74272/AuKfYcHfkoxYhRmYKiof3oA5G usWiH9uzdGmmYy3pJYmZiNg5bOvkc9X6fAGEKBgCBk6jm8jIn3XxKgN9zFwWr4Il72Cr i+Rt3+bSBq237oLQNfQs6Wt9l6lXEqafihoS3xxede4iDo/RY9OxrRrtpzmcA9ASg5ty IFGDSXAbtKotgxYG/wYhbYEubMn1m6grdZ9wteY52qU75P/+fNUz0FDaAuTVVNU9HE9b yITaGc38ktCzMWrtBRXy4HEyuxqUeMlHT3/A1YiVSjtt/euafEHkY4vK0UaFdAVr7fZ0 FKng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709088114; x=1709692914; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=SB7VJZrxLLlFGrRBdKBxwQcdc7hfIE4p4aRvNnIMDrw=; b=WOJKGj5SlqK4dvYCJnoxvttIf8ZUBKKB13mszgFPCqQ4lWr2QBd8Aa1p02cjXegoAy qU7C6cK7xZIuy5s5hsPhTygmoft1hf87FvpIkzsDQsV6M1WBsYp7EOyCPCROeGCV7f6g Joy7i1PEkhWZVWS0iXPqiFj1fY39Aytf/9hHxDYy/6Kcu+EgWx2hqqLDC+y4Revuq46z wJxb45O2AIeLnpmklJH8l/gNtQimEMh3T9v7pIhuoHJu0OLmuVpMKqcQ2ZDA2+oU+s3I lqHIMI+RE73wxw6dhaOugq9xo0K00dNovCF9oYSNgNq3nZ5fM7PL9rH0IVtuzYruqQ1h DtYg== X-Forwarded-Encrypted: i=1; AJvYcCVYfR/gNB/hx0N6BUCThFRS/Ojtboj8dwX0evJA8gR/cFmxlitpO2ES44+kx0m4kiNODYPl7j6YwfK6KDwj/HdPdYVTn2/H7kjfRx+z X-Gm-Message-State: AOJu0YwHRKAcown519W4sXVebu5v5Y8/bGXjBKLsmduZ48vpjV/jolWP i+qxoRUQD93+t/vgHxLegVh+6z0gyFMLHDoE/1Kyf3rjJlD1RJ51Q1SYJojmgFJeSiwfezoe+gG Y6g== X-Google-Smtp-Source: AGHT+IEtr1W2Og4WYHDO0Ie9yHqrThkRgrJTuHJgWLs/TyCSU0i16JcQQLlcsTydtmk6DAVKLy5zekb7Irs= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:21c1:0:b0:dcc:e1a6:aca9 with SMTP id h184-20020a2521c1000000b00dcce1a6aca9mr381616ybh.9.1709088114564; Tue, 27 Feb 2024 18:41:54 -0800 (PST) Reply-To: Sean Christopherson Date: Tue, 27 Feb 2024 18:41:33 -0800 In-Reply-To: <20240228024147.41573-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240228024147.41573-1-seanjc@google.com> X-Mailer: git-send-email 2.44.0.278.ge034bb2e1d-goog Message-ID: <20240228024147.41573-3-seanjc@google.com> Subject: [PATCH 02/16] KVM: x86: Remove separate "bit" defines for page fault error code masks From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yan Zhao , Isaku Yamahata , Michael Roth , Yu Zhang , Chao Peng , Fuad Tabba , David Matlack Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Open code the bit number directly in the PFERR_* masks and drop the intermediate PFERR_*_BIT defines, as having to bounce through two macros just to see which flag corresponds to which bit is quite annoying, as is having to define two macros just to add recognition of a new flag. Use ilog2() to derive the bit in permission_fault(), the one function that actually needs the bit number (it does clever shifting to manipulate flags in order to avoid conditional branches). No functional change intended. Signed-off-by: Sean Christopherson Reviewed-by: Paolo Bonzini --- arch/x86/include/asm/kvm_host.h | 32 ++++++++++---------------------- arch/x86/kvm/mmu.h | 4 ++-- 2 files changed, 12 insertions(+), 24 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index aaf5a25ea7ed..88cc523bafa8 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -254,28 +254,16 @@ enum x86_intercept_stage; KVM_GUESTDBG_INJECT_DB | \ KVM_GUESTDBG_BLOCKIRQ) =20 - -#define PFERR_PRESENT_BIT 0 -#define PFERR_WRITE_BIT 1 -#define PFERR_USER_BIT 2 -#define PFERR_RSVD_BIT 3 -#define PFERR_FETCH_BIT 4 -#define PFERR_PK_BIT 5 -#define PFERR_SGX_BIT 15 -#define PFERR_GUEST_FINAL_BIT 32 -#define PFERR_GUEST_PAGE_BIT 33 -#define PFERR_IMPLICIT_ACCESS_BIT 48 - -#define PFERR_PRESENT_MASK BIT(PFERR_PRESENT_BIT) -#define PFERR_WRITE_MASK BIT(PFERR_WRITE_BIT) -#define PFERR_USER_MASK BIT(PFERR_USER_BIT) -#define PFERR_RSVD_MASK BIT(PFERR_RSVD_BIT) -#define PFERR_FETCH_MASK BIT(PFERR_FETCH_BIT) -#define PFERR_PK_MASK BIT(PFERR_PK_BIT) -#define PFERR_SGX_MASK BIT(PFERR_SGX_BIT) -#define PFERR_GUEST_FINAL_MASK BIT_ULL(PFERR_GUEST_FINAL_BIT) -#define PFERR_GUEST_PAGE_MASK BIT_ULL(PFERR_GUEST_PAGE_BIT) -#define PFERR_IMPLICIT_ACCESS BIT_ULL(PFERR_IMPLICIT_ACCESS_BIT) +#define PFERR_PRESENT_MASK BIT(0) +#define PFERR_WRITE_MASK BIT(1) +#define PFERR_USER_MASK BIT(2) +#define PFERR_RSVD_MASK BIT(3) +#define PFERR_FETCH_MASK BIT(4) +#define PFERR_PK_MASK BIT(5) +#define PFERR_SGX_MASK BIT(15) +#define PFERR_GUEST_FINAL_MASK BIT_ULL(32) +#define PFERR_GUEST_PAGE_MASK BIT_ULL(33) +#define PFERR_IMPLICIT_ACCESS BIT_ULL(48) =20 #define PFERR_NESTED_GUEST_PAGE (PFERR_GUEST_PAGE_MASK | \ PFERR_WRITE_MASK | \ diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index 60f21bb4c27b..e8b620a85627 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -213,7 +213,7 @@ static inline u8 permission_fault(struct kvm_vcpu *vcpu= , struct kvm_mmu *mmu, */ u64 implicit_access =3D access & PFERR_IMPLICIT_ACCESS; bool not_smap =3D ((rflags & X86_EFLAGS_AC) | implicit_access) =3D=3D X86= _EFLAGS_AC; - int index =3D (pfec + (not_smap << PFERR_RSVD_BIT)) >> 1; + int index =3D (pfec + (not_smap << ilog2(PFERR_RSVD_MASK))) >> 1; u32 errcode =3D PFERR_PRESENT_MASK; bool fault; =20 @@ -235,7 +235,7 @@ static inline u8 permission_fault(struct kvm_vcpu *vcpu= , struct kvm_mmu *mmu, =20 /* clear present bit, replace PFEC.RSVD with ACC_USER_MASK. */ offset =3D (pfec & ~1) + - ((pte_access & PT_USER_MASK) << (PFERR_RSVD_BIT - PT_USER_SHIFT)); + ((pte_access & PT_USER_MASK) << (ilog2(PFERR_RSVD_MASK) - PT_USER_SHIFT= )); =20 pkru_bits &=3D mmu->pkru_mask >> offset; errcode |=3D -pkru_bits & PFERR_PK_MASK; --=20 2.44.0.278.ge034bb2e1d-goog From nobody Mon Nov 25 19:51:37 2024 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 580B12032C for ; Wed, 28 Feb 2024 02:41:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088118; cv=none; b=sDGxqBLPjach3qPJzfejTM4uU9SnNKFuYLGv1sfbXl+lljAULS+MHbc8m7fYPyqAa0IeXiZeUTMFb8j80NPQfL+V31enkfQ9LRnUBf8aZVFurP++MkLKZo7ab3jEgy2DxSyw5srX2iNwS+nD1udhnuj4SeIp9gh9+3e2my+rxSU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088118; c=relaxed/simple; bh=1Uryw7EIyGaADMbLf4vJNKsPqM+n7z9iSFQFR6i45EA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=P39aJS30NGrThQgaUhR8Q1FiabkCoSnRSPKvhRuEMRs3nw5es0IKWbuhFdq+d8XR1r6AX1cFzmG8AxfgqsaJqtNt/xzw9LjAff2SijS4en+5WWv75OQ5vXo5VaFL7pKf1hLIV7BkU9TyXeho+9QxvkQ13DPBCTG2Qt04h5d36js= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=VzOej303; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="VzOej303" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-dced704f17cso8087216276.1 for ; Tue, 27 Feb 2024 18:41:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709088116; x=1709692916; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=5/L3RUGH7+uAkBQWkVD/RBnetvTxq10PRbhVuv1Z2Es=; b=VzOej3036cWOSrvGRqAPgWv+JRFfg3UqbQbz7+hR+4Xemu0wsHujaiVIcyXVY5LpQ2 wG3YWgkutM/LxE5TSOTdY9TH2uI8+2q1sjiH413EJVakMlLz7CKbb1nT8Aw5xxdzvtXH HF5SckaMUWXFu4RteSnflqvKLchokpKsoAzVqKsqbrh52AX0ZuE6A3jwjbFgQ5Fh78wp xaUO5yO4JBPBUG3+RT+tuZpv49+Zv4U5UU18fyx2HFXJGDRfUQaUcjGX5baSFYAT6Ufv xNo9SrKH2LJgLH/5MQPJaB0t2Z7bdnvJz/OzolCk7oDEfwOuviut0uKLmf2jyXodz9+o IJuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709088116; x=1709692916; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=5/L3RUGH7+uAkBQWkVD/RBnetvTxq10PRbhVuv1Z2Es=; b=VYz95Lyx+0U8xSp/YnO6UT6uHLzI9wjAeDSIfdXccz440X+2N5dEXaC5D9DBdXRoBL wFBdM2V3pE+dnvOezlg2XyS/+UY55L/w7umtjR1fYhtePLubZbOvM7aPIa16m4CFNoNV ZKng3BsZeAd++XeA7Ch8diBNoZw4dZR8ZKWsUxhiKK0ozJbW0X0JdMS5d8teRlLV5G8b lmtjwnTTq0xQ+XHBeFQx5lJ0hcc6364dRrI+CKeOZazLPeXoqoOew5rv+7pCY90pLyZJ VPxDh1xr4xNACe3DtxVRV6vuNPwXnHKqUPRZk6qs0qhR7EzoQLfTUKUF/3AgeNXOaTBs 5S4Q== X-Forwarded-Encrypted: i=1; AJvYcCX7+kYNMfn+sGyL2bFIBlQMklUz/ie5fKfuRHmlzd9wCRItvtDI5wT9M4HV44h4gt0YQ2rSt/AFy3cy6Qmap2rjswKaKNNrhJK16+Cf X-Gm-Message-State: AOJu0YziJeKnur+zVkIzD60jJwpBOdxLbXwnEJXEI/gdaBP+o63parhD hSj1kd4WbbEE0hW4403IKDRhGnQrXBWqvkGswzl6YOZV9PQSpUJCHw61fv1voB00XOvzJHYY5Mq RHw== X-Google-Smtp-Source: AGHT+IHHsJM5lRH1Wgjj5L2FJEo06HzdA8kv6SKdx8rA1dzoB6Dy3aQiZHQWtax/vHxL1ZgCk9HefZp45BM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6902:1001:b0:dcd:b593:6503 with SMTP id w1-20020a056902100100b00dcdb5936503mr102444ybt.2.1709088116351; Tue, 27 Feb 2024 18:41:56 -0800 (PST) Reply-To: Sean Christopherson Date: Tue, 27 Feb 2024 18:41:34 -0800 In-Reply-To: <20240228024147.41573-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240228024147.41573-1-seanjc@google.com> X-Mailer: git-send-email 2.44.0.278.ge034bb2e1d-goog Message-ID: <20240228024147.41573-4-seanjc@google.com> Subject: [PATCH 03/16] KVM: x86: Define more SEV+ page fault error bits/flags for #NPF From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yan Zhao , Isaku Yamahata , Michael Roth , Yu Zhang , Chao Peng , Fuad Tabba , David Matlack Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Define more #NPF error code flags that are relevant to SEV+ (mostly SNP) guests, as specified by the APM: * Bit 34 (ENC): Set to 1 if the guest=E2=80=99s effective C-bit was 1, 0= otherwise. * Bit 35 (SIZEM): Set to 1 if the fault was caused by a size mismatch betw= een PVALIDATE or RMPADJUST and the RMP, 0 otherwise. * Bit 36 (VMPL): Set to 1 if the fault was caused by a VMPL permission check failure, 0 otherwise. * Bit 37 (SSS): Set to VMPL permission mask SSS (bit 4) value if VmplSSS= is enabled. Note, the APM is *extremely* misleading, and strongly implies that the above flags can _only_ be set for #NPF exits from SNP guests. That is a lie, as bit 34 (C-bit=3D1, i.e. was encrypted) can be set when running _any_ flavor of SEV guest on SNP capable hardware. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 88cc523bafa8..1e69743ef0fb 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -261,8 +261,12 @@ enum x86_intercept_stage; #define PFERR_FETCH_MASK BIT(4) #define PFERR_PK_MASK BIT(5) #define PFERR_SGX_MASK BIT(15) +#define PFERR_GUEST_RMP_MASK BIT_ULL(31) #define PFERR_GUEST_FINAL_MASK BIT_ULL(32) #define PFERR_GUEST_PAGE_MASK BIT_ULL(33) +#define PFERR_GUEST_ENC_MASK BIT_ULL(34) +#define PFERR_GUEST_SIZEM_MASK BIT_ULL(35) +#define PFERR_GUEST_VMPL_MASK BIT_ULL(36) #define PFERR_IMPLICIT_ACCESS BIT_ULL(48) =20 #define PFERR_NESTED_GUEST_PAGE (PFERR_GUEST_PAGE_MASK | \ --=20 2.44.0.278.ge034bb2e1d-goog From nobody Mon Nov 25 19:51:37 2024 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EAD7120B0E for ; Wed, 28 Feb 2024 02:41:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088121; cv=none; b=n/vnPZhBwI6KQFXIPsy2EOC794bhbq7lNhPUSBzyqL+LnWPw4bqOeu9IPDIsfYnffzaXn7g1P6Ay+pNXJEk8jsHCDCCjA0hD+eI8MvFkPtJOZeOFO9N6Mnyq55wG+fjSbwfkSygAAZdbYMGI1Yne5l1Tam6hegTMMXJQBIZCrNc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088121; c=relaxed/simple; bh=Ee4P1IGzSZAbqqoC45jHzPN8hF/0X+AxCf4vkLrZ8oo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Hee2S/uY15JjWQ0c7VFkZXZGdSqEikzc9H4zeDQkDP5RljzftsNvAauaNHBMrUr3Yw8NUDlQ+7RPgRbh1Ndxx3Fnj91ztIyoXLT5tn9D/pZFLCKQqxuKVjHzWpAYOV8tbEq/0Z5or4IjgHORefP6p6fH3JSX2pr1Fl3aMguJM2w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=gApBg6m9; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="gApBg6m9" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-608d6ffc64eso7155487b3.0 for ; Tue, 27 Feb 2024 18:41:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709088118; x=1709692918; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=vbRF992kdFc9tiyNuxItAbgdPRKbA7Cju3slwI3hgOA=; b=gApBg6m9/n4OByfbEvr+LNxa1FXhXQXl++lerlpZbBu/TZaJKN/Wrv5NAx6Y8OUYkn cEOnoUNmcEOBeR0upfproGY+rxMMBYjz+Q3gWoVVO0Cyc0l1n6BLT66+7GbK+6b16uez XvfylwSerawHTVsavbLcmWa14F7e4MqBiHV2eYY3M+M+pPlnJKVSHbDJGqmAIbOjDPug KKnSZueNOidQFLClvMQ4kdR8Pe25MkkKGQobd8efDml/+ZRkFjnkiPr4ary6U49yZr2n zMHJvS2ywYF4JwfZz2TPHZ0b4mouT7UoVQkXiZJRJUcbiYMYwueersvLCHGUPePJUk1R HgZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709088118; x=1709692918; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=vbRF992kdFc9tiyNuxItAbgdPRKbA7Cju3slwI3hgOA=; b=XEMcqCaIxj961uuESMHUi2DwFilyl/gvcekU+GlMSpptBeGEpTZqdzvG5afW7CFevM VDbUWOtm+Z2Joi6Mkr4E681zq7JgNb0JCewBh+Kj2Mo0YIWxvTVquWxu4V6/m4zL0owB I22jSjvKcKzuxML+hrucDYJ4D+Yc9F0RtafokIcrG18+2IbiAyS2XCLRxGVUiP++TyoB KQ/tIN888DFQaNhcnyCjk5yZMAqBucqvE4eWA19R8i/TsXdOth1TCMzEoka2r39Zis2o Jbmf3x1pdOUeijJ00n4lPmqQOBXZY0W8A6PH2hEYG2vnzhMvERLZ+5SgZLe84xVyZTHA Ebxw== X-Forwarded-Encrypted: i=1; AJvYcCVmGDYAYucsmQ+LOkQqVymxKc8dG3s083wbI8H4XmroDDTmwNiAzlW1XJ0g8t97OrZ6WT38gKKwPk3ddplXkEcaoTV8TNmuMiVH1cC8 X-Gm-Message-State: AOJu0Yz1WEmtTyNkXALgkHkXAvWVpIyxkbjSc+bLHvYA5xZy/nLRivk4 bpx5qii63uz0mx79AnBFjyFk+ZXgnUg0SLrzueeC/oqPzDmum8dVm3vl3Alb6645I6TTCvWbP4n wEw== X-Google-Smtp-Source: AGHT+IHCnZcr7IGx9UCKZXWtKEIsgWhnDoq9inJyUaTHMdVOsz14wV7APi/MVh0ZipKVxAEEBedkFmN5nhU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a81:4e8f:0:b0:608:ff12:4155 with SMTP id c137-20020a814e8f000000b00608ff124155mr292352ywb.0.1709088118014; Tue, 27 Feb 2024 18:41:58 -0800 (PST) Reply-To: Sean Christopherson Date: Tue, 27 Feb 2024 18:41:35 -0800 In-Reply-To: <20240228024147.41573-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240228024147.41573-1-seanjc@google.com> X-Mailer: git-send-email 2.44.0.278.ge034bb2e1d-goog Message-ID: <20240228024147.41573-5-seanjc@google.com> Subject: [PATCH 04/16] KVM: x86/mmu: Pass full 64-bit error code when handling page faults From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yan Zhao , Isaku Yamahata , Michael Roth , Yu Zhang , Chao Peng , Fuad Tabba , David Matlack Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Isaku Yamahata Plumb the full 64-bit error code throughout the page fault handling code so that KVM can use the upper 32 bits, e.g. SNP's PFERR_GUEST_ENC_MASK will be used to determine whether or not a fault is private vs. shared. Note, passing the 64-bit error code to FNAME(walk_addr)() does NOT change the behavior of permission_fault() when invoked in the page fault path, as KVM explicitly clears PFERR_IMPLICIT_ACCESS in kvm_mmu_page_fault(). Continue passing '0' from the async #PF worker, as guest_memfd() and thus private memory doesn't support async page faults. Signed-off-by: Isaku Yamahata [mdr: drop references/changes on rebase, update commit message] Signed-off-by: Michael Roth [sean: drop truncation in call to FNAME(walk_addr)(), rewrite changelog] Signed-off-by: Sean Christopherson Reviewed-by: Xiaoyao Li --- arch/x86/kvm/mmu/mmu.c | 3 +-- arch/x86/kvm/mmu/mmu_internal.h | 4 ++-- arch/x86/kvm/mmu/mmutrace.h | 2 +- 3 files changed, 4 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index e2fd74e06ff8..408969ac1291 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -5860,8 +5860,7 @@ int noinline kvm_mmu_page_fault(struct kvm_vcpu *vcpu= , gpa_t cr2_or_gpa, u64 err } =20 if (r =3D=3D RET_PF_INVALID) { - r =3D kvm_mmu_do_page_fault(vcpu, cr2_or_gpa, - lower_32_bits(error_code), false, + r =3D kvm_mmu_do_page_fault(vcpu, cr2_or_gpa, error_code, false, &emulation_type); if (KVM_BUG_ON(r =3D=3D RET_PF_INVALID, vcpu->kvm)) return -EIO; diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_interna= l.h index 0eea6c5a824d..1fab1f2359b5 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -190,7 +190,7 @@ static inline bool is_nx_huge_page_enabled(struct kvm *= kvm) struct kvm_page_fault { /* arguments to kvm_mmu_do_page_fault. */ const gpa_t addr; - const u32 error_code; + const u64 error_code; const bool prefetch; =20 /* Derived from error_code. */ @@ -288,7 +288,7 @@ static inline void kvm_mmu_prepare_memory_fault_exit(st= ruct kvm_vcpu *vcpu, } =20 static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_o= r_gpa, - u32 err, bool prefetch, int *emulation_type) + u64 err, bool prefetch, int *emulation_type) { struct kvm_page_fault fault =3D { .addr =3D cr2_or_gpa, diff --git a/arch/x86/kvm/mmu/mmutrace.h b/arch/x86/kvm/mmu/mmutrace.h index ae86820cef69..195d98bc8de8 100644 --- a/arch/x86/kvm/mmu/mmutrace.h +++ b/arch/x86/kvm/mmu/mmutrace.h @@ -260,7 +260,7 @@ TRACE_EVENT( TP_STRUCT__entry( __field(int, vcpu_id) __field(gpa_t, cr2_or_gpa) - __field(u32, error_code) + __field(u64, error_code) __field(u64 *, sptep) __field(u64, old_spte) __field(u64, new_spte) --=20 2.44.0.278.ge034bb2e1d-goog From nobody Mon Nov 25 19:51:37 2024 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7102720DF1 for ; Wed, 28 Feb 2024 02:42:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088122; cv=none; b=OReL+mC1Jh2cuj5dPxZNZOk/zj6PVc/vkjFR4+6sO5MoaD9OPaAPzaMiSLbpxG8XVOL2ROlO+QTVo45vuLqrQ9NHZDMCrralgbugthqbxBB1HWcY291E6ZPbeocfBcMNVRAxddpeZz4UdUXmN4gWSMT1rZlMhXL1We/I0AN+QY8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088122; c=relaxed/simple; bh=fw6fQCaxNyQVmsm0HwKPD91hojXxJNupzj4MzyZI3f0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=aWHVdmWIlgGRXtEGIiK5rsxPtYLj69JChoDVcvnCN0bTB7enMaxl94+Z14CSGOrjNgFl0rs/JsqgoRMWbhzg6VTujyz26eCIZjQctc3cXOXYKdTjWF2kaUlJQ0cQJh8P/7YH2LyNRd7tsBPmtIvSw8fVZMPXLFVa/MSZTmzBTds= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=PcHXFqab; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="PcHXFqab" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-6e5588db705so382405b3a.1 for ; Tue, 27 Feb 2024 18:42:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709088120; x=1709692920; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=UtgfH5mWUwUleAFxMU19P9nYVxmIT/Ru+pHd1FRZC+Q=; b=PcHXFqab6wfr2gPo40/6WpRR5Y4D22zIVOk8I9eZ4ln70tXNUJELztSRICOBRAaDuj B4MxQ0dre4sndiDi0+2aXU8XH6Pk4m7El8Zg1xgypt2hku2NWFXt+eUogAFHduBC3OCI ZVH7M83YfFE3EkfiFgs7pN4oA2lUUio6ot5V6XZvdTlTvsBYvdACnUzioFkbX88/Etrj jUwyIt1tqPPiBb7NvRPwIkQEMhL+7ZZbtBI3ehjguRGd+me4dW7gVxCHGHL7KL7vPcQf mnn1ZMea0ahhprXPuqsO9RawAKZ/gP2VIMESDbk/z7QnzFycYKWtPdm50DcN6odKzIXc KPTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709088120; x=1709692920; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=UtgfH5mWUwUleAFxMU19P9nYVxmIT/Ru+pHd1FRZC+Q=; b=avwowQJT+Dm81MbLYoZtqYwYm84BnWEBpfWn369XwFg6zuwNrTL4VJd6mtyDxx1qM6 jMsiAjr/dLk0rDV+x9fZybMQZJ3Y+vfigCCIYfSy3GX3NIGE4umn4UJke/oqgpJstnXU /XA0359EIX31iAxBBbGBuz00kkqp2iXqjuCZslcCOkQz2BbOsaUqHrS434QTySS9HDN0 BtrcB55Sh4ssLqJUljxMnWsk4z0IONQGc86VCrzF8Nv5StIpyC370wC0BH67Znieztd2 AO3C36BZQm16uh5SvepHR6Ng6KbTpTGSFHHaNjlxtJedPr+/MPxvzRJOF6/3phbw2Y2k WKHQ== X-Forwarded-Encrypted: i=1; AJvYcCWvTtsDeE5gjYSVKgnJ1phbBVAlnKAohLv1WNuRmUf0gwd6oBZtKyvO60P6jynt6rwr7rA1b0v+EY8gxKgLxGuzCrPcpqU44wLUidsY X-Gm-Message-State: AOJu0YzTHJK5NJoGsgmsQcfYCBaKG37OgUa9QuFMfgz8Hdfg7/Fy2Hk7 uaYamWhEjCAkxSGSpM7OB7C+1YzPUyvW+Yug1wyrHqRlobUda+Jj98ddhnIT0VME8ml444PRMqs jDw== X-Google-Smtp-Source: AGHT+IGqmp6QkrShsDKaoHDxyQmUjOsMVIaWkQmrr+uLwfys/m2SNtFhLMK3Ci20GlPI6cF86IvueWYbzkU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6a00:4708:b0:6e5:4142:ea1c with SMTP id df8-20020a056a00470800b006e54142ea1cmr3775pfb.3.1709088119796; Tue, 27 Feb 2024 18:41:59 -0800 (PST) Reply-To: Sean Christopherson Date: Tue, 27 Feb 2024 18:41:36 -0800 In-Reply-To: <20240228024147.41573-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240228024147.41573-1-seanjc@google.com> X-Mailer: git-send-email 2.44.0.278.ge034bb2e1d-goog Message-ID: <20240228024147.41573-6-seanjc@google.com> Subject: [PATCH 05/16] KVM: x86/mmu: Use synthetic page fault error code to indicate private faults From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yan Zhao , Isaku Yamahata , Michael Roth , Yu Zhang , Chao Peng , Fuad Tabba , David Matlack Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add and use a synthetic, KVM-defined page fault error code to indicate whether a fault is to private vs. shared memory. TDX and SNP have different mechanisms for reporting private vs. shared, and KVM's software-protected VMs have no mechanism at all. Usurp an error code flag to avoid having to plumb another parameter to kvm_mmu_page_fault() and friends. Alternatively, KVM could borrow AMD's PFERR_GUEST_ENC_MASK, i.e. set it for TDX and software-protected VMs as appropriate, but that would require *clearing* the flag for SEV and SEV-ES VMs, which support encrypted memory at the hardware layer, but don't utilize private memory at the KVM layer. Opportunistically add a comment to call out that the logic for software- protected VMs is (and was before this commit) broken for nested MMUs, i.e. for nested TDP, as the GPA is an L2 GPA. Punt on trying to play nice with nested MMUs as there is a _lot_ of functionality that simply doesn't work for software-protected VMs, e.g. all of the paths where KVM accesses guest memory need to be updated to be aware of private vs. shared memory. Signed-off-by: Sean Christopherson Reviewed-by: Binbin Wu --- arch/x86/include/asm/kvm_host.h | 11 +++++++++++ arch/x86/kvm/mmu/mmu.c | 26 +++++++++++++++++++------- arch/x86/kvm/mmu/mmu_internal.h | 2 +- 3 files changed, 31 insertions(+), 8 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 1e69743ef0fb..4077c46c61ab 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -267,7 +267,18 @@ enum x86_intercept_stage; #define PFERR_GUEST_ENC_MASK BIT_ULL(34) #define PFERR_GUEST_SIZEM_MASK BIT_ULL(35) #define PFERR_GUEST_VMPL_MASK BIT_ULL(36) + +/* + * IMPLICIT_ACCESS is a KVM-defined flag used to correctly perform SMAP ch= ecks + * when emulating instructions that triggers implicit access. + */ #define PFERR_IMPLICIT_ACCESS BIT_ULL(48) +/* + * PRIVATE_ACCESS is a KVM-defined flag us to indicate that a fault occurr= ed + * when the guest was accessing private memory. + */ +#define PFERR_PRIVATE_ACCESS BIT_ULL(49) +#define PFERR_SYNTHETIC_MASK (PFERR_IMPLICIT_ACCESS | PFERR_PRIVATE_ACCESS) =20 #define PFERR_NESTED_GUEST_PAGE (PFERR_GUEST_PAGE_MASK | \ PFERR_WRITE_MASK | \ diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 408969ac1291..7807bdcd87e8 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -5839,19 +5839,31 @@ int noinline kvm_mmu_page_fault(struct kvm_vcpu *vc= pu, gpa_t cr2_or_gpa, u64 err bool direct =3D vcpu->arch.mmu->root_role.direct; =20 /* - * IMPLICIT_ACCESS is a KVM-defined flag used to correctly perform SMAP - * checks when emulating instructions that triggers implicit access. * WARN if hardware generates a fault with an error code that collides - * with the KVM-defined value. Clear the flag and continue on, i.e. - * don't terminate the VM, as KVM can't possibly be relying on a flag - * that KVM doesn't know about. + * with KVM-defined sythentic flags. Clear the flags and continue on, + * i.e. don't terminate the VM, as KVM can't possibly be relying on a + * flag that KVM doesn't know about. */ - if (WARN_ON_ONCE(error_code & PFERR_IMPLICIT_ACCESS)) - error_code &=3D ~PFERR_IMPLICIT_ACCESS; + if (WARN_ON_ONCE(error_code & PFERR_SYNTHETIC_MASK)) + error_code &=3D ~PFERR_SYNTHETIC_MASK; =20 if (WARN_ON_ONCE(!VALID_PAGE(vcpu->arch.mmu->root.hpa))) return RET_PF_RETRY; =20 + /* + * Except for reserved faults (emulated MMIO is shared-only), set the + * private flag for software-protected VMs based on the gfn's current + * attributes, which are the source of truth for such VMs. Note, this + * wrong for nested MMUs as the GPA is an L2 GPA, but KVM doesn't + * currently supported nested virtualization (among many other things) + * for software-protected VMs. + */ + if (IS_ENABLED(CONFIG_KVM_SW_PROTECTED_VM) && + !(error_code & PFERR_RSVD_MASK) && + vcpu->kvm->arch.vm_type =3D=3D KVM_X86_SW_PROTECTED_VM && + kvm_mem_is_private(vcpu->kvm, gpa_to_gfn(cr2_or_gpa))) + error_code |=3D PFERR_PRIVATE_ACCESS; + r =3D RET_PF_INVALID; if (unlikely(error_code & PFERR_RSVD_MASK)) { r =3D handle_mmio_page_fault(vcpu, cr2_or_gpa, direct); diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_interna= l.h index 1fab1f2359b5..d7c10d338f14 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -306,7 +306,7 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu= *vcpu, gpa_t cr2_or_gpa, .max_level =3D KVM_MAX_HUGEPAGE_LEVEL, .req_level =3D PG_LEVEL_4K, .goal_level =3D PG_LEVEL_4K, - .is_private =3D kvm_mem_is_private(vcpu->kvm, cr2_or_gpa >> PAGE_SHIFT), + .is_private =3D err & PFERR_PRIVATE_ACCESS, }; int r; =20 --=20 2.44.0.278.ge034bb2e1d-goog From nobody Mon Nov 25 19:51:37 2024 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 660722231A for ; Wed, 28 Feb 2024 02:42:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088124; cv=none; b=atFH9PS3h58qUrbTXAxqdzLTfaLNcUI8eI0XV0Iwa8dNiEn2FC9S98SaNUsn60yyUAvVVwQR7poRZiHIpWzp7W+xsm7zWIhGzUEVr21ksW51tdQyHa5c+Xz3fF3LE6iyUIcCmAo0WRiB3YNvhO2k+wDy/MV7dSSxv1lh+tCu6fg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088124; c=relaxed/simple; bh=CoPJqaD56CaWccUD0GKW7vdTTZaLbgYVBxvvvw5siQg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=L50Dzw0+x2G9c2pVZHsqTilzwpiyU1S13jvZ+klFBm9t3RfqENzAPWmSb7N0OKGxt/PttQWRvC0Zflq3PDOFcEvadqucdTmw2yH+FI/Ls6zdBEtMjmlwoPd3oRD0va2cSLREq2LjBTJ8naptQu1BWDOjLUzcIOIESmViGs7t7Pg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=l0dW/RoA; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="l0dW/RoA" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-dce775fa8adso9608559276.1 for ; Tue, 27 Feb 2024 18:42:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709088121; x=1709692921; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=gfpkuG+ZVx2am4Y182O6tFhDfs/R2qPCe9zqwBMHRpo=; b=l0dW/RoAOtJbVNHxcxiV8U7Q1j379bGF3UnqtuIE8ngJoiF0UYzhU7nBY1PnwKdQXI dVUVYATYFzvhZlU0Tj0tuXClA+WyRU9sYlOK1y5TVjRCiGA7SAIQkT/QIaK2NsjdjDio Bz38DKgmuFyG/Hfxrw2S8NbkjU45xUXl+x/oQKao4VOcogwzBHbb0N0Wtn1L65p/70U6 l0m0J+6qPWgNd8WsDl2xi4ww9cUJaUDG4vh6YhwVn9dlu0ilcgY76QCfPFE1r3eeGfuO qNW9eZ51fOr2wbUsRyed2daDotzep+yjVbRD6Ef4uMQSm60y6i3pczVgzjqLLabcoBMx zucg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709088121; x=1709692921; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=gfpkuG+ZVx2am4Y182O6tFhDfs/R2qPCe9zqwBMHRpo=; b=jMgMq3i/dTSWpTS/2QHo1NtNz+4u9HD/12nndJZkT4uec5ZM3lQOhikpcjLqhdrmL6 0AyLJRrDo/oMOKvtQePSCAomV0CksvtkxmVYahtLiGxtFWG6G/BLnN3Gxrz/UzZiC1KM TqKNMgNWm0uZbcMKkvDPutVfIG6ozzYUn4rH6hbhSMtYrTsIMdj9b8G93tX/II1gvl40 r0bxlNXt53aokzjpY3IewwNla4ZIIaVL/rzebj9uHmIav6uZX5yECFs+iRpf7nTsZNoq cGcEu8CtoSqEAWmwFi1Kbcux+/ubeCLbd8644lvfbKg+wk7fUIYQbv4UfN36Iq83ZGb2 vxIQ== X-Forwarded-Encrypted: i=1; AJvYcCUOAuf8wHORSvrXv2BbVr1rfcYa/x09dHNyeb1bIq0kSbSouRreEZEMogg470LJMk0PoEydFxeVxJILNpMqGNhn1/RMn556PuXE7UVI X-Gm-Message-State: AOJu0YwkrwtXi0lo7x29ea0NlZ/qOUennpo/9u77YIgVje5vR7Ui3fRd nNhQS2ol2cGvWjLoIxDcwi2O4B4XtY3B/4ELRktbzmk5N1bo5cFsyC8PJ1WkK7+AnQSeQ+zb7zs Wyg== X-Google-Smtp-Source: AGHT+IF3KIx/8tFZn/wJwVNrNg1+yoGtgwdvNURhlI554RnWArO0MDtvM2JKBrSbo24e866hb0I37y5xSB0= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6902:100a:b0:dc7:53a0:83ad with SMTP id w10-20020a056902100a00b00dc753a083admr385599ybt.5.1709088121589; Tue, 27 Feb 2024 18:42:01 -0800 (PST) Reply-To: Sean Christopherson Date: Tue, 27 Feb 2024 18:41:37 -0800 In-Reply-To: <20240228024147.41573-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240228024147.41573-1-seanjc@google.com> X-Mailer: git-send-email 2.44.0.278.ge034bb2e1d-goog Message-ID: <20240228024147.41573-7-seanjc@google.com> Subject: [PATCH 06/16] KVM: x86/mmu: WARN if upper 32 bits of legacy #PF error code are non-zero From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yan Zhao , Isaku Yamahata , Michael Roth , Yu Zhang , Chao Peng , Fuad Tabba , David Matlack Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" WARN if bits 63:32 are non-zero when handling an intercepted legacy #PF, as the error code for #PF is limited to 32 bits (and in practice, 16 bits on Intel CPUS). This behavior is architectural, is part of KVM's ABI (see kvm_vcpu_events.error_code), and is explicitly documented as being preserved for intecerpted #PF in both the APM: The error code saved in EXITINFO1 is the same as would be pushed onto the stack by a non-intercepted #PF exception in protected mode. and even more explicitly in the SDM as VMCS.VM_EXIT_INTR_ERROR_CODE is a 32-bit field. Simply drop the upper bits of hardware provides garbage, as spurious information should do no harm (though in all likelihood hardware is buggy and the kernel is doomed). Handling all upper 32 bits in the #PF path will allow moving the sanity check on synthetic checks from kvm_mmu_page_fault() to npf_interception(), which in turn will allow deriving PFERR_PRIVATE_ACCESS from AMD's PFERR_GUEST_ENC_MASK without running afoul of the sanity check. Note, this also why Intel uses bit 15 for SGX (highest bit on Intel CPUs) and AMD uses bit 31 for RMP (highest bit on AMD CPUs); using the highest bit minimizes the probability of a collision with the "other" vendor, without needing to plumb more bits through microcode. Signed-off-by: Sean Christopherson Reviewed-by: Kai Huang --- arch/x86/kvm/mmu/mmu.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 7807bdcd87e8..5d892bd59c97 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4553,6 +4553,13 @@ int kvm_handle_page_fault(struct kvm_vcpu *vcpu, u64= error_code, if (WARN_ON_ONCE(fault_address >> 32)) return -EFAULT; #endif + /* + * Legacy #PF exception only have a 32-bit error code. Simply drop the + * upper bits as KVM doesn't use them for #PF (because they are never + * set), and to ensure there are no collisions with KVM-defined bits. + */ + if (WARN_ON_ONCE(error_code >> 32)) + error_code =3D lower_32_bits(error_code); =20 vcpu->arch.l1tf_flush_l1d =3D true; if (!flags) { --=20 2.44.0.278.ge034bb2e1d-goog From nobody Mon Nov 25 19:51:37 2024 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 63A3822F1D for ; Wed, 28 Feb 2024 02:42:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088125; cv=none; b=XLtpA4FLAPyIk//+9BywEP8R2TCZqIJ8xocS8FidcpYk8GMT+l/GrayRQN8hBbkG0r7JH5fSQn/wV5aMH6NW/Ls2QR5wJdz7fuLYhjF4l5HTAryqlYbh1jLUDi+/RCRw0TLAamcgEfsrtU0+DdmhCjCGXRc3m49dlWusT+UszRo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088125; c=relaxed/simple; bh=L+IvU42/MjztgCK7ijdWOl+7+6gX/ME9VFnO9RlLhcM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=F9ruXaH9f4ZAOffnrGSof7QOUYX5vAoUqrbQSLD/tL06U9/q6QE0Ch0iJb7mjlWYz9BmMaIY8JDlG4Dgno9SZKBKigUZ5qoZdRbPxS70nho+Hfq+OHiXWllCmuGP58mbgEhnArqWShojqs7485EfIeYUcnZxwWRX+CD1EEiR8EY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=36imo14K; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="36imo14K" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-6e55b676bc7so395375b3a.0 for ; Tue, 27 Feb 2024 18:42:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709088124; x=1709692924; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=K1OBRBk3Vku2bidIof7TOH5MpGEdlnRqjgwLLM8pKvQ=; b=36imo14K2mj75sRhHOFdmZqXxsk4fKgN5mdj1SxJvCB4aTXpxsKec8kRZ/s3y80NKg Cmb1nugw8vHc0/BoCO2dZiaceUfuCXGX1GIhOaTJVjD06w1nwstw86zDulVUEExiQug7 ttQm0KqKUi7+UByKYvmWuxKYMbvdn4XEKlnmgCMmD4Lps/qneG9vEJhM/RQMEV47tjEt Kt4d6c+kkwv4OTYDdCCFzYb7XeP/l2LfaE7XPQHywZXaBew+vAHi5WGmRU6zYwxkwA6N R8jbSS0BuuHRENoFmsPXfP540dJR4/RBNLymQkT58aMI83bMhThh66yKTmn2wdpmm2vD Xb/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709088124; x=1709692924; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=K1OBRBk3Vku2bidIof7TOH5MpGEdlnRqjgwLLM8pKvQ=; b=o6TllcFICTx2bU2MQrPiBMoTd7l8EZW0Ry9XOsBTLpBZmFmKJZJ9Rl3tTZSFy4EAp/ trni0eWXWdN29shjOmDWffPKYHx8KvcvFdOAq19c+nf0A+JZTlrxp3DMfc0aNhnRRmre bB5RVInt0GYc8E+etLEqGquQ04tlbBX0LClkhT+A89ScKTTMY/mEzTJIOdDKHSucZApW 4DsA5teWYqJB3JStPDBjO2rsD0hMKUduGJOzER5n8K2AQepy1wpzTfPxQNgNu2QLRYms SKxm5Opw5mR2AL8IrzpMRU4KVc4z7xFQSI3GkJpBRuT+FMkK1lrXUvkN5AkALHwR11Li 1Wqg== X-Forwarded-Encrypted: i=1; AJvYcCUWOoklZD6UhNbKhJ7EKEwL37nLLma+nqdm7xP92/nYEZ3ZqbIE5HYR7en9ygdPRaT1hyGg/4+vbun8rX4MN8IUtp2an4BNCJEZTVhf X-Gm-Message-State: AOJu0YzcEdZl/Bqv+OYIqUJNsylCqbADu1reULOnh5dAtx0DMTK8hlWx 4i9nkfaJUeOlDng36/hAYBNX6jrJKr+pHfMuXZhHmLIyPtiH/w8JebrdHmQ1I+2Vvok22swi/vj tEQ== X-Google-Smtp-Source: AGHT+IGQjZ5u9lsKdd55GvF9bKJDkjcEFySqssBNTDbI7xO0OMlG4FC/aQm6rWPNPLP5+jfE28ifC4/mWLY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6a00:1aca:b0:6e5:1196:1277 with SMTP id f10-20020a056a001aca00b006e511961277mr75263pfv.6.1709088123599; Tue, 27 Feb 2024 18:42:03 -0800 (PST) Reply-To: Sean Christopherson Date: Tue, 27 Feb 2024 18:41:38 -0800 In-Reply-To: <20240228024147.41573-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240228024147.41573-1-seanjc@google.com> X-Mailer: git-send-email 2.44.0.278.ge034bb2e1d-goog Message-ID: <20240228024147.41573-8-seanjc@google.com> Subject: [PATCH 07/16] KVM: x86: Move synthetic PFERR_* sanity checks to SVM's #NPF handler From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yan Zhao , Isaku Yamahata , Michael Roth , Yu Zhang , Chao Peng , Fuad Tabba , David Matlack Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move the sanity check that hardware never sets bits that collide with KVM- define synthetic bits from kvm_mmu_page_fault() to npf_interception(), i.e. make the sanity check #NPF specific. The legacy #PF path already WARNs if _any_ of bits 63:32 are set, and the error code that comes from VMX's EPT Violatation and Misconfig is 100% synthesized (KVM morphs VMX's EXIT_QUALIFICATION into error code flags). Add a compile-time assert in the legacy #PF handler to make sure that KVM- define flags are covered by its existing sanity check on the upper bits. Signed-off-by: Sean Christopherson Reviewed-by: Binbin Wu Reviewed-by: Kai Huang --- arch/x86/kvm/mmu/mmu.c | 12 +++--------- arch/x86/kvm/svm/svm.c | 9 +++++++++ 2 files changed, 12 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 5d892bd59c97..bd342ebd0809 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4561,6 +4561,9 @@ int kvm_handle_page_fault(struct kvm_vcpu *vcpu, u64 = error_code, if (WARN_ON_ONCE(error_code >> 32)) error_code =3D lower_32_bits(error_code); =20 + /* Ensure the above sanity check also covers KVM-defined flags. */ + BUILD_BUG_ON(lower_32_bits(PFERR_SYNTHETIC_MASK)); + vcpu->arch.l1tf_flush_l1d =3D true; if (!flags) { trace_kvm_page_fault(vcpu, fault_address, error_code); @@ -5845,15 +5848,6 @@ int noinline kvm_mmu_page_fault(struct kvm_vcpu *vcp= u, gpa_t cr2_or_gpa, u64 err int r, emulation_type =3D EMULTYPE_PF; bool direct =3D vcpu->arch.mmu->root_role.direct; =20 - /* - * WARN if hardware generates a fault with an error code that collides - * with KVM-defined sythentic flags. Clear the flags and continue on, - * i.e. don't terminate the VM, as KVM can't possibly be relying on a - * flag that KVM doesn't know about. - */ - if (WARN_ON_ONCE(error_code & PFERR_SYNTHETIC_MASK)) - error_code &=3D ~PFERR_SYNTHETIC_MASK; - if (WARN_ON_ONCE(!VALID_PAGE(vcpu->arch.mmu->root.hpa))) return RET_PF_RETRY; =20 diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index e90b429c84f1..199c4dd8d214 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -2055,6 +2055,15 @@ static int npf_interception(struct kvm_vcpu *vcpu) u64 fault_address =3D svm->vmcb->control.exit_info_2; u64 error_code =3D svm->vmcb->control.exit_info_1; =20 + /* + * WARN if hardware generates a fault with an error code that collides + * with KVM-defined sythentic flags. Clear the flags and continue on, + * i.e. don't terminate the VM, as KVM can't possibly be relying on a + * flag that KVM doesn't know about. + */ + if (WARN_ON_ONCE(error_code & PFERR_SYNTHETIC_MASK)) + error_code &=3D ~PFERR_SYNTHETIC_MASK; + trace_kvm_page_fault(vcpu, fault_address, error_code); return kvm_mmu_page_fault(vcpu, fault_address, error_code, static_cpu_has(X86_FEATURE_DECODEASSISTS) ? --=20 2.44.0.278.ge034bb2e1d-goog From nobody Mon Nov 25 19:51:37 2024 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E746D24A0A for ; Wed, 28 Feb 2024 02:42:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088127; cv=none; b=EGOO6AhNkeuW0Sa9jtnow+2nwIA5DvaqBw9l8SUWHVBwlhZkR3H7NYYHZE9WFLq0jOLJUaWz+uEGBXMQze1NR3cGwNxPkLzFarnD42EVFwxvpsPvtNjEWu1qpiM51sKTN4539z/Qjk1RIfkdVb4Rt/sPU3/taQ/kuVp+Q6EcDns= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088127; c=relaxed/simple; bh=10WzY2o+Flr57bQulGQh5SH//bSBY0IZ4G03BJuz+vI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ct3O/p2o2RaUvtw8yTgNVetJf8LGIx6hx0rnsCnmyXrk02OxG2h9BmsK2zg3kLOtwU2j/n15vyGQWyjwDYk2fGBKDH8fmcIXfQUVqXMWLL5IUdM74PnNmouLPZ0DL5I6q19YTPFvRxF6ME3vyxE+yaRW1PAxtrzFr1Ia8Ly25OA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=C2imXfn4; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="C2imXfn4" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-29ade776a78so1564146a91.1 for ; Tue, 27 Feb 2024 18:42:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709088125; x=1709692925; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=9j/IY1O+fiTDSKSqyI0HJx2iE7AMuX53NhuhIZhorfQ=; b=C2imXfn4SEpwwn8cx9cbELjtlxgLpNqvKJwrmQBiuqZKd/kOIAPHLnAuES//+XIDEV 23DauaSA9GocbUTI0NgSscwhCzPBg79p4/uUqp5LdYW90kKhyJ5VR+gwJDTdYE+UrTQX jiMPVaYNDMe56BbdYC1g8X9XMWlJdTe3coP/YLKW9rbp10BLTa2wbKalhSstO+VllRgd 9d5R/VY83bnyM3L+3QAQqSlda/zltRJScOlG1ReAlk7EMRRt4m32vXkON3fvjLZ6J4iB o7XVq39qGPC403Aj7a0H6ei3cR2Op92ewh/s5fTMqJDgd+syonFZa48MB2kTDhtdGYXc Fk2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709088125; x=1709692925; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9j/IY1O+fiTDSKSqyI0HJx2iE7AMuX53NhuhIZhorfQ=; b=GWrEo3W3hEcB+naD9kc8xh6tyDH5vN7vmSY++c+bHVoeXSvLQUM7iTDKCPLBvDjVKe pduV63bWdfoAyxhZjZVwzex/EOHHN1JugApy/MjBbu5DMcNavNDQiPd4KWl7kC1czwSF VDlawp3CReqG1CF292CAiXBnelWc2nzkwt3U/qdGOKxUROJmNOVpLdcr7Dri8wgujoMo 72DTrUG2eVJaz5t2nQOfbf+vFuyOUVdTomNJv2kwi8jmLMBJRBmBPlqq+5gEjZHQcAl7 yrtsZMd3xfXvpBSvN3RJ3F+eJD0smcA/IrCdmJNz2BVnLSRN3O66gxwpniftml4TZu7Z an/w== X-Forwarded-Encrypted: i=1; AJvYcCWzDeOmRkpUwxN//W/ao4YJWLA2N7XJWH0xl9sx2cAksYB/L8kkfwFqsKvkoDVp7/mxBovwe4lXcwPdGEhmKX04ixmVOcZSSCDY7co0 X-Gm-Message-State: AOJu0Yxi5P9PCYlw2apUpU9Z6ZvtsYYuXSmMK8BV4rPegdnen4vknU4o hCT985BZc/WkzN4vTBgSXRajGbN+YcvmOFwrVCj47kd7CHGczEXymt7hWSEGc9R42dtT2hGLexz elA== X-Google-Smtp-Source: AGHT+IF4cMKYA2HyWXFn/s6kzTHvnYGOu7L8q1paC0Vx85/7XZ8gBkVthsx01n5pIwYIBwPcox/9ac5Rv1s= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:90b:518f:b0:29a:b2e7:91d3 with SMTP id se15-20020a17090b518f00b0029ab2e791d3mr80809pjb.3.1709088125284; Tue, 27 Feb 2024 18:42:05 -0800 (PST) Reply-To: Sean Christopherson Date: Tue, 27 Feb 2024 18:41:39 -0800 In-Reply-To: <20240228024147.41573-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240228024147.41573-1-seanjc@google.com> X-Mailer: git-send-email 2.44.0.278.ge034bb2e1d-goog Message-ID: <20240228024147.41573-9-seanjc@google.com> Subject: [PATCH 08/16] KVM: x86/mmu: WARN and skip MMIO cache on private, reserved page faults From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yan Zhao , Isaku Yamahata , Michael Roth , Yu Zhang , Chao Peng , Fuad Tabba , David Matlack Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" WARN and skip the emulated MMIO fastpath if a private, reserved page fault is encountered, as private+reserved should be an impossible combination (KVM should never create an MMIO SPTE for a private access). Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index bd342ebd0809..9206cfa58feb 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -5866,7 +5866,8 @@ int noinline kvm_mmu_page_fault(struct kvm_vcpu *vcpu= , gpa_t cr2_or_gpa, u64 err error_code |=3D PFERR_PRIVATE_ACCESS; =20 r =3D RET_PF_INVALID; - if (unlikely(error_code & PFERR_RSVD_MASK)) { + if (unlikely((error_code & PFERR_RSVD_MASK) && + !WARN_ON_ONCE(error_code & PFERR_PRIVATE_ACCESS))) { r =3D handle_mmio_page_fault(vcpu, cr2_or_gpa, direct); if (r =3D=3D RET_PF_EMULATE) goto emulate; --=20 2.44.0.278.ge034bb2e1d-goog From nobody Mon Nov 25 19:51:37 2024 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C26072561F for ; Wed, 28 Feb 2024 02:42:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088129; cv=none; b=R/IxmF0gNPnDojD3uRspaxKqA8k1ArLGOwKfXsln+NU9YZ/Y27TsqrKwppQ5vb1AHUarsCLLFv3TauZ2NK/hyr3v6Nx204vYqafxc/wnHvz9hzfp6ujNZLegya2aZlcmsku/gN5BXaQw7ArnrtKklepIpvPlycGv8G+Ng+1JNfo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088129; c=relaxed/simple; bh=wTukbFZz6gTZ0AuLeXiBHYZaq0ngrSizsl1vey1TzwM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=DUi8eSeRfjv2zPN3hF2iu3mq9zJ2XMwaoTWvFKNoo2LySr+Nr2+OQHlNqq6zBfwaaA3p5g4Dlf5Q01OvQRlx+RiUqb5W3ORAAusR3QIGXRRsXG4lwmvLlPOyK2Fu0n02pQnuS8AMknzR8jGQz032Qf+L1l7/50TwH6CWaoF6TWA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=jm6G4Zku; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="jm6G4Zku" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-1dbe3ca6bb7so3200715ad.0 for ; Tue, 27 Feb 2024 18:42:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709088127; x=1709692927; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=DGgOzzQ1+jAk85gp0p6D7NJFf1xx8C5oqxWRvpff3B4=; b=jm6G4ZkuHsVYDoi4cc6dCfEHacaybn5wOU3GFpqSoz0boWYz0rUoOTFsbvU5/zfXd0 9M0JoaglZ0aT73cxmH3uR3FSrgqkThm5lG9ERhNuwGS1OyKsU3ufBa6wS8ShddvU3jIn os66jQ53NyvLRaYb65w9sLeU5HLz9IQZ8tcD2aETaYgecfgPrGbQDpQskq5i7l9n51I0 lXOlR4uR89lBzOl9zitud+SKEiEVDVKjz9DbjmD7zlYvPXCbVV6wQWjCrCAYTe424T86 sRdpoqBHVziaSbFkTw3qA3txAHoHenWoDU68M1OggvH+elix3TCsIkxObKeRE0u2bzO6 Sucw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709088127; x=1709692927; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=DGgOzzQ1+jAk85gp0p6D7NJFf1xx8C5oqxWRvpff3B4=; b=bnEzpXubC3oc1bpMWEpfBLDySXv99wwBRwI4awCbjSxDubgy5mCkibwapcGm9Eycxq rlxq2LLAlZJwpVxclShRbkVjbL0YRHJ4KLdWjsGmroe5unD6dMT44ILM6fJv+QrUsEDN Y2hxTcPMd5RHegvUUCB7dG33skmfFSHEANWoMmt8n7YwNNgaHtAG2dmjCuAWyBaDE0n1 33/jgj9tkc2XgDt/Euj52YsbBIBFBa4uRApFeMv67+X5b9xIscCsQ/77E2HLZBv45/rG FgvdPzYvTFjtSYl354+wORuOHHJ4lSFybV/BgFU92o4DjGECWIlDia2XN8C96P5nDhFo gZqQ== X-Forwarded-Encrypted: i=1; AJvYcCWykJMSxiy07cqUToHTeNv/mRyFld3TA3F44zbqCM3u85rgznniIhhjyEaNLw/7PdPcZz+NNURovAKpfQwlkAziZ4KePtNIEbzHP+yo X-Gm-Message-State: AOJu0YwRAYKf2mJlun9hJqtz+w1BW4JJkVy10EGhNjY3F8ox20H0+6QT hBWY28ttEok++TQh9j+2k1C9lwR2RL9Q7U4G68U72yLSlnheM9WNd+mEWpgq/gt7ZG3KNolNylp Z5Q== X-Google-Smtp-Source: AGHT+IHDf3xnx88tyDTwCRq6HMZJvOonccrd+E6zg1vRctJ9bCAsuL3M+j2Csp5VtZWLXjZfAOa9nJi8ie0= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:d114:b0:1dc:68a2:2cb6 with SMTP id w20-20020a170902d11400b001dc68a22cb6mr2554plw.6.1709088126987; Tue, 27 Feb 2024 18:42:06 -0800 (PST) Reply-To: Sean Christopherson Date: Tue, 27 Feb 2024 18:41:40 -0800 In-Reply-To: <20240228024147.41573-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240228024147.41573-1-seanjc@google.com> X-Mailer: git-send-email 2.44.0.278.ge034bb2e1d-goog Message-ID: <20240228024147.41573-10-seanjc@google.com> Subject: [PATCH 09/16] KVM: x86/mmu: Move private vs. shared check above slot validity checks From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yan Zhao , Isaku Yamahata , Michael Roth , Yu Zhang , Chao Peng , Fuad Tabba , David Matlack Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Prioritize private vs. shared gfn attribute checks above slot validity checks to ensure a consistent userspace ABI. E.g. as is, KVM will exit to userspace if there is no memslot, but emulate accesses to the APIC access page even if the attributes mismatch. Fixes: 8dd2eee9d526 ("KVM: x86/mmu: Handle page fault for private memory") Cc: Yu Zhang Cc: Chao Peng Cc: Fuad Tabba Cc: Michael Roth Cc: Isaku Yamahata Signed-off-by: Sean Christopherson Reviewed-by: Kai Huang --- arch/x86/kvm/mmu/mmu.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 9206cfa58feb..58c5ae8be66c 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4365,11 +4365,6 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, = struct kvm_page_fault *fault return RET_PF_EMULATE; } =20 - if (fault->is_private !=3D kvm_mem_is_private(vcpu->kvm, fault->gfn)) { - kvm_mmu_prepare_memory_fault_exit(vcpu, fault); - return -EFAULT; - } - if (fault->is_private) return kvm_faultin_pfn_private(vcpu, fault); =20 @@ -4410,6 +4405,16 @@ static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, st= ruct kvm_page_fault *fault, fault->mmu_seq =3D vcpu->kvm->mmu_invalidate_seq; smp_rmb(); =20 + /* + * Check for a private vs. shared mismatch *after* taking a snapshot of + * mmu_invalidate_seq, as changes to gfn attributes are guarded by the + * invalidation notifier. + */ + if (fault->is_private !=3D kvm_mem_is_private(vcpu->kvm, fault->gfn)) { + kvm_mmu_prepare_memory_fault_exit(vcpu, fault); + return -EFAULT; + } + /* * Check for a relevant mmu_notifier invalidation event before getting * the pfn from the primary MMU, and before acquiring mmu_lock. --=20 2.44.0.278.ge034bb2e1d-goog From nobody Mon Nov 25 19:51:37 2024 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BCF502C197 for ; Wed, 28 Feb 2024 02:42:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088132; cv=none; b=itnnpIABDboHX9l1v7BsUSOttmLff8E99jj8Y1ZET6QvfGERVsTk+1u/7hfon6wYGk8pLGwdNoQLReh6W02tfyC95VmjZY9cwUFddDQeDxUXbJ3XKXwuNf+mS4xflLqhsV5ATpw258VdwjXGtVLXsRrMLMrl+fhZ76adAclWvMY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088132; c=relaxed/simple; bh=9PNdeAstWFyBPPWEs7KhyiizgXxO3+J35k1TqWTitDo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=djjFrBMCvukE8Rib52wxZwLtdHAW6NWPR4ljRe4WSDTWYIcg/PQeoSLTDgM5YKXxzq2xMAcIQWyYFkkzhPzVWBbSJQbgvkXatgyNyVhlb/AUZhdrUIsEBIf5hFcQGwZB+0ebVh2pbjn7JR1SKbTVPz4sb6pbT9N9xGuuXSONuYw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=JZQPbhn/; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="JZQPbhn/" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-6e557bde036so877620b3a.0 for ; Tue, 27 Feb 2024 18:42:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709088129; x=1709692929; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=L6OrO+CyUKUmFb+nQO2BWBH5lLYiXnAmYHqUzxi5l7c=; b=JZQPbhn//+Ga12435LtjkrbT5ugjSrcSMCPTXKjZGsoV4QYeR37wszvWluIbpI2aIZ XZrJJRUhCZ/FkwzECVje7AnlwZs1EsUqGrUEj6jpRwZQ8At6WFRqTcmLgL3RBNTIjDYM F+Nv7LuSfc/dkW77SjR7MelG3ygjAuGHU7ipUYOGGWPpcNlLAMXJK36c15Oqzwg/0UH8 zR+uGGRr00a2iyq3RN9JZYXGnLRyG9mVFuzhSk7Eq1/nanepIT/Ouqqs6k0WTNFde3yC dIou95nF9WtE6J7m2JgQsKRFqTisE542GxbDn3Z8RXJyH+DoACu7M2APE7xGKCF10bYy Mu7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709088129; x=1709692929; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=L6OrO+CyUKUmFb+nQO2BWBH5lLYiXnAmYHqUzxi5l7c=; b=Je/birh5TwYxW3YGFqmxNj4496qoVYq7dScTWPx9v83wi6iMsswkCzuUTIXpm/aizW 5r++xDjx4hReuSrHua+11NfXy6wK4J4N2Pm6II9ZaJuREdWu+3262EXc+kM/ebGup9y/ qfD4GkcudoegNjlV0Mhece0t3FnokSgQoqzw0eeERIVnpz/LUbWjUp82PUon5fRuWw16 yFNryxum2hw4gA44s3iE0QFFcD7mxbM/CMXQztx1lqRMyCpMzqwEGbj0I6ai7t5FDUVl PcI5YfIxTtm4TPClqydSKn6XAj4kPh/W/G00De16gIshp1ucDFrUEkBRnSnXNg7Lerev kD5A== X-Forwarded-Encrypted: i=1; AJvYcCU3vb2PaMRjNlsjHIPB3xlPs23+sQn/TKvGA2ha7FL5L8+FDK6oPZXI4xLmi7eLF0Q0eRCPK6n1yG9Zu3TsIAC1jnCIJamGRN9+ivVG X-Gm-Message-State: AOJu0Yx22//XD6BeGbYraimsCgBZOaqaOU20NEvuav/IkqD7ddStYjEQ QrLdwY1R8IbfdQVFvg4lh27ssRmX5Mry4+uEO9g99NYS67r/yoMsOzhohKCVmnXlvYYdE3frrO1 BKA== X-Google-Smtp-Source: AGHT+IEHUj/do6mCm4DLRIfshMpNYR8umtAxls3inWMMZJ4H/OwZPKN3I4Jwqe17wSm4B2+jtdaIEYQIJPk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6a00:8917:b0:6e4:f310:1fd with SMTP id hw23-20020a056a00891700b006e4f31001fdmr362188pfb.4.1709088129052; Tue, 27 Feb 2024 18:42:09 -0800 (PST) Reply-To: Sean Christopherson Date: Tue, 27 Feb 2024 18:41:41 -0800 In-Reply-To: <20240228024147.41573-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240228024147.41573-1-seanjc@google.com> X-Mailer: git-send-email 2.44.0.278.ge034bb2e1d-goog Message-ID: <20240228024147.41573-11-seanjc@google.com> Subject: [PATCH 10/16] KVM: x86/mmu: Don't force emulation of L2 accesses to non-APIC internal slots From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yan Zhao , Isaku Yamahata , Michael Roth , Yu Zhang , Chao Peng , Fuad Tabba , David Matlack Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Allow mapping KVM's internal memslots used for EPT without unrestricted guest into L2, i.e. allow mapping the hidden TSS and the identity mapped page tables into L2. Unlike the APIC access page, there is no correctness issue with letting L2 access the "hidden" memory. Allowing these memslots to be mapped into L2 fixes a largely theoretical bug where KVM could incorrectly emulate subsequent _L1_ accesses as MMIO, and also ensures consistent KVM behavior for L2. If KVM is using TDP, but L1 is using shadow paging for L2, then routing through kvm_handle_noslot_fault() will incorrectly cache the gfn as MMIO, and create an MMIO SPTE. Creating an MMIO SPTE is ok, but only because kvm_mmu_page_role.guest_mode ensure KVM uses different roots for L1 vs. L2. But vcpu->arch.mmio_gfn will remain valid, and could cause KVM to incorrectly treat an L1 access to the hidden TSS or identity mapped page tables as MMIO. Furthermore, forcing L2 accesses to be treated as "no slot" faults doesn't actually prevent exposing KVM's internal memslots to L2, it simply forces KVM to emulate the access. In most cases, that will trigger MMIO, amusingly due to filling vcpu->arch.mmio_gfn, but also because vcpu_is_mmio_gpa() unconditionally treats APIC accesses as MMIO, i.e. APIC accesses are ok. But the hidden TSS and identity mapped page tables could go either way (MMIO or access the private memslot's backing memory). Alternatively, the inconsistent emulator behavior could be addressed by forcing MMIO emulation for L2 access to all internal memslots, not just to the APIC. But that's arguably less correct than letting L2 access the hidden TSS and identity mapped page tables, not to mention that it's *extremely* unlikely anyone cares what KVM does in this case. From L1's perspective there is R/W memory at those memslots, the memory just happens to be initialized with non-zero data. Making the memory disappear when it is accessed by L2 is far more magical and arbitrary than the memory existing in the first place. The APIC access page is special because KVM _must_ emulate the access to do the right thing (emulate an APIC access instead of reading/writing the APIC access page). And despite what commit 3a2936dedd20 ("kvm: mmu: Don't expose private memslots to L2") said, it's not just necessary when L1 is accelerating L2's virtual APIC, it's just as important (likely *more* imporant for correctness when L1 is passing through its own APIC to L2. Fixes: 3a2936dedd20 ("kvm: mmu: Don't expose private memslots to L2") Signed-off-by: Sean Christopherson Reviewed-by: Kai Huang --- arch/x86/kvm/mmu/mmu.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 58c5ae8be66c..5c8caab64ba2 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4346,8 +4346,18 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, = struct kvm_page_fault *fault if (slot && (slot->flags & KVM_MEMSLOT_INVALID)) return RET_PF_RETRY; =20 - if (!kvm_is_visible_memslot(slot)) { - /* Don't expose private memslots to L2. */ + if (slot && slot->id =3D=3D APIC_ACCESS_PAGE_PRIVATE_MEMSLOT) { + /* + * Don't map L1's APIC access page into L2, KVM doesn't support + * using APICv/AVIC to accelerate L2 accesses to L1's APIC, + * i.e. the access needs to be emulated. Emulating access to + * L1's APIC is also correct if L1 is accelerating L2's own + * virtual APIC, but for some reason L1 also maps _L1's_ APIC + * into L2. Note, vcpu_is_mmio_gpa() always treats access to + * the APIC as MMIO. Allow an MMIO SPTE to be created, as KVM + * uses different roots for L1 vs. L2, i.e. there is no danger + * of breaking APICv/AVIC for L1. + */ if (is_guest_mode(vcpu)) { fault->slot =3D NULL; fault->pfn =3D KVM_PFN_NOSLOT; @@ -4360,8 +4370,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, s= truct kvm_page_fault *fault * MMIO SPTE. That way the cache doesn't need to be purged * when the AVIC is re-enabled. */ - if (slot && slot->id =3D=3D APIC_ACCESS_PAGE_PRIVATE_MEMSLOT && - !kvm_apicv_activated(vcpu->kvm)) + if (!kvm_apicv_activated(vcpu->kvm)) return RET_PF_EMULATE; } =20 --=20 2.44.0.278.ge034bb2e1d-goog From nobody Mon Nov 25 19:51:37 2024 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ED68C2C1B6 for ; Wed, 28 Feb 2024 02:42:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088133; cv=none; b=fBQNrF+omhWZwYV97uPKm9VOLDaU9wiiXJ8CCEHq1nwO12AjayMTGqjP+veWBZN8fICgJxIrnWPN9TZzyzFRX09vCYM1vDDxRH2uzl9lfFiQSMf1KrGO13dsHqqSjFpXRXhBX7ylbGRJrMpuVEo3Mb7yR8p1I3Z9vttzhTNSnQ0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088133; c=relaxed/simple; bh=UGrkpK716aAHiw66iG1/DSWpe947kr0y+y0Afd7BmIg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=R75X57zlT/wsPEnVSMh1PRUJppzIhq7B9Tke5tDOdLnUKhPz1zOmeKzkK9cM/wGp0wblnN+XEMd7QtgRB5azleBt7vvPihj0kmWsv0Vhx/ZA6yYmkUmCGzxAR4BuSpZxXKgzKI3QQIJDZPi4IAj+lgzIR6+UPF/MBbAzhUH5Q+k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=edga/too; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="edga/too" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-60810219282so61210967b3.0 for ; Tue, 27 Feb 2024 18:42:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709088131; x=1709692931; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=f2pVTuxLJ1bF5JkBExEt38LFzKRRlG2j//10dt1o0l8=; b=edga/tooo62Ub6On2m2O1NUOjTM2zXNHq6BsT4vn89Q2jP+XHb+ii9oxXDO7TKkLjc 13mriBUiCJxtVKEbtq/+zZwEjfoQ4FIyY3atcTIZNTtS8ZQ9+X2d4IQcTeAT2IUln9OW X78TUvtzeZRJasqIg8qZwN8OMCMjCb7najP9MHk2NyWJ6eABftNwdvGr3F89Rm0pdZBF p174c5aDsey9R2lbEcAcQWsii4f5iQk4bK/d3gBF8P18P6YhW4Yh3yxbpxm3hRklAzfD fvy9LitG5Ncnhldf7ThRBbCetaCHq54huVnsRuWzEjACNXFtbMI+zLBR8IQd4UEtEIwC +/Ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709088131; x=1709692931; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=f2pVTuxLJ1bF5JkBExEt38LFzKRRlG2j//10dt1o0l8=; b=vKWIwwPv4kMd+1E6Y7qbcyKO27O0+pZZbEBc8a1rK3vtwaNuZkZSiFy14LZ+BafkCB oB7xq2PuTDyk0kp59HK0CSd2392ngqgs+qqG+REPChxSDaVL3c7lZWgTjlxmuwXF/63e rYQdCpJOt30/R+mzOKpXo1I9QHuYXNa5gTSd88PsOWaj4ndGfVJb/hrXE4BOQY5W/7vu yiuKqsmRZqKNExJjIOwHokDTUr9bpjcHkFRi3Zi/PwisUrxWQJ0pQofJMUlSwE8IbMf/ 98bQIBn33FS0oZBuIP14pBXQNmHyzzfXMXERygyYxbwq+gsIR8jiMfz6wdDBKfFT1m7y lx/g== X-Forwarded-Encrypted: i=1; AJvYcCW9zAV3zPBV7nDhX9zzpbjPOLaBnMl0aEMRbS2L3twma7soBTfGEl6MFOG1GWm5mYMvskDX/AWX2WLhCkkZQOwzD56H7n4Z1+nop8zS X-Gm-Message-State: AOJu0Yz1Wyb79bcld2pGFIx5FhxhEI+LMHJ2ODx4+IGnS37pTMoXTlyd Tu5tMGFkTtkng6AndlyEJLtxoXYcPLaPKYDwol+MH3JWzw2BkXk+2D4mjzJrsBTTpDNve1Y7RDO uww== X-Google-Smtp-Source: AGHT+IGV2/KMUT+5DS/RD3rbrnNhsUrJO/9cIjXu+5lli9rKXWp0tnf2EC/ZhZvoUmnwAPQ8D4qdy5ETdeA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a81:5714:0:b0:609:3bd3:31fd with SMTP id l20-20020a815714000000b006093bd331fdmr118940ywb.2.1709088131111; Tue, 27 Feb 2024 18:42:11 -0800 (PST) Reply-To: Sean Christopherson Date: Tue, 27 Feb 2024 18:41:42 -0800 In-Reply-To: <20240228024147.41573-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240228024147.41573-1-seanjc@google.com> X-Mailer: git-send-email 2.44.0.278.ge034bb2e1d-goog Message-ID: <20240228024147.41573-12-seanjc@google.com> Subject: [PATCH 11/16] KVM: x86/mmu: Explicitly disallow private accesses to emulated MMIO From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yan Zhao , Isaku Yamahata , Michael Roth , Yu Zhang , Chao Peng , Fuad Tabba , David Matlack Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Explicitly detect and disallow private accesses to emulated MMIO in kvm_handle_noslot_fault() instead of relying on kvm_faultin_pfn_private() to perform the check. This will allow the page fault path to go straight to kvm_handle_noslot_fault() without bouncing through __kvm_faultin_pfn(). Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 5c8caab64ba2..ebdb3fcce3dc 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3314,6 +3314,11 @@ static int kvm_handle_noslot_fault(struct kvm_vcpu *= vcpu, { gva_t gva =3D fault->is_tdp ? 0 : fault->addr; =20 + if (fault->is_private) { + kvm_mmu_prepare_memory_fault_exit(vcpu, fault); + return -EFAULT; + } + vcpu_cache_mmio_info(vcpu, gva, fault->gfn, access & shadow_mmio_access_mask); =20 --=20 2.44.0.278.ge034bb2e1d-goog From nobody Mon Nov 25 19:51:37 2024 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 854371BC27 for ; Wed, 28 Feb 2024 02:42:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088135; cv=none; b=BVfHOHzgiqpzQNAYTRJGg7cJceQwmPMG+odhVTZfjt7ks6jd3QJQLg5XPTO9AYcP2GpjFL2WiMmAuO3xNTkXezZmgxodAJP4SnfaYaXu0wsw9VWA+Yvk9ca62a6VukNn2oBs9iRF/faI1r2vXLlL1veGnuhNB0feX8/BN/jahak= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088135; c=relaxed/simple; bh=cV79rJWoCeFTUkciMMwroGk4yVBIOpdWOSpTM312CWI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=MapzSNlb2Y9e/Zd9V/UqHK2m/mHfaQUhL89UgEakc2dtpggLaCIHcfADytbtMz2B25WehAqTNUHBpxl7vo74qv+WHYj47W2kyE9/rWpX9PNxt8dqLjdZfykHxKX0AIGqvwq1SBTeVFSqRVGCEgaD+pra22JjMvZ78aJN3sOXhzg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=xp+mI7Zi; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="xp+mI7Zi" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-5d8bcf739e5so3246748a12.1 for ; Tue, 27 Feb 2024 18:42:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709088133; x=1709692933; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=dJc2s9t1awM79/Fy1pb8zawduPtJBJNE9uUwyg5GGwY=; b=xp+mI7ZiWfjE3IFH4YwD9lyAfW7sd1lMUxvyRB40HLiqXdbsQtw11iZ4Wzd5MotUfT JLhiaibdTS57YLVP19C4BrPT1nT+hGOEhJaN0m2kHIq/qO7zHq3+L4Z9ao/LNNvF8w0N I1kNTKBcddmnw3zCLcxuBdUTeTDHWZxz8gs69fWF7gUaRviaiO+Jz5qUu2I3uiWYEjYW qarlBaS/jpbSVeP+MmOo1WFL8TDOrv8RCECpLRTnyd5Er9DL1UVZqUZTxH+X7BEUJk8W 8mADb1cjVVEooZG9VYutd1Jyot9GgQHbng8BjQ6LFGth2y2kixRgZq8RkTVtXRJYej3M 9i7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709088133; x=1709692933; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=dJc2s9t1awM79/Fy1pb8zawduPtJBJNE9uUwyg5GGwY=; b=wo+yYkEIaqMR6g4xLQK1GPaJcH64sX04CQXauFCy7FK59DLJKgaHeqrjj0jjAxzcyL RZmaMNGWu5ZVE/sTiLeNsGs6YePmJ/F/rk5ACOz9z2pMsAJ9b1PTMcJ9dhHKjKcX0sUU DUHnhfZFDA1Ljf5t6nK6bQfXoO3egIZboXWqa/a1WwN1Gbxlc8jJtYvkwnUwbsMiZNUb qPhL/ScmzJONFp4IVs+j2cO4TkVry+qq1vOX70+LUSO+WQH8IWM81S5hD27TczL6feu5 bO19AT8dfsdqrxSAu01USO1pQub3jU1ZFkfaiVHKK3lnPdijZdqtbnKJyNFm2Dy5/z3o /+jw== X-Forwarded-Encrypted: i=1; AJvYcCU3AIUFU+pJWWgT7EWbEZrF6+cR3Arh8nDqn0SRp1m0ggDdCXw+ehGqw4a/+G7+3lTx+tioIGezseMo2we3OUzc+oTWW83H+/tgHV/n X-Gm-Message-State: AOJu0Yw7utPkC0sAi6PfvB5O55ILLDYtiwV2SNtxe5uC5NybaNlqZwtk lJS82F4WchAFv2toPA+GwIPuttlzndMfDx8XgD4o2OCKK7FfWwQvcVNvxmUUP088EXL8cXoV59y kAw== X-Google-Smtp-Source: AGHT+IEQrtR9z7mv6fKO+/I7F2jfZx7cwviWFpuAmB0BUMFbXR6LBdfKH1SzjUSCUVSMy+xt/hWaCnTFa+A= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:90b:3cc4:b0:299:979c:27e6 with SMTP id qd4-20020a17090b3cc400b00299979c27e6mr37082pjb.4.1709088132789; Tue, 27 Feb 2024 18:42:12 -0800 (PST) Reply-To: Sean Christopherson Date: Tue, 27 Feb 2024 18:41:43 -0800 In-Reply-To: <20240228024147.41573-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240228024147.41573-1-seanjc@google.com> X-Mailer: git-send-email 2.44.0.278.ge034bb2e1d-goog Message-ID: <20240228024147.41573-13-seanjc@google.com> Subject: [PATCH 12/16] KVM: x86/mmu: Move slot checks from __kvm_faultin_pfn() to kvm_faultin_pfn() From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yan Zhao , Isaku Yamahata , Michael Roth , Yu Zhang , Chao Peng , Fuad Tabba , David Matlack Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move the checks related to the validity of an access to a memslot from the inner __kvm_faultin_pfn() to its sole caller, kvm_faultin_pfn(). This allows emulating accesses to the APIC access page, which don't need to resolve a pfn, even if there is a relevant in-progress mmu_notifier invalidation. Ditto for accesses to KVM internal memslots from L2, which KVM also treats as emulated MMIO. More importantly, this will allow for future cleanup by having the "no memslot" case bail from kvm_faultin_pfn() very early on. Go to rather extreme and gross lengths to make the change a glorified nop, e.g. call into __kvm_faultin_pfn() even when there is no slot, as the related code is very subtle. E.g. fault->slot can be nullified if it points at the APIC access page, some flows in KVM x86 expect fault->pfn to be KVM_PFN_NOSLOT, while others check only fault->slot, etc. No functional change intended. Signed-off-by: Sean Christopherson Reviewed-by: Kai Huang --- arch/x86/kvm/mmu/mmu.c | 105 +++++++++++++++++++++-------------------- 1 file changed, 53 insertions(+), 52 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index ebdb3fcce3dc..8aa957f0a717 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4340,9 +4340,59 @@ static int kvm_faultin_pfn_private(struct kvm_vcpu *= vcpu, =20 static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault = *fault) { - struct kvm_memory_slot *slot =3D fault->slot; bool async; =20 + if (fault->is_private) + return kvm_faultin_pfn_private(vcpu, fault); + + async =3D false; + fault->pfn =3D __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, false, + &async, fault->write, + &fault->map_writable, &fault->hva); + if (!async) + return RET_PF_CONTINUE; /* *pfn has correct page already */ + + if (!fault->prefetch && kvm_can_do_async_pf(vcpu)) { + trace_kvm_try_async_get_page(fault->addr, fault->gfn); + if (kvm_find_async_pf_gfn(vcpu, fault->gfn)) { + trace_kvm_async_pf_repeated_fault(fault->addr, fault->gfn); + kvm_make_request(KVM_REQ_APF_HALT, vcpu); + return RET_PF_RETRY; + } else if (kvm_arch_setup_async_pf(vcpu, fault->addr, fault->gfn)) { + return RET_PF_RETRY; + } + } + + /* + * Allow gup to bail on pending non-fatal signals when it's also allowed + * to wait for IO. Note, gup always bails if it is unable to quickly + * get a page and a fatal signal, i.e. SIGKILL, is pending. + */ + fault->pfn =3D __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, + NULL, fault->write, + &fault->map_writable, &fault->hva); + return RET_PF_CONTINUE; +} + +static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *f= ault, + unsigned int access) +{ + struct kvm_memory_slot *slot =3D fault->slot; + int ret; + + fault->mmu_seq =3D vcpu->kvm->mmu_invalidate_seq; + smp_rmb(); + + /* + * Check for a private vs. shared mismatch *after* taking a snapshot of + * mmu_invalidate_seq, as changes to gfn attributes are guarded by the + * invalidation notifier. + */ + if (fault->is_private !=3D kvm_mem_is_private(vcpu->kvm, fault->gfn)) { + kvm_mmu_prepare_memory_fault_exit(vcpu, fault); + return -EFAULT; + } + /* * Retry the page fault if the gfn hit a memslot that is being deleted * or moved. This ensures any existing SPTEs for the old memslot will @@ -4367,7 +4417,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, s= truct kvm_page_fault *fault fault->slot =3D NULL; fault->pfn =3D KVM_PFN_NOSLOT; fault->map_writable =3D false; - return RET_PF_CONTINUE; + goto faultin_done; } /* * If the APIC access page exists but is disabled, go directly @@ -4379,56 +4429,6 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, = struct kvm_page_fault *fault return RET_PF_EMULATE; } =20 - if (fault->is_private) - return kvm_faultin_pfn_private(vcpu, fault); - - async =3D false; - fault->pfn =3D __gfn_to_pfn_memslot(slot, fault->gfn, false, false, &asyn= c, - fault->write, &fault->map_writable, - &fault->hva); - if (!async) - return RET_PF_CONTINUE; /* *pfn has correct page already */ - - if (!fault->prefetch && kvm_can_do_async_pf(vcpu)) { - trace_kvm_try_async_get_page(fault->addr, fault->gfn); - if (kvm_find_async_pf_gfn(vcpu, fault->gfn)) { - trace_kvm_async_pf_repeated_fault(fault->addr, fault->gfn); - kvm_make_request(KVM_REQ_APF_HALT, vcpu); - return RET_PF_RETRY; - } else if (kvm_arch_setup_async_pf(vcpu, fault->addr, fault->gfn)) { - return RET_PF_RETRY; - } - } - - /* - * Allow gup to bail on pending non-fatal signals when it's also allowed - * to wait for IO. Note, gup always bails if it is unable to quickly - * get a page and a fatal signal, i.e. SIGKILL, is pending. - */ - fault->pfn =3D __gfn_to_pfn_memslot(slot, fault->gfn, false, true, NULL, - fault->write, &fault->map_writable, - &fault->hva); - return RET_PF_CONTINUE; -} - -static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *f= ault, - unsigned int access) -{ - int ret; - - fault->mmu_seq =3D vcpu->kvm->mmu_invalidate_seq; - smp_rmb(); - - /* - * Check for a private vs. shared mismatch *after* taking a snapshot of - * mmu_invalidate_seq, as changes to gfn attributes are guarded by the - * invalidation notifier. - */ - if (fault->is_private !=3D kvm_mem_is_private(vcpu->kvm, fault->gfn)) { - kvm_mmu_prepare_memory_fault_exit(vcpu, fault); - return -EFAULT; - } - /* * Check for a relevant mmu_notifier invalidation event before getting * the pfn from the primary MMU, and before acquiring mmu_lock. @@ -4458,6 +4458,7 @@ static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, str= uct kvm_page_fault *fault, if (ret !=3D RET_PF_CONTINUE) return ret; =20 +faultin_done: if (unlikely(is_error_pfn(fault->pfn))) return kvm_handle_error_pfn(vcpu, fault); =20 --=20 2.44.0.278.ge034bb2e1d-goog From nobody Mon Nov 25 19:51:37 2024 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D8130200B8 for ; Wed, 28 Feb 2024 02:42:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088137; cv=none; b=DkTWAjqgR1oOtRQQJMXw7q47lShOXY1CM/9vYYQS7d89yavHSP1a+CqX9Q1VYn5IAGlAplqaPiAZyMKOqvPV/eBia7Yv+rZEDg/XQucL89Vww2iGA5PVt570hSgcgZJBbUO3EoFCOhhMpjCZ35/uMca1wh+pfj3lAeP5yiBU2Wc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088137; c=relaxed/simple; bh=0gEhOu+NvnccPPQj+DGkgmpKe+mDJdiQ8EYFWVgv6Yg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=KqdVeYRoJYJD0T/41Z8PBBi+ORPS3Xpt8IfdzIia3d2z8QZbA1ZXEBSmfYZiwUFqFmZCe/doqSfGESYlLl6L8I5UPIEGeMRQHEnNbZFmyh4UX2WOZ7dY04c+V/tXA7WAB8SZb8AcPkBmlciYADdQfyX1JnI9AmtzuUZ6ZGHKiJM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=K3vCi+W9; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="K3vCi+W9" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-60804b369c7so53335667b3.2 for ; Tue, 27 Feb 2024 18:42:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709088135; x=1709692935; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=l9B6PZq9lTAwCPQClSfUQyAcFb6buJcOcs8BLLLIFfI=; b=K3vCi+W9PP9mOXJVXEgpPk8GWlyWO7FsUI5EGYp9/KHl/81V5sfa2Ccl1HnuFMY4Uv CgxYNnEkV5Ky8lF7XRWzuGtVNb1olRoag5MKYejOOy0jy4FTBvf7AA8pyDYIG+Sa/ygg 9kgq1/g/fFWFBUqHJxX9xnyn5vQ4PEh2VV0Mq4ChxWj2ZrR1jM8c3XHVPgWyMr2jNtz0 +x4REkioKYUz5hHUmdpwEaNp9JWbQYagqq5uWE/upqWJp3KqhAEJO+IcCpdHKf2pYri8 xLoc+lxD2m4Htsydv27gwksD+ofGYKndcFQpgHW01YKvE2WIlFX+3zCMhHcr1Thn/RQI pGLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709088135; x=1709692935; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=l9B6PZq9lTAwCPQClSfUQyAcFb6buJcOcs8BLLLIFfI=; b=Tfg2sMsyQBEXgB9IydG1BZuzSzXflKO7DyoTadCNmnLLqSe8Vm9d8ADmDBtQkf20iX FNJKNQC8bv677zYyaOpkqp/V9TL8Z2V/wLzOlTYdsUrlLqpT+Yt0e0CqiStlHB7btocU B32IFLKP27wl+s0jS1dR+GiYHYZ2Tjl2QdheKuGbDcUpDSFA4TaCDasdSRoqkGHTr9S8 +5Ej0gXX3+0Kwy8VGRZrVIiTcytA7ctKkjbF0JfrJuu7b254mPu5C3gD4mvBVfwexgb8 O0fB0hKoA82zjgz6IL7CeLP900bTB8ntDzjnmlWB9pMol1EVHdoMtG2xaG/X/Ilw+Q8m G5ig== X-Forwarded-Encrypted: i=1; AJvYcCXpsiqWyuigY8Bk57iORLaOpqStxI3KwkZpBUpMoqDU2ZxXqBQvL5iNDl2Y+k5JuOmGUPHTG/2ej0poGMCilGk3ebyTQeNXOZZ1VxIF X-Gm-Message-State: AOJu0Yx5j9uurjIoqLv71fQjaeEKYGoUALANlbvkU29gZ4JFcNf5vBq6 qPeTiWy3dARURE33Rbo9KycNZItKvMCaTkUYF3adbNkueOJiiYzjJOfSw0rdh4pQ7QPg9H6DF0Z DFg== X-Google-Smtp-Source: AGHT+IEPzkpN558mdThb1mQJAX8w0FWfivalHkGQsaKz9YUi8JxhkESyegzZmqcp881opi3TCAgG1XKM99s= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a0d:d64e:0:b0:608:d0a4:75ea with SMTP id y75-20020a0dd64e000000b00608d0a475eamr905939ywd.7.1709088134972; Tue, 27 Feb 2024 18:42:14 -0800 (PST) Reply-To: Sean Christopherson Date: Tue, 27 Feb 2024 18:41:44 -0800 In-Reply-To: <20240228024147.41573-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240228024147.41573-1-seanjc@google.com> X-Mailer: git-send-email 2.44.0.278.ge034bb2e1d-goog Message-ID: <20240228024147.41573-14-seanjc@google.com> Subject: [PATCH 13/16] KVM: x86/mmu: Handle no-slot faults at the beginning of kvm_faultin_pfn() From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yan Zhao , Isaku Yamahata , Michael Roth , Yu Zhang , Chao Peng , Fuad Tabba , David Matlack Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Handle the "no memslot" case at the beginning of kvm_faultin_pfn(), just after the private versus shared check, so that there's no need to repeatedly query whether or not a slot exists. This also makes it more obvious that, except for private vs. shared attributes, the process of faulting in a pfn simply doesn't apply to gfns without a slot. Opportunistically stuff @fault's metadata in kvm_handle_noslot_fault() so that it doesn't need to be duplicated in all paths that invoke kvm_handle_noslot_fault(), and to minimize the probability of not stuffing the right fields. Leave the existing handle behind, but convert it to a WARN, to guard against __kvm_faultin_pfn() unexpectedly nullifying fault->slot. Cc: David Matlack Signed-off-by: Sean Christopherson Reviewed-by: Kai Huang --- arch/x86/kvm/mmu/mmu.c | 29 +++++++++++++++++------------ arch/x86/kvm/mmu/mmu_internal.h | 2 +- 2 files changed, 18 insertions(+), 13 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 8aa957f0a717..4dee0999a66e 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3322,6 +3322,10 @@ static int kvm_handle_noslot_fault(struct kvm_vcpu *= vcpu, vcpu_cache_mmio_info(vcpu, gva, fault->gfn, access & shadow_mmio_access_mask); =20 + fault->slot =3D NULL; + fault->pfn =3D KVM_PFN_NOSLOT; + fault->map_writable =3D false; + /* * If MMIO caching is disabled, emulate immediately without * touching the shadow page tables as attempting to install an @@ -4393,15 +4397,18 @@ static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, s= truct kvm_page_fault *fault, return -EFAULT; } =20 + if (unlikely(!slot)) + return kvm_handle_noslot_fault(vcpu, fault, access); + /* * Retry the page fault if the gfn hit a memslot that is being deleted * or moved. This ensures any existing SPTEs for the old memslot will * be zapped before KVM inserts a new MMIO SPTE for the gfn. */ - if (slot && (slot->flags & KVM_MEMSLOT_INVALID)) + if (slot->flags & KVM_MEMSLOT_INVALID) return RET_PF_RETRY; =20 - if (slot && slot->id =3D=3D APIC_ACCESS_PAGE_PRIVATE_MEMSLOT) { + if (slot->id =3D=3D APIC_ACCESS_PAGE_PRIVATE_MEMSLOT) { /* * Don't map L1's APIC access page into L2, KVM doesn't support * using APICv/AVIC to accelerate L2 accesses to L1's APIC, @@ -4413,12 +4420,9 @@ static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, st= ruct kvm_page_fault *fault, * uses different roots for L1 vs. L2, i.e. there is no danger * of breaking APICv/AVIC for L1. */ - if (is_guest_mode(vcpu)) { - fault->slot =3D NULL; - fault->pfn =3D KVM_PFN_NOSLOT; - fault->map_writable =3D false; - goto faultin_done; - } + if (is_guest_mode(vcpu)) + return kvm_handle_noslot_fault(vcpu, fault, access); + /* * If the APIC access page exists but is disabled, go directly * to emulation without caching the MMIO access or creating a @@ -4429,6 +4433,9 @@ static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, str= uct kvm_page_fault *fault, return RET_PF_EMULATE; } =20 + fault->mmu_seq =3D vcpu->kvm->mmu_invalidate_seq; + smp_rmb(); + /* * Check for a relevant mmu_notifier invalidation event before getting * the pfn from the primary MMU, and before acquiring mmu_lock. @@ -4450,19 +4457,17 @@ static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, s= truct kvm_page_fault *fault, * *guaranteed* to need to retry, i.e. waiting until mmu_lock is held * to detect retry guarantees the worst case latency for the vCPU. */ - if (fault->slot && - mmu_invalidate_retry_gfn_unsafe(vcpu->kvm, fault->mmu_seq, fault->gfn= )) + if (mmu_invalidate_retry_gfn_unsafe(vcpu->kvm, fault->mmu_seq, fault->gfn= )) return RET_PF_RETRY; =20 ret =3D __kvm_faultin_pfn(vcpu, fault); if (ret !=3D RET_PF_CONTINUE) return ret; =20 -faultin_done: if (unlikely(is_error_pfn(fault->pfn))) return kvm_handle_error_pfn(vcpu, fault); =20 - if (unlikely(!fault->slot)) + if (WARN_ON_ONCE(!fault->slot)) return kvm_handle_noslot_fault(vcpu, fault, access); =20 /* diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_interna= l.h index d7c10d338f14..74736d517e74 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -235,7 +235,7 @@ struct kvm_page_fault { /* The memslot containing gfn. May be NULL. */ struct kvm_memory_slot *slot; =20 - /* Outputs of kvm_faultin_pfn. */ + /* Outputs of kvm_faultin_pfn. */ unsigned long mmu_seq; kvm_pfn_t pfn; hva_t hva; --=20 2.44.0.278.ge034bb2e1d-goog From nobody Mon Nov 25 19:51:37 2024 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 97B9C25566 for ; Wed, 28 Feb 2024 02:42:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088139; cv=none; b=KocNDApRuuS/TEzmoPnpdr51nyaWQql4v0wOY3nau2fzzKRk5Ub5uk6uqdiesGbLadi8II4sWMcbQwOyFmtIzpE+nUtAy0S/gC5CC1f8dXcI7ybLOqfyzhD1Gg6atWm0IVkSTVTazFmkA/Cqcka6Rk7frj7PiZ7k8FfITSDBjQM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088139; c=relaxed/simple; bh=JLsN4xg251qE9a3MZGdWpvkmGCZsQVcBJtHNmsnHqx0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=D+8up5P0Cwk7Z13SaB82BFsiLSw1intS5mt9FsVk4eQIzkUg2G37rXYQ+AWa9nCjgDxwrYEnkvvLZ7oG3rkMiNfVs4Zns+dvS7GkqYe4OpbRyd1mPHq042OwS6lipGVijYyvIeTgprhaJaIQ3aVgAxN5gz0s5XZWegdnRKYMcig= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=B2DwrAqe; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="B2DwrAqe" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-5cf555b2a53so304953a12.1 for ; Tue, 27 Feb 2024 18:42:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709088137; x=1709692937; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=za8QaygzoKIY5MyMe/Z2Ubm1oLIN1MbBZtglXwIHigs=; b=B2DwrAqeZ3hxUE5PrHwB//zObnj3gOVUrBHZ7xlBCCpYJwsaZsujU1Wd6wRYLYKFVQ ciPVYnK1SHSFjRSfWcz7DzJ+KuBdcNZaVAp6YQxNlQrmZ9ozftnUOEf9bEVq6FwNyGZT uUcqGZbU2HNYM7KrHo3KmJ3Inmi3YIJoXDEDlrG5x1iddFnp9C48nEQ/1+7Hl1TrilfH 1qri28/ll7THvl6UBOzAy/IWkLi/YqZIdINLf52ijL2HEPWA9UL9M785vvTRjSWr1j7t FZnkQP1xBcX2LzHqHTFuGrsctfXMggNxh6HQiut930hQsKBmCOHK/+EpKSutmNoTP26H Gtuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709088137; x=1709692937; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=za8QaygzoKIY5MyMe/Z2Ubm1oLIN1MbBZtglXwIHigs=; b=e5ejnIR19Qolhbca8xPcAiE4CbjHHKo2gqnykfGQP4zQdHu+hk08fwW8bTEPMPWoso pBzXER+rdMAwcnFKgfZfayC5Y0tRgKQcY64ewfYy7bILtzapPoxjCJzt3rDfoFb+f02j JFoijE5mhM0K9wgj/TC1Oh6CYDR/GVWvdkcutsFssTH8GEGCisPxzE1IEWdIEA7ykhBE zauMEnHfQurhBYL9k24/t0Shu6YldrbIurO46m8kvoEGd3tCdU9slFjGbMCAd4R1024s WRdTrhlSxtxmp8TLVOh2wZfdyKSyYnTMA8V0m6Tjy9u0l/opz/CmPwupEhYSWMPYXxRz m6Lw== X-Forwarded-Encrypted: i=1; AJvYcCUsxrvWt2GowtecU3FClGzQ9D/+V1tCQhp8tFei36SlZA1l9M6+HZCcrTGgDPe5pSlxbewzX/Use7VU4KVAUIWfou0QmmW9IzAsByxM X-Gm-Message-State: AOJu0YyPEzk4xvq0m7NGWnYglDKa2dfpQYfSpRcgfoln/1LjOEagSyOx Kf+nfj956sqDynlN1KGbNWg4tmc1odlun0V/MEB0L+RBKbXMz7cPiLw+jP9CTMswPws1IPyCdxa 8cA== X-Google-Smtp-Source: AGHT+IEmZWdUDpfZFtSaGW+CNF5X12KqObi0fTpi5HyUpexKv6qJR0Hv72mZ7zAGZ81rRBqkZor7Gh8nLJE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:fa83:b0:1db:a739:d17b with SMTP id lc3-20020a170902fa8300b001dba739d17bmr2743plb.1.1709088136837; Tue, 27 Feb 2024 18:42:16 -0800 (PST) Reply-To: Sean Christopherson Date: Tue, 27 Feb 2024 18:41:45 -0800 In-Reply-To: <20240228024147.41573-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240228024147.41573-1-seanjc@google.com> X-Mailer: git-send-email 2.44.0.278.ge034bb2e1d-goog Message-ID: <20240228024147.41573-15-seanjc@google.com> Subject: [PATCH 14/16] KVM: x86/mmu: Set kvm_page_fault.hva to KVM_HVA_ERR_BAD for "no slot" faults From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yan Zhao , Isaku Yamahata , Michael Roth , Yu Zhang , Chao Peng , Fuad Tabba , David Matlack Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Explicitly set fault->hva to KVM_HVA_ERR_BAD when handling a "no slot" fault to ensure that KVM doesn't use a bogus virtual address, e.g. if there *was* a slot but it's unusable (APIC access page), or if there really was no slot, in which case fault->hva will be '0' (which is a legal address for x86). Signed-off-by: Sean Christopherson Reviewed-by: Kai Huang --- arch/x86/kvm/mmu/mmu.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 4dee0999a66e..43f24a74571a 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3325,6 +3325,7 @@ static int kvm_handle_noslot_fault(struct kvm_vcpu *v= cpu, fault->slot =3D NULL; fault->pfn =3D KVM_PFN_NOSLOT; fault->map_writable =3D false; + fault->hva =3D KVM_HVA_ERR_BAD; =20 /* * If MMIO caching is disabled, emulate immediately without --=20 2.44.0.278.ge034bb2e1d-goog From nobody Mon Nov 25 19:51:37 2024 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A34D2C6BB for ; Wed, 28 Feb 2024 02:42:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088141; cv=none; b=pWzCjtJsZunn1sMRfheVD7GEQsxBOGsq527a5h416WeM4rsr4yk/vdAgvz94haRLUrX8oZRbyqZ1OxPox/9MC825+MC3wqVLYKxyBVuCdJtl5wJ1CetYczTUODsSadbhIHxWOz96VXuqFVqCcA7mVPqm8L00c80VXJeetnC46XM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088141; c=relaxed/simple; bh=ixgVj0KGrSU4gTjyAiHEvZBk3jB06YOVx+ihjzW4I4g=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=WTvGO4PUpAQ3sY9HGkQuLzSl/1917k6K9YSQqllrRsDo9A/YmlNV8b55+swEqvt0GfWtRO/6dT4pBIbma8uRnFVI9oiEI77AtHGVIA+4OSkFmNf7SFS6l76nrvasL00OJDs0TzMFsD7OaYtdkLpstUltJWcw+w01vVsFRAuGE5Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=SV3xcdt8; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="SV3xcdt8" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-29ad35a8650so385893a91.0 for ; Tue, 27 Feb 2024 18:42:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709088138; x=1709692938; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=rtApIkKLSvdOXgbJmewj1zhLCz1lTULYuVLD/orXMmk=; b=SV3xcdt82+EU0XYkzm0TXdWcoFNN0pTij3VezjKwKwBoZpD4MP6Mp0kgJYA+lofPVO VFwFUzPt6pss/3YVmtc9cSRifd2El2If1O8h8rhhYQw9BOagtfntoxFuHF61YCc2PxIK M4/CeJegY2PrgfNmD2opHJn6WsKz2WvEPxopleQzdv4GQfDpgTawirY96olqYzRgkTgD fZs1WRev+q+L8k5m260El8BjflnMltpOkFWuNTfZaqJT9Aam5Mb/x34F4d18bVO8WZ2n Aag4ZsYZFdPOWO7TZpv8VJAEsxf7x+wLVk7nJ+0G4hVAUNmvyS7zBMsYLmq49d1iGo14 vxeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709088138; x=1709692938; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=rtApIkKLSvdOXgbJmewj1zhLCz1lTULYuVLD/orXMmk=; b=wFkGfOaQbhx1j3/sgK6jXHcsasnwbPABbvpbBPV4ASaeDyCQIawnr/M5WqQUH1yYTP EwL/x12DfroJBZF/KPumcqz4G/SaphRvdPG660d4csYkW0sww+MHJI7iZwYFmstKs2Fv gli8tl05wAutPS/F9VfPPlmhk+TJbluOMbbSA15ZoRMFDTuhAlyyK+LNjvcwsLcnIyF0 5DDl6JLHl1B+B+z5SHGh4pdHiTUWvL4fb3gdAJSjJhWSGRIcctC1tcb7KVoXe+uSFggs UJXcI4teZ3NqwcmCFmL1lNXbL05haz0d2zURUWdT6IC2B84dijuH5cMrNuOV7lcMCzZC xrHQ== X-Forwarded-Encrypted: i=1; AJvYcCWplM/YWW9+Z15GSsimw9WhSxShmRNrIkPiNYtUktNbpiCZNxRHG0OFYo85ib/afW6edM1/YfWSomdtaZ3qg4qp3CGQpVQwu+YSR0Zx X-Gm-Message-State: AOJu0YwcaZvfvnLATSmpJZ95a8TeBgdPE7JMIAmWfGufjCyhDoF6YgnI aIE9FYUPX815UKICaIeJ4xIHRuaqB5SnjA/tp5obZosMo3ruwyF3cI7JNaDKOHy8vn8RFaRh34l ZnA== X-Google-Smtp-Source: AGHT+IH9tXof/kR1AMzV1Lra2XibmJjG90HRu0dfNOBX/JbDNMXAO+L+Avq5RMTgEZhP13UN3jO7pyoCH3E= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:90a:17cc:b0:29a:61fa:e3fc with SMTP id q70-20020a17090a17cc00b0029a61fae3fcmr5365pja.2.1709088138588; Tue, 27 Feb 2024 18:42:18 -0800 (PST) Reply-To: Sean Christopherson Date: Tue, 27 Feb 2024 18:41:46 -0800 In-Reply-To: <20240228024147.41573-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240228024147.41573-1-seanjc@google.com> X-Mailer: git-send-email 2.44.0.278.ge034bb2e1d-goog Message-ID: <20240228024147.41573-16-seanjc@google.com> Subject: [PATCH 15/16] KVM: x86/mmu: Initialize kvm_page_fault's pfn and hva to error values From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yan Zhao , Isaku Yamahata , Michael Roth , Yu Zhang , Chao Peng , Fuad Tabba , David Matlack Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Explicitly set "pfn" and "hva" to error values in kvm_mmu_do_page_fault() to harden KVM against using "uninitialized" values. In quotes because the fields are actually zero-initialized, and zero is a legal value for both page frame numbers and virtual addresses. E.g. failure to set "pfn" prior to creating an SPTE could result in KVM pointing at physical address '0', which is far less desirable than KVM generating a SPTE with reserved PA bits set and thus effectively killing the VM. Signed-off-by: Sean Christopherson Reviewed-by: Kai Huang --- arch/x86/kvm/mmu/mmu_internal.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_interna= l.h index 74736d517e74..67e32dec9424 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -307,6 +307,9 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu= *vcpu, gpa_t cr2_or_gpa, .req_level =3D PG_LEVEL_4K, .goal_level =3D PG_LEVEL_4K, .is_private =3D err & PFERR_PRIVATE_ACCESS, + + .pfn =3D KVM_PFN_ERR_FAULT, + .hva =3D KVM_HVA_ERR_BAD, }; int r; =20 --=20 2.44.0.278.ge034bb2e1d-goog From nobody Mon Nov 25 19:51:37 2024 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 08EFE2D05D for ; Wed, 28 Feb 2024 02:42:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088146; cv=none; b=fb1qb5FKxu4jIeUFHcH0/IKMsIEVMFF7lx6NjBURJfrkhEFWVXJUNbs3YGNVUdIWFYz9p0h3h2FSOfpFw5mI9isvWlIBoT2rsU6PTa4q99urS3fX22PWOjhrxZpzI4cX0qbZgZdNQ96M5TaWpqG9bZ4saVQzkXxYNjh/ANTuOqA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709088146; c=relaxed/simple; bh=szLQWH4Y5Bzo6Uy7reBTywxc6dUFXHryHkO47zK9DXo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=OuL5yGe5S+omMHQDD8/IEq325uE4sjYbFC3Su3B9rhHVNYRJzJ75EkG7pYMFHaIa6W2zrzsNLU9BSOHxqYrXHoO2Ah+042Mr27S3L+LDFv+Jq01aBc0Uuug6/XDvNHeFoFUBcXFxA82/m3N/+cPGbXNDGIOz120sDSdv55ds7sU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=rpsnoFLQ; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="rpsnoFLQ" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2993efc802dso2200183a91.1 for ; Tue, 27 Feb 2024 18:42:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709088140; x=1709692940; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=fK/e/x3+Tf/3Rz4S7hZymU4b4heBfx/RABet1RXMTfk=; b=rpsnoFLQv8Hr66kEDU1R/DQUQ6CdvcALLXvKoaMxmlhmQQG0O67AeLnt9n4kiGI3r+ PWR7gPyio7d5kk9PHvrwe7XFTqnGtSBvBBUha/E58qdt+bIRrE6ajvnakAJv8ygpVNm0 ZVB22rxYNL2l03f1fOgK3fKGP2AMjMOFUenCCXtDI1rvr148vHTjobq17YZnIdfdOrty Z7LqtuCCEzC2upLTszhOf9oi1MamnqctqOUGgibMm3WrrOxMYmtZtV/76so88kR7Fwl2 KgJ8DDkgQakRo3CVqGSAEY/9qgpAiz6YRMPVu36M27+i2ImmG5uj2SU1X8ON65jaU2KD OIUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709088140; x=1709692940; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=fK/e/x3+Tf/3Rz4S7hZymU4b4heBfx/RABet1RXMTfk=; b=S2jsREXF+16tvZn+wmzZF2taq3zf98Uuu+BKxqGosvPQJsOKVtiLQU8aqW1XeACQrO Ynx8YM97sCZTYroOFyZB0NjgekD248oeBS3qHPDYKfmB168y655p7tCQQTQ8zXUsnmw1 /M3iMU8zD/foTHvlkWS0+dHA165VwvohQ2sle60mkzoyuGvi3zDZ2pyg0Ivg3NFjg08p YYsrkuEQ3AFECkm17PEZXnTawZ/6voYMlVjJQzyvDScWvm5P0BItqbpRnbktV7zc1vjD 4wVWBdT82dyfh9WpAJu1yTCFLHlPyYuOhz8WmQTgmdK3o+ZwPzhPYnp2KLmOLEkUtcjF PekA== X-Forwarded-Encrypted: i=1; AJvYcCV4gy9Ffufa+mXn2Y1GxwyumLJdHGKKAkM6Yo7VNEW4WC9cHPlDr5Mp7MI12v2nPs4QGJhtoNelM9tC0Sdbxb9TNtVHarVuBwCrYelv X-Gm-Message-State: AOJu0YyV9xfHfz/yqbZLOQvuMzia9i8Zs21dBaaVrEDKmhc25jQm6WcR s/jODDAipauaN4uokey+YzmVY9Y8VOfatF1YYSxz2pFbQt13cd5HTmCVE+UwNexWQ5ydUgsuwzo M6w== X-Google-Smtp-Source: AGHT+IFQ0g9E+Mk5m1PF8jqDNWlJr3OOxNFUEzGXFuClugZHZeqtSpftN+Bu/TNy5X1nKDvg9GP+3UI48k4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:90b:2d0b:b0:299:40c3:338f with SMTP id sy11-20020a17090b2d0b00b0029940c3338fmr38855pjb.1.1709088140382; Tue, 27 Feb 2024 18:42:20 -0800 (PST) Reply-To: Sean Christopherson Date: Tue, 27 Feb 2024 18:41:47 -0800 In-Reply-To: <20240228024147.41573-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240228024147.41573-1-seanjc@google.com> X-Mailer: git-send-email 2.44.0.278.ge034bb2e1d-goog Message-ID: <20240228024147.41573-17-seanjc@google.com> Subject: [PATCH 16/16] KVM: x86/mmu: Sanity check that __kvm_faultin_pfn() doesn't create noslot pfns From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yan Zhao , Isaku Yamahata , Michael Roth , Yu Zhang , Chao Peng , Fuad Tabba , David Matlack Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" WARN if __kvm_faultin_pfn() generates a "no slot" pfn, and gracefully handle the unexpected behavior instead of continuing on with dangerous state, e.g. tdp_mmu_map_handle_target_level() _only_ checks fault->slot, and so could install a bogus PFN into the guest. The existing code is functionally ok, because kvm_faultin_pfn() pre-checks all of the cases that result in KVM_PFN_NOSLOT, but it is unnecessarily unsafe as it relies on __gfn_to_pfn_memslot() getting the _exact_ same memslot, i.e. not a re-retrieved pointer with KVM_MEMSLOT_INVALID set. And checking only fault->slot would fall apart if KVM ever added a flag or condition that forced emulation, similar to how KVM handles writes to read-only memslots. Cc: David Matlack Signed-off-by: Sean Christopherson Reviewed-by: Kai Huang --- arch/x86/kvm/mmu/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 43f24a74571a..cedacb1b89c5 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4468,7 +4468,7 @@ static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, str= uct kvm_page_fault *fault, if (unlikely(is_error_pfn(fault->pfn))) return kvm_handle_error_pfn(vcpu, fault); =20 - if (WARN_ON_ONCE(!fault->slot)) + if (WARN_ON_ONCE(!fault->slot || is_noslot_pfn(fault->pfn))) return kvm_handle_noslot_fault(vcpu, fault, access); =20 /* --=20 2.44.0.278.ge034bb2e1d-goog