From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 236152EBBA4 for ; Tue, 28 Apr 2026 11:09:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374596; cv=none; b=RNaTOhdGZSGXv/OjOKvhLd6TMQgF8pw80HJcgH9AYeaNJGp6il+Vo4xaftUxyszvEvHSiymysMSMfbLrJ2NvLapp6U9iGt+nMaVSkepo2pFTF9wX726Ki0ZNzaFzQne0skv0dagtwY4C7rD3ENVWFyNRk3rEUF2QDZpF3NVjGMY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374596; c=relaxed/simple; bh=m/y1kMQz/0mZXMPtm+sW3IoQNR3ZtUfQB3hgxkbWXHQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Hh7juGr6xRiRBDZj5KgO/S3ZvAc6A1rQmUItPOflwKk3erZ7dBQnsyWR3N0nqwZL6ZFr3LglzXwlN0/HISm9usPsJT5SYxFhkHhxQSDYSZK0bZRuTxFBwTXaO0Zw7zVGJDG3ke93naWQaBdL1D/sP6zffhWeIOEwTh649uX1QTo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=YTSOztVu; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="YTSOztVu" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374593; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qxFFgrsB72Gtu2mLfBxxq44PkZ2W/bhE3LoVP54M540=; b=YTSOztVuQ8aeQCmU1SlKCfmdksktap6S/JBIJx/U/C99gBWeLMDfABTlQffzav6ciHP+CQ PzraAdaIQV4z7kV/L4ID0gPaxn2Tf7zPkT8/oTSOBbtpwTHvmjbYscYc5h0Ek7IsvozccQ +uKhy9EHTvRPogV97X6LjJQoJsujXrY= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-176-2P33H7KyNhiwBKcJK9tMDA-1; Tue, 28 Apr 2026 07:09:50 -0400 X-MC-Unique: 2P33H7KyNhiwBKcJK9tMDA-1 X-Mimecast-MFC-AGG-ID: 2P33H7KyNhiwBKcJK9tMDA_1777374589 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id AD0F51956048; Tue, 28 Apr 2026 11:09:48 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id BD2DA300757C; Tue, 28 Apr 2026 11:09:47 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com, Sean Christopherson Subject: [PATCH 01/28] KVM: TDX/VMX: rework EPT_VIOLATION_EXEC_FOR_RING3_LIN into PROT_MASK Date: Tue, 28 Apr 2026 07:09:19 -0400 Message-ID: <20260428110946.11466-2-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" From: Jon Kohler EPT exit qualification bit 6 is used when mode-based execute control is enabled, and reflects user executable addresses. Rework name to reflect the intention and add to EPT_VIOLATION_PROT_MASK, which allows simplifying the return evaluation in tdx_is_sept_violation_unexpected_pending a pinch. Rework handling in __vmx_handle_ept_violation to unconditionally clear EPT_VIOLATION_PROT_USER_EXEC until MBEC is implemented, as suggested by Sean [1]. Note: Intel SDM Table 29-7 defines bit 6 as: If the "mode-based execute control" VM-execution control is 0, the value of this bit is undefined. If that control is 1, this bit is the logical-AND of bit 10 in the EPT paging-structure entries used to translate the guest-physical address of the access causing the EPT violation. In this case, it indicates whether the guest-physical address was executable for user-mode linear addresses. [1] https://lore.kernel.org/all/aCJDzU1p_SFNRIJd@google.com/ Suggested-by: Sean Christopherson Signed-off-by: Jon Kohler Message-ID: <20251223054806.1611168-2-jon@nutanix.com> Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/include/asm/vmx.h | 5 +++-- arch/x86/kvm/vmx/common.h | 9 +++++++-- arch/x86/kvm/vmx/tdx.c | 2 +- 3 files changed, 11 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h index 37080382df54..b2291a766e3f 100644 --- a/arch/x86/include/asm/vmx.h +++ b/arch/x86/include/asm/vmx.h @@ -608,10 +608,11 @@ enum vm_entry_failure_code { #define EPT_VIOLATION_PROT_READ BIT(3) #define EPT_VIOLATION_PROT_WRITE BIT(4) #define EPT_VIOLATION_PROT_EXEC BIT(5) -#define EPT_VIOLATION_EXEC_FOR_RING3_LIN BIT(6) +#define EPT_VIOLATION_PROT_USER_EXEC BIT(6) #define EPT_VIOLATION_PROT_MASK (EPT_VIOLATION_PROT_READ | \ EPT_VIOLATION_PROT_WRITE | \ - EPT_VIOLATION_PROT_EXEC) + EPT_VIOLATION_PROT_EXEC | \ + EPT_VIOLATION_PROT_USER_EXEC) #define EPT_VIOLATION_GVA_IS_VALID BIT(7) #define EPT_VIOLATION_GVA_TRANSLATED BIT(8) =20 diff --git a/arch/x86/kvm/vmx/common.h b/arch/x86/kvm/vmx/common.h index 412d0829d7a2..adf925500b9e 100644 --- a/arch/x86/kvm/vmx/common.h +++ b/arch/x86/kvm/vmx/common.h @@ -94,8 +94,13 @@ static inline int __vmx_handle_ept_violation(struct kvm_= vcpu *vcpu, gpa_t gpa, /* Is it a fetch fault? */ error_code |=3D (exit_qualification & EPT_VIOLATION_ACC_INSTR) ? PFERR_FETCH_MASK : 0; - /* ept page table entry is present? */ - error_code |=3D (exit_qualification & EPT_VIOLATION_PROT_MASK) + /* + * ept page table entry is present? + * note: unconditionally clear USER_EXEC until mode-based + * execute control is implemented + */ + error_code |=3D (exit_qualification & + (EPT_VIOLATION_PROT_MASK & ~EPT_VIOLATION_PROT_USER_EXEC)) ? PFERR_PRESENT_MASK : 0; =20 if (exit_qualification & EPT_VIOLATION_GVA_IS_VALID) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 04ce321ebdf3..30cd5b19ecf9 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -1845,7 +1845,7 @@ static inline bool tdx_is_sept_violation_unexpected_p= ending(struct kvm_vcpu *vcp if (eeq_type !=3D TDX_EXT_EXIT_QUAL_TYPE_PENDING_EPT_VIOLATION) return false; =20 - return !(eq & EPT_VIOLATION_PROT_MASK) && !(eq & EPT_VIOLATION_EXEC_FOR_R= ING3_LIN); + return !(eq & EPT_VIOLATION_PROT_MASK); } =20 static int tdx_handle_ept_violation(struct kvm_vcpu *vcpu) --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 253F53F1643 for ; Tue, 28 Apr 2026 11:09:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374597; cv=none; b=mmLX+vq5p2dkhkgiPb9ZLLvucPUae/75mSRmBCiDipvkhmGrOYOw8btQOqQ9dCSN1Pj2+Mc4R+OgD5/qNxpE4toGLr0uL7VrAMtc7VfNeB2pZkw5+yI9xESo3d+XlBWOJjV8K0KPv7apkzW/qTTaWs4mVcS4Z10wWGjsrwzUTf4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374597; c=relaxed/simple; bh=BG1kAh7EtMGWmcMC1GlPnoHKnRNOqLKvCGNaN/4yEYg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=n/Br/bZ/nMNaVRr5T0jYNnGsgbtOsYZsB+kWM9amJpghvPAA7knwx0Oc3mM/M4b1j3f0c6hr6xg7xJTmtZRe0KX/V7F8auIbDr4+xYyR23GGdPjqQr6+oFReXfXB3lCi/A37pv7JBr/fXWkpuOBRp4BF7yhtoQ6x6B4iDbMr5L8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=WEhrjr3V; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="WEhrjr3V" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374594; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=COO67oamJrUqujxmroIoUz2hjP+GCsrSsINBeLnuCKQ=; b=WEhrjr3VsKXQE29QIgxsoSeSBpjtthP/ygt6uVBaMaAwVEFGqGeTpPEU1zfs6mOK7Hi4Je NxhTVJH2mU9lYVYxoyKxt1C1MRsD2Atbp6FSLi7qeOcx9XN1k83kN/q8fQdRBc2G0Px3Yg WpAX/crJv4X5IjJd6PXp/wiw5qZMnKg= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-202-YGQv8xdhPnuCk936_H7IZw-1; Tue, 28 Apr 2026 07:09:50 -0400 X-MC-Unique: YGQv8xdhPnuCk936_H7IZw-1 X-Mimecast-MFC-AGG-ID: YGQv8xdhPnuCk936_H7IZw_1777374589 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 721681956095; Tue, 28 Apr 2026 11:09:49 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D32E03000C22; Tue, 28 Apr 2026 11:09:48 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 02/28] KVM: x86/mmu: remove SPTE_PERM_MASK Date: Tue, 28 Apr 2026 07:09:20 -0400 Message-ID: <20260428110946.11466-3-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" From: Jon Kohler SPTE_PERM_MASK is no longer referenced by anything in the kernel. Signed-off-by: Jon Kohler Message-ID: <20251223054806.1611168-3-jon@nutanix.com> Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/spte.h | 3 --- 1 file changed, 3 deletions(-) diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index 8c0ffa2cded6..f4dfb8dc74de 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -42,9 +42,6 @@ static_assert(SPTE_TDP_AD_ENABLED =3D=3D 0); #define SPTE_BASE_ADDR_MASK (((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1)) #endif =20 -#define SPTE_PERM_MASK (PT_PRESENT_MASK | PT_WRITABLE_MASK | shadow_user_m= ask \ - | shadow_x_mask | shadow_nx_mask | shadow_me_mask) - #define ACC_EXEC_MASK 1 #define ACC_WRITE_MASK PT_WRITABLE_MASK #define ACC_USER_MASK PT_USER_MASK --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A7A803F1664 for ; Tue, 28 Apr 2026 11:09:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374598; cv=none; b=J7zoS+spCCLQlJkgpK31btdUluIhJwq4DcX8jnqqF1OSLungjLRn16XsR1AKPnxSmRBA/5drNxOjivDPGMfA3ZxtvTKijDqe12+9lPyoaDiXJTrsukS6Be7Pb29FSEOXkPcegiE66RycTD6A0l7xq8F2F1FxLIgE23B1KqWZVw0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374598; c=relaxed/simple; bh=kLEqS1nmC9j7BHGpqDgkveiR0cR3gllOF/Mq+O4SMZQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=LyIkMHH3kZF2aeY1vxtJQTGo4GQp474WcdBFn6Liubh/jx9YXIN5GIdUm1t1QJuNm0uWG9r5+nYv8QP12Bx2SdHp/BZRyNCcAWazfS7C78DMIs/lA5vzHr51ysQPd1cIDBtgTLKpbcxlwYQ88ID0CRvrlueE6zaFW94tvYCH3kA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=D0a5YodK; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="D0a5YodK" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374594; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Hh50KiakvUiqoDmpF5tROh/JlkwQTxMvtXgvfpCmvNU=; b=D0a5YodKpksoWAS2RrCvM+baABglaKn38l8pz/w5Ze73j3o+G4i4iGHjEE37hZYBed09h1 mjNJS2GzuWXxR+G3ROml+1AzMU6ai4xN5GFXLjkaIT0Y43yXZD1bMqrtbOvm7p9nJuNsQr HZ7Zm0yxqSOGH45vaiReuMrinSo12Tc= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-97-Wh_C5A1pORCN7chf0z0P8A-1; Tue, 28 Apr 2026 07:09:51 -0400 X-MC-Unique: Wh_C5A1pORCN7chf0z0P8A-1 X-Mimecast-MFC-AGG-ID: Wh_C5A1pORCN7chf0z0P8A_1777374590 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 40ADE195608B; Tue, 28 Apr 2026 11:09:50 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 96F73300757C; Tue, 28 Apr 2026 11:09:49 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com, Kai Huang Subject: [PATCH 03/28] KVM: x86/mmu: free up bit 10 of PTEs in preparation for MBEC Date: Tue, 28 Apr 2026 07:09:21 -0400 Message-ID: <20260428110946.11466-4-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" From: Jon Kohler Update SPTE_MMIO_ALLOWED_MASK to allow EPT user executable (bit 10) to be treated like EPT RWX bit2:0, as when mode-based execute control is enabled, bit 10 can act like a "present" bit. Likewise do not include it in FROZEN_SPTE. No functional changes intended, other than the reduction of the maximum MMIO generation that is stored in page tables. Cc: Kai Huang Signed-off-by: Jon Kohler Message-ID: <20251223054806.1611168-4-jon@nutanix.com> Reviewed-by: Kai Huang Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/include/asm/vmx.h | 2 ++ arch/x86/kvm/mmu/spte.h | 20 +++++++++++--------- 2 files changed, 13 insertions(+), 9 deletions(-) diff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h index b2291a766e3f..2b30b921b375 100644 --- a/arch/x86/include/asm/vmx.h +++ b/arch/x86/include/asm/vmx.h @@ -560,10 +560,12 @@ enum vmcs_field { #define VMX_EPT_ACCESS_BIT (1ull << 8) #define VMX_EPT_DIRTY_BIT (1ull << 9) #define VMX_EPT_SUPPRESS_VE_BIT (1ull << 63) + #define VMX_EPT_RWX_MASK (VMX_EPT_READABLE_MASK | = \ VMX_EPT_WRITABLE_MASK | \ VMX_EPT_EXECUTABLE_MASK) #define VMX_EPT_MT_MASK (7ull << VMX_EPT_MT_EPTE_SHIFT) +#define VMX_EPT_USER_EXECUTABLE_MASK (1ull << 10) =20 static inline u8 vmx_eptp_page_walk_level(u64 eptp) { diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index f4dfb8dc74de..a107ddb66601 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -96,11 +96,11 @@ static_assert(!(EPT_SPTE_MMU_WRITABLE & SHADOW_ACC_TRAC= K_SAVED_MASK)); #undef SHADOW_ACC_TRACK_SAVED_MASK =20 /* - * Due to limited space in PTEs, the MMIO generation is a 19 bit subset of + * Due to limited space in PTEs, the MMIO generation is an 18 bit subset of * the memslots generation and is derived as follows: * - * Bits 0-7 of the MMIO generation are propagated to spte bits 3-10 - * Bits 8-18 of the MMIO generation are propagated to spte bits 52-62 + * Bits 0-6 of the MMIO generation are propagated to spte bits 3-9 + * Bits 7-17 of the MMIO generation are propagated to spte bits 52-62 * * The KVM_MEMSLOT_GEN_UPDATE_IN_PROGRESS flag is intentionally not includ= ed in * the MMIO generation number, as doing so would require stealing a bit fr= om @@ -111,7 +111,7 @@ static_assert(!(EPT_SPTE_MMU_WRITABLE & SHADOW_ACC_TRAC= K_SAVED_MASK)); */ =20 #define MMIO_SPTE_GEN_LOW_START 3 -#define MMIO_SPTE_GEN_LOW_END 10 +#define MMIO_SPTE_GEN_LOW_END 9 =20 #define MMIO_SPTE_GEN_HIGH_START 52 #define MMIO_SPTE_GEN_HIGH_END 62 @@ -133,7 +133,8 @@ static_assert(!(SPTE_MMU_PRESENT_MASK & * and so they're off-limits for generation; additional checks ensure the = mask * doesn't overlap legal PA bits), and bit 63 (carved out for future usage= ). */ -#define SPTE_MMIO_ALLOWED_MASK (BIT_ULL(63) | GENMASK_ULL(51, 12) | GENMAS= K_ULL(2, 0)) +#define SPTE_MMIO_ALLOWED_MASK (BIT_ULL(63) | GENMASK_ULL(51, 12) | \ + BIT_ULL(10) | GENMASK_ULL(2, 0)) static_assert(!(SPTE_MMIO_ALLOWED_MASK & (SPTE_MMU_PRESENT_MASK | MMIO_SPTE_GEN_LOW_MASK | MMIO_SPTE_GEN_HIGH_MAS= K))); =20 @@ -141,7 +142,7 @@ static_assert(!(SPTE_MMIO_ALLOWED_MASK & #define MMIO_SPTE_GEN_HIGH_BITS (MMIO_SPTE_GEN_HIGH_END - MMIO_SPTE_GEN_H= IGH_START + 1) =20 /* remember to adjust the comment above as well if you change these */ -static_assert(MMIO_SPTE_GEN_LOW_BITS =3D=3D 8 && MMIO_SPTE_GEN_HIGH_BITS = =3D=3D 11); +static_assert(MMIO_SPTE_GEN_LOW_BITS =3D=3D 7 && MMIO_SPTE_GEN_HIGH_BITS = =3D=3D 11); =20 #define MMIO_SPTE_GEN_LOW_SHIFT (MMIO_SPTE_GEN_LOW_START - 0) #define MMIO_SPTE_GEN_HIGH_SHIFT (MMIO_SPTE_GEN_HIGH_START - MMIO_SPTE_GEN= _LOW_BITS) @@ -217,10 +218,11 @@ extern u64 __read_mostly shadow_nonpresent_or_rsvd_ma= sk; * * Only used by the TDP MMU. */ -#define FROZEN_SPTE (SHADOW_NONPRESENT_VALUE | 0x5a0ULL) +#define FROZEN_SPTE (SHADOW_NONPRESENT_VALUE | 0x1a0ULL) =20 -/* Frozen SPTEs must not be misconstrued as shadow present PTEs. */ -static_assert(!(FROZEN_SPTE & SPTE_MMU_PRESENT_MASK)); +/* Frozen SPTEs must not be misconstrued as shadow or MMU present PTEs. */ +static_assert(!(FROZEN_SPTE & (SPTE_MMU_PRESENT_MASK | + VMX_EPT_RWX_MASK | VMX_EPT_USER_EXECUTABLE_MASK))); =20 static inline bool is_frozen_spte(u64 spte) { --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4C6AD32ED2E for ; Tue, 28 Apr 2026 11:09:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374598; cv=none; b=brdOCLjYWUUD5yZ2iXJhDxiDRsBhLq3BO7taUabNFgUjfa8mQn7i4eZmIpaboDXaGQzsI26PNAr9WtqOkieF6tKF0iwFE1ZrD8SdrHOl1TJNbzNQtdZLpoSMBXoeFHoXdkvWyXaTKUw9gb8iECDQ2JojAtGR4a2pH+eFPNgGFGg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374598; c=relaxed/simple; bh=kZ5o17P9kuQLq3qxTyxpQUTRQOzBgubCy3Nch+ibymU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=F1MRVdZsuHKM+LUzoLTcvIWe2p3R4h7JhB2OE0Nth6lRKkzF7heajck/Oux42cu2GJRpTDO2osHDp6vK1f9JT84v/aoufoDm1H/f7BNYQkmgodoRA1wzvslmTJIqglCQcCb8rSWJ9gJ0tZKcbxWWBNBjuMpGI/sQXMMF0a8eA9I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=cCmUd+1H; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="cCmUd+1H" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374595; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NCcSrd+hKpzwGh0VZiUZWb0acdQRJpx8ZctDOWe/EHY=; b=cCmUd+1HLhv6zPavujTIkPnbuQrRJpdK6oP8J2mX6lJprH42RZPoLa7MUv6k9daOqoNMGB BBXr++jEtvKgYZxLSobiad2OlzOTS46iOR2vdZvSizSKGKv4VVYSuechY2VCqE3M/ffV5O pu8gmShlmHsPSppQ3unF/jmyVurpvpw= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-452-TASaEmlFNUWHZqxxnC5CBA-1; Tue, 28 Apr 2026 07:09:52 -0400 X-MC-Unique: TASaEmlFNUWHZqxxnC5CBA-1 X-Mimecast-MFC-AGG-ID: TASaEmlFNUWHZqxxnC5CBA_1777374591 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id F0711195608B; Tue, 28 Apr 2026 11:09:50 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 66EDB3000C22; Tue, 28 Apr 2026 11:09:50 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 04/28] KVM: x86/mmu: shuffle high bits of SPTEs in preparation for MBEC Date: Tue, 28 Apr 2026 07:09:22 -0400 Message-ID: <20260428110946.11466-5-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" Access tracking will need to save bit 10 when MBEC is enabled. Right now it is simply shifting the R and X bits into bits 54 and 56, but bit 10 would not fit with the same scheme. Reorganize the high bits so that access tracking will use bits 52, 54 and 62. As a side effect, the free bits are compacted slightly, with 56-59 still unused. Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/spte.h | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index a107ddb66601..7913672c0f6f 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -17,10 +17,20 @@ */ #define SPTE_MMU_PRESENT_MASK BIT_ULL(11) =20 +/* + * The ignored high bits are allocated as follows: + * - bits 52, 54: saved X-R bits for access tracking when EPT does not hav= e A/D + * - bits 53 (EPT only): host writable + * - bits 55 (EPT only): MMU-writable + * - bits 56-59: unused + * - bits 60-61: type of A/D tracking + * - bits 62: unused + */ + /* * TDP SPTES (more specifically, EPT SPTEs) may not have A/D bits, and may= also * be restricted to using write-protection (for L2 when CPU dirty logging,= i.e. - * PML, is enabled). Use bits 52 and 53 to hold the type of A/D tracking = that + * PML, is enabled). Use bits 60 and 61 to hold the type of A/D tracking = that * is must be employed for a given TDP SPTE. * * Note, the "enabled" mask must be '0', as bits 62:52 are _reserved_ for = PAE @@ -29,7 +39,7 @@ * TDP with CPU dirty logging (PML). If NPT ever gains PML-like support, = it * must be restricted to 64-bit KVM. */ -#define SPTE_TDP_AD_SHIFT 52 +#define SPTE_TDP_AD_SHIFT 60 #define SPTE_TDP_AD_MASK (3ULL << SPTE_TDP_AD_SHIFT) #define SPTE_TDP_AD_ENABLED (0ULL << SPTE_TDP_AD_SHIFT) #define SPTE_TDP_AD_DISABLED (1ULL << SPTE_TDP_AD_SHIFT) @@ -65,7 +75,7 @@ static_assert(SPTE_TDP_AD_ENABLED =3D=3D 0); */ #define SHADOW_ACC_TRACK_SAVED_BITS_MASK (SPTE_EPT_READABLE_MASK | \ SPTE_EPT_EXECUTABLE_MASK) -#define SHADOW_ACC_TRACK_SAVED_BITS_SHIFT 54 +#define SHADOW_ACC_TRACK_SAVED_BITS_SHIFT 52 #define SHADOW_ACC_TRACK_SAVED_MASK (SHADOW_ACC_TRACK_SAVED_BITS_MASK << \ SHADOW_ACC_TRACK_SAVED_BITS_SHIFT) static_assert(!(SPTE_TDP_AD_MASK & SHADOW_ACC_TRACK_SAVED_MASK)); @@ -84,8 +94,8 @@ static_assert(!(SPTE_TDP_AD_MASK & SHADOW_ACC_TRACK_SAVED= _MASK)); * to not overlap the A/D type mask or the saved access bits of access-tra= cked * SPTEs when A/D bits are disabled. */ -#define EPT_SPTE_HOST_WRITABLE BIT_ULL(57) -#define EPT_SPTE_MMU_WRITABLE BIT_ULL(58) +#define EPT_SPTE_HOST_WRITABLE BIT_ULL(53) +#define EPT_SPTE_MMU_WRITABLE BIT_ULL(55) =20 static_assert(!(EPT_SPTE_HOST_WRITABLE & SPTE_TDP_AD_MASK)); static_assert(!(EPT_SPTE_MMU_WRITABLE & SPTE_TDP_AD_MASK)); --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C0B083F54C6 for ; Tue, 28 Apr 2026 11:09:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374601; cv=none; b=tQEalMNFtOKJf7RRA3cRjXlAqtG3OxKreV7uqV4D/uQYGof0FcQLv2R1pIE1/RCtkRbkAu98mp2ayDOthDiEVkeXcPhaGPYgKfQmvyESAPwjOu8hZZ7bBYlX254+wPb0YwltwomVrHSlXRj1fr7svEh7rbQJ5pKW+pmmNyMkW9c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374601; c=relaxed/simple; bh=QAKg6QudMZTaAddgpjvoDFJ4zbQ1b9xWfxKfa0meuFY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=dITjZaTnSC1KHwoH0mjiX+VKjKc1WEvNPED3j3wUxTpilfr9ofKpj8K8lMSjk3ZDEUKmytiwgWSN+/MBCgr6UggcnJrulnlzZCtLQcwolP4YEQpg0nZmTBLidW5yIGBO9rDGIdiCoJDl3W8SbHxJPIup/69STumyhxxZvbGuvlY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=LgPnWsy1; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="LgPnWsy1" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374597; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=in6s9CvkfcYt5FUUicAvCzh+OuHOuujlss5A3mb2EV4=; b=LgPnWsy1bBwtXiQamQfe8z+vEtCg0y2KFRjXO3UIerqGf/HVs+/jDTFJJix87XL0eKZYzQ vegagW3bkwmil02VKxl+y5Ax4Esip+HQzQdIyZoYD7cqcPxRYtw/voye/PgSw7Xpjz55eI rtl2k3+OdnhPbKdtKszERidzD5IoKns= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-609-3BZpTUS7ObGUpECR8gtyqw-1; Tue, 28 Apr 2026 07:09:53 -0400 X-MC-Unique: 3BZpTUS7ObGUpECR8gtyqw-1 X-Mimecast-MFC-AGG-ID: 3BZpTUS7ObGUpECR8gtyqw_1777374592 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E0E491956054; Tue, 28 Apr 2026 11:09:51 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 20FA530078C0; Tue, 28 Apr 2026 11:09:51 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 05/28] KVM: x86/mmu: remove SPTE_EPT_* Date: Tue, 28 Apr 2026 07:09:23 -0400 Message-ID: <20260428110946.11466-6-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" spte.h is already including vmx.h, use the constants it defines. Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/spte.h | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index 7913672c0f6f..ac957bd7dab0 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -57,10 +57,6 @@ static_assert(SPTE_TDP_AD_ENABLED =3D=3D 0); #define ACC_USER_MASK PT_USER_MASK #define ACC_ALL (ACC_EXEC_MASK | ACC_WRITE_MASK | ACC_USER_MASK) =20 -/* The mask for the R/X bits in EPT PTEs */ -#define SPTE_EPT_READABLE_MASK 0x1ull -#define SPTE_EPT_EXECUTABLE_MASK 0x4ull - #define SPTE_LEVEL_BITS 9 #define SPTE_LEVEL_SHIFT(level) __PT_LEVEL_SHIFT(level, SPTE_LEVEL_BITS) #define SPTE_INDEX(address, level) __PT_INDEX(address, level, SPTE_LEVEL_B= ITS) @@ -73,8 +69,8 @@ static_assert(SPTE_TDP_AD_ENABLED =3D=3D 0); * restored only when a write is attempted to the page. This mask obvious= ly * must not overlap the A/D type mask. */ -#define SHADOW_ACC_TRACK_SAVED_BITS_MASK (SPTE_EPT_READABLE_MASK | \ - SPTE_EPT_EXECUTABLE_MASK) +#define SHADOW_ACC_TRACK_SAVED_BITS_MASK (VMX_EPT_READABLE_MASK | \ + VMX_EPT_EXECUTABLE_MASK) #define SHADOW_ACC_TRACK_SAVED_BITS_SHIFT 52 #define SHADOW_ACC_TRACK_SAVED_MASK (SHADOW_ACC_TRACK_SAVED_BITS_MASK << \ SHADOW_ACC_TRACK_SAVED_BITS_SHIFT) --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9D5A73F54BF for ; Tue, 28 Apr 2026 11:09:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374600; cv=none; b=Fpf/KN+blB+VNazRlsC6vsY3iWU+6eeA3SrecSqxZX1RYPTQF+eQVjEP1NPRxFMq3/fWk+N7/k4MmlwfYNwilefytE4lr8i+Wn7LFcehdi5rjErN9T3NsE621TUVwu+SMDhTNNUlL20Sf9nWPhI961QS5ENBnKI9Dpk4cOeFzfg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374600; c=relaxed/simple; bh=vQvPgt0Mq2Gsbr1Skf087n28m4o4mK3xVgjfrnI90KM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=UQMbZILhNW9Pe94TgrE3pLRGfKXfke3JUO+CGmS+r9Mgg+WfTtPben5t/GdBVCXqT+oMSsNXCmE1n6nWTzVP1HjxNjqx0VPaSidYb2GLkEuynZ1hGaM0FznnFs+vS6shcFof12xxPI5BZzJlOktqWKfrkBnWDUu/i2klbNRG1hI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=gOe/xSvH; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="gOe/xSvH" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374597; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pMPlsOACadmgUCBsX+kfM3jcuhxlgiVMWFCf2O3a/w0=; b=gOe/xSvH7tzB4C4MrUFlg6N/6xJDpnB7KsJQFFxYBWXiFH64BaQ1oS1sMcqTlH6JSrCQol 4dmr2hmDWtAlqFNxf8ZIsv+bUltJT1IpT22NFjDsUhRLumPtAVlOomFa08LMB0kalIzuvH WwQdSfHy/8a8UTkYaKO8oV4hLgkmjj4= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-425-naYV9K8HNC-iR2a4scXNhw-1; Tue, 28 Apr 2026 07:09:54 -0400 X-MC-Unique: naYV9K8HNC-iR2a4scXNhw-1 X-Mimecast-MFC-AGG-ID: naYV9K8HNC-iR2a4scXNhw_1777374592 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C32DD18002C0; Tue, 28 Apr 2026 11:09:52 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 11F1D3000C22; Tue, 28 Apr 2026 11:09:51 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 06/28] KVM: x86/mmu: merge make_spte_{non,}executable Date: Tue, 28 Apr 2026 07:09:24 -0400 Message-ID: <20260428110946.11466-7-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" As the logic will become more complicated with the introduction of MBEC, at least write it only once. Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/spte.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 85a0473809b0..e9dc0ae44274 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -317,14 +317,16 @@ static u64 modify_spte_protections(u64 spte, u64 set,= u64 clear) return spte; } =20 -static u64 make_spte_executable(u64 spte) +static u64 make_spte_executable(u64 spte, u8 access) { - return modify_spte_protections(spte, shadow_x_mask, shadow_nx_mask); -} + u64 set, clear; =20 -static u64 make_spte_nonexecutable(u64 spte) -{ - return modify_spte_protections(spte, shadow_nx_mask, shadow_x_mask); + if (access & ACC_EXEC_MASK) + set =3D shadow_x_mask; + else + set =3D shadow_nx_mask; + clear =3D set ^ (shadow_nx_mask | shadow_x_mask); + return modify_spte_protections(spte, set, clear); } =20 /* @@ -356,8 +358,8 @@ u64 make_small_spte(struct kvm *kvm, u64 huge_spte, * the page executable as the NX hugepage mitigation no longer * applies. */ - if ((role.access & ACC_EXEC_MASK) && is_nx_huge_page_enabled(kvm)) - child_spte =3D make_spte_executable(child_spte); + if (is_nx_huge_page_enabled(kvm)) + child_spte =3D make_spte_executable(child_spte, role.access); } =20 return child_spte; @@ -379,7 +381,7 @@ u64 make_huge_spte(struct kvm *kvm, u64 small_spte, int= level) huge_spte &=3D KVM_HPAGE_MASK(level) | ~PAGE_MASK; =20 if (is_nx_huge_page_enabled(kvm)) - huge_spte =3D make_spte_nonexecutable(huge_spte); + huge_spte =3D make_spte_executable(huge_spte, 0); =20 return huge_spte; } --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D83783F7A8C for ; Tue, 28 Apr 2026 11:10:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374603; cv=none; b=sRlwS3jqY4kgxTa1o28stgO9brDGALsS4ybXIOj5PgxFpsZ8+1M0WpEdFlogbH9AgYGFE1UCdhYWRuCET6A0kUFYLrHurTxDdhckvXj/oqfNblUIjlW5GyWTRXGEKEa6u+BSmLzJF6d8FJcgskLUs21KEa1aVFuhHKbkYqaLfx8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374603; c=relaxed/simple; bh=pYxi/j+e+ki5JQsTiKeKDXJukerh7jXKmGy6I5ArLY0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=HF+f4w8bYVoLpKgP+jpaDUk2Lu/e5hxduQtLZbCNTBCaccD0Fcmzl5HonbuXKiXBzuy1cT3QQWaLYPVYIMEMyrcrSmuB195Ks+jxsmsJ3WM+JpoaCfz/iAwI1loI+lQMP00AFR+BQT25a8C4kqymA+psPN0XawlDgKwmQTxgTfs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Qet27YJp; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Qet27YJp" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374599; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IqH4mMwuSaq5xJDDbQpqV0WrUZp6tq9mDH2+yND/Ss0=; b=Qet27YJpnqRTGXXIpdRTxY+SEgwiaerPDoOmeuOh/DxtKD0Y4olAxAh7AKqsup6L1OasTl RjVbBBoQiUjMyyQuU+7kLW7WOn2meXIZB/kTDpm3leqfpZtI2D185SoDFDhDLkQJAbVMiP 1HPWsPRdSfCpNx2OA+QdJVPDTOXZAmc= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-425-YCtJnhDJO32r7EOlGDcI_Q-1; Tue, 28 Apr 2026 07:09:54 -0400 X-MC-Unique: YCtJnhDJO32r7EOlGDcI_Q-1 X-Mimecast-MFC-AGG-ID: YCtJnhDJO32r7EOlGDcI_Q_1777374593 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7D01F19560B2; Tue, 28 Apr 2026 11:09:53 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E97D83000C22; Tue, 28 Apr 2026 11:09:52 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 07/28] KVM: x86/mmu: rename and clarify BYTE_MASK Date: Tue, 28 Apr 2026 07:09:25 -0400 Message-ID: <20260428110946.11466-8-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" The BYTE_MASK macro is the central point of the black magic in update_permission_bitmask(). Rename it to something that relates to how it is used, and add a comment explaining how it works. Using shifts instead of powers of two was actually suggested by David Hildenbrand back in 2017 for clarity[1] but I evidently forgot his suggestion when applying to kvm.git. [1] https://lore.kernel.org/kvm/e4b5df86-31ae-2f4e-0666-393753e256df@redhat= .com/ Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 55 ++++++++++++++++++++++++++++++------------ 1 file changed, 39 insertions(+), 16 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 24fbc9ea502a..70fd6868a555 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -5529,29 +5529,53 @@ reset_ept_shadow_zero_bits_mask(struct kvm_mmu *con= text, bool execonly) max_huge_page_level); } =20 -#define BYTE_MASK(access) \ - ((1 & (access) ? 2 : 0) | \ - (2 & (access) ? 4 : 0) | \ - (3 & (access) ? 8 : 0) | \ - (4 & (access) ? 16 : 0) | \ - (5 & (access) ? 32 : 0) | \ - (6 & (access) ? 64 : 0) | \ - (7 & (access) ? 128 : 0)) - +/* + * Build a mask with all combinations of PTE access rights that + * include the given access bit. The mask can be queried with + * "mask & (1 << access)", where access is a combination of + * ACC_* bits. + * + * By mixing and matching multiple masks returned by ACC_BITS_MASK, + * update_permission_bitmask() builds what is effectively a + * two-dimensional array of bools. The second dimension is + * provided by individual bits of permissions[pfec >> 1], and + * logical &, | and ~ operations operate on all the 8 possible + * combinations of ACC_* bits. + */ +#define ACC_BITS_MASK(access) \ + ((1 & (access) ? 1 << 1 : 0) | \ + (2 & (access) ? 1 << 2 : 0) | \ + (3 & (access) ? 1 << 3 : 0) | \ + (4 & (access) ? 1 << 4 : 0) | \ + (5 & (access) ? 1 << 5 : 0) | \ + (6 & (access) ? 1 << 6 : 0) | \ + (7 & (access) ? 1 << 7 : 0)) =20 static void update_permission_bitmask(struct kvm_mmu *mmu, bool ept) { unsigned byte; =20 - const u8 x =3D BYTE_MASK(ACC_EXEC_MASK); - const u8 w =3D BYTE_MASK(ACC_WRITE_MASK); - const u8 u =3D BYTE_MASK(ACC_USER_MASK); + const u8 x =3D ACC_BITS_MASK(ACC_EXEC_MASK); + const u8 w =3D ACC_BITS_MASK(ACC_WRITE_MASK); + const u8 u =3D ACC_BITS_MASK(ACC_USER_MASK); =20 bool cr4_smep =3D is_cr4_smep(mmu); bool cr4_smap =3D is_cr4_smap(mmu); bool cr0_wp =3D is_cr0_wp(mmu); bool efer_nx =3D is_efer_nx(mmu); =20 + /* + * In hardware, page fault error codes are generated (as the name + * suggests) on any kind of page fault. permission_fault() and + * paging_tmpl.h already use the same bits after a successful page + * table walk, to indicate the kind of access being performed. + * + * However, PFERR_PRESENT_MASK and PFERR_RSVD_MASK are never set here, + * exactly because the page walk is successful. PFERR_PRESENT_MASK is + * removed by the shift, while PFERR_RSVD_MASK is repurposed in + * permission_fault() to indicate accesses that are *not* subject to + * SMAP restrictions. + */ for (byte =3D 0; byte < ARRAY_SIZE(mmu->permissions); ++byte) { unsigned pfec =3D byte << 1; =20 @@ -5598,10 +5622,9 @@ static void update_permission_bitmask(struct kvm_mmu= *mmu, bool ept) * - The access is supervisor mode * - If implicit supervisor access or X86_EFLAGS_AC is clear * - * Here, we cover the first four conditions. - * The fifth is computed dynamically in permission_fault(); - * PFERR_RSVD_MASK bit will be set in PFEC if the access is - * *not* subject to SMAP restrictions. + * Here, we cover the first four conditions. The fifth + * is computed dynamically in permission_fault() and + * communicated by setting PFERR_RSVD_MASK. */ if (cr4_smap) smapf =3D (pfec & (PFERR_RSVD_MASK|PFERR_FETCH_MASK)) ? 0 : kf; --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B1D1F3F6601 for ; Tue, 28 Apr 2026 11:09:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374603; cv=none; b=Z7k1YdWDnIaNGeyGYaPDvwvfabW/xqATNl1n9cu5wXJrvTQH8NzluB5WQ/hdjHSiHbUFbPwvXxgd8I6UVwhWhsxfdH9Kdcke61VYOVPH1cfAYEAM6Rkw4Vxt1Fr70htUvJftbhCFoe3TLBj27kmuuAaBelp7srjI9ShDn7tN16M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374603; c=relaxed/simple; bh=rdanpFOBy0SSaC7Ew1+Ig05DS0LiTl54FOAkJ4H0duQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=I4jBhRDJntCAsq5VW1QOm8s+uB5ZaBW/pe/B4y5aRqA2ooqf6tCKBuHAyKJYdFi9veQ8ynMxd1EQ6RKOE0LWYtrwcRQZGuzqoJ4Fc2sclwbgbXphrA9yLByz+US/O0bfRKQD/uxsdvVFZPs/JY6cgFi4dDHdd2ddfzU/IVMhZsU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=XlLipa7T; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="XlLipa7T" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374598; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mKg1mu5rGrJ4DRnOkw8ahlYBVQObROt4nW+hmrJGgH4=; b=XlLipa7TzDw3QwIPMBCp/LFrYHg6OKjqxeHGMe9Ljc1GZMqbZkZ80tETFfRtFmcJQM6k/l boTAC0OvkOOWsjqcFf17Tf+hcL6a8EW7iWYdd2fS9XwBWKU08jU06RGQyT8IvQa16Ae400 ja2IMfI5CyzX0YuuYduvUSlU8inSffE= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-453-4kZOgqVFMnCixBldCLbDdg-1; Tue, 28 Apr 2026 07:09:55 -0400 X-MC-Unique: 4kZOgqVFMnCixBldCLbDdg-1 X-Mimecast-MFC-AGG-ID: 4kZOgqVFMnCixBldCLbDdg_1777374594 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 37AD9195608F; Tue, 28 Apr 2026 11:09:54 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id A3AEF3000C22; Tue, 28 Apr 2026 11:09:53 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 08/28] KVM: x86/mmu: introduce ACC_READ_MASK Date: Tue, 28 Apr 2026 07:09:26 -0400 Message-ID: <20260428110946.11466-9-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" Read permissions so far were only needed for EPT, which does not need ACC_USER_MASK. Therefore, for EPT page tables ACC_USER_MASK was repurposed as a read permission bit. In order to implement nested MBEC, EPT will genuinely have four kinds of accesses, and there will be no room for such hacks; bite the bullet at last, enlarging ACC_ALL to four bits and permissions[] to 2^4 bits (u16). The new code does not enforce that the XWR bits on non-execonly processors have their R bit set, even when running nested: none of the shadow_*_mask values have bit 0 set, and make_spte() genuinely relies on ACC_READ_MASK being requested! This works because, if execonly is not supported by the processor, shadow EPT will generate an EPT misconfig vmexit if the XWR bits represent a non-readable page, and therefore the pte_access argument to make_spte() will also always have ACC_READ_MASK set. Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/include/asm/kvm_host.h | 12 +++++----- arch/x86/kvm/mmu.h | 2 +- arch/x86/kvm/mmu/mmu.c | 39 +++++++++++++++++++++------------ arch/x86/kvm/mmu/mmutrace.h | 3 ++- arch/x86/kvm/mmu/paging_tmpl.h | 35 +++++++++++++++++------------ arch/x86/kvm/mmu/spte.c | 18 ++++++--------- arch/x86/kvm/mmu/spte.h | 5 +++-- arch/x86/kvm/vmx/capabilities.h | 5 ----- arch/x86/kvm/vmx/common.h | 5 +---- arch/x86/kvm/vmx/vmx.c | 3 +-- 10 files changed, 67 insertions(+), 60 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index c470e40a00aa..8f2a1b915df9 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -328,11 +328,11 @@ struct kvm_kernel_irq_routing_entry; * the number of unique SPs that can theoretically be created is 2^n, wher= e n * is the number of bits that are used to compute the role. * - * But, even though there are 20 bits in the mask below, not all combinati= ons + * But, even though there are 21 bits in the mask below, not all combinati= ons * of modes and flags are possible: * * - invalid shadow pages are not accounted, mirror pages are not shadow= ed, - * so the bits are effectively 18. + * so the bits are effectively 19. * * - quadrant will only be used if has_4_byte_gpte=3D1 (non-PAE paging); * execonly and ad_disabled are only used for nested EPT which has @@ -347,7 +347,7 @@ struct kvm_kernel_irq_routing_entry; * cr0_wp=3D0, therefore these three bits only give rise to 5 possibil= ities. * * Therefore, the maximum number of possible upper-level shadow pages for a - * single gfn is a bit less than 2^13. + * single gfn is a bit less than 2^14. */ union kvm_mmu_page_role { u32 word; @@ -356,7 +356,7 @@ union kvm_mmu_page_role { unsigned has_4_byte_gpte:1; unsigned quadrant:2; unsigned direct:1; - unsigned access:3; + unsigned access:4; unsigned invalid:1; unsigned efer_nx:1; unsigned cr0_wp:1; @@ -366,7 +366,7 @@ union kvm_mmu_page_role { unsigned guest_mode:1; unsigned passthrough:1; unsigned is_mirror:1; - unsigned :4; + unsigned:3; =20 /* * This is left at the top of the word so that @@ -492,7 +492,7 @@ struct kvm_mmu { * Byte index: page fault error code [4:1] * Bit index: pte permissions in ACC_* format */ - u8 permissions[16]; + u16 permissions[16]; =20 u64 *pae_root; u64 *pml4_root; diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index 830f46145692..23f37535c0ce 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -81,7 +81,7 @@ u8 kvm_mmu_get_max_tdp_level(void); void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mmio_mask, u64 access_= mask); void kvm_mmu_set_mmio_spte_value(struct kvm *kvm, u64 mmio_value); void kvm_mmu_set_me_spte_mask(u64 me_value, u64 me_mask); -void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only); +void kvm_mmu_set_ept_masks(bool has_ad_bits); =20 void kvm_init_mmu(struct kvm_vcpu *vcpu); void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, unsigned long cr0, diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 70fd6868a555..cd1f8a51867d 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2033,7 +2033,7 @@ static bool kvm_sync_page_check(struct kvm_vcpu *vcpu= , struct kvm_mmu_page *sp) */ const union kvm_mmu_page_role sync_role_ign =3D { .level =3D 0xf, - .access =3D 0x7, + .access =3D ACC_ALL, .quadrant =3D 0x3, .passthrough =3D 0x1, }; @@ -5539,7 +5539,7 @@ reset_ept_shadow_zero_bits_mask(struct kvm_mmu *conte= xt, bool execonly) * update_permission_bitmask() builds what is effectively a * two-dimensional array of bools. The second dimension is * provided by individual bits of permissions[pfec >> 1], and - * logical &, | and ~ operations operate on all the 8 possible + * logical &, | and ~ operations operate on all the 16 possible * combinations of ACC_* bits. */ #define ACC_BITS_MASK(access) \ @@ -5549,15 +5549,24 @@ reset_ept_shadow_zero_bits_mask(struct kvm_mmu *con= text, bool execonly) (4 & (access) ? 1 << 4 : 0) | \ (5 & (access) ? 1 << 5 : 0) | \ (6 & (access) ? 1 << 6 : 0) | \ - (7 & (access) ? 1 << 7 : 0)) + (7 & (access) ? 1 << 7 : 0) | \ + (8 & (access) ? 1 << 8 : 0) | \ + (9 & (access) ? 1 << 9 : 0) | \ + (10 & (access) ? 1 << 10 : 0) | \ + (11 & (access) ? 1 << 11 : 0) | \ + (12 & (access) ? 1 << 12 : 0) | \ + (13 & (access) ? 1 << 13 : 0) | \ + (14 & (access) ? 1 << 14 : 0) | \ + (15 & (access) ? 1 << 15 : 0)) =20 static void update_permission_bitmask(struct kvm_mmu *mmu, bool ept) { unsigned byte; =20 - const u8 x =3D ACC_BITS_MASK(ACC_EXEC_MASK); - const u8 w =3D ACC_BITS_MASK(ACC_WRITE_MASK); - const u8 u =3D ACC_BITS_MASK(ACC_USER_MASK); + const u16 x =3D ACC_BITS_MASK(ACC_EXEC_MASK); + const u16 w =3D ACC_BITS_MASK(ACC_WRITE_MASK); + const u16 u =3D ACC_BITS_MASK(ACC_USER_MASK); + const u16 r =3D ACC_BITS_MASK(ACC_READ_MASK); =20 bool cr4_smep =3D is_cr4_smep(mmu); bool cr4_smap =3D is_cr4_smap(mmu); @@ -5580,24 +5589,26 @@ static void update_permission_bitmask(struct kvm_mm= u *mmu, bool ept) unsigned pfec =3D byte << 1; =20 /* - * Each "*f" variable has a 1 bit for each UWX value + * Each "*f" variable has a 1 bit for each ACC_* combo * that causes a fault with the given PFEC. */ =20 + /* Faults from reads to non-readable pages */ + u16 rf =3D (pfec & (PFERR_WRITE_MASK|PFERR_FETCH_MASK)) ? 0 : (u16)~r; /* Faults from writes to non-writable pages */ - u8 wf =3D (pfec & PFERR_WRITE_MASK) ? (u8)~w : 0; + u16 wf =3D (pfec & PFERR_WRITE_MASK) ? (u16)~w : 0; /* Faults from user mode accesses to supervisor pages */ - u8 uf =3D (pfec & PFERR_USER_MASK) ? (u8)~u : 0; + u16 uf =3D (pfec & PFERR_USER_MASK) ? (u16)~u : 0; /* Faults from fetches of non-executable pages*/ - u8 ff =3D (pfec & PFERR_FETCH_MASK) ? (u8)~x : 0; + u16 ff =3D (pfec & PFERR_FETCH_MASK) ? (u16)~x : 0; /* Faults from kernel mode fetches of user pages */ - u8 smepf =3D 0; + u16 smepf =3D 0; /* Faults from kernel mode accesses of user pages */ - u8 smapf =3D 0; + u16 smapf =3D 0; =20 if (!ept) { /* Faults from kernel mode accesses to user pages */ - u8 kf =3D (pfec & PFERR_USER_MASK) ? 0 : u; + u16 kf =3D (pfec & PFERR_USER_MASK) ? 0 : u; =20 /* Not really needed: !nx will cause pte.nx to fault */ if (!efer_nx) @@ -5630,7 +5641,7 @@ static void update_permission_bitmask(struct kvm_mmu = *mmu, bool ept) smapf =3D (pfec & (PFERR_RSVD_MASK|PFERR_FETCH_MASK)) ? 0 : kf; } =20 - mmu->permissions[byte] =3D ff | uf | wf | smepf | smapf; + mmu->permissions[byte] =3D ff | uf | wf | rf | smepf | smapf; } } =20 diff --git a/arch/x86/kvm/mmu/mmutrace.h b/arch/x86/kvm/mmu/mmutrace.h index 764e3015d021..dcfdfedfc4e9 100644 --- a/arch/x86/kvm/mmu/mmutrace.h +++ b/arch/x86/kvm/mmu/mmutrace.h @@ -25,7 +25,8 @@ #define KVM_MMU_PAGE_PRINTK() ({ \ const char *saved_ptr =3D trace_seq_buffer_ptr(p); \ static const char *access_str[] =3D { \ - "---", "--x", "w--", "w-x", "-u-", "-ux", "wu-", "wux" \ + "----", "r---", "-w--", "rw--", "--u-", "r-u-", "-wu-", "rwu-", \ + "---x", "r--x", "-w-x", "rw-x", "--ux", "r-ux", "-wux", "rwux" \ }; \ union kvm_mmu_page_role role; \ \ diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 901cd2bd40b8..fb1b5d8b23e5 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -170,25 +170,24 @@ static bool FNAME(prefetch_invalid_gpte)(struct kvm_v= cpu *vcpu, return true; } =20 -/* - * For PTTYPE_EPT, a page table can be executable but not readable - * on supported processors. Therefore, set_spte does not automatically - * set bit 0 if execute only is supported. Here, we repurpose ACC_USER_MASK - * to signify readability since it isn't used in the EPT case - */ static inline unsigned FNAME(gpte_access)(u64 gpte) { unsigned access; #if PTTYPE =3D=3D PTTYPE_EPT access =3D ((gpte & VMX_EPT_WRITABLE_MASK) ? ACC_WRITE_MASK : 0) | ((gpte & VMX_EPT_EXECUTABLE_MASK) ? ACC_EXEC_MASK : 0) | - ((gpte & VMX_EPT_READABLE_MASK) ? ACC_USER_MASK : 0); + ((gpte & VMX_EPT_READABLE_MASK) ? ACC_READ_MASK : 0); #else - BUILD_BUG_ON(ACC_EXEC_MASK !=3D PT_PRESENT_MASK); - BUILD_BUG_ON(ACC_EXEC_MASK !=3D 1); + /* + * P is set here, so the page is always readable and W/U/!NX represent + * allowed accesses. + */ + BUILD_BUG_ON(ACC_READ_MASK !=3D PT_PRESENT_MASK); + BUILD_BUG_ON(ACC_WRITE_MASK !=3D PT_WRITABLE_MASK); + BUILD_BUG_ON(ACC_USER_MASK !=3D PT_USER_MASK); + BUILD_BUG_ON(ACC_EXEC_MASK & (PT_WRITABLE_MASK | PT_USER_MASK | PT_PRESEN= T_MASK)); access =3D gpte & (PT_WRITABLE_MASK | PT_USER_MASK | PT_PRESENT_MASK); - /* Combine NX with P (which is set here) to get ACC_EXEC_MASK. */ - access ^=3D (gpte >> PT64_NX_SHIFT); + access |=3D gpte & PT64_NX_MASK ? 0 : ACC_EXEC_MASK; #endif =20 return access; @@ -501,10 +500,18 @@ static int FNAME(walk_addr_generic)(struct guest_walk= er *walker, =20 if (write_fault) walker->fault.exit_qualification |=3D EPT_VIOLATION_ACC_WRITE; - if (user_fault) - walker->fault.exit_qualification |=3D EPT_VIOLATION_ACC_READ; - if (fetch_fault) + else if (fetch_fault) walker->fault.exit_qualification |=3D EPT_VIOLATION_ACC_INSTR; + else + walker->fault.exit_qualification |=3D EPT_VIOLATION_ACC_READ; + + /* + * Accesses to guest paging structures are either "reads" or + * "read+write" accesses, so consider them the latter if write_fault + * is true. + */ + if (access & PFERR_GUEST_PAGE_MASK) + walker->fault.exit_qualification |=3D EPT_VIOLATION_ACC_READ; =20 /* * Note, pte_access holds the raw RWX bits from the EPTE, not diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index e9dc0ae44274..7b5f118ae211 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -194,12 +194,6 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_p= age *sp, int is_host_mmio =3D -1; bool wrprot =3D false; =20 - /* - * For the EPT case, shadow_present_mask has no RWX bits set if - * exec-only page table entries are supported. In that case, - * ACC_USER_MASK and shadow_user_mask are used to represent - * read access. See FNAME(gpte_access) in paging_tmpl.h. - */ WARN_ON_ONCE((pte_access | shadow_present_mask) =3D=3D SHADOW_NONPRESENT_= VALUE); =20 if (sp->role.ad_disabled) @@ -228,6 +222,9 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_pa= ge *sp, pte_access &=3D ~ACC_EXEC_MASK; } =20 + if (pte_access & ACC_READ_MASK) + spte |=3D PT_PRESENT_MASK; /* or VMX_EPT_READABLE_MASK */ + if (pte_access & ACC_EXEC_MASK) spte |=3D shadow_x_mask; else @@ -391,6 +388,7 @@ u64 make_nonleaf_spte(u64 *child_pt, bool ad_disabled) u64 spte =3D SPTE_MMU_PRESENT_MASK; =20 spte |=3D __pa(child_pt) | shadow_present_mask | PT_WRITABLE_MASK | + PT_PRESENT_MASK /* or VMX_EPT_READABLE_MASK */ | shadow_user_mask | shadow_x_mask | shadow_me_value; =20 if (ad_disabled) @@ -491,18 +489,16 @@ void kvm_mmu_set_me_spte_mask(u64 me_value, u64 me_ma= sk) } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_mmu_set_me_spte_mask); =20 -void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only) +void kvm_mmu_set_ept_masks(bool has_ad_bits) { kvm_ad_enabled =3D has_ad_bits; =20 - shadow_user_mask =3D VMX_EPT_READABLE_MASK; + shadow_user_mask =3D 0; shadow_accessed_mask =3D VMX_EPT_ACCESS_BIT; shadow_dirty_mask =3D VMX_EPT_DIRTY_BIT; shadow_nx_mask =3D 0ull; shadow_x_mask =3D VMX_EPT_EXECUTABLE_MASK; - /* VMX_EPT_SUPPRESS_VE_BIT is needed for W or X violation. */ - shadow_present_mask =3D - (has_exec_only ? 0ull : VMX_EPT_READABLE_MASK) | VMX_EPT_SUPPRESS_VE_BIT; + shadow_present_mask =3D VMX_EPT_SUPPRESS_VE_BIT; =20 shadow_acc_track_mask =3D VMX_EPT_RWX_MASK; shadow_host_writable_mask =3D EPT_SPTE_HOST_WRITABLE; diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index ac957bd7dab0..c0055ede0132 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -52,10 +52,11 @@ static_assert(SPTE_TDP_AD_ENABLED =3D=3D 0); #define SPTE_BASE_ADDR_MASK (((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1)) #endif =20 -#define ACC_EXEC_MASK 1 +#define ACC_READ_MASK PT_PRESENT_MASK #define ACC_WRITE_MASK PT_WRITABLE_MASK #define ACC_USER_MASK PT_USER_MASK -#define ACC_ALL (ACC_EXEC_MASK | ACC_WRITE_MASK | ACC_USER_MASK) +#define ACC_EXEC_MASK 8 +#define ACC_ALL (ACC_EXEC_MASK | ACC_WRITE_MASK | ACC_USER_MASK |= ACC_READ_MASK) =20 #define SPTE_LEVEL_BITS 9 #define SPTE_LEVEL_SHIFT(level) __PT_LEVEL_SHIFT(level, SPTE_LEVEL_BITS) diff --git a/arch/x86/kvm/vmx/capabilities.h b/arch/x86/kvm/vmx/capabilitie= s.h index 56cacc06225e..7e59eb0f41bb 100644 --- a/arch/x86/kvm/vmx/capabilities.h +++ b/arch/x86/kvm/vmx/capabilities.h @@ -300,11 +300,6 @@ static inline bool cpu_has_vmx_flexpriority(void) cpu_has_vmx_virtualize_apic_accesses(); } =20 -static inline bool cpu_has_vmx_ept_execute_only(void) -{ - return vmx_capability.ept & VMX_EPT_EXECUTE_ONLY_BIT; -} - static inline bool cpu_has_vmx_ept_4levels(void) { return vmx_capability.ept & VMX_EPT_PAGE_WALK_4_BIT; diff --git a/arch/x86/kvm/vmx/common.h b/arch/x86/kvm/vmx/common.h index adf925500b9e..1afbf272efae 100644 --- a/arch/x86/kvm/vmx/common.h +++ b/arch/x86/kvm/vmx/common.h @@ -85,11 +85,8 @@ static inline int __vmx_handle_ept_violation(struct kvm_= vcpu *vcpu, gpa_t gpa, { u64 error_code; =20 - /* Is it a read fault? */ - error_code =3D (exit_qualification & EPT_VIOLATION_ACC_READ) - ? PFERR_USER_MASK : 0; /* Is it a write fault? */ - error_code |=3D (exit_qualification & EPT_VIOLATION_ACC_WRITE) + error_code =3D (exit_qualification & EPT_VIOLATION_ACC_WRITE) ? PFERR_WRITE_MASK : 0; /* Is it a fetch fault? */ error_code |=3D (exit_qualification & EPT_VIOLATION_ACC_INSTR) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index a29896a9ef14..337bbfecc021 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -8683,8 +8683,7 @@ __init int vmx_hardware_setup(void) set_bit(0, vmx_vpid_bitmap); /* 0 is reserved for host */ =20 if (enable_ept) - kvm_mmu_set_ept_masks(enable_ept_ad_bits, - cpu_has_vmx_ept_execute_only()); + kvm_mmu_set_ept_masks(enable_ept_ad_bits); else vt_x86_ops.get_mt_mask =3D NULL; =20 --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3D88E3F9F4C for ; Tue, 28 Apr 2026 11:10:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374605; cv=none; b=iY273edBkjbD1y1vLj9QZ54Q7U86Q91OHIR9vcOAaoUICHs9WO9BkHEtRjd53c2pCHE2r7P3Y8uOoUsm6eVof2FVTeQQ1WA7HniAUOjVztdwIv56QNgv1WiHVQi2UVkD9f2PfaHFyzLedYXa+naaHl4a10DAphn6Vl9LY9tFPnY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374605; c=relaxed/simple; bh=oAxHGH9SBIUZdnUc5NX6qtSfH3XRMOtG16yftL7zZQ8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=gRIl/8kkGnf6bVVOmxCkMWBA/wYkiwc8bOUlsTnoPwLTPVmbycnKkMmgDjagammIv791hGXEisIjlUiF258Di/0mbx75knlBBpTGtfsDrBuYKhawc7fEauaagURz/9o9FzPBroPXKx1S+GlyV5rB7Yj4tZ7kZRtwLvhvLwSe0jI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=V3w4dcMx; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="V3w4dcMx" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374601; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GZatRJJDY6bnjDKKoO77YMzv1PQpxJ9aBLIL+NZ9diQ=; b=V3w4dcMxifoZPT+pif0DQEONGZUPf9iXmiVXEEiq6kjcopuYzrbJJvxQLLqYxIJaSjT7D0 n5QjEMwUIGE+3HK/zn53G9dPqft14E1g5xawJCrxpsKH6rh0OkM1J06Xvlyyfd1K8UWGrK lLNS/5sGQmvA1SguFYlQh62jIzY6v2w= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-169-cNqvpuaxO_OCsqqQtXBptQ-1; Tue, 28 Apr 2026 07:09:56 -0400 X-MC-Unique: cNqvpuaxO_OCsqqQtXBptQ-1 X-Mimecast-MFC-AGG-ID: cNqvpuaxO_OCsqqQtXBptQ_1777374595 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E629E19560BB; Tue, 28 Apr 2026 11:09:54 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 5DE4C3000C22; Tue, 28 Apr 2026 11:09:54 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 09/28] KVM: x86/mmu: separate more EPT/non-EPT permission_fault() Date: Tue, 28 Apr 2026 07:09:27 -0400 Message-ID: <20260428110946.11466-10-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" Now that EPT is not abusing anymore ACC_USER_MASK, move its handling entirely in the !ept branch. Merge smepf and ff into a single variable because EPT's "SMEP" (actually MBEC) is defined differently. Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index cd1f8a51867d..fc1b17e22ea2 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -5565,7 +5565,6 @@ static void update_permission_bitmask(struct kvm_mmu = *mmu, bool ept) =20 const u16 x =3D ACC_BITS_MASK(ACC_EXEC_MASK); const u16 w =3D ACC_BITS_MASK(ACC_WRITE_MASK); - const u16 u =3D ACC_BITS_MASK(ACC_USER_MASK); const u16 r =3D ACC_BITS_MASK(ACC_READ_MASK); =20 bool cr4_smep =3D is_cr4_smep(mmu); @@ -5598,21 +5597,24 @@ static void update_permission_bitmask(struct kvm_mm= u *mmu, bool ept) /* Faults from writes to non-writable pages */ u16 wf =3D (pfec & PFERR_WRITE_MASK) ? (u16)~w : 0; /* Faults from user mode accesses to supervisor pages */ - u16 uf =3D (pfec & PFERR_USER_MASK) ? (u16)~u : 0; - /* Faults from fetches of non-executable pages*/ - u16 ff =3D (pfec & PFERR_FETCH_MASK) ? (u16)~x : 0; - /* Faults from kernel mode fetches of user pages */ - u16 smepf =3D 0; + u16 uf =3D 0; + /* Faults from fetches of non-executable pages */ + u16 ff =3D 0; /* Faults from kernel mode accesses of user pages */ u16 smapf =3D 0; =20 - if (!ept) { + if (ept) { + ff =3D (pfec & PFERR_FETCH_MASK) ? (u16)~x : 0; + } else { + const u16 u =3D ACC_BITS_MASK(ACC_USER_MASK); + /* Faults from kernel mode accesses to user pages */ u16 kf =3D (pfec & PFERR_USER_MASK) ? 0 : u; =20 - /* Not really needed: !nx will cause pte.nx to fault */ - if (!efer_nx) - ff =3D 0; + uf =3D (pfec & PFERR_USER_MASK) ? (u16)~u : 0; + + if (efer_nx) + ff =3D (pfec & PFERR_FETCH_MASK) ? (u16)~x : 0; =20 /* Allow supervisor writes if !cr0.wp */ if (!cr0_wp) @@ -5620,7 +5622,7 @@ static void update_permission_bitmask(struct kvm_mmu = *mmu, bool ept) =20 /* Disallow supervisor fetches of user code if cr4.smep */ if (cr4_smep) - smepf =3D (pfec & PFERR_FETCH_MASK) ? kf : 0; + ff |=3D (pfec & PFERR_FETCH_MASK) ? kf : 0; =20 /* * SMAP:kernel-mode data accesses from user-mode @@ -5641,7 +5643,7 @@ static void update_permission_bitmask(struct kvm_mmu = *mmu, bool ept) smapf =3D (pfec & (PFERR_RSVD_MASK|PFERR_FETCH_MASK)) ? 0 : kf; } =20 - mmu->permissions[byte] =3D ff | uf | wf | rf | smepf | smapf; + mmu->permissions[byte] =3D ff | uf | wf | rf | smapf; } } =20 --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE7483F54C3 for ; Tue, 28 Apr 2026 11:10:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374605; cv=none; b=J6inCm1TA277M/x0geiMBQr7HuNL8/JP64Q965B2i8LpDr2SeRJNd9hNipZpu9WLJ3og8ieL4lablsuuwBhfYCu9vnGtTFO7P4ENF87Du9pNt3nbQ9fxaQyJvVgz6obpfhxS8xy6/Husgy3HflnY1YQheEqyJuTgoLB2dIOylMA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374605; c=relaxed/simple; bh=wXjuhSz6nVLjdUFmzOE30XGcCV0H2i3wuwB7AhkrVDs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=FTnkaVEviDZGfa34rpnmPZUKl60AQPU+H2bYrf2cDLhTQ9kXOMQfejrFGygkVmUMqDF5yXAwbZDNKrjiL5P50VHKYjRzZa3BfSdysAKuFkKiXb2f9O93DbuVtm0i78bStrgTuIwMY/hIScSjOuV/KQxYYnQ1AMSgwNV2ELVmawQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=gifpbivf; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="gifpbivf" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374600; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lwmBYA9ksrwWQcFcUn5nZOkE9O0tPmBRjesg8bFOvsg=; b=gifpbivfGzJ5vR74gbmP+VhvQZ0+pkwkeZIOFLR5VcjNuGjlGNRvfu2wXXh+IPWLmjy7FC iAYEBbme+ydPyWnD6Ii949QT7HPw/LRAEcKbUL9E/rYNN9SXknTQVUEsfbqHC5S8tIAVV7 q2rWRfnoY0ACvHvWK+AsJhcOAS9nXGw= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-61-XBfhnQg8PeWntidSZy-4wg-1; Tue, 28 Apr 2026 07:09:57 -0400 X-MC-Unique: XBfhnQg8PeWntidSZy-4wg-1 X-Mimecast-MFC-AGG-ID: XBfhnQg8PeWntidSZy-4wg_1777374596 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2F8DA180034A; Tue, 28 Apr 2026 11:09:56 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9C9AA180045E; Tue, 28 Apr 2026 11:09:55 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 10/28] KVM: x86/mmu: pass PFERR_GUEST_PAGE/FINAL_MASK to kvm_translate_gpa Date: Tue, 28 Apr 2026 07:09:28 -0400 Message-ID: <20260428110946.11466-11-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Content-Type: text/plain; charset="utf-8" The XS/XU bit for EPT are only applied to final accesses, and use the U bit from the page walk itself. While strictly speaking not necessary (any value of PFERR_USER_MASK would be the same for page table accesses, because they're reads and writes only), it is clearer and less hackish to only apply MBEC to PFERR_GUEST_FINAL_MASK. Allow kvm-intel.ko to distinguish the two cases. Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/kvm/hyperv.c | 3 ++- arch/x86/kvm/mmu/mmu.c | 3 ++- arch/x86/kvm/mmu/paging_tmpl.h | 7 +++++-- arch/x86/kvm/x86.c | 3 ++- 4 files changed, 11 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 9b140bbdc1d8..cf9dd565b894 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -2041,7 +2041,8 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, st= ruct kvm_hv_hcall *hc) * read with kvm_read_guest(). */ if (!hc->fast && is_guest_mode(vcpu)) { - hc->ingpa =3D translate_nested_gpa(vcpu, hc->ingpa, 0, NULL); + hc->ingpa =3D translate_nested_gpa(vcpu, hc->ingpa, + PFERR_GUEST_FINAL_MASK, NULL); if (unlikely(hc->ingpa =3D=3D INVALID_GPA)) return HV_STATUS_INVALID_HYPERCALL_INPUT; } diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index fc1b17e22ea2..0fc362508a19 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4348,7 +4348,8 @@ static gpa_t nonpaging_gva_to_gpa(struct kvm_vcpu *vc= pu, struct kvm_mmu *mmu, { if (exception) exception->error_code =3D 0; - return kvm_translate_gpa(vcpu, mmu, vaddr, access, exception); + return kvm_translate_gpa(vcpu, mmu, vaddr, access | PFERR_GUEST_FINAL_MAS= K, + exception); } =20 static bool mmio_info_in_cache(struct kvm_vcpu *vcpu, u64 addr, bool direc= t) diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index fb1b5d8b23e5..567f8b77ffe0 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -376,7 +376,8 @@ static int FNAME(walk_addr_generic)(struct guest_walker= *walker, walker->pte_gpa[walker->level - 1] =3D pte_gpa; =20 real_gpa =3D kvm_translate_gpa(vcpu, mmu, gfn_to_gpa(table_gfn), - nested_access, &walker->fault); + nested_access | PFERR_GUEST_PAGE_MASK, + &walker->fault); =20 /* * FIXME: This can happen if emulation (for of an INS/OUTS @@ -444,7 +445,9 @@ static int FNAME(walk_addr_generic)(struct guest_walker= *walker, gfn +=3D pse36_gfn_delta(pte); #endif =20 - real_gpa =3D kvm_translate_gpa(vcpu, mmu, gfn_to_gpa(gfn), access, &walke= r->fault); + real_gpa =3D kvm_translate_gpa(vcpu, mmu, gfn_to_gpa(gfn), + access | PFERR_GUEST_FINAL_MASK, + &walker->fault); if (real_gpa =3D=3D INVALID_GPA) return 0; =20 diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 0a1b63c63d1a..ef1e3ae13887 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1072,7 +1072,8 @@ int load_pdptrs(struct kvm_vcpu *vcpu, unsigned long = cr3) * to an L1 GPA. */ real_gpa =3D kvm_translate_gpa(vcpu, mmu, gfn_to_gpa(pdpt_gfn), - PFERR_USER_MASK | PFERR_WRITE_MASK, NULL); + PFERR_USER_MASK | PFERR_WRITE_MASK | + PFERR_GUEST_PAGE_MASK, NULL); if (real_gpa =3D=3D INVALID_GPA) return 0; =20 --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 659CB3F7ABD for ; Tue, 28 Apr 2026 11:10:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374605; cv=none; b=gtiwnpyviHJsy1T9QHGMaH8grzzcZqrPAXcyswAFGTb0Lchd/ffh1ZOw1dK5lMsAWulBRgjd4nlZ6FXgAG9nxiyX3zKljcDhCmwf8fEp0kfc1xudJK1oqK1fUs9q/EhlFxr8fNo+qUJtu4wKMsS2Q5JgDzLe4ISBG4KCK85dPK4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374605; c=relaxed/simple; bh=/Xuv4Z/ODmL8+SKD6v1Y+oSc6EwRPGoFExshXhgI+Cs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=V/P2uPRjQgZlPkOOvjoKGxbplXENRaG48No76vag/4tuR7Z0g9FxFbs6OEHAiimbx6RAgn3Z5eER6f2Rn97NVts9ukLSK9A7kknd2YNLBo7pmVh7EmU2PIrFcVXxNoE5+B0Qa3DCgu9ZCox/NcN2tKsqM4eqo4c5K/5B4indhCQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=eGCqkj2Q; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="eGCqkj2Q" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374601; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gvFnzMoilyTLMnArk/yAEZs4ahqff0Le86aytRVv5cE=; b=eGCqkj2QzEyE2nf6+jDcKUZOOpDRYuBDkL7j9PWx69HOI2E3LM+jDcFn4F5h0zsAuAhghc KBNSA2U0NhtshH9/QWSFY86GPVideJLGWT0X82nJ6J2xEyXw0iHENP59rFnvRSdOS8lnDT Do112j/DLdVT6xE0QQXxCwCrNugYAvo= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-14-ikiJJoKKPIe6g9MjGQms7A-1; Tue, 28 Apr 2026 07:09:58 -0400 X-MC-Unique: ikiJJoKKPIe6g9MjGQms7A-1 X-Mimecast-MFC-AGG-ID: ikiJJoKKPIe6g9MjGQms7A_1777374597 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id DD18F195608B; Tue, 28 Apr 2026 11:09:56 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 56D68180045E; Tue, 28 Apr 2026 11:09:56 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 11/28] KVM: x86/mmu: pass pte_access for final nGPA->GPA walk Date: Tue, 28 Apr 2026 07:09:29 -0400 Message-ID: <20260428110946.11466-12-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Content-Type: text/plain; charset="utf-8" The XS/XU bit for EPT are only applied to final accesses, and use the U bit from the page walk itself. This is available in the page walker as pte_access & ACC_USER_MASK but not available to translate_nested_gpa, so pass it down. Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/kvm/hyperv.c | 2 +- arch/x86/kvm/mmu.h | 15 ++++++++++++--- arch/x86/kvm/mmu/mmu.c | 2 +- arch/x86/kvm/mmu/paging_tmpl.h | 4 ++-- arch/x86/kvm/mmu/spte.h | 6 ------ arch/x86/kvm/x86.c | 5 +++-- 6 files changed, 19 insertions(+), 15 deletions(-) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index cf9dd565b894..53688f7b76eb 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -2042,7 +2042,7 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, st= ruct kvm_hv_hcall *hc) */ if (!hc->fast && is_guest_mode(vcpu)) { hc->ingpa =3D translate_nested_gpa(vcpu, hc->ingpa, - PFERR_GUEST_FINAL_MASK, NULL); + PFERR_GUEST_FINAL_MASK, NULL, 0); if (unlikely(hc->ingpa =3D=3D INVALID_GPA)) return HV_STATUS_INVALID_HYPERCALL_INPUT; } diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index 23f37535c0ce..635c2e5d8513 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -37,6 +37,12 @@ extern bool __read_mostly enable_mmio_caching; #define PT32_ROOT_LEVEL 2 #define PT32E_ROOT_LEVEL 3 =20 +#define ACC_READ_MASK PT_PRESENT_MASK +#define ACC_WRITE_MASK PT_WRITABLE_MASK +#define ACC_USER_MASK PT_USER_MASK +#define ACC_EXEC_MASK 8 +#define ACC_ALL (ACC_EXEC_MASK | ACC_WRITE_MASK | ACC_USER_MASK |= ACC_READ_MASK) + #define KVM_MMU_CR4_ROLE_BITS (X86_CR4_PSE | X86_CR4_PAE | X86_CR4_LA57 | \ X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_PKE) =20 @@ -289,16 +295,19 @@ static inline void kvm_update_page_stats(struct kvm *= kvm, int level, int count) } =20 gpa_t translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa, u64 access, - struct x86_exception *exception); + struct x86_exception *exception, + u64 pte_access); =20 static inline gpa_t kvm_translate_gpa(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, gpa_t gpa, u64 access, - struct x86_exception *exception) + struct x86_exception *exception, + u64 pte_access) { if (mmu !=3D &vcpu->arch.nested_mmu) return gpa; - return translate_nested_gpa(vcpu, gpa, access, exception); + return translate_nested_gpa(vcpu, gpa, access, exception, + pte_access); } =20 static inline bool kvm_has_mirrored_tdp(const struct kvm *kvm) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 0fc362508a19..c82d151ca6c1 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4349,7 +4349,7 @@ static gpa_t nonpaging_gva_to_gpa(struct kvm_vcpu *vc= pu, struct kvm_mmu *mmu, if (exception) exception->error_code =3D 0; return kvm_translate_gpa(vcpu, mmu, vaddr, access | PFERR_GUEST_FINAL_MAS= K, - exception); + exception, ACC_ALL); } =20 static bool mmio_info_in_cache(struct kvm_vcpu *vcpu, u64 addr, bool direc= t) diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 567f8b77ffe0..de8770d2fcfc 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -377,7 +377,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker= *walker, =20 real_gpa =3D kvm_translate_gpa(vcpu, mmu, gfn_to_gpa(table_gfn), nested_access | PFERR_GUEST_PAGE_MASK, - &walker->fault); + &walker->fault, 0); =20 /* * FIXME: This can happen if emulation (for of an INS/OUTS @@ -447,7 +447,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker= *walker, =20 real_gpa =3D kvm_translate_gpa(vcpu, mmu, gfn_to_gpa(gfn), access | PFERR_GUEST_FINAL_MASK, - &walker->fault); + &walker->fault, pte_access); if (real_gpa =3D=3D INVALID_GPA) return 0; =20 diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index c0055ede0132..fe71ae131ec1 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -52,12 +52,6 @@ static_assert(SPTE_TDP_AD_ENABLED =3D=3D 0); #define SPTE_BASE_ADDR_MASK (((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1)) #endif =20 -#define ACC_READ_MASK PT_PRESENT_MASK -#define ACC_WRITE_MASK PT_WRITABLE_MASK -#define ACC_USER_MASK PT_USER_MASK -#define ACC_EXEC_MASK 8 -#define ACC_ALL (ACC_EXEC_MASK | ACC_WRITE_MASK | ACC_USER_MASK |= ACC_READ_MASK) - #define SPTE_LEVEL_BITS 9 #define SPTE_LEVEL_SHIFT(level) __PT_LEVEL_SHIFT(level, SPTE_LEVEL_BITS) #define SPTE_INDEX(address, level) __PT_INDEX(address, level, SPTE_LEVEL_B= ITS) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index ef1e3ae13887..67979b7de5d6 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1073,7 +1073,7 @@ int load_pdptrs(struct kvm_vcpu *vcpu, unsigned long = cr3) */ real_gpa =3D kvm_translate_gpa(vcpu, mmu, gfn_to_gpa(pdpt_gfn), PFERR_USER_MASK | PFERR_WRITE_MASK | - PFERR_GUEST_PAGE_MASK, NULL); + PFERR_GUEST_PAGE_MASK, NULL, 0); if (real_gpa =3D=3D INVALID_GPA) return 0; =20 @@ -7849,7 +7849,8 @@ void kvm_get_segment(struct kvm_vcpu *vcpu, } =20 gpa_t translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa, u64 access, - struct x86_exception *exception) + struct x86_exception *exception, + u64 pte_access) { struct kvm_mmu *mmu =3D vcpu->arch.mmu; gpa_t t_gpa; --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3E6673F9F4D for ; Tue, 28 Apr 2026 11:10:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374607; cv=none; b=pHbgb0z+iBHrIkhVhqQKO7pntrfEIQvQYU4dRKkW9ZQZt8FEAUmi5KAUhGaWeFsUJrwHvk/9wf8MiQy9T840gJ8YGapZrjI/HcJHftyDaD0qhZjF7EyKg1GLNrVB0HfoKnOsvG5Guc/i5LxzgfoOH7Wq1tK//Qx+VaS+IYDDcH8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374607; c=relaxed/simple; bh=bSg7XmRbrruQa2PqC5StFoRIwS95D3bE1el8XO+L7IM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=JeeOTENpAc9y2wLQB1Z7FEI4tHtD047M1edc5+tJbhDyz0Tz3wej0JXG/QEOVmT17diqAlFDtlncL9ip51VHq64A/vfOpJdJrvnzqvAybpEoYGQ+7WbH/vbYlIKeePbVLOH61KT1j3QIR/kfBVb0DkBffEASTZ0qHNK3M265YOo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Z2O9MjPT; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Z2O9MjPT" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374602; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=q4I+ErKCcFAyo+V91TRlTUsn1L5S+t2tgXLY5cfXULw=; b=Z2O9MjPTXij26The1Nn2Q/hkCdCiJTn5bKZsNlYbo/10DOuixoy6PZkeKmWjWwgIo0mSVr U/2V6pQfw0Pd0vRZ4VTst4HE0Pp0DvJfoiTNHyGMxPvhwpAZ99nYONhPMM7hydeEg2uy6q 4rAxzkEQNP9WNokEAY6nfbDn9Uf9/uw= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-225-NACe8r25O3OKAZ_NhnQ2kw-1; Tue, 28 Apr 2026 07:09:58 -0400 X-MC-Unique: NACe8r25O3OKAZ_NhnQ2kw-1 X-Mimecast-MFC-AGG-ID: NACe8r25O3OKAZ_NhnQ2kw_1777374597 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id ACE4418005AA; Tue, 28 Apr 2026 11:09:57 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0FC15180045E; Tue, 28 Apr 2026 11:09:56 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 12/28] KVM: x86: make translate_nested_gpa vendor-specific Date: Tue, 28 Apr 2026 07:09:30 -0400 Message-ID: <20260428110946.11466-13-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Content-Type: text/plain; charset="utf-8" EPT and NPT have different rules for passing PFERR_USER_MASK to the nested page table walk. In particular, for final addresses EPT uses the U bit of the guest (nGVA->nGPA) walk. While at it, remove PFERR_USER_MASK from the VMX version of the function, since it is actually ignored by the tables that update_permission_bitmask() generates for EPT. Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/include/asm/kvm_host.h | 4 ++++ arch/x86/kvm/hyperv.c | 3 ++- arch/x86/kvm/mmu.h | 9 +++------ arch/x86/kvm/svm/nested.c | 15 +++++++++++++++ arch/x86/kvm/vmx/nested.c | 12 ++++++++++++ arch/x86/kvm/x86.c | 16 ---------------- 6 files changed, 36 insertions(+), 23 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 8f2a1b915df9..62dc782b2dd3 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -2010,6 +2010,10 @@ struct kvm_x86_nested_ops { struct kvm_nested_state *kvm_state); bool (*get_nested_state_pages)(struct kvm_vcpu *vcpu); int (*write_log_dirty)(struct kvm_vcpu *vcpu, gpa_t l2_gpa); + gpa_t (*translate_nested_gpa)(struct kvm_vcpu *vcpu, gpa_t gpa, + u64 access, + struct x86_exception *exception, + u64 pte_access); =20 int (*enable_evmcs)(struct kvm_vcpu *vcpu, uint16_t *vmcs_version); diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 53688f7b76eb..f35fae3a7b3d 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -2041,7 +2041,8 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, st= ruct kvm_hv_hcall *hc) * read with kvm_read_guest(). */ if (!hc->fast && is_guest_mode(vcpu)) { - hc->ingpa =3D translate_nested_gpa(vcpu, hc->ingpa, + hc->ingpa =3D kvm_x86_ops.nested_ops->translate_nested_gpa( + vcpu, hc->ingpa, PFERR_GUEST_FINAL_MASK, NULL, 0); if (unlikely(hc->ingpa =3D=3D INVALID_GPA)) return HV_STATUS_INVALID_HYPERCALL_INPUT; diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index 635c2e5d8513..63be5c5efed9 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -294,10 +294,6 @@ static inline void kvm_update_page_stats(struct kvm *k= vm, int level, int count) atomic64_add(count, &kvm->stat.pages[level - 1]); } =20 -gpa_t translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa, u64 access, - struct x86_exception *exception, - u64 pte_access); - static inline gpa_t kvm_translate_gpa(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, gpa_t gpa, u64 access, @@ -306,8 +302,9 @@ static inline gpa_t kvm_translate_gpa(struct kvm_vcpu *= vcpu, { if (mmu !=3D &vcpu->arch.nested_mmu) return gpa; - return translate_nested_gpa(vcpu, gpa, access, exception, - pte_access); + return kvm_x86_ops.nested_ops->translate_nested_gpa(vcpu, gpa, access, + exception, + pte_access); } =20 static inline bool kvm_has_mirrored_tdp(const struct kvm *kvm) diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index 961804df5f45..df232153eb24 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -2071,8 +2071,23 @@ static bool svm_get_nested_state_pages(struct kvm_vc= pu *vcpu) return true; } =20 +static gpa_t svm_translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa, + u64 access, + struct x86_exception *exception, + u64 pte_access) +{ + struct kvm_mmu *mmu =3D vcpu->arch.mmu; + + BUG_ON(!mmu_is_nested(vcpu)); + + /* NPT walks are always user-walks */ + access |=3D PFERR_USER_MASK; + return mmu->gva_to_gpa(vcpu, mmu, gpa, access, exception); +} + struct kvm_x86_nested_ops svm_nested_ops =3D { .leave_nested =3D svm_leave_nested, + .translate_nested_gpa =3D svm_translate_nested_gpa, .is_exception_vmexit =3D nested_svm_is_exception_vmexit, .check_events =3D svm_check_nested_events, .triple_fault =3D nested_svm_triple_fault, diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 3fe88f29be7a..cd1924c6e075 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -7438,8 +7438,20 @@ __init int nested_vmx_hardware_setup(int (*exit_hand= lers[])(struct kvm_vcpu *)) return 0; } =20 +static gpa_t vmx_translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa, + u64 access, + struct x86_exception *exception, + u64 pte_access) +{ + struct kvm_mmu *mmu =3D vcpu->arch.mmu; + + BUG_ON(!mmu_is_nested(vcpu)); + return mmu->gva_to_gpa(vcpu, mmu, gpa, access, exception); +} + struct kvm_x86_nested_ops vmx_nested_ops =3D { .leave_nested =3D vmx_leave_nested, + .translate_nested_gpa =3D vmx_translate_nested_gpa, .is_exception_vmexit =3D nested_vmx_is_exception_vmexit, .check_events =3D vmx_check_nested_events, .has_events =3D vmx_has_nested_events, diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 67979b7de5d6..7c6942afae81 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -7848,22 +7848,6 @@ void kvm_get_segment(struct kvm_vcpu *vcpu, kvm_x86_call(get_segment)(vcpu, var, seg); } =20 -gpa_t translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa, u64 access, - struct x86_exception *exception, - u64 pte_access) -{ - struct kvm_mmu *mmu =3D vcpu->arch.mmu; - gpa_t t_gpa; - - BUG_ON(!mmu_is_nested(vcpu)); - - /* NPT walks are always user-walks */ - access |=3D PFERR_USER_MASK; - t_gpa =3D mmu->gva_to_gpa(vcpu, mmu, gpa, access, exception); - - return t_gpa; -} - gpa_t kvm_mmu_gva_to_gpa_read(struct kvm_vcpu *vcpu, gva_t gva, struct x86_exception *exception) { --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 039E63FA5E8 for ; Tue, 28 Apr 2026 11:10:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374608; cv=none; b=R3EttrDhse60Tm9q+9ejqdHyFTtETTIiExljUrv6nm6TtBGSF8CcvnPGw+nBglc39bIxnP2Nxcb4CHj8mmgGtkV+nFTLUqNvnhJjFtPP5ckYXrwzjZt2hiXsfacIaikdomWynCx7YeUzTwE8HuPP9VvCv1FOry4nddI05b+qAv8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374608; c=relaxed/simple; bh=UxCHQEBZ5HLm+rsI3qNAgxCjeTCORALu2NuHST1ynwo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=VLpeNwGPnePIE99G5k1tUJ52OMLFHfCIgRfurHrNoTCKKFXqX3Rg+ZXVtj3/k+ZmUkwIkuzQqTZoZNhqfH/JMbShss10qhQVcnUMQR89dup5a6P6cH6OuqTeHHs36ALlZtD1XQeOLncTTw0+r+KeW/ly3InanJjqoW/sfK8S7lQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=BARbU9AQ; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BARbU9AQ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374603; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LTcRs9TwppRL59fZPXP1OJc3O2pgW5ClzGiNpIcG0qw=; b=BARbU9AQjKKs5U2qn+ztqTd7K+o4mr+AWliAgyIgIgqMgy9GH+rlPHsMQt6nCluN8DTEA1 tjurUK9LHOYiRFU4w9m2U2rxKmJlxvES4T+GHrD+5EQD5qTZf4Ttr0MwS3aHDf1LVJaYxl RrcYG2gseQP3w0gMSqcVO04PXn2lqV8= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-193-NBXUJ1juPBejsrqUKIZi3w-1; Tue, 28 Apr 2026 07:09:59 -0400 X-MC-Unique: NBXUJ1juPBejsrqUKIZi3w-1 X-Mimecast-MFC-AGG-ID: NBXUJ1juPBejsrqUKIZi3w_1777374598 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4F88E19560A6; Tue, 28 Apr 2026 11:09:58 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id BD222180045E; Tue, 28 Apr 2026 11:09:57 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 13/28] KVM: x86/mmu: split XS/XU bits for EPT Date: Tue, 28 Apr 2026 07:09:31 -0400 Message-ID: <20260428110946.11466-14-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Content-Type: text/plain; charset="utf-8" When EPT is in use, replace ACC_USER_MASK with ACC_USER_EXEC_MASK, so that supervisor and user-mode execution can be controlled independently (ACC_USER_MASK would not allow a setting similar to XU=3D0 XS=3D1 W=3D1 R=3D1). Replace shadow_x_mask with shadow_xs_mask/shadow_xu_mask, to allow setting XS and XU bits separately in EPT entries. Note that ACC_USER_EXEC_MASK is already set through ACC_ALL in the kvm_mmu_page roles, but it does not propagate to the XU bit because (for now) shadow_xs_mask =3D=3D shadow_xu_mask. Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu.h | 3 ++- arch/x86/kvm/mmu/mmu.c | 2 +- arch/x86/kvm/mmu/mmutrace.h | 6 ++--- arch/x86/kvm/mmu/spte.c | 44 +++++++++++++++++++++++-------------- arch/x86/kvm/mmu/spte.h | 11 ++++++++-- 5 files changed, 42 insertions(+), 24 deletions(-) diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index 63be5c5efed9..d8c13e43c2d7 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -39,7 +39,8 @@ extern bool __read_mostly enable_mmio_caching; =20 #define ACC_READ_MASK PT_PRESENT_MASK #define ACC_WRITE_MASK PT_WRITABLE_MASK -#define ACC_USER_MASK PT_USER_MASK +#define ACC_USER_MASK PT_USER_MASK /* non EPT */ +#define ACC_USER_EXEC_MASK ACC_USER_MASK /* EPT only */ #define ACC_EXEC_MASK 8 #define ACC_ALL (ACC_EXEC_MASK | ACC_WRITE_MASK | ACC_USER_MASK |= ACC_READ_MASK) =20 diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index c82d151ca6c1..72f56791122e 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -5485,7 +5485,7 @@ static void reset_shadow_zero_bits_mask(struct kvm_vc= pu *vcpu, static inline bool boot_cpu_is_amd(void) { WARN_ON_ONCE(!tdp_enabled); - return shadow_x_mask =3D=3D 0; + return shadow_xs_mask =3D=3D 0; } =20 /* diff --git a/arch/x86/kvm/mmu/mmutrace.h b/arch/x86/kvm/mmu/mmutrace.h index dcfdfedfc4e9..3429c1413f42 100644 --- a/arch/x86/kvm/mmu/mmutrace.h +++ b/arch/x86/kvm/mmu/mmutrace.h @@ -357,8 +357,8 @@ TRACE_EVENT( __entry->sptep =3D virt_to_phys(sptep); __entry->level =3D level; __entry->r =3D shadow_present_mask || (__entry->spte & PT_PRESENT_MASK); - __entry->x =3D is_executable_pte(__entry->spte); - __entry->u =3D shadow_user_mask ? !!(__entry->spte & shadow_user_mask) := -1; + __entry->x =3D (__entry->spte & (shadow_xs_mask | shadow_nx_mask)) =3D= =3D shadow_xs_mask; + __entry->u =3D !!(__entry->spte & (shadow_xu_mask | shadow_user_mask)); ), =20 TP_printk("gfn %llx spte %llx (%s%s%s%s) level %d at %llx", @@ -366,7 +366,7 @@ TRACE_EVENT( __entry->r ? "r" : "-", __entry->spte & PT_WRITABLE_MASK ? "w" : "-", __entry->x ? "x" : "-", - __entry->u =3D=3D -1 ? "" : (__entry->u ? "u" : "-"), + __entry->u ? "u" : "-", __entry->level, __entry->sptep ) ); diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 7b5f118ae211..779ee44893b0 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -29,8 +29,9 @@ bool __read_mostly kvm_ad_enabled; u64 __read_mostly shadow_host_writable_mask; u64 __read_mostly shadow_mmu_writable_mask; u64 __read_mostly shadow_nx_mask; -u64 __read_mostly shadow_x_mask; /* mutual exclusive with nx_mask */ u64 __read_mostly shadow_user_mask; +u64 __read_mostly shadow_xs_mask; /* mutual exclusive with nx_mask and use= r_mask */ +u64 __read_mostly shadow_xu_mask; /* mutual exclusive with nx_mask and use= r_mask */ u64 __read_mostly shadow_accessed_mask; u64 __read_mostly shadow_dirty_mask; u64 __read_mostly shadow_mmio_value; @@ -217,21 +218,26 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_= page *sp, * would tie make_spte() further to vCPU/MMU state, and add complexity * just to optimize a mode that is anything but performance critical. */ - if (level > PG_LEVEL_4K && (pte_access & ACC_EXEC_MASK) && - is_nx_huge_page_enabled(vcpu->kvm)) { + if (level > PG_LEVEL_4K && is_nx_huge_page_enabled(vcpu->kvm)) { pte_access &=3D ~ACC_EXEC_MASK; + if (shadow_xu_mask) + pte_access &=3D ~ACC_USER_EXEC_MASK; } =20 if (pte_access & ACC_READ_MASK) spte |=3D PT_PRESENT_MASK; /* or VMX_EPT_READABLE_MASK */ =20 - if (pte_access & ACC_EXEC_MASK) - spte |=3D shadow_x_mask; - else - spte |=3D shadow_nx_mask; - - if (pte_access & ACC_USER_MASK) - spte |=3D shadow_user_mask; + if (shadow_nx_mask) { + if (!(pte_access & ACC_EXEC_MASK)) + spte |=3D shadow_nx_mask; + if (pte_access & ACC_USER_MASK) + spte |=3D shadow_user_mask; + } else { + if (pte_access & ACC_EXEC_MASK) + spte |=3D shadow_xs_mask; + if (pte_access & ACC_USER_EXEC_MASK) + spte |=3D shadow_xu_mask; + } =20 if (level > PG_LEVEL_4K) spte |=3D PT_PAGE_SIZE_MASK; @@ -318,11 +324,13 @@ static u64 make_spte_executable(u64 spte, u8 access) { u64 set, clear; =20 - if (access & ACC_EXEC_MASK) - set =3D shadow_x_mask; + if (shadow_nx_mask) + set =3D (access & ACC_EXEC_MASK) ? 0 : shadow_nx_mask; else - set =3D shadow_nx_mask; - clear =3D set ^ (shadow_nx_mask | shadow_x_mask); + set =3D + (access & ACC_EXEC_MASK ? shadow_xs_mask : 0) | + (access & ACC_USER_EXEC_MASK ? shadow_xu_mask : 0); + clear =3D set ^ (shadow_nx_mask | shadow_xs_mask | shadow_xu_mask); return modify_spte_protections(spte, set, clear); } =20 @@ -389,7 +397,7 @@ u64 make_nonleaf_spte(u64 *child_pt, bool ad_disabled) =20 spte |=3D __pa(child_pt) | shadow_present_mask | PT_WRITABLE_MASK | PT_PRESENT_MASK /* or VMX_EPT_READABLE_MASK */ | - shadow_user_mask | shadow_x_mask | shadow_me_value; + shadow_user_mask | shadow_xs_mask | shadow_xu_mask | shadow_me_value; =20 if (ad_disabled) spte |=3D SPTE_TDP_AD_DISABLED; @@ -497,7 +505,8 @@ void kvm_mmu_set_ept_masks(bool has_ad_bits) shadow_accessed_mask =3D VMX_EPT_ACCESS_BIT; shadow_dirty_mask =3D VMX_EPT_DIRTY_BIT; shadow_nx_mask =3D 0ull; - shadow_x_mask =3D VMX_EPT_EXECUTABLE_MASK; + shadow_xs_mask =3D VMX_EPT_EXECUTABLE_MASK; + shadow_xu_mask =3D VMX_EPT_EXECUTABLE_MASK; shadow_present_mask =3D VMX_EPT_SUPPRESS_VE_BIT; =20 shadow_acc_track_mask =3D VMX_EPT_RWX_MASK; @@ -548,7 +557,8 @@ void kvm_mmu_reset_all_pte_masks(void) shadow_accessed_mask =3D PT_ACCESSED_MASK; shadow_dirty_mask =3D PT_DIRTY_MASK; shadow_nx_mask =3D PT64_NX_MASK; - shadow_x_mask =3D 0; + shadow_xs_mask =3D 0; + shadow_xu_mask =3D 0; shadow_present_mask =3D PT_PRESENT_MASK; =20 shadow_acc_track_mask =3D 0; diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index fe71ae131ec1..958605c6a5ea 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -178,8 +178,9 @@ extern bool __read_mostly kvm_ad_enabled; extern u64 __read_mostly shadow_host_writable_mask; extern u64 __read_mostly shadow_mmu_writable_mask; extern u64 __read_mostly shadow_nx_mask; -extern u64 __read_mostly shadow_x_mask; /* mutual exclusive with nx_mask */ extern u64 __read_mostly shadow_user_mask; +extern u64 __read_mostly shadow_xs_mask; /* mutual exclusive with nx_mask = and user_mask */ +extern u64 __read_mostly shadow_xu_mask; /* mutual exclusive with nx_mask = and user_mask */ extern u64 __read_mostly shadow_accessed_mask; extern u64 __read_mostly shadow_dirty_mask; extern u64 __read_mostly shadow_mmio_value; @@ -357,7 +358,13 @@ static inline bool is_last_spte(u64 pte, int level) =20 static inline bool is_executable_pte(u64 spte) { - return (spte & (shadow_x_mask | shadow_nx_mask)) =3D=3D shadow_x_mask; + /* + * For now, return true if either the XS or XU bit is set + * This function is only used for fast_page_fault, + * which never processes shadow EPT, and regular page + * tables always have XS=3D=3DXU. + */ + return (spte & (shadow_xs_mask | shadow_xu_mask | shadow_nx_mask)) !=3D s= hadow_nx_mask; } =20 static inline kvm_pfn_t spte_to_pfn(u64 pte) --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DF1DA3F9F41 for ; Tue, 28 Apr 2026 11:10:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374606; cv=none; b=cp+C+CVIz0O1AuEJcsgIszWRYVWcZfIp7e0Rjx0qC1xo7Wb3sQ7s3qr8OIcy1mbp4w2HyS0CrxaraoVXLs9e8HkYKRSgfJHAMBf0RkW13Y4ToDddKNoEOn5xWyk1gMC6SwF1XKZetho0bwK2OF/87ea0lA6u+Mp6ObXWV2F6pwQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374606; c=relaxed/simple; bh=u/T/XnFEYI8jb2x+BMcXW1IpRiElxCRg/RvL4rLmSSM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=T4VH57HHfie3QJLOedNFjE1LyX4M7Y/IY65CgI7BFJ3kDFTUJXbQ2RuYvcqYZ/mGJFWxN+rwH3ROk2YdD39RhorvUF52MKSffTrBZpnMb8iK+/BpzLBo93n0z+6PvSXclUEsTg0EXNt6RTUGOguu+jtM81J6Pe6aHFkqcdk+HgA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Z4SL6FG1; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Z4SL6FG1" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374601; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/5JGWhDyupao8iQS7w6vNtoMXP+MNkr3VzEL5bu/Qt0=; b=Z4SL6FG1qnfguqXnhbE8SXhyaVMDZrj8rTIb4vN9t+vKadVZgqhJMu7Z8HtHoWxU3ejlmW vAgDm3oJgvmWx+l/KJ8YgsaOpdjCEY04BmydrArGMhkYs2eoFKf1zNgLrKfzoBp2t50jyL R5cI6Z8QpFKsjQHaOTPqAMK2qDBBG2g= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-64-ea44xMwNOI6HhxDtM0M77A-1; Tue, 28 Apr 2026 07:10:00 -0400 X-MC-Unique: ea44xMwNOI6HhxDtM0M77A-1 X-Mimecast-MFC-AGG-ID: ea44xMwNOI6HhxDtM0M77A_1777374599 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 092DD18002C6; Tue, 28 Apr 2026 11:09:59 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 76109180045E; Tue, 28 Apr 2026 11:09:58 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 14/28] KVM: x86/mmu: move cr4_smep to base role Date: Tue, 28 Apr 2026 07:09:32 -0400 Message-ID: <20260428110946.11466-15-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Content-Type: text/plain; charset="utf-8" Guest page tables can be reused independent of the value of CR4.SMEP (at least if WP=3D1). However, this is not true of EPT MBEC pages, because presence of EPT entries is signaled by bits 0-2 when MBEC is off, and bits 0-2 + bit 10 when MBEC is on. In preparation for enabling MBEC, move cr4_smep to the base role. This makes the smep_andnot_wp bit redundant, so remove it. Tested-by: David Riley Signed-off-by: Paolo Bonzini --- Documentation/virt/kvm/x86/mmu.rst | 10 ++++------ arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 23 +++++++++++++++-------- arch/x86/kvm/mmu/mmu.c | 6 +++--- 4 files changed, 23 insertions(+), 17 deletions(-) diff --git a/Documentation/virt/kvm/x86/mmu.rst b/Documentation/virt/kvm/x8= 6/mmu.rst index 2b3b6d442302..666aa179601a 100644 --- a/Documentation/virt/kvm/x86/mmu.rst +++ b/Documentation/virt/kvm/x86/mmu.rst @@ -184,10 +184,8 @@ Shadow pages contain the following information: Contains the value of efer.nx for which the page is valid. role.cr0_wp: Contains the value of cr0.wp for which the page is valid. - role.smep_andnot_wp: - Contains the value of cr4.smep && !cr0.wp for which the page is valid - (pages for which this is true are different from other pages; see the - treatment of cr0.wp=3D0 below). + role.cr4_smep: + Contains the value of cr4.smep for which the page is valid. role.smap_andnot_wp: Contains the value of cr4.smap && !cr0.wp for which the page is valid (pages for which this is true are different from other pages; see the @@ -435,8 +433,8 @@ from being written by the kernel after cr0.wp has chang= ed to 1, we make the value of cr0.wp part of the page role. This means that an spte created with one value of cr0.wp cannot be used when cr0.wp has a different value - it will simply be missed by the shadow page lookup code. A similar issue -exists when an spte created with cr0.wp=3D0 and cr4.smep=3D0 is used after -changing cr4.smep to 1. To avoid this, the value of !cr0.wp && cr4.smep +exists when an spte created with cr0.wp=3D0 and cr4.smap=3D0 is used after +changing cr4.smap to 1. To avoid this, the value of !cr0.wp && cr4.smap is also made a part of the page role. =20 Large pages diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-= x86-ops.h index 3776cf5382a2..e4fca997ec79 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -94,6 +94,7 @@ KVM_X86_OP_OPTIONAL(sync_pir_to_irr) KVM_X86_OP_OPTIONAL_RET0(set_tss_addr) KVM_X86_OP_OPTIONAL_RET0(set_identity_map_addr) KVM_X86_OP_OPTIONAL_RET0(get_mt_mask) +KVM_X86_OP_OPTIONAL_RET0(tdp_has_smep) KVM_X86_OP(load_mmu_pgd) KVM_X86_OP_OPTIONAL(link_external_spt) KVM_X86_OP_OPTIONAL(set_external_spte) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 62dc782b2dd3..23a7ac8d7fbe 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -343,8 +343,8 @@ struct kvm_kernel_irq_routing_entry; * paging has exactly one upper level, making level completely redunda= nt * when has_4_byte_gpte=3D1. * - * - on top of this, smep_andnot_wp and smap_andnot_wp are only set if - * cr0_wp=3D0, therefore these three bits only give rise to 5 possibil= ities. + * - on top of this, smap_andnot_wp is only set if cr0_wp=3D0, + * therefore these two bits only give rise to 3 possibilities. * * Therefore, the maximum number of possible upper-level shadow pages for a * single gfn is a bit less than 2^14. @@ -360,12 +360,19 @@ union kvm_mmu_page_role { unsigned invalid:1; unsigned efer_nx:1; unsigned cr0_wp:1; - unsigned smep_andnot_wp:1; unsigned smap_andnot_wp:1; unsigned ad_disabled:1; unsigned guest_mode:1; unsigned passthrough:1; unsigned is_mirror:1; + + /* + * cr4_smep is also set for EPT MBEC. Because it affects + * which pages are considered non-present (bit 10 additionally + * must be zero if MBEC is on) it has to be in the base role. + */ + unsigned cr4_smep:1; + unsigned:3; =20 /* @@ -392,10 +399,10 @@ union kvm_mmu_page_role { * tables (because KVM doesn't support Protection Keys with shadow paging)= , and * CR0.PG, CR4.PAE, and CR4.PSE are indirectly reflected in role.level. * - * Note, SMEP and SMAP are not redundant with sm*p_andnot_wp in the page r= ole. - * If CR0.WP=3D1, KVM can reuse shadow pages for the guest regardless of S= MEP and - * SMAP, but the MMU's permission checks for software walks need to be SME= P and - * SMAP aware regardless of CR0.WP. + * Note, SMAP is not redundant with smap_andnot_wp in the page role. If + * CR0.WP=3D1, KVM can reuse shadow pages for the guest regardless of SMAP, + * but the MMU's permission checks for software walks need to be SMAP + * aware regardless of CR0.WP. */ union kvm_mmu_extended_role { u32 word; @@ -405,7 +412,6 @@ union kvm_mmu_extended_role { unsigned int cr4_pse:1; unsigned int cr4_pke:1; unsigned int cr4_smap:1; - unsigned int cr4_smep:1; unsigned int cr4_la57:1; unsigned int efer_lma:1; }; @@ -1887,6 +1893,7 @@ struct kvm_x86_ops { int (*set_tss_addr)(struct kvm *kvm, unsigned int addr); int (*set_identity_map_addr)(struct kvm *kvm, u64 ident_addr); u8 (*get_mt_mask)(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio); + bool (*tdp_has_smep)(struct kvm *kvm); =20 void (*load_mmu_pgd)(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level); diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 72f56791122e..7ec29979a209 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -227,7 +227,7 @@ static inline bool __maybe_unused is_##reg##_##name(str= uct kvm_mmu *mmu) \ } BUILD_MMU_ROLE_ACCESSOR(base, cr0, wp); BUILD_MMU_ROLE_ACCESSOR(ext, cr4, pse); -BUILD_MMU_ROLE_ACCESSOR(ext, cr4, smep); +BUILD_MMU_ROLE_ACCESSOR(base, cr4, smep); BUILD_MMU_ROLE_ACCESSOR(ext, cr4, smap); BUILD_MMU_ROLE_ACCESSOR(ext, cr4, pke); BUILD_MMU_ROLE_ACCESSOR(ext, cr4, la57); @@ -5758,7 +5758,7 @@ static union kvm_cpu_role kvm_calc_cpu_role(struct kv= m_vcpu *vcpu, =20 role.base.efer_nx =3D ____is_efer_nx(regs); role.base.cr0_wp =3D ____is_cr0_wp(regs); - role.base.smep_andnot_wp =3D ____is_cr4_smep(regs) && !____is_cr0_wp(regs= ); + role.base.cr4_smep =3D ____is_cr4_smep(regs); role.base.smap_andnot_wp =3D ____is_cr4_smap(regs) && !____is_cr0_wp(regs= ); role.base.has_4_byte_gpte =3D !____is_cr4_pae(regs); =20 @@ -5770,7 +5770,6 @@ static union kvm_cpu_role kvm_calc_cpu_role(struct kv= m_vcpu *vcpu, else role.base.level =3D PT32_ROOT_LEVEL; =20 - role.ext.cr4_smep =3D ____is_cr4_smep(regs); role.ext.cr4_smap =3D ____is_cr4_smap(regs); role.ext.cr4_pse =3D ____is_cr4_pse(regs); =20 @@ -5829,6 +5828,7 @@ kvm_calc_tdp_mmu_root_page_role(struct kvm_vcpu *vcpu, =20 role.access =3D ACC_ALL; role.cr0_wp =3D true; + role.cr4_smep =3D kvm_x86_call(tdp_has_smep)(vcpu->kvm); role.efer_nx =3D true; role.smm =3D cpu_role.base.smm; role.guest_mode =3D cpu_role.base.guest_mode; --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 46FF83FA5C5 for ; Tue, 28 Apr 2026 11:10:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374608; cv=none; b=uCoSHDF5M4Bso0UvKeZ5eAGf0D0kklU4wOQJH3NTwtkbPfkO5R/Id78LsC33VyZWE3I2159/+n9lSunHfc7464RKbRcWWiyzyk6GZicr2UdJ3uksHbnOpBRgv7D7i/vNLGFhSCNbR2LjRDt3/ugIokqGmMoA84HK7YZY29W7YEI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374608; c=relaxed/simple; bh=CeVJdQrpMNewl6kbY7DOoOFc0qOOqb7KKzRI0hbw3C0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=DrYJFBHkNORYcwza4AuzpOVAdJqIQtY+1HQP3DlID+znfWYdRABOEtHRxXX3kqJ9t0lA+1a9JP81a8s/C7NYrMhLRNT0pMQhlGlJnFm5EEeeFIWHPrWa8GYyDjvLQEFgJBD/Jxk0B/GNa1oTPlwdx4Z/ISpZn51iaxQLjr8Aock= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=YQZEOWMG; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="YQZEOWMG" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374602; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XjlRO/sYKTHoG35nIXOBvfqYt2OqrvgxHk73uv4xoxQ=; b=YQZEOWMGhgg2DdVMZ/E31O+VR05hdiVu6hse+AEv0IPt/ZeulCNKo+M3zLgbYq/CUAWekm /84q3ME40CL/bqMnlXbUr//S+F7/3c/O9GHE7Vpzt2UlunZ3BuhyXJS6jG/okG8kD05gZm uIlvSld1z+imyamsujlY/6sLzyR8lvo= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-292-83K9qVKoM4OqUo2bqqqBFQ-1; Tue, 28 Apr 2026 07:10:00 -0400 X-MC-Unique: 83K9qVKoM4OqUo2bqqqBFQ-1 X-Mimecast-MFC-AGG-ID: 83K9qVKoM4OqUo2bqqqBFQ_1777374599 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B640419560B4; Tue, 28 Apr 2026 11:09:59 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2F134180047F; Tue, 28 Apr 2026 11:09:59 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 15/28] KVM: VMX: enable use of MBEC Date: Tue, 28 Apr 2026 07:09:33 -0400 Message-ID: <20260428110946.11466-16-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Content-Type: text/plain; charset="utf-8" If available, set SECONDARY_EXEC_MODE_BASED_EPT_EXEC in the secondary execution controls and configure XS and XU separately (even if they are always used together). Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/include/asm/vmx.h | 3 +++ arch/x86/kvm/mmu.h | 7 ++++++- arch/x86/kvm/mmu/spte.c | 6 +++--- arch/x86/kvm/mmu/spte.h | 5 +++-- arch/x86/kvm/vmx/capabilities.h | 7 +++++++ arch/x86/kvm/vmx/common.h | 10 +++++----- arch/x86/kvm/vmx/main.c | 9 +++++++++ arch/x86/kvm/vmx/nested.c | 1 + arch/x86/kvm/vmx/vmx.c | 16 +++++++++++++++- arch/x86/kvm/vmx/vmx.h | 1 + arch/x86/kvm/vmx/x86_ops.h | 1 + 11 files changed, 54 insertions(+), 12 deletions(-) diff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h index 2b30b921b375..54aa5be50df9 100644 --- a/arch/x86/include/asm/vmx.h +++ b/arch/x86/include/asm/vmx.h @@ -619,9 +619,12 @@ enum vm_entry_failure_code { #define EPT_VIOLATION_GVA_TRANSLATED BIT(8) =20 #define EPT_VIOLATION_RWX_TO_PROT(__epte) (((__epte) & VMX_EPT_RWX_MASK) <= < 3) +#define EPT_VIOLATION_USER_EXEC_TO_PROT(__epte) (((__epte) & VMX_EPT_USER_= EXECUTABLE_MASK) >> 4) =20 static_assert(EPT_VIOLATION_RWX_TO_PROT(VMX_EPT_RWX_MASK) =3D=3D (EPT_VIOLATION_PROT_READ | EPT_VIOLATION_PROT_WRITE | EPT_VIOLATION= _PROT_EXEC)); +static_assert(EPT_VIOLATION_USER_EXEC_TO_PROT(VMX_EPT_USER_EXECUTABLE_MASK= ) =3D=3D + (EPT_VIOLATION_PROT_USER_EXEC)); =20 /* * Exit Qualifications for NOTIFY VM EXIT diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index d8c13e43c2d7..d15f908d048f 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -83,12 +83,17 @@ static inline gfn_t kvm_mmu_max_gfn(void) return (1ULL << (max_gpa_bits - PAGE_SHIFT)) - 1; } =20 +static inline bool mmu_has_mbec(struct kvm_mmu *mmu) +{ + return mmu->root_role.cr4_smep; +} + u8 kvm_mmu_get_max_tdp_level(void); =20 void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mmio_mask, u64 access_= mask); void kvm_mmu_set_mmio_spte_value(struct kvm *kvm, u64 mmio_value); void kvm_mmu_set_me_spte_mask(u64 me_value, u64 me_mask); -void kvm_mmu_set_ept_masks(bool has_ad_bits); +void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_mbec); =20 void kvm_init_mmu(struct kvm_vcpu *vcpu); void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, unsigned long cr0, diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 779ee44893b0..6da5c73d1004 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -497,7 +497,7 @@ void kvm_mmu_set_me_spte_mask(u64 me_value, u64 me_mask) } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_mmu_set_me_spte_mask); =20 -void kvm_mmu_set_ept_masks(bool has_ad_bits) +void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_mbec) { kvm_ad_enabled =3D has_ad_bits; =20 @@ -506,10 +506,10 @@ void kvm_mmu_set_ept_masks(bool has_ad_bits) shadow_dirty_mask =3D VMX_EPT_DIRTY_BIT; shadow_nx_mask =3D 0ull; shadow_xs_mask =3D VMX_EPT_EXECUTABLE_MASK; - shadow_xu_mask =3D VMX_EPT_EXECUTABLE_MASK; + shadow_xu_mask =3D has_mbec ? VMX_EPT_USER_EXECUTABLE_MASK : VMX_EPT_EXE= CUTABLE_MASK; shadow_present_mask =3D VMX_EPT_SUPPRESS_VE_BIT; =20 - shadow_acc_track_mask =3D VMX_EPT_RWX_MASK; + shadow_acc_track_mask =3D VMX_EPT_RWX_MASK | shadow_xu_mask; shadow_host_writable_mask =3D EPT_SPTE_HOST_WRITABLE; shadow_mmu_writable_mask =3D EPT_SPTE_MMU_WRITABLE; =20 diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index 958605c6a5ea..22923ddd0617 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -24,7 +24,7 @@ * - bits 55 (EPT only): MMU-writable * - bits 56-59: unused * - bits 60-61: type of A/D tracking - * - bits 62: unused + * - bits 62 (EPT only): saved XU bit for disabled AD */ =20 /* @@ -65,7 +65,8 @@ static_assert(SPTE_TDP_AD_ENABLED =3D=3D 0); * must not overlap the A/D type mask. */ #define SHADOW_ACC_TRACK_SAVED_BITS_MASK (VMX_EPT_READABLE_MASK | \ - VMX_EPT_EXECUTABLE_MASK) + VMX_EPT_EXECUTABLE_MASK | \ + VMX_EPT_USER_EXECUTABLE_MASK) #define SHADOW_ACC_TRACK_SAVED_BITS_SHIFT 52 #define SHADOW_ACC_TRACK_SAVED_MASK (SHADOW_ACC_TRACK_SAVED_BITS_MASK << \ SHADOW_ACC_TRACK_SAVED_BITS_SHIFT) diff --git a/arch/x86/kvm/vmx/capabilities.h b/arch/x86/kvm/vmx/capabilitie= s.h index 7e59eb0f41bb..07469d1cfe74 100644 --- a/arch/x86/kvm/vmx/capabilities.h +++ b/arch/x86/kvm/vmx/capabilities.h @@ -15,6 +15,7 @@ extern bool __read_mostly enable_ept; extern bool __read_mostly enable_unrestricted_guest; extern bool __read_mostly enable_ept_ad_bits; extern bool __read_mostly enable_pml; +extern bool __read_mostly enable_mbec; extern int __read_mostly pt_mode; =20 #define PT_MODE_SYSTEM 0 @@ -406,4 +407,10 @@ static inline bool cpu_has_notify_vmexit(void) SECONDARY_EXEC_NOTIFY_VM_EXITING; } =20 +static inline bool cpu_has_ept_mbec(void) +{ + return vmcs_config.cpu_based_2nd_exec_ctrl & + SECONDARY_EXEC_MODE_BASED_EPT_EXEC; +} + #endif /* __KVM_X86_VMX_CAPS_H */ diff --git a/arch/x86/kvm/vmx/common.h b/arch/x86/kvm/vmx/common.h index 1afbf272efae..40fa72f31fc7 100644 --- a/arch/x86/kvm/vmx/common.h +++ b/arch/x86/kvm/vmx/common.h @@ -91,15 +91,15 @@ static inline int __vmx_handle_ept_violation(struct kvm= _vcpu *vcpu, gpa_t gpa, /* Is it a fetch fault? */ error_code |=3D (exit_qualification & EPT_VIOLATION_ACC_INSTR) ? PFERR_FETCH_MASK : 0; - /* - * ept page table entry is present? - * note: unconditionally clear USER_EXEC until mode-based - * execute control is implemented - */ + /* ept page table entry is present? */ error_code |=3D (exit_qualification & (EPT_VIOLATION_PROT_MASK & ~EPT_VIOLATION_PROT_USER_EXEC)) ? PFERR_PRESENT_MASK : 0; =20 + if (mmu_has_mbec(vcpu->arch.mmu)) + error_code |=3D (exit_qualification & EPT_VIOLATION_PROT_USER_EXEC) + ? PFERR_PRESENT_MASK : 0; + if (exit_qualification & EPT_VIOLATION_GVA_IS_VALID) error_code |=3D (exit_qualification & EPT_VIOLATION_GVA_TRANSLATED) ? PFERR_GUEST_FINAL_MASK : PFERR_GUEST_PAGE_MASK; diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index dbebddf648be..83d9921277ea 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -755,6 +755,14 @@ static int vt_set_identity_map_addr(struct kvm *kvm, u= 64 ident_addr) return vmx_set_identity_map_addr(kvm, ident_addr); } =20 +static bool vt_tdp_has_smep(struct kvm *kvm) +{ + if (is_td(kvm)) + return false; + + return vmx_tdp_has_smep(kvm); +} + static u64 vt_get_l2_tsc_offset(struct kvm_vcpu *vcpu) { /* TDX doesn't support L2 guest at the moment. */ @@ -966,6 +974,7 @@ struct kvm_x86_ops vt_x86_ops __initdata =3D { .set_tss_addr =3D vt_op(set_tss_addr), .set_identity_map_addr =3D vt_op(set_identity_map_addr), .get_mt_mask =3D vmx_get_mt_mask, + .tdp_has_smep =3D vt_op(tdp_has_smep), =20 .get_exit_info =3D vt_op(get_exit_info), .get_entry_info =3D vt_op(get_entry_info), diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index cd1924c6e075..299d4ca60fb3 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -2440,6 +2440,7 @@ static void prepare_vmcs02_early(struct vcpu_vmx *vmx= , struct loaded_vmcs *vmcs0 SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY | SECONDARY_EXEC_APIC_REGISTER_VIRT | SECONDARY_EXEC_ENABLE_VMFUNC | + SECONDARY_EXEC_MODE_BASED_EPT_EXEC | SECONDARY_EXEC_DESC); =20 if (nested_cpu_has(vmcs12, diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 337bbfecc021..72a75fa33c93 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -114,6 +114,9 @@ module_param(emulate_invalid_guest_state, bool, 0444); static bool __read_mostly fasteoi =3D 1; module_param(fasteoi, bool, 0444); =20 +bool __read_mostly enable_mbec =3D 1; +module_param_named(mbec, enable_mbec, bool, 0444); + module_param(enable_apicv, bool, 0444); module_param(enable_ipiv, bool, 0444); =20 @@ -2773,6 +2776,7 @@ static int setup_vmcs_config(struct vmcs_config *vmcs= _conf, return -EIO; =20 vmx_cap->ept =3D 0; + _cpu_based_2nd_exec_control &=3D ~SECONDARY_EXEC_MODE_BASED_EPT_EXEC; _cpu_based_2nd_exec_control &=3D ~SECONDARY_EXEC_EPT_VIOLATION_VE; } if (!(_cpu_based_2nd_exec_control & SECONDARY_EXEC_ENABLE_VPID) && @@ -4735,6 +4739,9 @@ static u32 vmx_secondary_exec_control(struct vcpu_vmx= *vmx) */ exec_control &=3D ~SECONDARY_EXEC_ENABLE_VMFUNC; =20 + if (!enable_mbec) + exec_control &=3D ~SECONDARY_EXEC_MODE_BASED_EPT_EXEC; + /* SECONDARY_EXEC_DESC is enabled/disabled on writes to CR4.UMIP, * in vmx_set_cr4. */ exec_control &=3D ~SECONDARY_EXEC_DESC; @@ -7823,6 +7830,11 @@ u8 vmx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn,= bool is_mmio) return (MTRR_TYPE_WRBACK << VMX_EPT_MT_EPTE_SHIFT); } =20 +bool vmx_tdp_has_smep(struct kvm *kvm) +{ + return enable_mbec; +} + static void vmcs_set_secondary_exec_control(struct vcpu_vmx *vmx, u32 new_= ctl) { /* @@ -8622,6 +8634,8 @@ __init int vmx_hardware_setup(void) =20 if (!cpu_has_vmx_ept_ad_bits() || !enable_ept) enable_ept_ad_bits =3D 0; + if (!cpu_has_ept_mbec() || !enable_ept) + enable_mbec =3D 0; =20 if (!cpu_has_vmx_unrestricted_guest() || !enable_ept) enable_unrestricted_guest =3D 0; @@ -8683,7 +8697,7 @@ __init int vmx_hardware_setup(void) set_bit(0, vmx_vpid_bitmap); /* 0 is reserved for host */ =20 if (enable_ept) - kvm_mmu_set_ept_masks(enable_ept_ad_bits); + kvm_mmu_set_ept_masks(enable_ept_ad_bits, enable_mbec); else vt_x86_ops.get_mt_mask =3D NULL; =20 diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index db84e8001da5..0a4e263c4095 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -567,6 +567,7 @@ static inline u8 vmx_get_rvi(void) SECONDARY_EXEC_ENABLE_VMFUNC | \ SECONDARY_EXEC_BUS_LOCK_DETECTION | \ SECONDARY_EXEC_NOTIFY_VM_EXITING | \ + SECONDARY_EXEC_MODE_BASED_EPT_EXEC | \ SECONDARY_EXEC_ENCLS_EXITING | \ SECONDARY_EXEC_EPT_VIOLATION_VE) =20 diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index d09abeac2b56..69cf276be88e 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -103,6 +103,7 @@ void vmx_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *e= oi_exit_bitmap); int vmx_set_tss_addr(struct kvm *kvm, unsigned int addr); int vmx_set_identity_map_addr(struct kvm *kvm, u64 ident_addr); u8 vmx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio); +bool vmx_tdp_has_smep(struct kvm *kvm); =20 void vmx_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, u64 *info1, u64 *info2, u32 *intr_info, u32 *error_code); --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 334E63F23BE for ; Tue, 28 Apr 2026 11:10:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374609; cv=none; b=a7jJy+y3fMAebZSc4zjnfoxn51hkClJzNAV8UVYIjaJv/rdCOHdbtWCwNfzygzsdalAe0O+vN6V8CraVbrS3nPSK6a7bGcrXS+g3fmvGbTR1TvUmP22MF7bTSKG4jlozBSWSJBdJlZ7WpZofA06VJ2OJWTt/Zm4Z7wAu3OxowdA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374609; c=relaxed/simple; bh=uWRDnS/vHDd52rjGMERq6SsAiryYF/CgZnXspFuYvHw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=j7v+IMgECa3BezpTspcbGHhPF0gON1Dluf4n8MFBVoQlGf9avd2l7o6L4Hh49+Eh770c+jUP4xeYYIccyaXYlloiVy7/msa+HMgzoodcMhuKDmDxdb7SxBslJbTNmP7xMnkdMLId9/ztLZusDfX8WbRxNp0XImOUGUbcppAgTGo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=DSf6WUGS; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="DSf6WUGS" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374604; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Kuf4IDR3Ydbimabid6qV/JghvOduDmfrEOhZ/hgfers=; b=DSf6WUGSs52CO8NcwNW9ZZPWCDehtwVw7BvOELODk3wwOiyixZV3qDg4yjmP21MQcfWvzf g309okNRByzXu8oGWVU4LZ1Dx674sIrP2xsjs/yNmvHum6Llh6Z4pBkd7aG3Cujc292Ik+ S1D81/4eBoPH3Rez9PKWehDUX3Ujx90= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-460-hmPNU8CTNky9muvwOoZq8A-1; Tue, 28 Apr 2026 07:10:01 -0400 X-MC-Unique: hmPNU8CTNky9muvwOoZq8A-1 X-Mimecast-MFC-AGG-ID: hmPNU8CTNky9muvwOoZq8A_1777374600 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6F7531800367; Tue, 28 Apr 2026 11:10:00 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id DC6FF180047F; Tue, 28 Apr 2026 11:09:59 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 16/28] KVM: nVMX: pass advanced EPT violation vmexit info to guest Date: Tue, 28 Apr 2026 07:09:34 -0400 Message-ID: <20260428110946.11466-17-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Content-Type: text/plain; charset="utf-8" KVM will use advanced vmexit information for EPT violations to virtualize MBEC. Pass it to the guest since it is easy and allows testing nested nested. Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/include/asm/vmx.h | 4 ++++ arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/vmx/nested.c | 13 +++++++++---- 3 files changed, 14 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h index 54aa5be50df9..ed2ded531e55 100644 --- a/arch/x86/include/asm/vmx.h +++ b/arch/x86/include/asm/vmx.h @@ -535,6 +535,7 @@ enum vmcs_field { #define VMX_EPT_1GB_PAGE_BIT (1ull << 17) #define VMX_EPT_INVEPT_BIT (1ull << 20) #define VMX_EPT_AD_BIT (1ull << 21) +#define VMX_EPT_ADVANCED_VMEXIT_INFO_BIT (1ull << 22) #define VMX_EPT_EXTENT_CONTEXT_BIT (1ull << 25) #define VMX_EPT_EXTENT_GLOBAL_BIT (1ull << 26) =20 @@ -617,6 +618,9 @@ enum vm_entry_failure_code { EPT_VIOLATION_PROT_USER_EXEC) #define EPT_VIOLATION_GVA_IS_VALID BIT(7) #define EPT_VIOLATION_GVA_TRANSLATED BIT(8) +#define EPT_VIOLATION_GVA_USER BIT(9) +#define EPT_VIOLATION_GVA_WRITABLE BIT(10) +#define EPT_VIOLATION_GVA_NX BIT(11) =20 #define EPT_VIOLATION_RWX_TO_PROT(__epte) (((__epte) & VMX_EPT_RWX_MASK) <= < 3) #define EPT_VIOLATION_USER_EXEC_TO_PROT(__epte) (((__epte) & VMX_EPT_USER_= EXECUTABLE_MASK) >> 4) diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index de8770d2fcfc..f5f9e745f21d 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -494,7 +494,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker= *walker, * [2:0] - Derive from the access bits. The exit_qualification might be * out of date if it is serving an EPT misconfiguration. * [5:3] - Calculated by the page walk of the guest EPT page tables - * [7:8] - Derived from [7:8] of real exit_qualification + * [7:11] - Derived from [7:11] of real exit_qualification * * The other bits are set to 0. */ diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 299d4ca60fb3..46b65475765d 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -443,10 +443,14 @@ static void nested_ept_inject_page_fault(struct kvm_v= cpu *vcpu, vm_exit_reason =3D EXIT_REASON_EPT_MISCONFIG; exit_qualification =3D 0; } else { + u64 mask =3D EPT_VIOLATION_GVA_IS_VALID | + EPT_VIOLATION_GVA_TRANSLATED; + if (vmx->nested.msrs.ept_caps & VMX_EPT_ADVANCED_VMEXIT_INFO_BIT) + mask |=3D EPT_VIOLATION_GVA_USER | + EPT_VIOLATION_GVA_WRITABLE | + EPT_VIOLATION_GVA_NX; exit_qualification =3D fault->exit_qualification; - exit_qualification |=3D vmx_get_exit_qual(vcpu) & - (EPT_VIOLATION_GVA_IS_VALID | - EPT_VIOLATION_GVA_TRANSLATED); + exit_qualification |=3D vmx_get_exit_qual(vcpu) & mask; vm_exit_reason =3D EXIT_REASON_EPT_VIOLATION; } =20 @@ -7240,7 +7244,8 @@ static void nested_vmx_setup_secondary_ctls(u32 ept_c= aps, VMX_EPT_PAGE_WALK_5_BIT | VMX_EPTP_WB_BIT | VMX_EPT_INVEPT_BIT | - VMX_EPT_EXECUTE_ONLY_BIT; + VMX_EPT_EXECUTE_ONLY_BIT | + VMX_EPT_ADVANCED_VMEXIT_INFO_BIT; =20 msrs->ept_caps &=3D ept_caps; msrs->ept_caps |=3D VMX_EPT_EXTENT_GLOBAL_BIT | --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E554401A0E for ; Tue, 28 Apr 2026 11:10:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374622; cv=none; b=DFO7ZyiCMjkc3F3kUN8D5Axg6E24UscRTJgBF8oMRRg3hvzVA5jZdrbdvk1Vz5EfMbqnq9l0KETtg9H4dYj0Yb29FuD9fGHeLXS8llmMW3oKAyMdTCOGht9iy6jgjGEm+q2LffK0i7Spz+VkPQ521njh/i+0/bDGSRsEVMESabw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374622; c=relaxed/simple; bh=+u1fMN6nd8KUKMbyFT62MQ4AptYZ2XnnW9NGV9ACRcY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Eg9VwW1wi9TY46SQlkq1miqn7rlW4YUi/UGVW+MbK1sutUKdzGw9go/O/z/YRzaWeFDT+QOmiwKxkCgJznNXwM2JYrLUtx4TZQq24sb+JLQfWKBnRhVbDL6o0JKVEt+LF02sPdGrEZxVp9jBIGRdzF3vb/dns5OvtfRaDNS7dgo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=X8HjMKgw; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="X8HjMKgw" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374615; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0LpoYunsVnYZDVJp7EOWJeICXFOg6+wbjkFfeG7udrI=; b=X8HjMKgwOoYm0IFBcHHtIAdQAmtagXo95xY6fOZzbv+9xaaReye8INvJa6TnjbG+ENWO69 y1EcGebjqgl+WDcK2RcYXVMnRWPT+MZfkSkTfMqbknNnt3YSx/tGOAyhsYJXSTZYA5dQGC jTFqGHqzlwyYgcQB8dhSnLu6AAcZlzk= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-288-nqghDRKJPCyBbrBknjSguw-1; Tue, 28 Apr 2026 07:10:04 -0400 X-MC-Unique: nqghDRKJPCyBbrBknjSguw-1 X-Mimecast-MFC-AGG-ID: nqghDRKJPCyBbrBknjSguw_1777374601 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 28390195609E; Tue, 28 Apr 2026 11:10:01 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 95C0D180047F; Tue, 28 Apr 2026 11:10:00 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 17/28] KVM: nVMX: pass PFERR_USER_MASK to MMU on EPT violations Date: Tue, 28 Apr 2026 07:09:35 -0400 Message-ID: <20260428110946.11466-18-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Content-Type: text/plain; charset="utf-8" For EPT, PFERR_USER_MASK refers not to the CPL of the guest, but to the AND of the U bits encountered while walking guest page tables; this is consistent with how MBEC differentiates between XS and XU. This is available through the "advanced vmexit information for EPT violations" feature. Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/kvm/vmx/common.h | 6 +++++- arch/x86/kvm/vmx/vmx.c | 10 ++++++++++ 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/vmx/common.h b/arch/x86/kvm/vmx/common.h index 40fa72f31fc7..48520fa1c8e8 100644 --- a/arch/x86/kvm/vmx/common.h +++ b/arch/x86/kvm/vmx/common.h @@ -100,9 +100,13 @@ static inline int __vmx_handle_ept_violation(struct kv= m_vcpu *vcpu, gpa_t gpa, error_code |=3D (exit_qualification & EPT_VIOLATION_PROT_USER_EXEC) ? PFERR_PRESENT_MASK : 0; =20 - if (exit_qualification & EPT_VIOLATION_GVA_IS_VALID) + if (exit_qualification & EPT_VIOLATION_GVA_IS_VALID) { error_code |=3D (exit_qualification & EPT_VIOLATION_GVA_TRANSLATED) ? PFERR_GUEST_FINAL_MASK : PFERR_GUEST_PAGE_MASK; + if ((exit_qualification & (EPT_VIOLATION_GVA_TRANSLATED|EPT_VIOLATION_GV= A_USER)) + =3D=3D (EPT_VIOLATION_GVA_TRANSLATED|EPT_VIOLATION_GVA_USER)) + error_code |=3D PFERR_USER_MASK; + } =20 if (vt_is_tdx_private_gpa(vcpu->kvm, gpa)) error_code |=3D PFERR_PRIVATE_ACCESS; diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 72a75fa33c93..5e4aa519423c 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -2790,6 +2790,16 @@ static int setup_vmcs_config(struct vmcs_config *vmc= s_conf, vmx_cap->vpid =3D 0; } =20 + /* + * Virtualizing MBEC requires advanced vmexit information in order to + * distinguish supervisor and user accesses. For simplicity and clarity + * disable MBEC entirely if advanced vmexit information is not available, + * this way mbec=3D1 in the kvm_intel module parameters implies availabil= ity + * to nested guests as well. + */ + if (!(vmx_cap->ept & VMX_EPT_ADVANCED_VMEXIT_INFO_BIT)) + _cpu_based_2nd_exec_control &=3D ~SECONDARY_EXEC_MODE_BASED_EPT_EXEC; + if (!cpu_has_sgx()) _cpu_based_2nd_exec_control &=3D ~SECONDARY_EXEC_ENCLS_EXITING; =20 --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C89223FADF6 for ; Tue, 28 Apr 2026 11:10:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374609; cv=none; b=BF2Lfn+yJIFfrjLf37d3Mi+xWTS552icdfBzClEFw5F1q3vZAricW8xFYCyXbbvyuMpFM2sCXiaT/6pFIh30oQ/md58vqoNgARhTL98UZa6TGo6ivbRqIYjTmKlmnWaDziYEulfSr+7/QCKnBZEcefbByEDjFr+hUbJ0/7O0+dA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374609; c=relaxed/simple; bh=YZ+NH/Su/iMvJfRF2cllUnPDZcsTovpVe+jJ+QNs4IM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=UX8KEexio3Z/a3CcpSw123gOaiI6XHfHBwOlqP+Ep3LM261Lhd3izrj8RVHsNjljXt6FowY6OJbNOr1m28G6zY1KXwWpNPmKaOPuGcHyADJ89LxCsXGStgFFtsObx809Ehhf1brwJ1TcGJdQxDX0ukJTcChEAZnWvcJ7eTMLvF0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=PXNAO+mW; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="PXNAO+mW" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374604; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mFEpUt5c3ApyGuHZCnAkReNh/DxsHyLuo8RY9+G+nT4=; b=PXNAO+mWX6SRLXvri+2xwmktK9lzfqgzOXO6G3YzwZtI+HBRJepRM6RquOqRUuvTeYIBYJ f1hjsBeGDbRMHgHyauqwogWQEzfNYQlpuXH3gwzpz3DG509wokhmOcE2rlZHb1KBhXw2FU FNuf9YtIHfbGFpCV5qhLPcDZAdTDV8U= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-504-x4wgZ4pmOrG63rYPRT-8pQ-1; Tue, 28 Apr 2026 07:10:03 -0400 X-MC-Unique: x4wgZ4pmOrG63rYPRT-8pQ-1 X-Mimecast-MFC-AGG-ID: x4wgZ4pmOrG63rYPRT-8pQ_1777374602 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D55ED19560A7; Tue, 28 Apr 2026 11:10:01 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 4EC2D180045E; Tue, 28 Apr 2026 11:10:01 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 18/28] KVM: x86/mmu: add support for MBEC to EPT page table walks Date: Tue, 28 Apr 2026 07:09:36 -0400 Message-ID: <20260428110946.11466-19-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Content-Type: text/plain; charset="utf-8" Extend the page walker to support moving bit 10 of the PTEs into ACC_USER_EXEC_MASK and bit 6 of the exit qualification of EPT violation VM exits. Note that while mmu_has_mbec()/cr4_smep affect the interpretation of ACC_USER_EXEC_MASK and add bit 10 as a "present bit" in guest EPT page table entries, they do not affect how KVM operates on SPTEs. That's because the MMU uses explicit ACC_USER_EXEC_MASK/shadow_xu_mask even for the non-nested EPT; the only difference is that ACC_USER_EXEC_MASK and ACC_EXEC_MASK will always be set in tandem outside the nested scenario. Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 13 +++++++++++-- arch/x86/kvm/mmu/paging_tmpl.h | 27 +++++++++++++++++++++------ arch/x86/kvm/mmu/spte.h | 2 ++ arch/x86/kvm/vmx/nested.c | 9 +++++++++ 4 files changed, 43 insertions(+), 8 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 7ec29979a209..6de28ac0b454 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -5564,7 +5564,6 @@ static void update_permission_bitmask(struct kvm_mmu = *mmu, bool ept) { unsigned byte; =20 - const u16 x =3D ACC_BITS_MASK(ACC_EXEC_MASK); const u16 w =3D ACC_BITS_MASK(ACC_WRITE_MASK); const u16 r =3D ACC_BITS_MASK(ACC_READ_MASK); =20 @@ -5605,8 +5604,18 @@ static void update_permission_bitmask(struct kvm_mmu= *mmu, bool ept) u16 smapf =3D 0; =20 if (ept) { - ff =3D (pfec & PFERR_FETCH_MASK) ? (u16)~x : 0; + const u16 xs =3D ACC_BITS_MASK(ACC_EXEC_MASK); + const u16 xu =3D ACC_BITS_MASK(ACC_USER_EXEC_MASK); + + if (pfec & PFERR_FETCH_MASK) { + /* Ignore XU unless MBEC is enabled. */ + if (cr4_smep) + ff =3D pfec & PFERR_USER_MASK ? (u16)~xu : (u16)~xs; + else + ff =3D (u16)~xs; + } } else { + const u16 x =3D ACC_BITS_MASK(ACC_EXEC_MASK); const u16 u =3D ACC_BITS_MASK(ACC_USER_MASK); =20 /* Faults from kernel mode accesses to user pages */ diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index f5f9e745f21d..7d7c617885fa 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -124,12 +124,17 @@ static inline void FNAME(protect_clean_gpte)(struct k= vm_mmu *mmu, unsigned *acce *access &=3D mask; } =20 -static inline int FNAME(is_present_gpte)(unsigned long pte) +static inline int FNAME(is_present_gpte)(struct kvm_mmu *mmu, + unsigned long pte) { #if PTTYPE !=3D PTTYPE_EPT return pte & PT_PRESENT_MASK; #else - return pte & 7; + /* + * For EPT, an entry is present if any of bits 2:0 are set. + * With mode-based execute control, bit 10 also indicates presence. + */ + return pte & (7 | (mmu_has_mbec(mmu) ? VMX_EPT_USER_EXECUTABLE_MASK : 0)); #endif } =20 @@ -152,7 +157,7 @@ static bool FNAME(prefetch_invalid_gpte)(struct kvm_vcp= u *vcpu, struct kvm_mmu_page *sp, u64 *spte, u64 gpte) { - if (!FNAME(is_present_gpte)(gpte)) + if (!FNAME(is_present_gpte)(vcpu->arch.mmu, gpte)) goto no_present; =20 /* Prefetch only accessed entries (unless A/D bits are disabled). */ @@ -173,10 +178,17 @@ static bool FNAME(prefetch_invalid_gpte)(struct kvm_v= cpu *vcpu, static inline unsigned FNAME(gpte_access)(u64 gpte) { unsigned access; + /* + * Set bits in ACC_*_MASK even if they might not be used in the + * actual checks. For example, if EFER.NX is clear permission_fault() + * will ignore ACC_EXEC_MASK, and if MBEC is disabled it will + * ignore ACC_USER_EXEC_MASK. + */ #if PTTYPE =3D=3D PTTYPE_EPT access =3D ((gpte & VMX_EPT_WRITABLE_MASK) ? ACC_WRITE_MASK : 0) | ((gpte & VMX_EPT_EXECUTABLE_MASK) ? ACC_EXEC_MASK : 0) | - ((gpte & VMX_EPT_READABLE_MASK) ? ACC_READ_MASK : 0); + ((gpte & VMX_EPT_READABLE_MASK) ? ACC_READ_MASK : 0) | + ((gpte & VMX_EPT_USER_EXECUTABLE_MASK) ? ACC_USER_EXEC_MASK : 0); #else /* * P is set here, so the page is always readable and W/U/!NX represent @@ -331,7 +343,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker= *walker, if (walker->level =3D=3D PT32E_ROOT_LEVEL) { pte =3D mmu->get_pdptr(vcpu, (addr >> 30) & 3); trace_kvm_mmu_paging_element(pte, walker->level); - if (!FNAME(is_present_gpte)(pte)) + if (!FNAME(is_present_gpte)(mmu, pte)) goto error; --walker->level; } @@ -414,7 +426,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker= *walker, */ pte_access =3D pt_access & (pte ^ walk_nx_mask); =20 - if (unlikely(!FNAME(is_present_gpte)(pte))) + if (unlikely(!FNAME(is_present_gpte)(mmu, pte))) goto error; =20 if (unlikely(FNAME(is_rsvd_bits_set)(mmu, pte, walker->level))) { @@ -521,6 +533,9 @@ static int FNAME(walk_addr_generic)(struct guest_walker= *walker, * ACC_*_MASK flags! */ walker->fault.exit_qualification |=3D EPT_VIOLATION_RWX_TO_PROT(pte_acce= ss); + if (mmu_has_mbec(mmu)) + walker->fault.exit_qualification |=3D + EPT_VIOLATION_USER_EXEC_TO_PROT(pte_access); } #endif walker->fault.address =3D addr; diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index 22923ddd0617..13eea94dd212 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -395,6 +395,8 @@ static inline bool __is_rsvd_bits_set(struct rsvd_bits_= validate *rsvd_check, static inline bool __is_bad_mt_xwr(struct rsvd_bits_validate *rsvd_check, u64 pte) { + if (pte & VMX_EPT_USER_EXECUTABLE_MASK) + pte |=3D VMX_EPT_EXECUTABLE_MASK; return rsvd_check->bad_mt_xwr & BIT_ULL(pte & 0x3f); } =20 diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 46b65475765d..84f5c25a1f12 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -7452,6 +7452,15 @@ static gpa_t vmx_translate_nested_gpa(struct kvm_vcp= u *vcpu, gpa_t gpa, struct kvm_mmu *mmu =3D vcpu->arch.mmu; =20 BUG_ON(!mmu_is_nested(vcpu)); + + /* + * MBEC differentiates based on the effective U/S bit of + * the guest page tables; not the processor CPL. + */ + access &=3D ~PFERR_USER_MASK; + if ((pte_access & ACC_USER_MASK) && (access & PFERR_GUEST_FINAL_MASK)) + access |=3D PFERR_USER_MASK; + return mmu->gva_to_gpa(vcpu, mmu, gpa, access, exception); } =20 --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 12ED03FE34C for ; Tue, 28 Apr 2026 11:10:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374614; cv=none; b=ifOq3rkXNxyY1Wmrex6xhKBe3AvxAHXso5MBN1ALR1QESryX/rhmBh3SEKbvcT9rYWaqAio80Chgh0UDaPHqv4j2q4El53bHSiv7zEHFhFyH89B8fi3r8Jerk8KV3FiiTlACgp7h/6iaNLBSvQGdeiVdAGtIvWlET743QJpT+Wc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374614; c=relaxed/simple; bh=MFRv8YL91NfqrF0yv+deqRutOH+ME54n++4JMbFv+6Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=bgp5Y5dvDuPuJ2R8qLnjFzJV/o9JSsJFgtfebTB0AQcjWPOwlVfMGcazSP2RNE2dX9ixPJoYYfcBgU8+LilLFCjd+7coqUyLAfgNsbDEWGYqOEVFQHCCnbO1/BhDRvDN4mOEwd56frnjx6iEuqfvcN3gfnKX7swsDP13Pbva+mI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=DgbMebE1; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="DgbMebE1" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374611; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qZX0QX2so685aU57YTa+NP9hVn+oNCIHSJx5FvyUvSI=; b=DgbMebE1Zj/a5lhAtk3yyTeu7E3msESHFBdn7MJBQbZ/q/YmjOxK97ZjNAHxgAgXZxS1ww JT3cSW064VM4VtqW5E1XVjy6vCKK1i3lGWdFK6vYBwZSNZEBwZH8/oyGdfOAyMdgTm+A1K Dim976FEF99gFzVsMOZBwG7nwbDIH5Q= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-279-V0c8Md7kNUyg0TnNouQHzA-1; Tue, 28 Apr 2026 07:10:08 -0400 X-MC-Unique: V0c8Md7kNUyg0TnNouQHzA-1 X-Mimecast-MFC-AGG-ID: V0c8Md7kNUyg0TnNouQHzA_1777374607 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A914A1800370; Tue, 28 Apr 2026 11:10:07 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 07C72180047F; Tue, 28 Apr 2026 11:10:01 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 19/28] KVM: nVMX: advertise MBEC to nested guests Date: Tue, 28 Apr 2026 07:09:37 -0400 Message-ID: <20260428110946.11466-20-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Content-Type: text/plain; charset="utf-8" From: Jon Kohler Advertise SECONDARY_EXEC_MODE_BASED_EPT_EXEC (MBEC) to userspace, which allows userspace to expose and advertise the feature to the guest. When MBEC is enabled by the guest, it is passed to the MMU via cr4_smep, and to the processor by the merging of vmcs12->secondary_vm_exec_control into the VMCS02's secondary VM execution controls. Signed-off-by: Jon Kohler Message-ID: <20251223054806.1611168-9-jon@nutanix.com> Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu.h | 2 +- arch/x86/kvm/mmu/mmu.c | 7 ++++--- arch/x86/kvm/vmx/nested.c | 11 +++++++++++ 3 files changed, 16 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index d15f908d048f..9031f7ef8e4e 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -100,7 +100,7 @@ void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, uns= igned long cr0, unsigned long cr4, u64 efer, gpa_t nested_cr3); void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu, bool execonly, int huge_page_level, bool accessed_dirty, - gpa_t new_eptp); + bool mbec, gpa_t new_eptp); bool kvm_can_do_async_pf(struct kvm_vcpu *vcpu); int kvm_handle_page_fault(struct kvm_vcpu *vcpu, u64 error_code, u64 fault_address, char *insn, int insn_len); diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 6de28ac0b454..0d364a984aea 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -5953,7 +5953,7 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_init_shadow_npt_mm= u); =20 static union kvm_cpu_role kvm_calc_shadow_ept_root_page_role(struct kvm_vcpu *vcpu, bool accessed_di= rty, - bool execonly, u8 level) + bool execonly, u8 level, bool mbec) { union kvm_cpu_role role =3D {0}; =20 @@ -5963,6 +5963,7 @@ kvm_calc_shadow_ept_root_page_role(struct kvm_vcpu *v= cpu, bool accessed_dirty, */ WARN_ON_ONCE(is_smm(vcpu)); role.base.level =3D level; + role.base.cr4_smep =3D mbec; role.base.has_4_byte_gpte =3D false; role.base.direct =3D false; role.base.ad_disabled =3D !accessed_dirty; @@ -5978,13 +5979,13 @@ kvm_calc_shadow_ept_root_page_role(struct kvm_vcpu = *vcpu, bool accessed_dirty, =20 void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu, bool execonly, int huge_page_level, bool accessed_dirty, - gpa_t new_eptp) + bool mbec, gpa_t new_eptp) { struct kvm_mmu *context =3D &vcpu->arch.guest_mmu; u8 level =3D vmx_eptp_page_walk_level(new_eptp); union kvm_cpu_role new_mode =3D kvm_calc_shadow_ept_root_page_role(vcpu, accessed_dirty, - execonly, level); + execonly, level, mbec); =20 if (new_mode.as_u64 !=3D context->cpu_role.as_u64) { /* EPT, and thus nested EPT, does not consume CR0, CR4, nor EFER. */ diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 84f5c25a1f12..bc1046f32ebc 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -469,6 +469,13 @@ static void nested_ept_inject_page_fault(struct kvm_vc= pu *vcpu, vmcs12->guest_physical_address =3D fault->address; } =20 +static inline bool nested_ept_mbec_enabled(struct kvm_vcpu *vcpu) +{ + struct vmcs12 *vmcs12 =3D get_vmcs12(vcpu); + + return nested_cpu_has2(vmcs12, SECONDARY_EXEC_MODE_BASED_EPT_EXEC); +} + static void nested_ept_new_eptp(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx =3D to_vmx(vcpu); @@ -477,6 +484,7 @@ static void nested_ept_new_eptp(struct kvm_vcpu *vcpu) =20 kvm_init_shadow_ept_mmu(vcpu, execonly, ept_lpage_level, nested_ept_ad_enabled(vcpu), + nested_ept_mbec_enabled(vcpu), nested_ept_get_eptp(vcpu)); } =20 @@ -7257,6 +7265,9 @@ static void nested_vmx_setup_secondary_ctls(u32 ept_c= aps, msrs->ept_caps |=3D VMX_EPT_AD_BIT; } =20 + if (enable_mbec) + msrs->secondary_ctls_high |=3D + SECONDARY_EXEC_MODE_BASED_EPT_EXEC; /* * Advertise EPTP switching irrespective of hardware support, * KVM emulates it in software so long as VMFUNC is supported. --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1BB643FF89B for ; Tue, 28 Apr 2026 11:10:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374617; cv=none; b=rYQwAX2WN+W0b9mGAtu7kIrwrHti6PKYvKKYeRT6pM9JM0E8ceEEfG4rg58ptmt5UGc5Et5gG5O4lNf2HOSMTqfS6z4+dq0P8UGvcfzqKfydi0wCq0SAXDT85n3ok9xHhzbkTNmavMv4fd3GxnJXxPXEBQ90ggstFKvHgZ/56uk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374617; c=relaxed/simple; bh=i3jQrAyLxnhA/t/BHsxb3+sqwKCc2Ml3j5/foser9aU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=UwanyU62ziKfQO3ulsoi0zr4X5r+yPD/cVIxij/G5EYoraSc2Ic5tEe3nJFPUQiH7k54V06XX71NJ+iuBDxHxZ94P6G5gvRpqTFQbHocJT04WDzdJ2Udf2nXiR5dzJETh+Gu5HirxGEP7UEnCW3sNWKAB4sriadrtmVFggYo2/E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=AWKBeI+A; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AWKBeI+A" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374613; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+YJGyvsFomEhH/bKnG38bbVfFx34TKz1rxdBHf//6Lw=; b=AWKBeI+A6dIGuKmVMaCqadclJK5Yx5EaoaRiG2kdTLfMWRKXkcmEOD+UxiHWcU+Memk+AK QierLemLkAVzi66Cge2SM6vG9taCVPspJWKS980iHA6eQeP5pVbKVvQL2gJBvsakBLo4Hv Wbe8GSDIi+nnoWoZV6AKWflw0V/7POo= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-318-BepGP0SEOv6Lr627bbFYMw-1; Tue, 28 Apr 2026 07:10:10 -0400 X-MC-Unique: BepGP0SEOv6Lr627bbFYMw-1 X-Mimecast-MFC-AGG-ID: BepGP0SEOv6Lr627bbFYMw_1777374609 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id EDDEF1800576; Tue, 28 Apr 2026 11:10:08 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 6207D19560B1; Tue, 28 Apr 2026 11:10:08 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 20/28] KVM: nVMX: allow MBEC with EVMCS Date: Tue, 28 Apr 2026 07:09:38 -0400 Message-ID: <20260428110946.11466-21-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" From: Jon Kohler Extend EVMCS1_SUPPORTED_2NDEXEC to allow MBEC and EVMCS to coexist. Presenting both EVMCS and MBEC simultaneously causes KVM to filter out MBEC and not present it as a supported control to the guest, preventing performance gains from MBEC when Windows HVCI is enabled. The guest may choose not to use MBEC (e.g., if the admin does not enable Windows HVCI / Memory Integrity), but if they use traditional nested virt (Hyper-V, WSL2, etc.), having EVMCS exposed is important for improving nested guest performance. IOW allowing MBEC and EVMCS to coexist provides maximum optionality to Windows users without overcomplicating VM administration. Signed-off-by: Jon Kohler Message-ID: <20251223054806.1611168-8-jon@nutanix.com> Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/kvm/vmx/hyperv_evmcs.h | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/kvm/vmx/hyperv_evmcs.h b/arch/x86/kvm/vmx/hyperv_evmc= s.h index fc7c4e7bd1bf..bc08fe40590e 100644 --- a/arch/x86/kvm/vmx/hyperv_evmcs.h +++ b/arch/x86/kvm/vmx/hyperv_evmcs.h @@ -87,6 +87,7 @@ SECONDARY_EXEC_PT_CONCEAL_VMX | \ SECONDARY_EXEC_BUS_LOCK_DETECTION | \ SECONDARY_EXEC_NOTIFY_VM_EXITING | \ + SECONDARY_EXEC_MODE_BASED_EPT_EXEC | \ SECONDARY_EXEC_ENCLS_EXITING) =20 #define EVMCS1_SUPPORTED_3RDEXEC (0ULL) --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3BDAA3FE377 for ; Tue, 28 Apr 2026 11:10:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374619; cv=none; b=rQ/MTNaRWLcUqsWDMZ1Ds9ynjkTw2TT1bA6fzo/mfeehVuJLVwmnzJV8jzE+JpjMHEEPgsYsNd8wsoFD5949+T5TqgCFWrInlVWdQvDERWc/uUlvG0qHM5kwWRS1OA0kMCE5qo+vpvnouBUjrKd9s6B9hd98PNkfCfalqs77lSo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374619; c=relaxed/simple; bh=R6fnUtGGWuDbmS/uH5fZV+1MDGX8VjCrLTMw73Y7/j8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=mmZP3XEQc12mhyrybKZntVPRRYz9Fp9Uus2Sw/vUZ+ofyNEWpQol+lLo2hbp4aLNLO0S/P6GZLvH4eDq9Y519wmyJZ4AURc+fC5cPqC2Ukb6l3xD/tMn2P722k7J+QNYNmJ+XUEu0glAHYHeY5w73fOsRpji6DnteSGbTiwFDj8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=CCS7vfUj; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="CCS7vfUj" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374615; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LsFanCnOqbUMXznO7d4RZrJ9HoECVpa0y6LWuiw5g8g=; b=CCS7vfUjcQW88vhNXwU7q9XoIgBCFd5hcX02WkxIoLYrRuYrcJLnO2AfjfOxkNppJXaCS0 /kZgsW/JHI+JtxgBMwInbESCf29RwTY3xr0yTgGv8xSVmDGib6K1imCzzVY3z5Nv1vf5ax 15LDdHVcfhMYKfuBn7VBsPBc6+qtiMY= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-259-3m6ZHa50M6O-jDTbQ8eb3Q-1; Tue, 28 Apr 2026 07:10:10 -0400 X-MC-Unique: 3m6ZHa50M6O-jDTbQ8eb3Q-1 X-Mimecast-MFC-AGG-ID: 3m6ZHa50M6O-jDTbQ8eb3Q_1777374609 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A519819560B4; Tue, 28 Apr 2026 11:10:09 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1D8A819560AB; Tue, 28 Apr 2026 11:10:09 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 21/28] KVM: x86/mmu: propagate access mask from root pages down Date: Tue, 28 Apr 2026 07:09:39 -0400 Message-ID: <20260428110946.11466-22-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" Until now, all SPTEs have had all kinds of access allowed; however, for GMET to be enabled all the pages have to have ACC_USER_MASK disabled. By marking them as supervisor pages, the processor allows execution from either user or supervisor mode (unlike for normal paging, NPT ignores the U bit for reads and writes). That will mean that the root page's role has ACC_USER_MASK cleared and that has to be propagated down through the kvm_mmu_page tree. Do that, and pass the required access to the kvm_mmu_spte_requested tracepoint since it's not ACC_ALL anymore. Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 9 +++++---- arch/x86/kvm/mmu/mmutrace.h | 10 ++++++---- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/mmu/tdp_mmu.c | 6 +++--- 4 files changed, 15 insertions(+), 12 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 0d364a984aea..8deeb89d88c4 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3446,12 +3446,13 @@ static int direct_map(struct kvm_vcpu *vcpu, struct= kvm_page_fault *fault) { struct kvm_shadow_walk_iterator it; struct kvm_mmu_page *sp; - int ret; + int ret, access; gfn_t base_gfn =3D fault->gfn; =20 kvm_mmu_hugepage_adjust(vcpu, fault); =20 - trace_kvm_mmu_spte_requested(fault); + access =3D vcpu->arch.mmu->root_role.access; + trace_kvm_mmu_spte_requested(fault, access); for_each_shadow_entry(vcpu, fault->addr, it) { /* * We cannot overwrite existing page tables with an NX @@ -3464,7 +3465,7 @@ static int direct_map(struct kvm_vcpu *vcpu, struct k= vm_page_fault *fault) if (it.level =3D=3D fault->goal_level) break; =20 - sp =3D kvm_mmu_get_child_sp(vcpu, it.sptep, base_gfn, true, ACC_ALL); + sp =3D kvm_mmu_get_child_sp(vcpu, it.sptep, base_gfn, true, access); if (sp =3D=3D ERR_PTR(-EEXIST)) continue; =20 @@ -3477,7 +3478,7 @@ static int direct_map(struct kvm_vcpu *vcpu, struct k= vm_page_fault *fault) if (WARN_ON_ONCE(it.level !=3D fault->goal_level)) return -EFAULT; =20 - ret =3D mmu_set_spte(vcpu, fault->slot, it.sptep, ACC_ALL, + ret =3D mmu_set_spte(vcpu, fault->slot, it.sptep, access, base_gfn, fault->pfn, fault); if (ret =3D=3D RET_PF_SPURIOUS) return ret; diff --git a/arch/x86/kvm/mmu/mmutrace.h b/arch/x86/kvm/mmu/mmutrace.h index 3429c1413f42..fa01719baf8d 100644 --- a/arch/x86/kvm/mmu/mmutrace.h +++ b/arch/x86/kvm/mmu/mmutrace.h @@ -373,23 +373,25 @@ TRACE_EVENT( =20 TRACE_EVENT( kvm_mmu_spte_requested, - TP_PROTO(struct kvm_page_fault *fault), - TP_ARGS(fault), + TP_PROTO(struct kvm_page_fault *fault, u8 access), + TP_ARGS(fault, access), =20 TP_STRUCT__entry( __field(u64, gfn) __field(u64, pfn) __field(u8, level) + __field(u8, access) ), =20 TP_fast_assign( __entry->gfn =3D fault->gfn; __entry->pfn =3D fault->pfn | (fault->gfn & (KVM_PAGES_PER_HPAGE(fault->= goal_level) - 1)); __entry->level =3D fault->goal_level; + __entry->access =3D access; ), =20 - TP_printk("gfn %llx pfn %llx level %d", - __entry->gfn, __entry->pfn, __entry->level + TP_printk("gfn %llx pfn %llx level %d access %x", + __entry->gfn, __entry->pfn, __entry->level, __entry->access ) ); =20 diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 7d7c617885fa..9ad2b05a0f44 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -734,7 +734,7 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, struct k= vm_page_fault *fault, */ kvm_mmu_hugepage_adjust(vcpu, fault); =20 - trace_kvm_mmu_spte_requested(fault); + trace_kvm_mmu_spte_requested(fault, gw->pte_access); =20 for (; shadow_walk_okay(&it); shadow_walk_next(&it)) { /* diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 7b1102d26f9c..5a2f8ce9a32b 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -1185,9 +1185,9 @@ static int tdp_mmu_map_handle_target_level(struct kvm= _vcpu *vcpu, } =20 if (unlikely(!fault->slot)) - new_spte =3D make_mmio_spte(vcpu, iter->gfn, ACC_ALL); + new_spte =3D make_mmio_spte(vcpu, iter->gfn, sp->role.access); else - wrprot =3D make_spte(vcpu, sp, fault->slot, ACC_ALL, iter->gfn, + wrprot =3D make_spte(vcpu, sp, fault->slot, sp->role.access, iter->gfn, fault->pfn, iter->old_spte, fault->prefetch, false, fault->map_writable, &new_spte); =20 @@ -1272,7 +1272,7 @@ int kvm_tdp_mmu_map(struct kvm_vcpu *vcpu, struct kvm= _page_fault *fault) =20 kvm_mmu_hugepage_adjust(vcpu, fault); =20 - trace_kvm_mmu_spte_requested(fault); + trace_kvm_mmu_spte_requested(fault, root->role.access); =20 rcu_read_lock(); =20 --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 583D737C0F3 for ; Tue, 28 Apr 2026 11:10:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374618; cv=none; b=g2HdvFyPQUVNv3Ii9tLosVC2SeUE9VNzjnDOmh9m1wmffuBH11pmu655Ec6Gz4Je3ocD1p3F20d/d8VmZBUawfem8x0G9w6X4fAJEZt5IFEOhgysTmFexPNTPynmgfptJi3neaXYIpxsAmwei2JFMXJPdt1UEsKlR+yYs2YvwYI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374618; c=relaxed/simple; bh=Bbnd3BUnQXTYnytKJzY6LoFx9CgYX19VSh5xco7y51Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ip6QBuJAtlqI2iTJcP0AbageufzuOcokO3uUvJpJv28Snd3TCt+KM2bZSHnN26fBLu1ebSQcjHWPyqTJAlVMfk+gPeYn63aEUGL4HTWzyTE5bBQdKViTZI3RizAq8izXZ5+K2JIFfjFJC9qQmays1O+se+V90uE5ICOfPIDcWDU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=L3EiJJOS; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="L3EiJJOS" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374615; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1OvKZw2ZCZ+pTGKnaYxanCgyg8mnQ5aZNhL8dTXgrDw=; b=L3EiJJOS5I212Gh2zb8LymCf+44O0LKFagxOKQGgoZYoLcTQVZa+ZR2fsGbXgRQ4Xf4vXy lE50V3/Yt44yRo8GnpuGIok1gE/0yzEewoweJdNXlACOZC1Dkhi+pUPZE02Ql/9eZ9Jped 4dlF+Hs3rgUBMEBvntByvq8SWN1UScg= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-515-C9F2Xj4yM_WuP4ozzkSdCA-1; Tue, 28 Apr 2026 07:10:11 -0400 X-MC-Unique: C9F2Xj4yM_WuP4ozzkSdCA-1 X-Mimecast-MFC-AGG-ID: C9F2Xj4yM_WuP4ozzkSdCA_1777374610 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5FF2D19560B1; Tue, 28 Apr 2026 11:10:10 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id CBB1019560AB; Tue, 28 Apr 2026 11:10:09 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 22/28] KVM: x86/mmu: introduce cpu_role bit for availability of PFEC.I/D Date: Tue, 28 Apr 2026 07:09:40 -0400 Message-ID: <20260428110946.11466-23-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" While GMET looks a lot like SMEP, it has several annoying differences. The main one is that the availability of the I/D bit in the page fault error code still depends on the host CR4.SMEP and EFER.NXE bits. If the base.cr4_smep bit of the cpu_role is (ab)used to enable GMET, there needs to be another place where the host CR4.SMEP is read from; just merge it with EFER.NXE into a new cpu_role bit that tells paging_tmpl.h whether to set the I/D bit at all. Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/include/asm/kvm_host.h | 7 +++++++ arch/x86/kvm/mmu/mmu.c | 8 ++++++++ arch/x86/kvm/mmu/paging_tmpl.h | 2 +- 3 files changed, 16 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 23a7ac8d7fbe..7dde4ca87752 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -414,6 +414,13 @@ union kvm_mmu_extended_role { unsigned int cr4_smap:1; unsigned int cr4_la57:1; unsigned int efer_lma:1; + + /* + * True if either CR4.SMEP or EFER.NXE are set. For AMD NPT + * this is the "real" host CR4.SMEP whereas cr4_smep is + * actually GMET. + */ + unsigned int has_pferr_fetch:1; }; }; =20 diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 8deeb89d88c4..07b64df8519d 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -234,6 +234,11 @@ BUILD_MMU_ROLE_ACCESSOR(ext, cr4, la57); BUILD_MMU_ROLE_ACCESSOR(base, efer, nx); BUILD_MMU_ROLE_ACCESSOR(ext, efer, lma); =20 +static inline bool has_pferr_fetch(struct kvm_mmu *mmu) +{ + return mmu->cpu_role.ext.has_pferr_fetch; +} + static inline bool is_cr0_pg(struct kvm_mmu *mmu) { return mmu->cpu_role.base.level > 0; @@ -5787,6 +5792,8 @@ static union kvm_cpu_role kvm_calc_cpu_role(struct kv= m_vcpu *vcpu, role.ext.cr4_pke =3D ____is_efer_lma(regs) && ____is_cr4_pke(regs); role.ext.cr4_la57 =3D ____is_efer_lma(regs) && ____is_cr4_la57(regs); role.ext.efer_lma =3D ____is_efer_lma(regs); + + role.ext.has_pferr_fetch =3D role.base.efer_nx | role.base.cr4_smep; return role; } =20 @@ -5940,6 +5947,7 @@ void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, u= nsigned long cr0, =20 /* NPT requires CR0.PG=3D1. */ WARN_ON_ONCE(cpu_role.base.direct || !cpu_role.base.guest_mode); + cpu_role.base.cr4_smep =3D false; =20 root_role =3D cpu_role.base; root_role.level =3D kvm_mmu_get_tdp_level(vcpu); diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 9ad2b05a0f44..ad0e5f43b8ad 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -489,7 +489,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker= *walker, =20 error: errcode |=3D write_fault | user_fault; - if (fetch_fault && (is_efer_nx(mmu) || is_cr4_smep(mmu))) + if (fetch_fault && has_pferr_fetch(mmu)) errcode |=3D PFERR_FETCH_MASK; =20 walker->fault.vector =3D PF_VECTOR; --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 75C6C3FFACF for ; Tue, 28 Apr 2026 11:10:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374619; cv=none; b=oZNSgMGxWw1c66jtez45f5d0iHEpyjbfTRNaZrmvZ2mFt9ry8mpbCzw4cvhA0IYCrVMvNSPkkU4Ke7SdLZHSp+LKw3UoL9e59cj33OROebKxEISyONOT/fr79Qa8avP+Ig1gFSTXtGcdlCERBbzwOockqqdrDZg9hZOfYK1ghrs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374619; c=relaxed/simple; bh=hoTatyRYXoIBfLRHwPcxGG1+pSL5waPss30kc6PgoV4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=GIdzVyBMaVFgXIL3hqI5c+DdRxC7i84O+V4i7hsdQVpz0Y8Ds0TkRrprk+yd5x8TY0Cn2i+xiYc4gNWnu53Yx2NjxyCJYAtvhw5Vn2nz4zo2UONkCR3No9MN6H8p88vmiGcFWteMJ5Fy7n9TXgPxY+dG7KPec9RFL87fmDGJO5o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=E0kyFTP3; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="E0kyFTP3" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374616; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=d1r/8zG7HzE0J0RSifY4zJKxgWsdJGMYRBoWC/B6KdE=; b=E0kyFTP3TYThR59JT7c7c8uZ/jG7VVihKnmUEihVjBpNrtzlkwDsyvynFE0dsboXmUc791 B0keZyayIoLKVx7ku7onY30CFEot518P2xwHzcitvHE2c4Pbl0cKZ/I3wwwURMuaXCMR9R +ItngLSqU5Qwui+5MZwH2927KTii48I= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-517-mrbD650FMNaRnX_etdjdYw-1; Tue, 28 Apr 2026 07:10:13 -0400 X-MC-Unique: mrbD650FMNaRnX_etdjdYw-1 X-Mimecast-MFC-AGG-ID: mrbD650FMNaRnX_etdjdYw_1777374611 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6B82F195608F; Tue, 28 Apr 2026 11:10:11 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 856E219560AB; Tue, 28 Apr 2026 11:10:10 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com, "Borislav Petkov (AMD)" Subject: [PATCH 23/28] KVM: SVM: add GMET bit definitions Date: Tue, 28 Apr 2026 07:09:41 -0400 Message-ID: <20260428110946.11466-24-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" GMET (Guest Mode Execute Trap) is an AMD virtualization feature, essentially the nested paging version of SMEP. Hyper-V uses it; add it in preparation for making it available to hypervisors running under KVM. Acked-by: Borislav Petkov (AMD) Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/svm.h | 1 + 2 files changed, 2 insertions(+) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpuf= eatures.h index 1d506e5d6f46..35a2a0f9ab32 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -379,6 +379,7 @@ #define X86_FEATURE_AVIC (15*32+13) /* "avic" Virtual Interrupt Controlle= r */ #define X86_FEATURE_V_VMSAVE_VMLOAD (15*32+15) /* "v_vmsave_vmload" Virtua= l VMSAVE VMLOAD */ #define X86_FEATURE_VGIF (15*32+16) /* "vgif" Virtual GIF */ +#define X86_FEATURE_GMET (15*32+17) /* Guest Mode Execution Trap */ #define X86_FEATURE_X2AVIC (15*32+18) /* "x2avic" Virtual x2apic */ #define X86_FEATURE_V_SPEC_CTRL (15*32+20) /* "v_spec_ctrl" Virtual SPEC_= CTRL */ #define X86_FEATURE_VNMI (15*32+25) /* "vnmi" Virtual NMI */ diff --git a/arch/x86/include/asm/svm.h b/arch/x86/include/asm/svm.h index bcfeb5e7c0ed..aa63431ba92c 100644 --- a/arch/x86/include/asm/svm.h +++ b/arch/x86/include/asm/svm.h @@ -243,6 +243,7 @@ struct __attribute__ ((__packed__)) vmcb_control_area { #define SVM_MISC_ENABLE_NP BIT(0) #define SVM_MISC_ENABLE_SEV BIT(1) #define SVM_MISC_ENABLE_SEV_ES BIT(2) +#define SVM_MISC_ENABLE_GMET BIT(3) =20 #define SVM_MISC2_ENABLE_V_LBR BIT_ULL(0) #define SVM_MISC2_ENABLE_V_VMLOAD_VMSAVE BIT_ULL(1) --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5D7AC3FFAC8 for ; Tue, 28 Apr 2026 11:10:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374619; cv=none; b=X2n18a5+ZpGq6mRGoSpmRRnCpEMT4HWkDS568ahx2v0fK3UokwtNP5NI6YmvooSq6uicliGzp2B4IIhHLyxIZK0F4H7GNNcUAOvlQpdQBfLp4/Etzxhq6HU9B+ssJTcwcUJPdrr7TkbIfoxjTB3MysJFWkkp3JJv59TDlxpfvGk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374619; c=relaxed/simple; bh=+9/MBdn3/HXIcU82TkwlbEMRTavJuREuP3RSFkLHOcg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Re1XozpnxXlVCTQC3/MOYVevMfY+VeQCx+lTdjme22mLP2SZUDxWkQ7rI6fUi4BUUkbU1hIsW0deBO2HqMHJHR+1+HEqxm9URbp2/Hc2mM7GBKPDsBYfIAb5hpjGh5O/HzObJJP9+Z+w7pcBPRJ39oV0eOaSyIlWFPutxufUrAg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=UBe+tENP; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="UBe+tENP" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374616; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2Xq5JwhvAgFZv5T+bpA+ShHdO+d4ms6wcusJmPMECzA=; b=UBe+tENPxSFLyUbQFXqRNR5+terobUSmKUDCzssOsUvYnKVgCk3of8yHGLDns7K1yis4n0 rO0u4ZYooXAn29/ay+5Ps9Ze6x4Ai7fJlEIJUioft22p2dDvCbX59FQqxeih47VzBl+Eih B3gJ5dzZAsy2WOymXWlPr3R7S5B5lp8= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-584-kIGohtFZMVWXqtP021DCWQ-1; Tue, 28 Apr 2026 07:10:13 -0400 X-MC-Unique: kIGohtFZMVWXqtP021DCWQ-1 X-Mimecast-MFC-AGG-ID: kIGohtFZMVWXqtP021DCWQ_1777374612 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2B06E19560BB; Tue, 28 Apr 2026 11:10:12 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 912FB19560B1; Tue, 28 Apr 2026 11:10:11 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 24/28] KVM: x86/mmu: set CR0.WP=1 for shadow NPT MMU Date: Tue, 28 Apr 2026 07:09:42 -0400 Message-ID: <20260428110946.11466-25-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" The host CR0 does not really reflect onto the NPT format because hCR0.PG=3D1 must be set and hCR0.WP is ignored. Carve that in stone by removing the cr0 argument from kvm_init_shadow_npt_mmu, and while at it pass in WP=3D1 as well. It does not matter for GMET disabled because PFERR_USER_MASK is always set, but a cleared W bit in the nested page tables cannot be overridden in supervisor mode when GMET is enabled, either. In fact, since CR0.WP=3D0 is the weird "extra accesses allowed" mode, it is actually easier think about it being always set. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu.h | 4 ++-- arch/x86/kvm/mmu/mmu.c | 6 +++--- arch/x86/kvm/svm/nested.c | 2 +- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index 9031f7ef8e4e..29aa3994a8c2 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -96,8 +96,8 @@ void kvm_mmu_set_me_spte_mask(u64 me_value, u64 me_mask); void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_mbec); =20 void kvm_init_mmu(struct kvm_vcpu *vcpu); -void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, unsigned long cr0, - unsigned long cr4, u64 efer, gpa_t nested_cr3); +void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, unsigned long cr4, + u64 efer, gpa_t nested_cr3); void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu, bool execonly, int huge_page_level, bool accessed_dirty, bool mbec, gpa_t new_eptp); diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 07b64df8519d..432f0d3ce632 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -5933,12 +5933,12 @@ static void kvm_init_shadow_mmu(struct kvm_vcpu *vc= pu, shadow_mmu_init_context(vcpu, context, cpu_role, root_role); } =20 -void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, unsigned long cr0, - unsigned long cr4, u64 efer, gpa_t nested_cr3) +void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, unsigned long cr4, + u64 efer, gpa_t nested_cr3) { struct kvm_mmu *context =3D &vcpu->arch.guest_mmu; struct kvm_mmu_role_regs regs =3D { - .cr0 =3D cr0, + .cr0 =3D X86_CR0_PG | X86_CR0_WP, .cr4 =3D cr4 & ~X86_CR4_PKE, .efer =3D efer, }; diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index df232153eb24..a1cffd274000 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -93,7 +93,7 @@ static void nested_svm_init_mmu_context(struct kvm_vcpu *= vcpu) * when called via KVM_SET_NESTED_STATE, that state may _not_ match curre= nt * vCPU state. CR0.WP is explicitly ignored, while CR0.PG is required. */ - kvm_init_shadow_npt_mmu(vcpu, X86_CR0_PG, svm->vmcb01.ptr->save.cr4, + kvm_init_shadow_npt_mmu(vcpu, svm->vmcb01.ptr->save.cr4, svm->vmcb01.ptr->save.efer, svm->nested.ctl.nested_cr3); vcpu->arch.mmu->get_guest_pgd =3D nested_svm_get_tdp_cr3; --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 60A4D40148B for ; Tue, 28 Apr 2026 11:10:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374620; cv=none; b=BVOn9Jv3LL2knj7EsOpM7Txrt2gnU2MfT2KwiKbr33bDLedoM7AW+ML6bJ5M61U3HQ1PhoPDrO0SStu4aW3sIdjS1ip3vbNoDZGj/99TRcBy0DdSIRH8p6sZMrezRw6IZ+aXJ7zrnS2L5ZHepcWUEja40x+EUA2MP47zlsCcYGI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374620; c=relaxed/simple; bh=3RDSRsqKIiXzCKqJTZM0Wa0waf9XJs5AZ9g+uxeM/GA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=HaERCe/x1nCLFT4902vBp1K0oEZcLjUQ6Q1Z2di7KmDyuNAeOZKupsgUv7i3rWiCYx3tNCAa1tzlfL739lkUg8wsUGc1jRx7PY9YOjJt2HsTNktRsHoss5+5w2HmKfJPlto7aC4nIC2BvA51p9DHvuQRZMhMP3XcGoKy6MhwAxQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=RsBwKIiH; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="RsBwKIiH" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374617; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KG0c/pWf1iTcb4uBQ0RsG3mVRz8LJHqFdc81qjcAOpU=; b=RsBwKIiH4LxSyC6T95fzxWp9AuKzS4DBbg047Iy6E9FVTTXd53FjEwDFWrD5qinoh6I2qP ktRktld6mNAMWPJofXawyyAtkg0/3h6z+b4d8XLFOj+c2XJM86KkiLdmxcE89ac0accO8f KqiFLFiDPon9XAAbLB4fKAo1CJJoaMs= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-584-30mnfmLsM-O9or55t9C0iw-1; Tue, 28 Apr 2026 07:10:14 -0400 X-MC-Unique: 30mnfmLsM-O9or55t9C0iw-1 X-Mimecast-MFC-AGG-ID: 30mnfmLsM-O9or55t9C0iw_1777374612 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D25AE19560AB; Tue, 28 Apr 2026 11:10:12 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 4ACA019560B1; Tue, 28 Apr 2026 11:10:12 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 25/28] KVM: x86/mmu: add support for GMET to NPT page table walks Date: Tue, 28 Apr 2026 07:09:43 -0400 Message-ID: <20260428110946.11466-26-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" GMET allows page table entries to be created with U=3D0 in NPT. However, when GMET=3D1 U=3D0 only affects execution, not reads or writes. Ignore user faults on non-fetch accesses for NPT GMET. Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/mmu.h | 2 +- arch/x86/kvm/mmu/mmu.c | 18 ++++++++++++------ arch/x86/kvm/svm/nested.c | 10 +++++++--- 4 files changed, 22 insertions(+), 10 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 7dde4ca87752..1da3d5c59e15 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -370,6 +370,8 @@ union kvm_mmu_page_role { * cr4_smep is also set for EPT MBEC. Because it affects * which pages are considered non-present (bit 10 additionally * must be zero if MBEC is on) it has to be in the base role. + * It also has to be in the base role for AMD GMET because + * kernel-executable pages need to have U=3D0 with GMET enabled. */ unsigned cr4_smep:1; =20 diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index 29aa3994a8c2..9a8040e09d92 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -97,7 +97,7 @@ void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_mbe= c); =20 void kvm_init_mmu(struct kvm_vcpu *vcpu); void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, unsigned long cr4, - u64 efer, gpa_t nested_cr3); + u64 efer, gpa_t nested_cr3, u64 misc_ctl); void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu, bool execonly, int huge_page_level, bool accessed_dirty, bool mbec, gpa_t new_eptp); diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 432f0d3ce632..6cf0e5f28ba5 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -55,6 +55,7 @@ #include #include #include +#include #include =20 #include "trace.h" @@ -5566,7 +5567,7 @@ reset_ept_shadow_zero_bits_mask(struct kvm_mmu *conte= xt, bool execonly) (14 & (access) ? 1 << 14 : 0) | \ (15 & (access) ? 1 << 15 : 0)) =20 -static void update_permission_bitmask(struct kvm_mmu *mmu, bool ept) +static void update_permission_bitmask(struct kvm_mmu *mmu, bool tdp, bool = ept) { unsigned byte; =20 @@ -5627,7 +5628,12 @@ static void update_permission_bitmask(struct kvm_mmu= *mmu, bool ept) /* Faults from kernel mode accesses to user pages */ u16 kf =3D (pfec & PFERR_USER_MASK) ? 0 : u; =20 - uf =3D (pfec & PFERR_USER_MASK) ? (u16)~u : 0; + /* + * For NPT GMET, U=3D0 does not affect reads and writes. Fetches + * are handled below via cr4_smep. + */ + if (!(tdp && cr4_smep)) + uf =3D (pfec & PFERR_USER_MASK) ? (u16)~u : 0; =20 if (efer_nx) ff =3D (pfec & PFERR_FETCH_MASK) ? (u16)~x : 0; @@ -5738,7 +5744,7 @@ static void reset_guest_paging_metadata(struct kvm_vc= pu *vcpu, return; =20 reset_guest_rsvds_bits_mask(vcpu, mmu); - update_permission_bitmask(mmu, false); + update_permission_bitmask(mmu, mmu =3D=3D &vcpu->arch.guest_mmu, false); update_pkru_bitmask(mmu); } =20 @@ -5934,7 +5940,7 @@ static void kvm_init_shadow_mmu(struct kvm_vcpu *vcpu, } =20 void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, unsigned long cr4, - u64 efer, gpa_t nested_cr3) + u64 efer, gpa_t nested_cr3, u64 misc_ctl) { struct kvm_mmu *context =3D &vcpu->arch.guest_mmu; struct kvm_mmu_role_regs regs =3D { @@ -5947,7 +5953,7 @@ void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, u= nsigned long cr4, =20 /* NPT requires CR0.PG=3D1. */ WARN_ON_ONCE(cpu_role.base.direct || !cpu_role.base.guest_mode); - cpu_role.base.cr4_smep =3D false; + cpu_role.base.cr4_smep =3D (misc_ctl & SVM_MISC_ENABLE_GMET) !=3D 0; =20 root_role =3D cpu_role.base; root_role.level =3D kvm_mmu_get_tdp_level(vcpu); @@ -6005,7 +6011,7 @@ void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu, b= ool execonly, context->gva_to_gpa =3D ept_gva_to_gpa; context->sync_spte =3D ept_sync_spte; =20 - update_permission_bitmask(context, true); + update_permission_bitmask(context, true, true); context->pkru_mask =3D 0; reset_rsvds_bits_mask_ept(vcpu, context, execonly, huge_page_level); reset_ept_shadow_zero_bits_mask(context, execonly); diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index a1cffd274000..7adfa7da210d 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -95,7 +95,8 @@ static void nested_svm_init_mmu_context(struct kvm_vcpu *= vcpu) */ kvm_init_shadow_npt_mmu(vcpu, svm->vmcb01.ptr->save.cr4, svm->vmcb01.ptr->save.efer, - svm->nested.ctl.nested_cr3); + svm->nested.ctl.nested_cr3, + svm->nested.ctl.misc_ctl); vcpu->arch.mmu->get_guest_pgd =3D nested_svm_get_tdp_cr3; vcpu->arch.mmu->get_pdptr =3D nested_svm_get_tdp_pdptr; vcpu->arch.mmu->inject_page_fault =3D nested_svm_inject_npf_exit; @@ -2076,12 +2077,15 @@ static gpa_t svm_translate_nested_gpa(struct kvm_vc= pu *vcpu, gpa_t gpa, struct x86_exception *exception, u64 pte_access) { + struct vcpu_svm *svm =3D to_svm(vcpu); struct kvm_mmu *mmu =3D vcpu->arch.mmu; =20 BUG_ON(!mmu_is_nested(vcpu)); =20 - /* NPT walks are always user-walks */ - access |=3D PFERR_USER_MASK; + /* Non-GMET walks are always user-walks */ + if (!(svm->nested.ctl.misc_ctl & SVM_MISC_ENABLE_GMET)) + access |=3D PFERR_USER_MASK; + return mmu->gva_to_gpa(vcpu, mmu, gpa, access, exception); } =20 --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E97E940149C for ; Tue, 28 Apr 2026 11:10:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374621; cv=none; b=XANJohklqvqs3eyW724tUZYy0gbY7qvJgb+x4MWFVu3E+Sd+jgD9CfBhpFW9WrX0jiB3K/Bl+8yzVXR/uE46e4KnYPexBY4Tayls3WVswtKmEDtyDQuaBJX3XG5+9T9V0iHHCC5eNXdYssrckRmlBxiUKW8A7Zu+h1Vus8vHgiw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374621; c=relaxed/simple; bh=j4Zp27lfKLT47cSt9lEA0iytxlktaR4GD6hEueak65w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=dPjeW6UG+QX1D6YU027r3lDvTy3ucxXyGAqsY/jinkEheIqvif92famxzs9fX04N2CMOlljI3DJdwVWmW6aVAi2NZKvj2dc96QLxqu4Ei3dJHeDufHWFe/Gs2+bY8sRgKTE5KYF5eczo6W/3OD9jNfHnRrKajUbai/1LIfoyR1k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=HZZvEy+9; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="HZZvEy+9" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374618; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EOywN7PsB9nok987GKX74XlPtbgpdmTNkk2VBiQDudI=; b=HZZvEy+9aHt3XKgD3FXB+4T9Pzw52nAk+1kqzp2orD1Zf3pEKpBW4K6zQUFfk+oy7oEUPF wGkH9F+K/jL6b5sbYpZFXEGZVIiUSME/lOrPG++rRq83Y6pM4m+BVvq4jst2rUZPTzvW2E BZhbyx/NmJCHxWIk0jpCCd8As3Ic+Wg= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-541--2peKKXMPwC8g2fvNBuu7g-1; Tue, 28 Apr 2026 07:10:14 -0400 X-MC-Unique: -2peKKXMPwC8g2fvNBuu7g-1 X-Mimecast-MFC-AGG-ID: -2peKKXMPwC8g2fvNBuu7g_1777374613 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8EDCE18003F6; Tue, 28 Apr 2026 11:10:13 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 04D1619560B1; Tue, 28 Apr 2026 11:10:12 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 26/28] KVM: SVM: enable GMET and set it in MMU role Date: Tue, 28 Apr 2026 07:09:44 -0400 Message-ID: <20260428110946.11466-27-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" Set the GMET bit in the nested control field. This has effectively no impact as long as NPT page tables are changed to have U=3D0. Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 6 +++++- arch/x86/kvm/svm/nested.c | 7 +++++-- arch/x86/kvm/svm/svm.c | 16 ++++++++++++++++ arch/x86/kvm/svm/svm.h | 1 + 4 files changed, 27 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 6cf0e5f28ba5..4ac5d25c5616 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -5849,7 +5849,6 @@ kvm_calc_tdp_mmu_root_page_role(struct kvm_vcpu *vcpu, { union kvm_mmu_page_role role =3D {0}; =20 - role.access =3D ACC_ALL; role.cr0_wp =3D true; role.cr4_smep =3D kvm_x86_call(tdp_has_smep)(vcpu->kvm); role.efer_nx =3D true; @@ -5860,6 +5859,11 @@ kvm_calc_tdp_mmu_root_page_role(struct kvm_vcpu *vcp= u, role.direct =3D true; role.has_4_byte_gpte =3D false; =20 + /* All TDP pages are supervisor-executable */ + role.access =3D ACC_ALL; + if (role.cr4_smep && shadow_user_mask) + role.access &=3D ~ACC_USER_MASK; + return role; } =20 diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index 7adfa7da210d..a28c5e8d017b 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -895,9 +895,12 @@ static void nested_vmcb02_prepare_control(struct vcpu_= svm *svm) /* Also overwritten later if necessary. */ vmcb02->control.tlb_ctl =3D TLB_CONTROL_DO_NOTHING; =20 - /* nested_cr3. */ - if (nested_npt_enabled(svm)) + /* Use vmcb01 MMU and format if guest does not use nNPT */ + if (nested_npt_enabled(svm)) { + vmcb02->control.misc_ctl &=3D ~SVM_MISC_ENABLE_GMET; + nested_svm_init_mmu_context(vcpu); + } =20 vcpu->arch.tsc_offset =3D kvm_calc_nested_tsc_offset(vcpu->arch.l1_tsc_of= fset, vmcb12_ctrl->tsc_offset, diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index e7fdd7a9c280..3895d8794366 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -138,6 +138,9 @@ module_param(pause_filter_count_max, ushort, 0444); bool __ro_after_init npt_enabled =3D true; module_param_named(npt, npt_enabled, bool, 0444); =20 +bool gmet_enabled =3D true; +module_param_named(gmet, gmet_enabled, bool, 0444); + /* allow nested virtualization in KVM/SVM */ static int __ro_after_init nested =3D true; module_param(nested, int, 0444); @@ -1209,6 +1212,10 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool in= it_event) save->g_pat =3D vcpu->arch.pat; save->cr3 =3D 0; } + + if (gmet_enabled) + control->misc_ctl |=3D SVM_MISC_ENABLE_GMET; + svm->current_vmcb->asid_generation =3D 0; svm->asid =3D 0; =20 @@ -4612,6 +4619,11 @@ svm_patch_hypercall(struct kvm_vcpu *vcpu, unsigned = char *hypercall) hypercall[2] =3D 0xd9; } =20 +static bool svm_tdp_has_smep(struct kvm *kvm) +{ + return gmet_enabled; +} + /* * The kvm parameter can be NULL (module initialization, or invocation bef= ore * VM creation). Be sure to check the kvm parameter before using it. @@ -5355,6 +5367,7 @@ struct kvm_x86_ops svm_x86_ops __initdata =3D { .write_tsc_multiplier =3D svm_write_tsc_multiplier, =20 .load_mmu_pgd =3D svm_load_mmu_pgd, + .tdp_has_smep =3D svm_tdp_has_smep, =20 .check_intercept =3D svm_check_intercept, .handle_exit_irqoff =3D svm_handle_exit_irqoff, @@ -5588,6 +5601,9 @@ static __init int svm_hardware_setup(void) if (!boot_cpu_has(X86_FEATURE_NPT)) npt_enabled =3D false; =20 + if (!npt_enabled || !boot_cpu_has(X86_FEATURE_GMET)) + gmet_enabled =3D false; + /* Force VM NPT level equal to the host's paging level */ kvm_configure_mmu(npt_enabled, get_npt_level(), get_npt_level(), PG_LEVEL_1G); diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index a10668d17a16..dd93b3daefa9 100644 --- a/arch/x86/kvm/svm/svm.h +++ b/arch/x86/kvm/svm/svm.h @@ -44,6 +44,7 @@ static inline struct page *__sme_pa_to_page(unsigned long= pa) #define IOPM_SIZE PAGE_SIZE * 3 #define MSRPM_SIZE PAGE_SIZE * 2 =20 +extern bool gmet_enabled; extern bool npt_enabled; extern int nrips; extern int vgif; --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BB6624014BB for ; Tue, 28 Apr 2026 11:10:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374625; cv=none; b=JQY73eZe5GDEkWs4nq+jkmZp9ljQwx7tzecmrFtHuil+EgzCqtRigQ/uAoOPUGPEtMu490BZRMVy6gKgWEXs1v+2c714tCSuerq1wRW/n7xbeUgWYkGOyMuO/JCYpGauVZ/6K/PGOlAbGAGqpRwmg/StBmKg/xEUzvnxTRQdBUo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374625; c=relaxed/simple; bh=GR++9H5p0i1D5sIKFL2DvhkQ9RXdFtz3TIvO90pRMw0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=FrggBjDcm8UywRte9XyLB1zEBaCyyN5Tt8Vq3V4zCdVoHHsGBjCyPKWLBbIWzN+zBk7sivMOtEgDNZATbxptwBJJZhAy2894HTUSGXVeZANoGY5gJ1gk7r1tKWSKU0x1fUNjT+Sfy+7PxRUybbdDz1KIvfbqnrok5qjBEwffIxE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=QWFOscaJ; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QWFOscaJ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374618; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RQb5rHk5eMELl/+vuZzClPskfe+ABko0k3vSQpQtlOY=; b=QWFOscaJCX8t1mjsRp+aIhAwRx8o1hZqwmk4M/bwRdxFHF/Bxb1NYtZr2xlpNdzLX3zqHN L/lZosKlwd6DgnMMsbBBLHTI+mYbcns97/bDi3XDT4yRy5ZcNwIc85NgxW7Wpx6PJqXa59 pLYa2v3C/MVh1D1f1wXY1W2GPP/PkK8= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-582-dNhLvYQlM1y3JZ4jKftN3w-1; Tue, 28 Apr 2026 07:10:15 -0400 X-MC-Unique: dNhLvYQlM1y3JZ4jKftN3w-1 X-Mimecast-MFC-AGG-ID: dNhLvYQlM1y3JZ4jKftN3w_1777374614 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4944F1956080; Tue, 28 Apr 2026 11:10:14 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B4DBF19560AB; Tue, 28 Apr 2026 11:10:13 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 27/28] KVM: SVM: work around errata 1218 Date: Tue, 28 Apr 2026 07:09:45 -0400 Message-ID: <20260428110946.11466-28-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" According to AMD, the hypervisor may not be able to determine whether a fault was a GMET fault or an NX fault based on EXITINFO1, and software "must read the relevant VMCB to determine whether a fault was a GMET fault or an NX fault". The APM further details that they meant the CPL field. KVM uses the page fault error code to distinguish the causes of a nested page fault, so recalculate the PFERR_USER_MASK bit of the vmexit information. Only do it for fetches and only if GMET is in use, because KVM does not differentiate based on PFERR_USER_MASK for other nested NPT page faults. Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/kvm/svm/svm.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 3895d8794366..fd79874c5f4b 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -1993,6 +1993,18 @@ static int npf_interception(struct kvm_vcpu *vcpu) } } =20 + if (!is_sev_es_guest(vcpu) && + (svm->vmcb->control.misc_ctl & SVM_MISC_ENABLE_GMET) && + (error_code & PFERR_FETCH_MASK)) { + /* + * Work around errata 1218: EXITINFO1[2] May Be Incorrectly Set + * When GMET (Guest Mode Execute Trap extension) is Enabled + */ + error_code |=3D PFERR_USER_MASK; + if (svm_get_cpl(vcpu) !=3D 3) + error_code &=3D ~PFERR_USER_MASK; + } + if (is_sev_snp_guest(vcpu) && (error_code & PFERR_GUEST_ENC_MASK)) error_code |=3D PFERR_PRIVATE_ACCESS; =20 --=20 2.52.0 From nobody Wed Jun 17 02:51:47 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 515D5401A16 for ; Tue, 28 Apr 2026 11:10:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374622; cv=none; b=DH0IvS/CmxduD0T98oy5ZBgpdfcv9DV/+hnsgO5MHSYPAuYKnunfC9E1RDNyHG6UgxXkrp3o7nUHxDIHn+svYklWb2lcRsalgG31a6w9Om1V9UwdA3sCol6jje/e492CehpsmTXdvCgPqDiK2Z2oZXgx9/2ywfED5bhvO8FBh0A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777374622; c=relaxed/simple; bh=9YKrVHMy/Is4wk0GV6lHiSaj/teDs4EBo29b2+RM9G4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=M10YvANfYivnJ7TeU7qQUwl2ajjDPaGGqh++tQ7Plmm9wZwWjuJqvEOuYdj5tJTi7FXfcuV4JonGTdnNkXHUaZ/q+tX57IFOBLw2IumidoH9QM+7wv+t32spNTeDXTlwu+i9+hjlXMM1iR6AhZC4PLs+v+DsG8B1XNXqsbCeMOU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=NflEKUUF; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="NflEKUUF" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777374619; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bX86XKGVtckHkBiIxzqAKcxicpJAva5O8kkQH2k0ltk=; b=NflEKUUFMU4ATq0fBAJDQ3GcR92bvf8I2BHyoC/lOq5Aql0f6oTWtz6AAYVSazDG25P13x c/z7pRFULELyKKjagTIevihBw5JiN0TZQ6CS9IjpQAzgrkkIzo3xiEgQ0cQUba2SbmiTsc /wP3aNJLcK3YLKc3gw+BMIchDSF1ab4= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-638-cXda99BJO7-TXY2qCP_fMw-1; Tue, 28 Apr 2026 07:10:16 -0400 X-MC-Unique: cXda99BJO7-TXY2qCP_fMw-1 X-Mimecast-MFC-AGG-ID: cXda99BJO7-TXY2qCP_fMw_1777374615 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0315A180034F; Tue, 28 Apr 2026 11:10:15 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 6EA4719560B1; Tue, 28 Apr 2026 11:10:14 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jon@nutanix.com, d.riley@proxmox.com Subject: [PATCH 28/28] KVM: nSVM: enable GMET for guests Date: Tue, 28 Apr 2026 07:09:46 -0400 Message-ID: <20260428110946.11466-29-pbonzini@redhat.com> In-Reply-To: <20260428110946.11466-1-pbonzini@redhat.com> References: <20260428110946.11466-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" All that needs to be done is moving the GMET bit from vmcs12 to vmcs02. The only new thing is that __nested_copy_vmcb_control_to_cache now ensures that ignored-if-unavailable bits are zero in svm->nested.ctl. Tested-by: David Riley Signed-off-by: Paolo Bonzini --- arch/x86/kvm/svm/nested.c | 7 +++++++ arch/x86/kvm/svm/svm.c | 3 +++ 2 files changed, 10 insertions(+) diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index a28c5e8d017b..1d791918efb3 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -508,6 +508,12 @@ void __nested_copy_vmcb_control_to_cache(struct kvm_vc= pu *vcpu, to->exit_int_info =3D from->exit_int_info; to->exit_int_info_err =3D from->exit_int_info_err; to->event_inj =3D from->event_inj & ~SVM_EVTINJ_RESERVED_BITS; + + to->misc_ctl =3D from->misc_ctl; + if (!gmet_enabled || !guest_cpu_cap_has(vcpu, X86_FEATURE_GMET)) + to->misc_ctl &=3D ~SVM_MISC_ENABLE_GMET; + + to->event_inj =3D from->event_inj; to->event_inj_err =3D from->event_inj_err; to->next_rip =3D from->next_rip; to->nested_cr3 =3D from->nested_cr3; @@ -898,6 +904,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_s= vm *svm) /* Use vmcb01 MMU and format if guest does not use nNPT */ if (nested_npt_enabled(svm)) { vmcb02->control.misc_ctl &=3D ~SVM_MISC_ENABLE_GMET; + vmcb02->control.misc_ctl |=3D (svm->nested.ctl.misc_ctl & SVM_MISC_ENABL= E_GMET); =20 nested_svm_init_mmu_context(vcpu); } diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index fd79874c5f4b..a82471a6d3ea 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -5504,6 +5504,9 @@ static __init void svm_set_cpu_caps(void) if (boot_cpu_has(X86_FEATURE_PFTHRESHOLD)) kvm_cpu_cap_set(X86_FEATURE_PFTHRESHOLD); =20 + if (gmet_enabled) + kvm_cpu_cap_set(X86_FEATURE_GMET); + if (vgif) kvm_cpu_cap_set(X86_FEATURE_VGIF); =20 --=20 2.52.0