From nobody Thu Oct 2 07:44:01 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 03BB81F3FEC for ; Fri, 19 Sep 2025 00:59:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758243601; cv=none; b=Wz8feKwZ97QZEGZ8r3YAX17ADv7+MA4Y59uaUPZEj+/Btcjw93YkZ3TQGYkX6xhVbbhhp0NzgaiHERfHo8aCU1My76qc/lCvmTVH+ic2DSsO3TKIwpZQxFabAYGDzYwWEcz04Vorx2JZfIO8iss8GpPboVBPws2lpfFnrLw+seA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758243601; c=relaxed/simple; bh=FfCh13Q+m92+CymbRlQmSA5l2N0xV/rmT0iRiDdm/D4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=e53TAmQEWLkVTZNfoGQoSg+3Lj9XkmIkgHcinAKuZrhAsiQ3YyaPHAa9c9hzTKYDGzRo3ezKa2iJz+GUAiPr+zA4WEYoiVlsPkCKMtc5GKC1DXX0mzq+tbKpZgfExfALRGw2rxjrnQlyFs72i1MgWj4rQrHtVruUFkmSGVyz1xw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=vhN5csWE; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="vhN5csWE" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-24ae30bd2d0so12337955ad.3 for ; Thu, 18 Sep 2025 17:59:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1758243599; x=1758848399; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=OvlGI1x2xd9nnaxn6+YbYZ5V941T2eRm4PgITqh2qd4=; b=vhN5csWEC1zq5+4TiEGdenNHhy2vI3koxy08I2MTowtxFAArPMaqWPtAmH0J9emdeS A5je756RXM2gs64Zpgoslhu3iNOQQYChwj7+McQ6r+716CRsp5e7Xk+C9H6EE/O5+VWZ H7wH8rvFuCLJtu4aZzs0/mOmuk1EOSMafoYuavabUFKYi0qFZBmtLCXTfqNqwi7AxKhS SXfjKBMw0Wj1Ke26N0gUerm0ce4Mxh850wTJqDiucBE5RRNrkZRyOscOcjmv7mNEIgAH YyUNuTp3tz9+uxwrKRnU2zjeUmrMU7KEQV1cCbCirwqrx/Qb9/akj1GRb3doXG/hxS+j JkRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1758243599; x=1758848399; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OvlGI1x2xd9nnaxn6+YbYZ5V941T2eRm4PgITqh2qd4=; b=DMp9rvZLo1D5C4gh8zIkItJusRtZZgOcyVvdZ+SEO8MTZb6/rLtFyW/qxFlRVAWaz/ vhq5NBLNdaEKvy8l1WBtWd6jxe9pBjuTxgLd9lQq9SADzkjT0QRTGWUgOSYUnHqsm4oc c9R/DvKpx+FQSM60yBHE0e0JNlfVwkjkrZXDHmaLSc3YLdkQrhrs+kdtbt598cMRBoi3 +Jv6496/JDjrGD2nqCCUIGvYQ01LubNlPraLWfRK1/YtHKCzOOHT+zOotA9OCav23yG3 P2J6mTkW5ml/pl80g6cnp74p1RZiSHcs0FwAp8tmvAcRE8hJ3fEfC7+zhe5E+pq28V6q AaMw== X-Forwarded-Encrypted: i=1; AJvYcCUMZb7p/W3dcMrZ6rWHfVnyprsaGv3lBiyPPNWXTtDznS5/DjhXSXvN2TZ799xUKe7U9CKQG038Vdwvg5E=@vger.kernel.org X-Gm-Message-State: AOJu0YxNgMo2v40xYZ44cvz8Or4haRNCg6O9EJ+qNfYphW/D7c1Meknk NSQIh71+vYe4X6qUeV+t91TAlj+XdPnO964pedsB5jjMb48MDIPX7Q9HvzYTlIluEcnGKqwswBH Y3GBASw== X-Google-Smtp-Source: AGHT+IF9CdEIDOE/UKd3QCqc4VwIlDG3jXKLfgBVhQjS9OcGyBwSpjhRCVv6E83o2ykIT0BdPjNtPktfrEY= X-Received: from pjvv5.prod.google.com ([2002:a17:90b:5885:b0:32b:58d1:a610]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ef44:b0:25c:343a:12eb with SMTP id d9443c01a7336-269ba402095mr18635305ad.4.1758243599336; Thu, 18 Sep 2025 17:59:59 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 18 Sep 2025 17:59:47 -0700 In-Reply-To: <20250919005955.1366256-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250919005955.1366256-1-seanjc@google.com> X-Mailer: git-send-email 2.51.0.470.ga7dc726c21-goog Message-ID: <20250919005955.1366256-2-seanjc@google.com> Subject: [PATCH 1/9] KVM: VMX: Hoist construct_eptp() "up" in vmx.c From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move construct_eptp() further up in vmx.c so that it's above vmx_flush_tlb_current(), its "first" user in vmx.c. This will allow a future patch to opportunistically make construct_eptp() local to vmx.c. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/vmx.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 35037fc326e5..3c622c91cbc5 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -3201,6 +3201,20 @@ static inline int vmx_get_current_vpid(struct kvm_vc= pu *vcpu) return to_vmx(vcpu)->vpid; } =20 +u64 construct_eptp(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level) +{ + u64 eptp =3D VMX_EPTP_MT_WB; + + eptp |=3D (root_level =3D=3D 5) ? VMX_EPTP_PWL_5 : VMX_EPTP_PWL_4; + + if (enable_ept_ad_bits && + (!is_guest_mode(vcpu) || nested_ept_ad_enabled(vcpu))) + eptp |=3D VMX_EPTP_AD_ENABLE_BIT; + eptp |=3D root_hpa; + + return eptp; +} + void vmx_flush_tlb_current(struct kvm_vcpu *vcpu) { struct kvm_mmu *mmu =3D vcpu->arch.mmu; @@ -3378,20 +3392,6 @@ static int vmx_get_max_ept_level(void) return 4; } =20 -u64 construct_eptp(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level) -{ - u64 eptp =3D VMX_EPTP_MT_WB; - - eptp |=3D (root_level =3D=3D 5) ? VMX_EPTP_PWL_5 : VMX_EPTP_PWL_4; - - if (enable_ept_ad_bits && - (!is_guest_mode(vcpu) || nested_ept_ad_enabled(vcpu))) - eptp |=3D VMX_EPTP_AD_ENABLE_BIT; - eptp |=3D root_hpa; - - return eptp; -} - void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_leve= l) { struct kvm *kvm =3D vcpu->kvm; --=20 2.51.0.470.ga7dc726c21-goog From nobody Thu Oct 2 07:44:01 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 79D93244686 for ; Fri, 19 Sep 2025 01:00:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758243603; cv=none; b=ESJdd9RGCpf8+2/RUZVnApiUMKqdMlEqpweOhShVrfIixBZaQyFaiJ/QWKrgOo1XtmWGhjLp5hiJDo9VptDpgLLnaXLPlCvuNpPl1WS4zRSxurpLEAvVBf8JfK2GAn6h7Cuo2hsB6ES36h5BQYGruxWLvIqD8DMHxSIWIwu72b8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758243603; c=relaxed/simple; bh=KtmjwUQKEDaGma8aD7fkjztjTOgXgTDisEJmE5V5YC0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=fHLnHD9SkRabHxaLWYyqnoLlpdnRLl3nr891yMvMQ5zzpoQzsS3oK7mhR+jlTcWLT1VJX29AZ/g3tf2mlTYUlxMm7rVzlYC0NBGJP+vH9o/8M8bYZCHJzlOgXBJPaR+vzoIUcnFzD7EKzhZSUVk3vWJgD/HztoY3K7MSXCo1jug= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=IXo3+5uU; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="IXo3+5uU" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-76e6e71f7c6so1518013b3a.0 for ; Thu, 18 Sep 2025 18:00:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1758243601; x=1758848401; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=UWy3i2fp1q6nRdPbeRuCID8VdI7QyY4h6hWBylbyQSw=; b=IXo3+5uUYMbJkGwxfullqYznBYsGxp2pkMJENqwMov6kkdpHc37kAghpKXvCT4BxQ2 wgG+Y3ZGJ4W3Z+1q38ngQ2J0ZTWN+erqt791Tm2O+J8jAtsOWZFZqZeVoS+lzMnFZoSg tTdK5JSzOoCTUigqCVDNL40f4zjtIMgFuzPQsvWazDppGgZA767C8W8O5NaQu6Xaovg/ QGmIcfSemJqNP4yJPtLituXdmSXv0Nxp4TH1IXWyhaTC3xL+prJwktFubuVEKJCKyvgG BZ6TXMeX2r1TK3THhJZZHO/Ahgna1mE3DtTonk0QbRX0fTQdecbxCLynssQDWYixElqo S34Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1758243601; x=1758848401; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=UWy3i2fp1q6nRdPbeRuCID8VdI7QyY4h6hWBylbyQSw=; b=LFLJf901U7jDVoAmfn013W9rGYkCApyj1cNCt+C7cN9MCnoQuTaOq7TeuVpA1ITP/G SQkutBFxp6tjXfTPRBg4jHygiNSnqVbc2IJShuspeJebyLhjocc9LLYWvXDw5CJiB6w1 LNj0/3n8dLoqThsu/Tr5Hib6cx/hy72J2E7JFL7ReimHMZ8JESN34sYeVgjImOiTXj+g 4KY8zBTT+y6edENyqGHzvBS2tOm3ao+YDTclMasJoaY44WKiBLEfJ//TDaDJdnpy9Nbg uyj9TO0IbebWeR33mTk7xpFdqfzJS4lN4R71geC9jR/wh+aAaYReeXlfbTbXr9wNZl2A LpNw== X-Forwarded-Encrypted: i=1; AJvYcCXqqOxYL9o+VYROJT+X5lgrOnQjmDtQS9quD5+YXh8wb19PzapbdKx+KJUwMCU6nIyoVmYdFHwS52AY0gI=@vger.kernel.org X-Gm-Message-State: AOJu0Yw6ms5/55SWj7/8QFF6ZBTaz6Z0qj1zep0maXEzK3pH8omqy9TQ /JPHYIOQBzMkAR55xqBULkjPTxJAv2GO3i08qVjZeuA6uk4XE8kvS+EdDDxoIERgbL2JYszGZ9r NvpAdfQ== X-Google-Smtp-Source: AGHT+IFnWENHzvN/NcFf/IxWJBYR1Ml2tRSnK0NlBzCXck73s2Te+54DEYFubEJyD7rnzsffuRVAm7qyw50= X-Received: from pjbnw4.prod.google.com ([2002:a17:90b:2544:b0:32b:65c6:661a]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:e082:b0:243:97c7:a013 with SMTP id adf61e73a8af0-2926e379d27mr1860284637.34.1758243600810; Thu, 18 Sep 2025 18:00:00 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 18 Sep 2025 17:59:48 -0700 In-Reply-To: <20250919005955.1366256-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250919005955.1366256-1-seanjc@google.com> X-Mailer: git-send-email 2.51.0.470.ga7dc726c21-goog Message-ID: <20250919005955.1366256-3-seanjc@google.com> Subject: [PATCH 2/9] KVM: nVMX: Hardcode dummy EPTP used for early nested consistency checks From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Hardcode the dummy EPTP used for "early" consistency checks as there's no need to use 5-level EPT based on the guest.MAXPHYADDR (the EPTP just needs to be valid, it's never truly consumed). This will allow breaking construct_eptp()'s dependency on having access to the vCPU, which in turn will (much further in the future) allow for eliding per-root TLB flushes when a vCPU is migrated between pCPUs (a flush is need if and only if that particular pCPU hasn't already flushed the vCPU's roots). Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 8 +++----- arch/x86/kvm/vmx/vmx.c | 2 +- arch/x86/kvm/vmx/vmx.h | 1 - 3 files changed, 4 insertions(+), 7 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 2156c9a854f4..253e93ced9dc 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -2278,13 +2278,11 @@ static void prepare_vmcs02_constant_state(struct vc= pu_vmx *vmx) vmx->nested.vmcs02_initialized =3D true; =20 /* - * We don't care what the EPTP value is we just need to guarantee - * it's valid so we don't get a false positive when doing early - * consistency checks. + * If early consistency checks are enabled, stuff the EPT Pointer with + * a dummy *legal* value to avoid false positives on bad control state. */ if (enable_ept && nested_early_check) - vmcs_write64(EPT_POINTER, - construct_eptp(&vmx->vcpu, 0, PT64_ROOT_4LEVEL)); + vmcs_write64(EPT_POINTER, VMX_EPTP_MT_WB | VMX_EPTP_PWL_4); =20 if (vmx->ve_info) vmcs_write64(VE_INFORMATION_ADDRESS, __pa(vmx->ve_info)); diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 3c622c91cbc5..74dba9f1d098 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -3201,7 +3201,7 @@ static inline int vmx_get_current_vpid(struct kvm_vcp= u *vcpu) return to_vmx(vcpu)->vpid; } =20 -u64 construct_eptp(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level) +static u64 construct_eptp(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_= level) { u64 eptp =3D VMX_EPTP_MT_WB; =20 diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index 23d6e89b96f2..e912a82a1d14 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -366,7 +366,6 @@ void set_cr4_guest_host_mask(struct vcpu_vmx *vmx); void ept_save_pdptrs(struct kvm_vcpu *vcpu); void vmx_get_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int s= eg); void __vmx_set_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int= seg); -u64 construct_eptp(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level); =20 bool vmx_guest_inject_ac(struct kvm_vcpu *vcpu); void vmx_update_exception_bitmap(struct kvm_vcpu *vcpu); --=20 2.51.0.470.ga7dc726c21-goog From nobody Thu Oct 2 07:44:01 2025 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1BFC01EDA0E for ; Fri, 19 Sep 2025 01:00:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758243604; cv=none; b=lmuah0m03yx1CMB1Haeof/MafA45YjwFqqgFyDyZZ0UQNpJ/gahkfgf++APshSX9F0ABQcb9ySNb6nLxNi+DiDiIYFEa0xqgxD0cDi1VcAz57+yyjjW1CkiPuit+WNB/I21bArcU1EsdINCrcuGPsCGvCYu9EcOOffrrf0Eprl4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758243604; c=relaxed/simple; bh=YmO5ZhLn3vGz8lgRuFa6XK7iRPoUAovIdCgjt6bcB3k=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=PR+MwWcD7yWoZRRDxTv/uC3TiSKxaidrN24YyhKx425YsR1CT8LPL3mAZuLOVK74XdKfD0qsxMlqIrkpwDdhzTM8ubgPRBQ7hFVJq+m+ECIz2sQqOUJFPEnaVT6mjBgAOW9um5I3rRS/Q/aMVpfaFfYX00A5J7FUYDOa1wSoURo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=aPCkW+R5; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="aPCkW+R5" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2445805d386so17720345ad.1 for ; Thu, 18 Sep 2025 18:00:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1758243602; x=1758848402; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=lH98c26oi0iG7hKKc9Hul2ePNiDjTOk5eR1Ays5VvX8=; b=aPCkW+R5esZpJH+em8tEcdjH1VdXpzHoc2g9Bhn9Z3jrCP8L80e5BqFb/ZrTsi8jB2 bSDxCTpfU0XzTjvIXggX83SostRHoktEwdLdhXJK52LjyVMBojilv2xkPmuRbc/7L7NO E1yWJ7gPq161V9rmiLfCYghdtOasTK3noYcI5bBLog9gI2fjalu9Y60UWhNAmOZI3gRu xmMgR9mkCzFg4MbVwHFwTwzvvN18h4F9DMPSVLum65Y5QIxHAodMrMhx1Cs752RaS6bJ oUsZFE0cmSs55zAtAvCmC4DNg5v6/Ntwn5JjgAUlqUMdEQ42KeHBRAVk6dmCp9SjElSj BuSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1758243602; x=1758848402; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=lH98c26oi0iG7hKKc9Hul2ePNiDjTOk5eR1Ays5VvX8=; b=PTD+R+xi/TFgiTeoWzoU6FY5P2pmhxHobbJDJxzSrqG6iBW+/FsOWPndHNbeVuPo3Q 1guSSiivu5++q4PYmJ2mnWk9dxtkTR+bw/tLHYFrwOH0JHUUPL8UsGSbPS2UQQ+P30Ij llvmCwvCk/cJvOD/Vq5zbCLU7L88HV5R27JtqlaWzewBKHdc3m63mhs+xVj1Gf55Ujru hYaCl2XdNK5J26pMkAyG5mAEF1n1QmihDnaXu2y6qLolwx5lh8kxpWYCltW2yFWbeYmU 9N3X8nP05bwttG5zcEjUDG1hrffRaDwJZQ+wBWd1MJLuiC1uH91cViQbhi+IajD+X4D6 FnaA== X-Forwarded-Encrypted: i=1; AJvYcCWFepOaWztbNXuiwWhSssk3fIyM7Xgm1GUu6kKPzcdoJlntUyI/q5uMU+ldmoQjNQ4MlKbf/+Csz07a9hM=@vger.kernel.org X-Gm-Message-State: AOJu0Yz6DQSlEeMTNCu2WmX2hc4z4wn/GZd9VI55YRV2GzAXbhCmWWtR D9GfbqxWAgoBEVgzP3OnkpMJs+9dvfwPsixTnk52v2lHhClOFX94p/l5BIuIYzndUImpm6jY63N 5E2qP1Q== X-Google-Smtp-Source: AGHT+IEVhfZo+VuCff+zD2U0D2h648aLSOlkoBvlJxDLxKHMbodux6YoSgiIo14EdAHAY8tP2J5+PU7+TcY= X-Received: from pjac15.prod.google.com ([2002:a17:90a:108f:b0:32e:e4e6:ecfe]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:b07:b0:264:f3ed:ee2d with SMTP id d9443c01a7336-269ba48383cmr16975795ad.27.1758243602373; Thu, 18 Sep 2025 18:00:02 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 18 Sep 2025 17:59:49 -0700 In-Reply-To: <20250919005955.1366256-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250919005955.1366256-1-seanjc@google.com> X-Mailer: git-send-email 2.51.0.470.ga7dc726c21-goog Message-ID: <20250919005955.1366256-4-seanjc@google.com> Subject: [PATCH 3/9] KVM: x86/mmu: Move "dummy root" helpers to spte.h From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move the helpers to get/query a dummy root from mmu_internal.h to spte.h so that VMX can detect and handle dummy roots when constructing EPTPs. This will allow using the root's role to build the EPTP instead of pulling equivalent information out of the vCPU structure. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu_internal.h | 10 ---------- arch/x86/kvm/mmu/spte.h | 10 ++++++++++ 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_interna= l.h index ed5c01df21ba..73cdcbccc89e 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -39,16 +39,6 @@ #define INVALID_PAE_ROOT 0 #define IS_VALID_PAE_ROOT(x) (!!(x)) =20 -static inline hpa_t kvm_mmu_get_dummy_root(void) -{ - return my_zero_pfn(0) << PAGE_SHIFT; -} - -static inline bool kvm_mmu_is_dummy_root(hpa_t shadow_page) -{ - return is_zero_pfn(shadow_page >> PAGE_SHIFT); -} - typedef u64 __rcu *tdp_ptep_t; =20 struct kvm_mmu_page { diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index 3133f066927e..91ce29fd6f1b 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -246,6 +246,16 @@ static inline int spte_index(u64 *sptep) */ extern u64 __read_mostly shadow_nonpresent_or_rsvd_lower_gfn_mask; =20 +static inline hpa_t kvm_mmu_get_dummy_root(void) +{ + return my_zero_pfn(0) << PAGE_SHIFT; +} + +static inline bool kvm_mmu_is_dummy_root(hpa_t shadow_page) +{ + return is_zero_pfn(shadow_page >> PAGE_SHIFT); +} + static inline struct kvm_mmu_page *to_shadow_page(hpa_t shadow_page) { struct page *page =3D pfn_to_page((shadow_page) >> PAGE_SHIFT); --=20 2.51.0.470.ga7dc726c21-goog From nobody Thu Oct 2 07:44:01 2025 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 979B02566D2 for ; Fri, 19 Sep 2025 01:00:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758243606; cv=none; b=BOZ2eRNjggwJqF58vkbXU33fLFT59BT5FtlMOz3lj5g+cM9KAmuzpWGSj2Mjvydgu76yZfdT630XmXYy1xHcuI+H4C56Z6xzGgcCaQjskZPEkYSzU72vtnSfryF+VwxzAUeLvS7wFLGBSyXUFO6MllIvIOL7sx/UJ8DPp8baSxQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758243606; c=relaxed/simple; bh=YJ2VFrZhM4qIUjqB49CAb9i42QiiedUh7facuEGSF6I=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=rp308IZbZqMIMUn4s73fZZJlDWYZ8/0lnAFWHFI1cmUfpNe9p8Iet/rpKJGaposc0ExSR15/zt6vbjo6QoDb1MDc4jbPFuk7bfZ3OVanNWoPWBcJNisgC0a7UTdy7sW/g0w109EX4gq1SDLdD7WhERWwBwqq6FHnKpFjme04tpc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=chiraHq+; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="chiraHq+" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-24456ebed7bso19398695ad.0 for ; Thu, 18 Sep 2025 18:00:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1758243604; x=1758848404; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=aSaDH6qLLomdBTa7b9zuhz08i6UmGSxbGqu+z1jTuOI=; b=chiraHq+z+5LT9T+kas3pnpFj0mBRKB5sO74/b77jfM1YbF9YlramoXwbUImiibxdF 2d9xPMBQLe1Sk0K0S116300b42XmfKayiAmR2UEvRgfiOtzJPQbb8bgUQG6hDpRytz69 p8FfHpIFMBTVNLtgFAzcCN1QlbEjt6ZW6uxj5GrV8msxoowazPUmh0mRwMjb+Q6k13KU Ttsu4RqnZ5qzoUy4yUKmMMLORX7gRaENL8RQTCk+b1IeAzG7540ZYM+G0zxkNxODqTPy ShfMc8pEbUrbCGxvgjwOzINDc1LjTFBaLL5xGQYfaD2lgjm/3B62+rQ1SaNwTjzZL6ad EG+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1758243604; x=1758848404; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=aSaDH6qLLomdBTa7b9zuhz08i6UmGSxbGqu+z1jTuOI=; b=kjIA4OVm7sNqr6H9RyyHVsR18SrdHOKaju6PiRR46dnKeTBgMeNstd4Qyl7z9U8EA5 AofNwMB52txHU5KsOD6PHzWsUSIci2SUBYrCmJcI0kx7ozpP+Xv2qdJqBtXlOX5+ZYMg G0bi8J7+yZBkgdIiBR/p1drs+oGKBwrXMPF5QYpSHjtFMnEXbuVQqcU2a/F44lJXTFme if+sqQ20h9FzuUEnyZNHrryWBTPnqmMqutbvTiLrXl0N3gnjTVCUPukutyaGqNMIRd1b iGq2FjkzUVzqh83yj8nk/YQbtd4JM4egCEicrd1bqU0zT3nf6ShJ6Uwft6/3Ssi1TFIy 3vTw== X-Forwarded-Encrypted: i=1; AJvYcCXO3U7GJ2COi7N45+01MClaSuIEHtuePrm6kbsrD29KWPDHFOeb+tsY8I2Y3jQiF11K3aufX+xpgfsd588=@vger.kernel.org X-Gm-Message-State: AOJu0YxOgTmI2f7MdmhzEBehNnFxs7v8AfXPv6ub9yZZjpbCS6AyPnFa I5yzwF3ZmIhnXNlU/+Ad58EMwan8P0pMEHWXJ7OAKSDv/59NTsklKbRZ426CoKmF51SZ1K/wX4F k5tyMaQ== X-Google-Smtp-Source: AGHT+IHkbfANFhJs0yTpbydDEuN4sXTG76yXOnVp4T9Fql8goBq4reBhUHVOl+kAMRkA9URfcpUB44v/ZHg= X-Received: from plae4.prod.google.com ([2002:a17:902:e0c4:b0:267:fa7d:b637]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:fc8f:b0:267:fa8d:29a6 with SMTP id d9443c01a7336-269b9cc7179mr19973895ad.25.1758243603881; Thu, 18 Sep 2025 18:00:03 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 18 Sep 2025 17:59:50 -0700 In-Reply-To: <20250919005955.1366256-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250919005955.1366256-1-seanjc@google.com> X-Mailer: git-send-email 2.51.0.470.ga7dc726c21-goog Message-ID: <20250919005955.1366256-5-seanjc@google.com> Subject: [PATCH 4/9] KVM: VMX: Use kvm_mmu_page role to construct EPTP, not current vCPU state From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the role for the to-be-loaded/invalidated EPT root to compute the root's level and A/D enablement instead of pulling the information from the vCPU (e.g. by passing in the root level and querying vmcs12). Not making unnecessary assumptions about the root will allow invalidating arbitrary EPT roots (which sadly requires a full EPTP) at any given time. No functional change intended (the end result should be the same). Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/vmx.c | 41 ++++++++++++++++++++++++++++++----------- 1 file changed, 30 insertions(+), 11 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 74dba9f1d098..cf2d44044da5 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -3201,20 +3201,40 @@ static inline int vmx_get_current_vpid(struct kvm_v= cpu *vcpu) return to_vmx(vcpu)->vpid; } =20 -static u64 construct_eptp(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_= level) +static u64 construct_eptp(hpa_t root_hpa) { - u64 eptp =3D VMX_EPTP_MT_WB; + u64 eptp =3D root_hpa | VMX_EPTP_MT_WB; + struct kvm_mmu_page *root; =20 - eptp |=3D (root_level =3D=3D 5) ? VMX_EPTP_PWL_5 : VMX_EPTP_PWL_4; + if (kvm_mmu_is_dummy_root(root_hpa)) + return eptp | VMX_EPTP_PWL_4; =20 - if (enable_ept_ad_bits && - (!is_guest_mode(vcpu) || nested_ept_ad_enabled(vcpu))) + /* + * EPT roots should always have an associated MMU page. Return a "bad" + * EPTP to induce VM-Fail instead of continuing on in a unknown state. + */ + root =3D root_to_sp(root_hpa); + if (WARN_ON_ONCE(!root)) + return INVALID_PAGE; + + eptp |=3D (root->role.level =3D=3D 5) ? VMX_EPTP_PWL_5 : VMX_EPTP_PWL_4; + + if (enable_ept_ad_bits && !root->role.ad_disabled) eptp |=3D VMX_EPTP_AD_ENABLE_BIT; - eptp |=3D root_hpa; =20 return eptp; } =20 +static void vmx_flush_tlb_ept_root(hpa_t root_hpa) +{ + u64 eptp =3D construct_eptp(root_hpa); + + if (VALID_PAGE(eptp)) + ept_sync_context(eptp); + else + ept_sync_global(); +} + void vmx_flush_tlb_current(struct kvm_vcpu *vcpu) { struct kvm_mmu *mmu =3D vcpu->arch.mmu; @@ -3225,8 +3245,7 @@ void vmx_flush_tlb_current(struct kvm_vcpu *vcpu) return; =20 if (enable_ept) - ept_sync_context(construct_eptp(vcpu, root_hpa, - mmu->root_role.level)); + vmx_flush_tlb_ept_root(root_hpa); else vpid_sync_context(vmx_get_current_vpid(vcpu)); } @@ -3397,11 +3416,11 @@ void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t = root_hpa, int root_level) struct kvm *kvm =3D vcpu->kvm; bool update_guest_cr3 =3D true; unsigned long guest_cr3; - u64 eptp; =20 if (enable_ept) { - eptp =3D construct_eptp(vcpu, root_hpa, root_level); - vmcs_write64(EPT_POINTER, eptp); + KVM_MMU_WARN_ON(root_to_sp(root_hpa) && + root_level !=3D root_to_sp(root_hpa)->role.level); + vmcs_write64(EPT_POINTER, construct_eptp(root_hpa)); =20 hv_track_root_tdp(vcpu, root_hpa); =20 --=20 2.51.0.470.ga7dc726c21-goog From nobody Thu Oct 2 07:44:01 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 50901258CCC for ; Fri, 19 Sep 2025 01:00:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758243607; cv=none; b=rhBw2qZDp/LcyKl4Z9/YR9pp4VTwczz8ri7b+NshxYSgfKu7H7JKaCefXky3jJTHGIleF7bB0bnsU06r9T6O2t6cRfLIppiPoX+DtA8NSH5np7FpFPTD4zFFzSCsDPEnakLjKUK1uK/IZGz5eL/BVwVLC4aKu2nKGPLs+30YEy8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758243607; c=relaxed/simple; bh=LsLpFAxR2r/kYmw7xHJtCSmeqll4la2sZk2Tqpzc9oc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=MEG7CvyjHtbDH8wM4i/n2+tqQaOP5q+FWD9LRtBUEDyYD8cxQY2yed6W06ZmmuxSiqtMjZFF+Y8TRInsuXtg2ryKOjrTPnHlUu5HFWmKVeILKAAPyKHv97ILNVx3Un8HbhU+kjf6wzxd1+WonvX/W9Wb4dtKFIc7f8wrUCWa51w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=QNaMHQKw; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="QNaMHQKw" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-32ec2211659so1547731a91.0 for ; Thu, 18 Sep 2025 18:00:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1758243606; x=1758848406; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=2SCjX/C9Pl/3F4W6bLgbAPLvNsr2cRYsD9OSEcrvFjM=; b=QNaMHQKwdDRuKv/nJSRn9hSD+jnscU1OLxtt6X3XdulK6AxSPy5K4tPHsU6yftI33L hF1PmpzG6I0Ge2VO88F5967IljV+ImmAAkkKyWbLuBjFIY94mcSnpBt4uvQiuli+H8/X gW71nsR/ggpF/MvOxljHCY9UXyRs24d3QY4SgL/GRPVkWWPt023gdm8XLe+CdTR/ZIDD jfulmXfdkczbib+sIXXjEBc5anNOQx4trgnUX6HioO4hxfU5wvaf66TqYcy+exIeo/M9 rdqkbWYBUbEbjhyycQmWIHiCPXrN3Cd/YuSxJIRlpksRZyU7qtGbs9KjtmQul6Ny4O7b PGhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1758243606; x=1758848406; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=2SCjX/C9Pl/3F4W6bLgbAPLvNsr2cRYsD9OSEcrvFjM=; b=KCO407yvjrv+VmGsNhJte8Gd55MSurLa96oFQ8g7R5V5tmfXHhTe9+cLFleda3UYKu pxzew6oJdB9/68PkNdN+Kv7lDLBuHJxnuGOBF9LUKFqRlSGAzG2I+NDYlwxiN3R6PMJk Vpsn/B9wpozW+jLq+8NGLSafH0PiDDFQr3tYBjeGFpIPBMjO7ljYMWCXofNPdm74NNpF wULiNzbVeZ0Hp89SoscSrN5eSbCy32EtOY0ptwlS0WqoH+G46YKnJ/yFtVPpE51A78Uk imCk+acNO3S/tZ945R2i9H0LWdEixPzJ8vjeewGH+j+pkLc6o0Skx6rOkKVXlDyMwka2 Dyng== X-Forwarded-Encrypted: i=1; AJvYcCVYXAjdxeKrzPYcU++99av+OBdYNGI2ES9OJJp65JfYvWaXn3CgXGXi6fO05XQbjrazHtVQdL68g+VOMPA=@vger.kernel.org X-Gm-Message-State: AOJu0Yzf+18DTMb7xBTHnI39n2GzN/J9/9TzQ6KbT+jc+zEsB5ZqG3SK mPFqLFqYhEuVcNgPrzeF5nNCCJcLecnc3gW7YALxM5UAhzSh6hYNJnEDgPryIsXGzc2cCLEuh0b oN/vRfg== X-Google-Smtp-Source: AGHT+IFUBVqY6omtVwJGu2LCApfEsfDNlkWkBfC9sU/rre8XJpYtigGsjXv/+phjEKVu5Y5A7dIXEO9/TmQ= X-Received: from pjbqi4.prod.google.com ([2002:a17:90b:2744:b0:327:50fa:eff9]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1e12:b0:32e:a59e:21c9 with SMTP id 98e67ed59e1d1-33098379102mr1492674a91.26.1758243605678; Thu, 18 Sep 2025 18:00:05 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 18 Sep 2025 17:59:51 -0700 In-Reply-To: <20250919005955.1366256-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250919005955.1366256-1-seanjc@google.com> X-Mailer: git-send-email 2.51.0.470.ga7dc726c21-goog Message-ID: <20250919005955.1366256-6-seanjc@google.com> Subject: [PATCH 5/9] KVM: nVMX: Add consistency check for TPR_THRESHOLD[31:4]!=0 without VID From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a missing consistency check on the TPR Threshold. Per the SDM If the "use TPR shadow" VM-execution control is 1 and the "virtual- interrupt delivery" VM-execution control is 0, bits 31:4 of the TPR threshold VM-execution control field must be 0. Note, nested_vmx_check_tpr_shadow_controls() bails early if "use TPR shadow" is 0. Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 253e93ced9dc..5ac7ad207ef7 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -555,6 +555,9 @@ static int nested_vmx_check_tpr_shadow_controls(struct = kvm_vcpu *vcpu, if (CC(!page_address_valid(vcpu, vmcs12->virtual_apic_page_addr))) return -EINVAL; =20 + if (CC(!nested_cpu_has_vid(vmcs12) && vmcs12->tpr_threshold >> 4)) + return -EINVAL; + return 0; } =20 --=20 2.51.0.470.ga7dc726c21-goog From nobody Thu Oct 2 07:44:01 2025 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 309D625BEF1 for ; Fri, 19 Sep 2025 01:00:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758243609; cv=none; b=CgPJK4JRUuIjCrJu7o8yVfprvExcffVs9whJNxKVrHAb3C58BU2ABW7IFWLHfqJ+3lPcdqdWyLfeRLwJCmP5FRXFB3RKLHv7cBHvJTW7arfD7zY3JOnDZsMxnxT+W36dIzq4niNFZRqCbBRVd3igdPMNbONTjeyv69Mq6XueVYg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758243609; c=relaxed/simple; bh=5TMk5+/OdEv6qy5r28RZbus9hXPlrKPCWHJzfl/drj0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=F8XaDP17gmjfv3CgqGFNm88EhD7V7DR2PIINNL5oFMpIcZzVVuhfzRjy4KB+KC5CgxM4QyP3iGp/onmbCaKsgvV/axAFDQix5b+2Kh3kuMBH1mz/9SHkCOsbrwkK+oemFO9hgR/xcjT2B46u1wbNnWfB/M7Z8DKGm2l5U7tLsfo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Z7Malv2d; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Z7Malv2d" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-b54fe1d849aso2054869a12.2 for ; Thu, 18 Sep 2025 18:00:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1758243607; x=1758848407; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=EXeqkUK7BZpzEGXbHHASAXnUJ0fsVfhCvA9M9Xum88E=; b=Z7Malv2d4Ym2VGBHoEq/BrldxltpTkaHqG2HqCJIHbjYWqgjoggreZMdWeu+qigUGw Z1oNbyoh4pNUzRSawZWzY6h5nidc9wao2xkagtcmM/oJpEtDjEkyL2uiSPYiDRefawLQ 06iWMpLzj1bCJ/VCH+xcc3KdR0YGOPzDTenQsOP5tlA388DmhslIqfFE0o8YubIqf1e4 N3vlRqnzcXUF2cf5/zeZ3p+9fETtDFiBa18INkdw6UDll9w1VtRp8YC0Mj83q229CgJ6 WDN3sMuG/QAPqVbn/SPX9A3zNoppXq96HpnfX6aPw798vncNdIqVV9Axb4s1iZ2i5I/S F+Lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1758243607; x=1758848407; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=EXeqkUK7BZpzEGXbHHASAXnUJ0fsVfhCvA9M9Xum88E=; b=v98EtyD0TMIcVJMl06JEf90VcPL4+6XLw0rfxBHG3Sq2B4UNS3MJAn31MXxtOGolfJ rDEHUOZD948ToqDshkRwKfwLzi7SQCcrCOFCvSyn93jXBwB5GZNXgGo7OhTtqZksqp1a OFYD1cyW93pB6NGB3rYFIe9zCqMEF+XWnnW/+4pSpPmF12x+hF+EJWhz40yUatTi0Shf VIB5eTBEj6NHfPwKC4Ah8NenHwj4wK8OpYir9bTUWvRuUxoJLs5bmbLl2/bp4fySHkPb ap4tHjkss/LdQlA3SrOpj0KkRkuQCg4fUPjZpAyEdc7rPhxxHwohfCeuGSBSL3KJkZ8h cE9A== X-Forwarded-Encrypted: i=1; AJvYcCW+24ntr3D5HBVrZSFb5aNF0x0L2Gaa5JgxF52P9ew9hImB8pmjJF67xzTU7AsOlJB+R+Dl6WKOgozM32w=@vger.kernel.org X-Gm-Message-State: AOJu0Yyhp/v5azXfnK/hAF98G0JaqmQkY6p2uUpscDM4aQ7/8005b468 6B8CrUGILFCQ4DVs2AjUx0bQVZVuZ0d7FvwJzMeWk9D2g58FMejfNKjtkXQwoEAhr6MglY+c4XM oNLmxFg== X-Google-Smtp-Source: AGHT+IFuwdL0e2irjXtVkdjRkTslOZU4HngmhB1oZFxTINPzEJyjeR2iQRAUtSAuEOfNTlToftODDWWwzwY= X-Received: from pjc6.prod.google.com ([2002:a17:90b:2f46:b0:329:d461:9889]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:329e:b0:24e:84c9:e986 with SMTP id adf61e73a8af0-2925d0dd333mr2318282637.15.1758243607520; Thu, 18 Sep 2025 18:00:07 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 18 Sep 2025 17:59:52 -0700 In-Reply-To: <20250919005955.1366256-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250919005955.1366256-1-seanjc@google.com> X-Mailer: git-send-email 2.51.0.470.ga7dc726c21-goog Message-ID: <20250919005955.1366256-7-seanjc@google.com> Subject: [PATCH 6/9] KVM: nVMX: Add consistency check for TSC_MULTIPLIER=0 From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a missing consistency check on the TSC Multiplier being '0'. Per the SDM: If the "use TSC scaling" VM-execution control is 1, the TSC-multiplier must not be zero. Fixes: d041b5ea9335 ("KVM: nVMX: Enable nested TSC scaling") Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 5ac7ad207ef7..eb838ebeff0f 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -2906,6 +2906,10 @@ static int nested_check_vm_execution_controls(struct= kvm_vcpu *vcpu, } } =20 + if (nested_cpu_has2(vmcs12, SECONDARY_EXEC_TSC_SCALING) && + CC(!vmcs12->tsc_multiplier)) + return -EINVAL; + return 0; } =20 --=20 2.51.0.470.ga7dc726c21-goog From nobody Thu Oct 2 07:44:01 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CDC1225A659 for ; Fri, 19 Sep 2025 01:00:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758243611; cv=none; b=gJtvoPn4Q1GdGKV1g2BhOWKGuvMVeD1USK6vQsqWV9iK4ESNJ5dzfVga5mBnZubklhSs2n8sYc1UfWpDafiQFEWlEJcYGxrLmNnhUBqb1NrVRTD3l2Xnojw+osv5S55z6L13yFXNoiVGs0ARnB+4iqLOLM8H1Crqt8JvyeJTzBU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758243611; c=relaxed/simple; bh=auDj64V/QQf5SopFijWR/IMUAn6qtFIRsS4N0dYQj5c=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=QWT5jeSaEsKXK+k1E2oclBt8C9JfvUwa/ZLJPOph5Drm6R1BdfQz7hR//4sKymQdOPI+IKuIcCC6f9WLXoh5epKDOHqiDnVfpFAJc4bL21kBdKUbS6givK6epnXYwo35/JS6ZwbRVfzXy2hwR4m6VUcKkh3Ixc2HF6MQZNj/zYc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=g1riA2h4; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="g1riA2h4" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-329cb4c3f78so1362492a91.2 for ; Thu, 18 Sep 2025 18:00:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1758243609; x=1758848409; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=OH/pNnK8EA9gjbd5XB2CxKFgoHiyKEJc22tQWHZ69/k=; b=g1riA2h4HqLsD09hXy0INwwZDKZXuGnm0I/C89gmyFZCIHrQ+cOg9pHcte6ibY2fL0 G4xwC+rMRJigEqcJA2kckmlPeqEN98QsYEl1mSxvXG8d/OJCB8Aw1PgojavtJG21egXx fjAv3iymVLUaK6yuFMp6Ty5snQsIhDa6Ou9xjwaMVQ4js8BaaFdtFPiYYYciFrMt5CeA Xb8z+9ZHoZIyoyveYeuptCLlmhlMMGsy8HjCsiTahpMfvCZEP9m0IEe8Ts7aVb/JxZwi 7vF/P3gaN2DlyP3/WJANUsjEMiaBNf2dO7U6iKDU4uG9+uLHh2rQjzuqmBtQQfvvEfBU 993A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1758243609; x=1758848409; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OH/pNnK8EA9gjbd5XB2CxKFgoHiyKEJc22tQWHZ69/k=; b=aEgKRygpM48aB6orMbqCGd2PDUHfC0xiFJMEf1v7cb+eb+hcL8MbZdprpx+S3ET8rh 3EyO6mTlvVdNyi/DmN1XL4pZilbus9dEI0HFs+USfI2QRPJKUxZM0bmWIGqsddZyEuyO tgQpC69BH47luzfI+AWLhMuHowoRjrKeQ2s5NLEw8wjoKY0rlk+C6aBH3FPTTKAYdLBL a4QCsR/Ccl9V4VNQjA310szPbHkDsu8WN1SsnKrCUo3GxFZgrG/awn0hEqS4qCHGuCYe 5I0oTxVbZWzkha/TriLIKuwPyhlEm99Bf+RzCxWIsvo3I01mx0Facr6YBgMTIrGwX52L yUkQ== X-Forwarded-Encrypted: i=1; AJvYcCU52FnZb7ZrARNmp9UIFQ8rxq9uyXPz4vTivcSrXx9pIw/hMH61XpPkv6kC0cTXpI7I5w7o2ZuT8Tak0VA=@vger.kernel.org X-Gm-Message-State: AOJu0YxOSP7s9/BvsYixBRZ+mIbzGwnli7Q/zsD1YmyF2VzDEIigHsNh y/bT4S7OG50d6fWL2Cj6lqqfC7bvSyAIo0MMShQNz0gt/t+z8VlWEkOWWAqxFq5/UVf2N+QEBqx dqETPJg== X-Google-Smtp-Source: AGHT+IHsi2EjszUT3jAjgKI3JLYJtiSogDVJ76Mn3rgKOAbzRdi9ZeWvwaxOp1oFOIYATZqbqd/20ux0O7M= X-Received: from pjbsp8.prod.google.com ([2002:a17:90b:52c8:b0:330:6513:c709]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3f08:b0:32e:2798:9064 with SMTP id 98e67ed59e1d1-3309838e23bmr1737178a91.35.1758243609216; Thu, 18 Sep 2025 18:00:09 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 18 Sep 2025 17:59:53 -0700 In-Reply-To: <20250919005955.1366256-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250919005955.1366256-1-seanjc@google.com> X-Mailer: git-send-email 2.51.0.470.ga7dc726c21-goog Message-ID: <20250919005955.1366256-8-seanjc@google.com> Subject: [PATCH 7/9] KVM: nVMX: Stuff vmcs02.TSC_MULTIPLIER early on for nested early checks From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" If KVM is doing "early" nested VM-Enter consistency checks and TSC scaling is supported, stuff vmcs02's TSC Multiplier early on to avoid getting a false positive VM-Fail due to trying to do VM-Enter with TSC_MULTIPLIER=3D0. To minimize complexity around L1 vs. L2 TSC, KVM sets the actual TSC Multiplier rather late during VM-Entry, i.e. may have '0' at the time of early consistency checks. If vmcs12 has TSC Scaling enabled, use the multiplier from vmcs12 so that nested early checks actually check vmcs12 state, otherwise throw in an arbitrary value of '1' (anything non-zero is legal). Fixes: d041b5ea9335 ("KVM: nVMX: Enable nested TSC scaling") Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index eb838ebeff0f..e3a94bf6d269 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -2352,6 +2352,13 @@ static void prepare_vmcs02_early_rare(struct vcpu_vm= x *vmx, else vmcs_write16(VIRTUAL_PROCESSOR_ID, vmx->vpid); } + + if (kvm_caps.has_tsc_control && nested_early_check) { + if (nested_cpu_has2(vmcs12, SECONDARY_EXEC_TSC_SCALING)) + vmcs_write64(TSC_MULTIPLIER, vmcs12->tsc_multiplier); + else + vmcs_write64(TSC_MULTIPLIER, 1); + } } =20 static void prepare_vmcs02_early(struct vcpu_vmx *vmx, struct loaded_vmcs = *vmcs01, --=20 2.51.0.470.ga7dc726c21-goog From nobody Thu Oct 2 07:44:01 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 630D1265CCB for ; Fri, 19 Sep 2025 01:00:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758243613; cv=none; b=WUM34qq48P92ekIV94+p+L2j/BA81gu+05xIbn836MJ5YP9Vj1LAsXvZ83CkfHw8UyFtfjg05Sy0v9dGI8IGOEyhU33r5SHj2qgYUwzeOEdUSKG6JzQdxqREdvjJwQ3arHuTQVqizUdWS7LZTm3j1qGQGJeIZLxqPBIdvRdVPYY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758243613; c=relaxed/simple; bh=EAAcphAPmqx0XNYCWxq3+wsF+A2zq9F9El6fEDAJCPk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=deHlsbK6TOQZ5UhS6jDhtJSdAU0b5EpPvXtAI8isHmLxvKE8QyU4zRRJEkkMC0C6uWG5cBoghqYEEsyZH4XFly55upuYc1TMumV1GL3ShsMUbJ4gv4ft9LNZuMX0pz6Vh1K9umZkp2yk0kvAKZ72UtMFUowhlfAqh8M5waeZCKI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=JoWB6KFZ; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="JoWB6KFZ" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-77241858ec1so1583622b3a.0 for ; Thu, 18 Sep 2025 18:00:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1758243611; x=1758848411; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=nC4lqy1VsaE/VYQOJkDaZdGDXi2NadHlztaUIfGpZG8=; b=JoWB6KFZy76/YcxDfSoX6uuPGiib7+tfb8Z01nThshonBR6eveH0GtQcbYipWlBKP5 vUjbcXiW6FM1j7UOfH3NriKlI7ETrTQqSW0xhgRK8k0tF0dNiRmXnFRAIMpb/Ny/p49b mahasYf8aYhH9E8e5MA00AGkEQMHzApHWPHTdLK43SPL3IphGdb8JULigLMSi7lLdzBI HyGbzXwtkrPmTOl6zoEoJFbXoGVYxQvVMXOniq3p+qi56b90kS0n8Ilv3Xpi3eLclhy5 rJhsTG6mNK8KycnhvFUvV5E9Va0ofC6jyf/bqH7titkHJwKbCnOmBAvnXDJzkCoNVvt9 eqZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1758243611; x=1758848411; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=nC4lqy1VsaE/VYQOJkDaZdGDXi2NadHlztaUIfGpZG8=; b=LbRYKVaEXQ9lE2L7ULHarEMVIxtv6g+BkE4mddMhl/T7RBZ3ZWZrBwB2ntFeHwRNji 18WO85nyM7sFeB7Uq2CObl2Pt0sjJm7B94lPK45Av5tBb0Xb9nhS3lA9ETxfGhoeTKsN b7oyjaKdV9CVPSTydhBfWdFU0LWmrNqZdxd2hJPLkuy4N90snfoKCE8WVH57aK8wiYiu W72KJpXkM9VbV/p0KaaQFocOzbtTC06HjqT0o0mDZMH3YenL2aXSNOu8hY5JOe65Y4/t XiLEo1MLFXQ3lWYVAyKl6jnD6iFPTy+CLaMKALecJ2+LjMcxBvCwfd66I2RVqhjRs1qR XeKg== X-Forwarded-Encrypted: i=1; AJvYcCXG9gO8AzfIB46HLFf1u1EucMYEuGGKqcgA1Cfyb1Pfauo9LBC3nNn11OFxD0rttGRdc2S85ikUGJynci0=@vger.kernel.org X-Gm-Message-State: AOJu0YzOtVgZxtrOFU9zjVbUpN8BneWSH0s06hoFqhgZn3aLyhyBJNmq pDa1kquSm6FL2uLUo4cFfEBWewENWnlB655gFX/Ipi+RNhcWodoKH9DX5tMke/8EtvsEHxdA03+ EKD6JJw== X-Google-Smtp-Source: AGHT+IEWT+b51PWDhVROnjj4MNHmNfWEi53pHuIcr7HOX8gmTcp1M+N2doO80KvMuo3RHEvn5affyawMJrw= X-Received: from pgdy24.prod.google.com ([2002:a65:6418:0:b0:b55:1604:9969]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:3287:b0:263:b547:d0a6 with SMTP id adf61e73a8af0-2926ec18aa0mr2573153637.36.1758243610714; Thu, 18 Sep 2025 18:00:10 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 18 Sep 2025 17:59:54 -0700 In-Reply-To: <20250919005955.1366256-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250919005955.1366256-1-seanjc@google.com> X-Mailer: git-send-email 2.51.0.470.ga7dc726c21-goog Message-ID: <20250919005955.1366256-9-seanjc@google.com> Subject: [PATCH 8/9] KVM: nVMX: Remove support for "early" consistency checks via hardware From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove nested_early_check and all associated code, as it's quite obviously not being used or tested (it's been broken for 4+ years without a single bug report). More importantly, KVM's software-based consistency checks have matured since the option to do hardware-based checks was added; KVM appears to be missing only _one_ consistency check, on vTPR. And even *more* importantly, that consistency check can't be prevented by an early hardware check due to L1 being able to modify the virtual APIC at any time, i.e. there's an inherent TOCTOU flaw that could cause KVM to "miss" a consistency check VM-Fail, regardless of whether the check is performed by software or by hardware. In other words, KVM _must_ be able to unwind from a late VM-Fail (which was a big motivation for doing early checks). I.e. now that KVM provides (almost) all necessary consistency checks, what's really needed is a way to detect missing checks in KVM, not a way to avoid having to unwind from a late VM-Fail. And that can be done much more simply, e.g. by an simple module param to guard a WARN (which, sadly, must be off-by-default to avoid splats due to the aforementioned TOCTOU issue). For all intents and purposes, this reverts commit 52017608da33 ("KVM: nVMX: add option to perform early consistency checks via H/W"). Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 130 ++++---------------------------------- 1 file changed, 12 insertions(+), 118 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index e3a94bf6d269..a1ffaccf317d 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -23,9 +23,6 @@ static bool __read_mostly enable_shadow_vmcs =3D 1; module_param_named(enable_shadow_vmcs, enable_shadow_vmcs, bool, S_IRUGO); =20 -static bool __read_mostly nested_early_check =3D 0; -module_param(nested_early_check, bool, S_IRUGO); - #define CC KVM_NESTED_VMENTER_CONSISTENCY_CHECK =20 /* @@ -2280,13 +2277,6 @@ static void prepare_vmcs02_constant_state(struct vcp= u_vmx *vmx) return; vmx->nested.vmcs02_initialized =3D true; =20 - /* - * If early consistency checks are enabled, stuff the EPT Pointer with - * a dummy *legal* value to avoid false positives on bad control state. - */ - if (enable_ept && nested_early_check) - vmcs_write64(EPT_POINTER, VMX_EPTP_MT_WB | VMX_EPTP_PWL_4); - if (vmx->ve_info) vmcs_write64(VE_INFORMATION_ADDRESS, __pa(vmx->ve_info)); =20 @@ -2352,13 +2342,6 @@ static void prepare_vmcs02_early_rare(struct vcpu_vm= x *vmx, else vmcs_write16(VIRTUAL_PROCESSOR_ID, vmx->vpid); } - - if (kvm_caps.has_tsc_control && nested_early_check) { - if (nested_cpu_has2(vmcs12, SECONDARY_EXEC_TSC_SCALING)) - vmcs_write64(TSC_MULTIPLIER, vmcs12->tsc_multiplier); - else - vmcs_write64(TSC_MULTIPLIER, 1); - } } =20 static void prepare_vmcs02_early(struct vcpu_vmx *vmx, struct loaded_vmcs = *vmcs01, @@ -3229,84 +3212,6 @@ static int nested_vmx_check_guest_state(struct kvm_v= cpu *vcpu, return 0; } =20 -static int nested_vmx_check_vmentry_hw(struct kvm_vcpu *vcpu) -{ - struct vcpu_vmx *vmx =3D to_vmx(vcpu); - unsigned long cr3, cr4; - bool vm_fail; - - if (!nested_early_check) - return 0; - - if (vmx->msr_autoload.host.nr) - vmcs_write32(VM_EXIT_MSR_LOAD_COUNT, 0); - if (vmx->msr_autoload.guest.nr) - vmcs_write32(VM_ENTRY_MSR_LOAD_COUNT, 0); - - preempt_disable(); - - vmx_prepare_switch_to_guest(vcpu); - - /* - * Induce a consistency check VMExit by clearing bit 1 in GUEST_RFLAGS, - * which is reserved to '1' by hardware. GUEST_RFLAGS is guaranteed to - * be written (by prepare_vmcs02()) before the "real" VMEnter, i.e. - * there is no need to preserve other bits or save/restore the field. - */ - vmcs_writel(GUEST_RFLAGS, 0); - - cr3 =3D __get_current_cr3_fast(); - if (unlikely(cr3 !=3D vmx->loaded_vmcs->host_state.cr3)) { - vmcs_writel(HOST_CR3, cr3); - vmx->loaded_vmcs->host_state.cr3 =3D cr3; - } - - cr4 =3D cr4_read_shadow(); - if (unlikely(cr4 !=3D vmx->loaded_vmcs->host_state.cr4)) { - vmcs_writel(HOST_CR4, cr4); - vmx->loaded_vmcs->host_state.cr4 =3D cr4; - } - - vm_fail =3D __vmx_vcpu_run(vmx, (unsigned long *)&vcpu->arch.regs, - __vmx_vcpu_run_flags(vmx)); - - if (vmx->msr_autoload.host.nr) - vmcs_write32(VM_EXIT_MSR_LOAD_COUNT, vmx->msr_autoload.host.nr); - if (vmx->msr_autoload.guest.nr) - vmcs_write32(VM_ENTRY_MSR_LOAD_COUNT, vmx->msr_autoload.guest.nr); - - if (vm_fail) { - u32 error =3D vmcs_read32(VM_INSTRUCTION_ERROR); - - preempt_enable(); - - trace_kvm_nested_vmenter_failed( - "early hardware check VM-instruction error: ", error); - WARN_ON_ONCE(error !=3D VMXERR_ENTRY_INVALID_CONTROL_FIELD); - return 1; - } - - /* - * VMExit clears RFLAGS.IF and DR7, even on a consistency check. - */ - if (hw_breakpoint_active()) - set_debugreg(__this_cpu_read(cpu_dr7), 7); - local_irq_enable(); - preempt_enable(); - - /* - * A non-failing VMEntry means we somehow entered guest mode with - * an illegal RIP, and that's just the tip of the iceberg. There - * is no telling what memory has been modified or what state has - * been exposed to unknown code. Hitting this all but guarantees - * a (very critical) hardware issue. - */ - WARN_ON(!(vmcs_read32(VM_EXIT_REASON) & - VMX_EXIT_REASONS_FAILED_VMENTRY)); - - return 0; -} - #ifdef CONFIG_KVM_HYPERV static bool nested_get_evmcs_page(struct kvm_vcpu *vcpu) { @@ -3557,22 +3462,18 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_= mode(struct kvm_vcpu *vcpu, vmx->nested.pre_vmenter_bndcfgs =3D vmcs_read64(GUEST_BNDCFGS); =20 /* - * Overwrite vmcs01.GUEST_CR3 with L1's CR3 if EPT is disabled *and* - * nested early checks are disabled. In the event of a "late" VM-Fail, - * i.e. a VM-Fail detected by hardware but not KVM, KVM must unwind its - * software model to the pre-VMEntry host state. When EPT is disabled, - * GUEST_CR3 holds KVM's shadow CR3, not L1's "real" CR3, which causes - * nested_vmx_restore_host_state() to corrupt vcpu->arch.cr3. Stuffing - * vmcs01.GUEST_CR3 results in the unwind naturally setting arch.cr3 to - * the correct value. Smashing vmcs01.GUEST_CR3 is safe because nested - * VM-Exits, and the unwind, reset KVM's MMU, i.e. vmcs01.GUEST_CR3 is - * guaranteed to be overwritten with a shadow CR3 prior to re-entering - * L1. Don't stuff vmcs01.GUEST_CR3 when using nested early checks as - * KVM modifies vcpu->arch.cr3 if and only if the early hardware checks - * pass, and early VM-Fails do not reset KVM's MMU, i.e. the VM-Fail - * path would need to manually save/restore vmcs01.GUEST_CR3. + * Overwrite vmcs01.GUEST_CR3 with L1's CR3 if EPT is disabled. In the + * event of a "late" VM-Fail, i.e. a VM-Fail detected by hardware but + * not KVM, KVM must unwind its software model to the pre-VM-Entry host + * state. When EPT is disabled, GUEST_CR3 holds KVM's shadow CR3, not + * L1's "real" CR3, which causes nested_vmx_restore_host_state() to + * corrupt vcpu->arch.cr3. Stuffing vmcs01.GUEST_CR3 results in the + * unwind naturally setting arch.cr3 to the correct value. Smashing + * vmcs01.GUEST_CR3 is safe because nested VM-Exits, and the unwind, + * reset KVM's MMU, i.e. vmcs01.GUEST_CR3 is guaranteed to be + * overwritten with a shadow CR3 prior to re-entering L1. */ - if (!enable_ept && !nested_early_check) + if (!enable_ept) vmcs_writel(GUEST_CR3, vcpu->arch.cr3); =20 vmx_switch_vmcs(vcpu, &vmx->nested.vmcs02); @@ -3585,11 +3486,6 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_m= ode(struct kvm_vcpu *vcpu, return NVMX_VMENTRY_KVM_INTERNAL_ERROR; } =20 - if (nested_vmx_check_vmentry_hw(vcpu)) { - vmx_switch_vmcs(vcpu, &vmx->vmcs01); - return NVMX_VMENTRY_VMFAIL; - } - if (nested_vmx_check_guest_state(vcpu, vmcs12, &entry_failure_code)) { exit_reason.basic =3D EXIT_REASON_INVALID_STATE; @@ -5038,12 +4934,10 @@ void __nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32= vm_exit_reason, /* * The only expected VM-instruction error is "VM entry with * invalid control field(s)." Anything else indicates a - * problem with L0. And we should never get here with a - * VMFail of any type if early consistency checks are enabled. + * problem with L0. */ WARN_ON_ONCE(vmcs_read32(VM_INSTRUCTION_ERROR) !=3D VMXERR_ENTRY_INVALID_CONTROL_FIELD); - WARN_ON_ONCE(nested_early_check); } =20 /* --=20 2.51.0.470.ga7dc726c21-goog From nobody Thu Oct 2 07:44:01 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 48B9026D4F9 for ; Fri, 19 Sep 2025 01:00:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758243616; cv=none; b=USNT19KIH1HUjcA2/dmRNNxmv1Yo+10CgNpPah6w+LCXk4/NSJyRtlSMGrRn+KbrxA4n4QlLoqF+j9UeQ7iVx1MxRJ/rbjiRvjGnXWsTTUXD6UWqp0EBJJGNTZ1LqIym71OAzgOTmXtwVxlZkgAaXLhuJq1dmkx08N/OTW/EpZc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758243616; c=relaxed/simple; bh=Z4EL755VnzBscigAIdyatLEqCV+FJKoinGi/veRQqjM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=BbzD2IwKLv5NZs5GtqTx+Llt8gc3g7eaKMqLzzir6SVHT4i0IkNq4WiIm47ti2A5Jd9+AjgNk1f9sEUZcQbKa1OrvXd/SvXrcZC7jMgokg5gYXPTFrhS0VE1rkgayMiwvhiFOIPflYmIPqu5nWfhbttPreIbkz0GZJZj3ip9FUc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=qoPk0s8H; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="qoPk0s8H" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-32e09eaf85dso1863924a91.1 for ; Thu, 18 Sep 2025 18:00:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1758243613; x=1758848413; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=IwTU2RLfH1ndjCsGiuua5mF+21jUufiSblpqQgvpKww=; b=qoPk0s8HK9bxDaJ4WkHOvarSzrhP8DemxqEfQqKO933jGsdSfyy/B8z7qFkWzGDuQS F5XSmUw3E5qkox0ESC1/CYSFqsU33BN0pdFGAt1OINbYtT+HcXrjTK171vwaz4Mo0kmY 5rkJM5NC2BDWJ6/Lt6vw0U2PV/VY0Ev8qIfCME3305xWIF2v0+4lkfcD2oDHxdWVkxMD ZzuDa7DkaALHTJim7Pp1PHzHvX8FDVKTvbRn9HtyJqKYax4uzbkxlvKHggFkPizIbqW+ OkEDjU1G4KsWafeJfDbrEbtb1idgqQzUPgOn2vpICjy4ojF70JSzwQu9nubB544t7YZv dLYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1758243613; x=1758848413; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=IwTU2RLfH1ndjCsGiuua5mF+21jUufiSblpqQgvpKww=; b=ueM61Mg9PVa7pUi6fuqoUQJei5CLnB1k7kgh4clO5rlH+NUkdjVX9ALzNOwaPlh2EX UzlUdIdi5S3bX+1/S64hM/OlcM8Ge3Pfde8w7FysOQwGEuIgVDYEQnUx0I4IQYNjyADd ErpcyZMlgAi1sJfq4jjys/C345+lRxB1GDRT10nyqBKNchbKZEahA/KM0jNGq0mbQ4ug IWVJuB7AHTAhldOyHQYk9FQv/8Cf6xnlu0PMXmMXNj2TS5J6m9WIbFEv3bbJsLAd3TkV DB1Vn6fT094UlJfBG3FjIaGDvs9KJBAX0MBS7qwKuzUFAtfLuh+wCoGtt8EahcQ2Tpbd s3HQ== X-Forwarded-Encrypted: i=1; AJvYcCUn7szgThyU23tVo/AOTmYuTztPpHwgnsrTgKzKhwPeMK7YwkUY8XXBJDmlNyO/SXkVK5TJgBsqUlPy+zM=@vger.kernel.org X-Gm-Message-State: AOJu0Yxsu9J14rhE3H+ln+ErAwRHgvH7VpbyLuP9hwFLTVH13yEPlI5R TKu/84THW3+oq1FEgJzCvZXfZxaKldxXFXUSghhDEcb0av1hQ2o3ZN5pIPZsMKAM53DqiPNYGCp rSvQzsA== X-Google-Smtp-Source: AGHT+IEAZT+E47BnTePmN6hFXJmIPS6TcZZYUv4vZ7Anz6HCUr6zaXKBeKpHvYaaVVYa3EzZIUzZoF+4fto= X-Received: from pjev9.prod.google.com ([2002:a17:90a:e09:b0:32e:ae63:2947]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:e703:b0:32d:17ce:49de with SMTP id 98e67ed59e1d1-33097fdd577mr1493427a91.4.1758243613562; Thu, 18 Sep 2025 18:00:13 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 18 Sep 2025 17:59:55 -0700 In-Reply-To: <20250919005955.1366256-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250919005955.1366256-1-seanjc@google.com> X-Mailer: git-send-email 2.51.0.470.ga7dc726c21-goog Message-ID: <20250919005955.1366256-10-seanjc@google.com> Subject: [PATCH 9/9] KVM: nVMX: Add an off-by-default module param to WARN on missed consistency checks From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add an off-by-default param, "warn_on_missed_cc", to have KVM WARN on a missed VMX Consistency Check on nested VM-Enter, specifically so that KVM developers and maintainers can more easily detect missing checks. KVM's goal/intent is that KVM detect *all* VM-Fail conditions in software, as relying on hardware leads to false passes when KVM's nested support is a subset of hardware support, e.g. see commit 095686e6fcb4 ("KVM: nVMX: Check vmcs12->guest_ia32_debugctl on nested VM-Enter"). With one notable exception, KVM now detects all VM-Fail scenarios for which there is known test coverage, i.e. KVM developers can enable the param and expect a clean run, and thus can use the param to detect missed checks, e.g. when enabling new features, when writing new tests, etc. The one exception is an unfortunate consistency check on vTPR. Because the vTPR for L2 comes from the virtual APIC page provided by L1, L2's vTPR is fully writable at all times, i.e. is inherently subject to TOCTOU issues with respect to checks in software versus consumption in hardware. Further complicating matters is KVM's deferred handling of vmcs12 pages when loading nested state; KVM flat out cannot check vTPR during KVM_SET_NESTED_STATE without breaking setups that do on-demand paging, e.g. for live migration and/or live update. To fudge around the vTPR issue, add a "late" controls check for vTPR and also treat an invalid virtual APIC as VM-Fail, but gate the check on warn_on_missed_cc being enabled to avoid unwanted false positives, i.e. to avoid breaking KVM in production. Cc: Jim Mattson Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 43 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index a1ffaccf317d..a9f48493ad72 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -23,6 +23,9 @@ static bool __read_mostly enable_shadow_vmcs =3D 1; module_param_named(enable_shadow_vmcs, enable_shadow_vmcs, bool, S_IRUGO); =20 +static bool __ro_after_init warn_on_missed_cc; +module_param(warn_on_missed_cc, bool, 0444); + #define CC KVM_NESTED_VMENTER_CONSISTENCY_CHECK =20 /* @@ -3011,6 +3014,38 @@ static int nested_vmx_check_controls(struct kvm_vcpu= *vcpu, return 0; } =20 +static int nested_vmx_check_controls_late(struct kvm_vcpu *vcpu, + struct vmcs12 *vmcs12) +{ + void *vapic =3D to_vmx(vcpu)->nested.virtual_apic_map.hva; + u32 vtpr =3D vapic ? (*(u32 *)(vapic + APIC_TASKPRI)) >> 4 : 0; + + /* + * Don't bother with the consistency checks if KVM isn't configured to + * WARN on missed consistency checks, as KVM needs to rely on hardware + * to fully detect an illegal vTPR vs. TRP Threshold combination due to + * the vTPR being writable by L1 at all times (it's an in-memory value, + * not a VMCS field). I.e. even if the check passes now, it might fail + * at the actual VM-Enter. + * + * Keying off the module param also allows treating an invalid vAPIC + * mapping as a consistency check failure without increasing the risk + * of breaking a "real" VM. + */ + if (!warn_on_missed_cc) + return 0; + + if ((exec_controls_get(to_vmx(vcpu)) & CPU_BASED_TPR_SHADOW) && + nested_cpu_has(vmcs12, CPU_BASED_TPR_SHADOW) && + !nested_cpu_has_vid(vmcs12) && + !nested_cpu_has2(vmcs12, SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES) && + (CC(!vapic) || + CC((vmcs12->tpr_threshold & GENMASK(3, 0)) > (vtpr & GENMASK(3, 0)))= )) + return -EINVAL; + + return 0; +} + static int nested_vmx_check_address_space_size(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12) { @@ -3486,6 +3521,11 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_m= ode(struct kvm_vcpu *vcpu, return NVMX_VMENTRY_KVM_INTERNAL_ERROR; } =20 + if (nested_vmx_check_controls_late(vcpu, vmcs12)) { + vmx_switch_vmcs(vcpu, &vmx->vmcs01); + return NVMX_VMENTRY_VMFAIL; + } + if (nested_vmx_check_guest_state(vcpu, vmcs12, &entry_failure_code)) { exit_reason.basic =3D EXIT_REASON_INVALID_STATE; @@ -4938,6 +4978,9 @@ void __nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 v= m_exit_reason, */ WARN_ON_ONCE(vmcs_read32(VM_INSTRUCTION_ERROR) !=3D VMXERR_ENTRY_INVALID_CONTROL_FIELD); + + /* VM-Fail at VM-Entry means KVM missed a consistency check. */ + WARN_ON_ONCE(warn_on_missed_cc); } =20 /* --=20 2.51.0.470.ga7dc726c21-goog