From nobody Sat Apr 18 07:42:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C199ACCA480 for ; Fri, 15 Jul 2022 23:00:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230513AbiGOXA3 (ORCPT ); Fri, 15 Jul 2022 19:00:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59650 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230063AbiGOXAX (ORCPT ); Fri, 15 Jul 2022 19:00:23 -0400 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BAE1D49B42 for ; Fri, 15 Jul 2022 16:00:21 -0700 (PDT) Received: by mail-pl1-x649.google.com with SMTP id n2-20020a170902e54200b0016c16832828so2718223plf.5 for ; Fri, 15 Jul 2022 16:00:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=4mRUe9k6N29Y8TY671ys1SOzt46CXKbNlnzYh7VBMGg=; b=BY+ofGKbdhxI2uO5Nyqd5dE57NYG6j6ZefdgDGiAMMy15vVMXVMCrAxkz/HmhpImK+ Grv+x99LKg2/2a7c8R/phTxEvCikiuo0xcEF+jCO7cRYnO7Jz86w6MRWzcHiUpAcmpse 0iTfrjC1O28pwL1VrBCVYAXfp8Eo9G1UJBI8jOC23NXXn0vDG7ZLCMOn05PaeAjhfClh wO0w2V7+MZ1Bl/hlLPcPW45z8QlYCjNcKvOAYamGbWsL9h7BLrpWxBRhmOYPC/bkhtLM qUA7RMiIQSituv4YUDa3XRxArvCtVaN2kUHvfSpAQ6HYdnbb4MjeiIsma9YMnf6/n2C4 1sNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=4mRUe9k6N29Y8TY671ys1SOzt46CXKbNlnzYh7VBMGg=; b=bf85g6GsIyms0VG66SpJw2VXTdHnbKarxSY8LiiGxztlrdUOxbE9QKqCmvQjJ53y9v ulswkVQeYXWK9lxSOky3/XtjRPPeo1KK2HEsQn1BphyN+8sRyROc7gaYxxaxwSo7AgEK 1MOFU+kHRcTjmgNQ9r1pObuB6z2VILOBtJVQKfmDoUU2CH1eTuADf/UxSWxJOv96MjN2 o/UsQY19BA0I74BwgxMqRfihUGfTt5YXSVvLW7Nbr8BkaLPU4JEfbbKBmStNnwcXTP+6 fcpCEe6QTCO93M3NhjLBpxenMGLybQo20qpebATYr3I7PgRhJ0RhDti6AkAyAhwTPu6V beYg== X-Gm-Message-State: AJIora84Af7EnzBJiV7ZZlI/BPVnArKMavU6b9wGRiMS0bItshkfFOlF bjmWyeFl9V5kYiLPhJ5YOg5d2yabh9M= X-Google-Smtp-Source: AGRyM1smbVwsZvafhg6YAk0nSxXgCOlfpSnfKN9p9HtlEhf/jQcGocWbnbcv1Ozn+bbDrtBXCdl//YT/YPo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:d54b:b0:16b:eea4:77da with SMTP id z11-20020a170902d54b00b0016beea477damr15632782plf.45.1657926021275; Fri, 15 Jul 2022 16:00:21 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 15 Jul 2022 23:00:13 +0000 In-Reply-To: <20220715230016.3762909-1-seanjc@google.com> Message-Id: <20220715230016.3762909-2-seanjc@google.com> Mime-Version: 1.0 References: <20220715230016.3762909-1-seanjc@google.com> X-Mailer: git-send-email 2.37.0.170.g444d1eabd0-goog Subject: [PATCH 1/4] KVM: x86: Reject loading KVM if host.PAT[0] != WB From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Reject KVM if entry '0' in the host's IA32_PAT MSR is not programmed to writeback (WB) memtype. KVM subtly relies on IA32_PAT entry '0' to be programmed to WB by leaving the PAT bits in shadow paging and NPT SPTEs as '0'. If something other than WB is in PAT[0], at _best_ guests will suffer very poor performance, and at worst KVM will crash the system by breaking cache-coherency expecations (e.g. using WC for guest memory). Signed-off-by: Sean Christopherson Reviewed-by: Maxim Levitsky --- arch/x86/kvm/x86.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index f389691d8c04..12199c40f2bc 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9141,6 +9141,7 @@ static struct notifier_block pvclock_gtod_notifier = =3D { int kvm_arch_init(void *opaque) { struct kvm_x86_init_ops *ops =3D opaque; + u64 host_pat; int r; =20 if (kvm_x86_ops.hardware_enable) { @@ -9179,6 +9180,20 @@ int kvm_arch_init(void *opaque) goto out; } =20 + /* + * KVM assumes that PAT entry '0' encodes WB memtype and simply zeroes + * the PAT bits in SPTEs. Bail if PAT[0] is programmed to something + * other than WB. Note, EPT doesn't utilize the PAT, but don't bother + * with an exception. PAT[0] is set to WB on RESET and also by the + * kernel, i.e. failure indicates a kernel bug or broken firmware. + */ + if (rdmsrl_safe(MSR_IA32_CR_PAT, &host_pat) || + (host_pat & GENMASK(2, 0)) !=3D 6) { + pr_err("kvm: host PAT[0] is not WB\n"); + r =3D -EIO; + goto out; + } + r =3D -ENOMEM; =20 x86_emulator_cache =3D kvm_alloc_emulator_cache(); --=20 2.37.0.170.g444d1eabd0-goog From nobody Sat Apr 18 07:42:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B876CCA480 for ; Fri, 15 Jul 2022 23:00:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231231AbiGOXAy (ORCPT ); Fri, 15 Jul 2022 19:00:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59686 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230114AbiGOXAY (ORCPT ); Fri, 15 Jul 2022 19:00:24 -0400 Received: from mail-pl1-x64a.google.com (mail-pl1-x64a.google.com [IPv6:2607:f8b0:4864:20::64a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 452FC4AD69 for ; Fri, 15 Jul 2022 16:00:23 -0700 (PDT) Received: by mail-pl1-x64a.google.com with SMTP id b10-20020a170902d50a00b0016c56d1f90fso2790751plg.21 for ; Fri, 15 Jul 2022 16:00:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=V+ijd0IpTmqqqLjto4/xJJjuTl9mPDgkHleQw8xA900=; b=g2f2U6dAMh2svtMNa0Gg+5Wo0SupEJFUQDZb4iOR1xe/N8YGAQMOLjhFPelZfKgo/c ey5P2g+Qwc12BJCqJLeZZ27863S5UOGpryHde4e5Z1+5/gk0cH07cYblwVoMeGKubPug Q5SWn8SZOfKaTQKlEECe/t/qWODjgKTW49O97PDpDPhy/9oa1bTWWSYNTjALE9yfqGFH D1aT+2F0D63s/iwgfYnUdeFxR+XZXlEIlp0EgfgjQsue3DRFxWn70QkWMilm+nOsOXu+ 67VYZ/MdkbCDsj6eJon214a52MGGbx3XX2e2rZwMTAy0CE4OHw8PNeudcRN+Bf2KWINS Y24w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=V+ijd0IpTmqqqLjto4/xJJjuTl9mPDgkHleQw8xA900=; b=xYMw4rkKze0OqCcMU723x6BvXiRAqHcl+uY095K0nnB3/h/j/3+1g65FX826vbb0xe ccvWvRJ/h4+0cBtFWUzmizrPBwc1TYx6k21ntUS+MhqWhMM9yVljWoXZCb2bRoUFSIGq yCQ6B259VD/7Jsl/MZrdf1KhFvLzCUSDAWiF1h8Rm704IvU7DX16hN0nNdJufG3G9xXH mkXZ2UtDxWFRSes3QJQadxDmBSD97m+tonPTtra8aMqG+3LK1ojI9hYXU8zWRkUWnzMN 65bztIECb2NRGgJwqHHPFAF2uS526P5UkzuVGHFAfngZ98TTz2CFQP3E6T6+vNw5nvrm QBRQ== X-Gm-Message-State: AJIora/GunTFDEmoobVrCKVK6yySHmvanYj52yB7r2YHWFjzaLufr7gd CEKzLrb3mSKdw5NcUh+BChnEpMeW5xs= X-Google-Smtp-Source: AGRyM1tWkRhKexMwNka8pMZLwhRDlIvKyPaqRa57ONwmleNE0JM3aPXeIWSQNInSCOIPyAw8GygZDNqnLQU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a62:6d05:0:b0:528:99a2:b10 with SMTP id i5-20020a626d05000000b0052899a20b10mr16101370pfc.72.1657926022849; Fri, 15 Jul 2022 16:00:22 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 15 Jul 2022 23:00:14 +0000 In-Reply-To: <20220715230016.3762909-1-seanjc@google.com> Message-Id: <20220715230016.3762909-3-seanjc@google.com> Mime-Version: 1.0 References: <20220715230016.3762909-1-seanjc@google.com> X-Mailer: git-send-email 2.37.0.170.g444d1eabd0-goog Subject: [PATCH 2/4] KVM: x86: Drop unnecessary goto+label in kvm_arch_init() From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Return directly if kvm_arch_init() detects an error before doing any real work, jumping through a label obfuscates what's happening and carries the unnecessary risk of leaving 'r' uninitialized. No functional change intended. Signed-off-by: Sean Christopherson Reviewed-by: Maxim Levitsky --- arch/x86/kvm/x86.c | 24 ++++++++---------------- 1 file changed, 8 insertions(+), 16 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 12199c40f2bc..41aa3137665c 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9146,21 +9146,18 @@ int kvm_arch_init(void *opaque) =20 if (kvm_x86_ops.hardware_enable) { pr_err("kvm: already loaded vendor module '%s'\n", kvm_x86_ops.name); - r =3D -EEXIST; - goto out; + return -EEXIST; } =20 if (!ops->cpu_has_kvm_support()) { pr_err_ratelimited("kvm: no hardware support for '%s'\n", ops->runtime_ops->name); - r =3D -EOPNOTSUPP; - goto out; + return -EOPNOTSUPP; } if (ops->disabled_by_bios()) { pr_err_ratelimited("kvm: support for '%s' disabled by bios\n", ops->runtime_ops->name); - r =3D -EOPNOTSUPP; - goto out; + return -EOPNOTSUPP; } =20 /* @@ -9170,14 +9167,12 @@ int kvm_arch_init(void *opaque) */ if (!boot_cpu_has(X86_FEATURE_FPU) || !boot_cpu_has(X86_FEATURE_FXSR)) { printk(KERN_ERR "kvm: inadequate fpu\n"); - r =3D -EOPNOTSUPP; - goto out; + return -EOPNOTSUPP; } =20 if (IS_ENABLED(CONFIG_PREEMPT_RT) && !boot_cpu_has(X86_FEATURE_CONSTANT_T= SC)) { pr_err("RT requires X86_FEATURE_CONSTANT_TSC\n"); - r =3D -EOPNOTSUPP; - goto out; + return -EOPNOTSUPP; } =20 /* @@ -9190,21 +9185,19 @@ int kvm_arch_init(void *opaque) if (rdmsrl_safe(MSR_IA32_CR_PAT, &host_pat) || (host_pat & GENMASK(2, 0)) !=3D 6) { pr_err("kvm: host PAT[0] is not WB\n"); - r =3D -EIO; - goto out; + return -EIO; } =20 - r =3D -ENOMEM; - x86_emulator_cache =3D kvm_alloc_emulator_cache(); if (!x86_emulator_cache) { pr_err("kvm: failed to allocate cache for x86 emulator\n"); - goto out; + return -ENOMEM; } =20 user_return_msrs =3D alloc_percpu(struct kvm_user_return_msrs); if (!user_return_msrs) { printk(KERN_ERR "kvm: failed to allocate percpu kvm_user_return_msrs\n"); + r =3D -ENOMEM; goto out_free_x86_emulator_cache; } kvm_nr_uret_msrs =3D 0; @@ -9235,7 +9228,6 @@ int kvm_arch_init(void *opaque) free_percpu(user_return_msrs); out_free_x86_emulator_cache: kmem_cache_destroy(x86_emulator_cache); -out: return r; } =20 --=20 2.37.0.170.g444d1eabd0-goog From nobody Sat Apr 18 07:42:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33AD5C433EF for ; Fri, 15 Jul 2022 23:01:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231530AbiGOXA7 (ORCPT ); Fri, 15 Jul 2022 19:00:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59746 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230241AbiGOXA1 (ORCPT ); Fri, 15 Jul 2022 19:00:27 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 296844D4EB for ; Fri, 15 Jul 2022 16:00:25 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-31cbcba2f28so49521767b3.19 for ; Fri, 15 Jul 2022 16:00:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=yUjqGaXxNRIkWFGwl+2pUdgstDXS94lxvTblB9yzYuE=; b=d0HtqcngCGEWckrmfolh8CY2i0XJnzrFb5U6PGDeVR1h0nBWTx0oQovQksQedcqaw5 rNIEvehYP/e2WByw7UXCle8v5tnX9eUF2ixPX/LeaN32FU0RS43PhyHsktjm7mRl9kCx doOlYb7OJsbiz3CvdzNyLZJfaxmHMCvjtfJF3wTxPbYjHHoz9N2QJslDF/oOxCp1ysxU sEAKUl41bbj251CIbRBdBNFCB3nPXrHNA1IXyyN8rh9ycMtnsuGKxk9kPqTN/nUgyl6t ibjUEGvETaptRy5VB8ovSHwrKh1+6LxooBefTD6Tv8D2FL9GLC7UBbdFtd3ghJwAqq47 0eAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=yUjqGaXxNRIkWFGwl+2pUdgstDXS94lxvTblB9yzYuE=; b=kGoGXYoOafQctrzz39rP9nQLHSMN1ivxnV6uWoHADEtov+v3UmM/Kkp70Un+V1tngp 6YpCEXaWtlyVJGN/SH8dlJZxYPZfOsKPLsZlZm3XgphPvILzUHldm/0H3W75AQLe9D56 1axY2sl/lnJLda1Cvl/Ag+Prkcxpy+RhYAmUJdBGQTDyWgi1KhWtsQCCZ4TuJY+cV95f EYEdF3B7solZcLIY/gbQHAeL0I5n7bKJs/gUd1wlI2s97SNeN/RykoF6fC6aCVx3pQ6C ZkZZN0pIFJtpeH1ABE2Zw09zj8+u99a/VphippelM6X9WbuXharAHF6Tgfm8mtj2A/Yj W+xQ== X-Gm-Message-State: AJIora8smm1/a8QMEms4TTGTEEUd5LFAd/mMqWrxXTEvcGfJmyHLTeT0 +qU8op7bdq8EeaiBh8XcI0LLP/W4/Og= X-Google-Smtp-Source: AGRyM1tq+cKZgY4d++EmVCawqmZ05GuVHpsnrWVt/0hYjlpfSVIk4z9NrsOkzJTeXpY7o67QjtJoJD4vVbc= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a81:13d7:0:b0:31c:c22b:4727 with SMTP id 206-20020a8113d7000000b0031cc22b4727mr19236561ywt.38.1657926024502; Fri, 15 Jul 2022 16:00:24 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 15 Jul 2022 23:00:15 +0000 In-Reply-To: <20220715230016.3762909-1-seanjc@google.com> Message-Id: <20220715230016.3762909-4-seanjc@google.com> Mime-Version: 1.0 References: <20220715230016.3762909-1-seanjc@google.com> X-Mailer: git-send-email 2.37.0.170.g444d1eabd0-goog Subject: [PATCH 3/4] KVM: x86/mmu: Add shadow mask for effective host MTRR memtype From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add shadow_memtype_mask to capture that EPT needs a non-zero memtype mask instead of relying on TDP being enabled, as NPT doesn't need a non-zero mask. This is a glorified nop as kvm_x86_ops.get_mt_mask() returns zero for NPT anyways. No functional change intended. Signed-off-by: Sean Christopherson Reviewed-by: Maxim Levitsky --- arch/x86/kvm/mmu/spte.c | 21 ++++++++++++++++++--- arch/x86/kvm/mmu/spte.h | 1 + 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index fb1f17504138..7314d27d57a4 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -33,6 +33,7 @@ u64 __read_mostly shadow_mmio_value; u64 __read_mostly shadow_mmio_mask; u64 __read_mostly shadow_mmio_access_mask; u64 __read_mostly shadow_present_mask; +u64 __read_mostly shadow_memtype_mask; u64 __read_mostly shadow_me_value; u64 __read_mostly shadow_me_mask; u64 __read_mostly shadow_acc_track_mask; @@ -161,10 +162,10 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_= page *sp, =20 if (level > PG_LEVEL_4K) spte |=3D PT_PAGE_SIZE_MASK; - if (tdp_enabled) + + if (shadow_memtype_mask) spte |=3D static_call(kvm_x86_get_mt_mask)(vcpu, gfn, - kvm_is_mmio_pfn(pfn)); - + kvm_is_mmio_pfn(pfn)); if (host_writable) spte |=3D shadow_host_writable_mask; else @@ -391,6 +392,13 @@ void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_= exec_only) shadow_nx_mask =3D 0ull; shadow_x_mask =3D VMX_EPT_EXECUTABLE_MASK; shadow_present_mask =3D has_exec_only ? 0ull : VMX_EPT_READABLE_MASK; + /* + * EPT overrides the host MTRRs, and so KVM must program the desired + * memtype directly into the SPTEs. Note, this mask is just the mask + * of all bits that factor into the memtype, the actual memtype must be + * dynamically calculated, e.g. to ensure host MMIO is mapped UC. + */ + shadow_memtype_mask =3D VMX_EPT_MT_MASK | VMX_EPT_IPAT_BIT; shadow_acc_track_mask =3D VMX_EPT_RWX_MASK; shadow_host_writable_mask =3D EPT_SPTE_HOST_WRITABLE; shadow_mmu_writable_mask =3D EPT_SPTE_MMU_WRITABLE; @@ -441,6 +449,13 @@ void kvm_mmu_reset_all_pte_masks(void) shadow_nx_mask =3D PT64_NX_MASK; shadow_x_mask =3D 0; shadow_present_mask =3D PT_PRESENT_MASK; + + /* + * For shadow paging and NPT, KVM uses PAT entry '0' to encode WB + * memtype in the SPTEs, i.e. relies on host MTRRs to provide the + * correct memtype (WB is the "weakest" memtype). + */ + shadow_memtype_mask =3D 0; shadow_acc_track_mask =3D 0; shadow_me_mask =3D 0; shadow_me_value =3D 0; diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index ba3dccb202bc..cabe3fbb4f39 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -147,6 +147,7 @@ extern u64 __read_mostly shadow_mmio_value; extern u64 __read_mostly shadow_mmio_mask; extern u64 __read_mostly shadow_mmio_access_mask; extern u64 __read_mostly shadow_present_mask; +extern u64 __read_mostly shadow_memtype_mask; extern u64 __read_mostly shadow_me_value; extern u64 __read_mostly shadow_me_mask; =20 --=20 2.37.0.170.g444d1eabd0-goog From nobody Sat Apr 18 07:42:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83BD6C43334 for ; Fri, 15 Jul 2022 23:01:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231655AbiGOXBD (ORCPT ); Fri, 15 Jul 2022 19:01:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230371AbiGOXA2 (ORCPT ); Fri, 15 Jul 2022 19:00:28 -0400 Received: from mail-pg1-x54a.google.com (mail-pg1-x54a.google.com [IPv6:2607:f8b0:4864:20::54a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6AF844E851 for ; Fri, 15 Jul 2022 16:00:27 -0700 (PDT) Received: by mail-pg1-x54a.google.com with SMTP id l71-20020a63914a000000b00419deb1ba88so1533623pge.5 for ; Fri, 15 Jul 2022 16:00:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=qyJeBdXwF5dNNWMBDELyZ0rPk0zOo8cNBsAR84NI3Pc=; b=s9nFvsxyVGJKS1GkkPlQocaGQPmzqZ8uonf6tpQY6iPa8ARVd585TI3w+2MAYJChA6 RL8d3MEoa4I/hstCMnfcU73XTiSG9gtZ4upagsnWDVM6B5KevwWwwSPCGqYRrtcSdRnw Foh0IHfWfS2P5NERSQOrvWXjXllKqbvD5mjBoawU6eGxdc194RqjecJKFmBAl8JQuMmA uj6mJb3MhGsGETgB2WJn5DSmZkZv1p8LRakMXhYPd6QzzyvZOyQts7GIuwB6/jKgCdtI JcTfbAOBThkzGrTnkEaSsVcPmzYsTXvyNEvGauh13geubQFqttp8ZimYmHjPdSvIDVDZ x6mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=qyJeBdXwF5dNNWMBDELyZ0rPk0zOo8cNBsAR84NI3Pc=; b=2yUDUNtFeEjMtm8sKP9Nyx0WF40GNBJG8+GJ5b9uWxZ/aoAzUJLHVYLH3pjAgUkHFA v0H0ifCnOUUXNaUJrNkBIb/Xm4bLrsQaKzu/jvmihR7U85fXdSsVAzEHM9uQPXGwXN1H KESYCAAoqk6iDUWuY7ObeZoYcZ94xnva26Hl6p3U+tWqF0QiAorE8js2NowjfI4CKjX5 OFTV/pY/Vi428ZJwWZVoImGGVNc8ESqwckDt8ZlEceTgI5FK8xPoAgMur3e523uvrpc1 3E+i9bm5Hic3An9vsEsJgfYzxL2BPBILpSaVfvF+GmIS9gDnK9XEO+pLg+870vnShQJ5 UUEw== X-Gm-Message-State: AJIora+Nqc5okr1Dgzl8kr17KvG6cYN/NBXRGCUX6dUjvBrNFc9D7eBV Egnv8ratOK/DW8UjQrMIisTlzFfyaqc= X-Google-Smtp-Source: AGRyM1sOQH8hQ8hf4xM2w/u1OrR0ibxbHFPEKSoDDrbPxn2n+mONs5jxdLyEZrumJUwzh4RWPB/CyyZIFsk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:90b:181:b0:1ef:c348:6835 with SMTP id t1-20020a17090b018100b001efc3486835mr913262pjs.1.1657926026078; Fri, 15 Jul 2022 16:00:26 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 15 Jul 2022 23:00:16 +0000 In-Reply-To: <20220715230016.3762909-1-seanjc@google.com> Message-Id: <20220715230016.3762909-5-seanjc@google.com> Mime-Version: 1.0 References: <20220715230016.3762909-1-seanjc@google.com> X-Mailer: git-send-email 2.37.0.170.g444d1eabd0-goog Subject: [PATCH 4/4] KVM: x86/mmu: Restrict mapping level based on guest MTRR iff they're used From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Restrict the mapping level for SPTEs based on the guest MTRRs if and only if KVM may actually use the guest MTRRs to compute the "real" memtype. For all forms of paging, guest MTRRs are purely virtual in the sense that they are completely ignored by hardware, i.e. they affect the memtype only if software manually consumes them. The only scenario where KVM consumes the guest MTRRs is when shadow_memtype_mask is non-zero and the guest has non-coherent DMA, in all other cases KVM simply leaves the PAT field in SPTEs as '0' to encode WB memtype. Note, KVM may still ultimately ignore guest MTRRs, e.g. if the backing pfn is host MMIO, but false positives are ok as they only cause a slight performance blip (unless the guest is doing weird things with its MTRRs, which is extremely unlikely). Signed-off-by: Sean Christopherson Reviewed-by: Maxim Levitsky --- arch/x86/kvm/mmu/mmu.c | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 52664c3caaab..82f38af06f5c 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4295,14 +4295,26 @@ EXPORT_SYMBOL_GPL(kvm_handle_page_fault); =20 int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { - while (fault->max_level > PG_LEVEL_4K) { - int page_num =3D KVM_PAGES_PER_HPAGE(fault->max_level); - gfn_t base =3D (fault->addr >> PAGE_SHIFT) & ~(page_num - 1); + /* + * If the guest's MTRRs may be used to compute the "real" memtype, + * restrict the mapping level to ensure KVM uses a consistent memtype + * across the entire mapping. If the host MTRRs are ignored by TDP + * (shadow_memtype_mask is non-zero), and the VM has non-coherent DMA + * (DMA doesn't snoop CPU caches), KVM's ABI is to honor the memtype + * from the guest's MTRRs so that guest accesses to memory that is + * DMA'd aren't cached against the guest's wishes. + * + * Note, KVM may still ultimately ignore guest MTRRs for certain PFNs, + * e.g. KVM will force UC memtype for host MMIO. + */ + if (shadow_memtype_mask && kvm_arch_has_noncoherent_dma(vcpu->kvm)) { + for ( ; fault->max_level > PG_LEVEL_4K; --fault->max_level) { + int page_num =3D KVM_PAGES_PER_HPAGE(fault->max_level); + gfn_t base =3D (fault->addr >> PAGE_SHIFT) & ~(page_num - 1); =20 - if (kvm_mtrr_check_gfn_range_consistency(vcpu, base, page_num)) - break; - - --fault->max_level; + if (kvm_mtrr_check_gfn_range_consistency(vcpu, base, page_num)) + break; + } } =20 return direct_page_fault(vcpu, fault); --=20 2.37.0.170.g444d1eabd0-goog